Spectral Entropies as Information-Theoretic Tools for Complex Network Comparison

Manlio De Domenico, Jacob Biamonte

Research output: Contribution to journalArticle

25 Citations (Scopus)

Abstract

Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we use techniques inspired by quantum statistical mechanics to define an entropy measure for complex networks and to develop a set of information-theoretic tools, based on network spectral properties, such as Rényi q entropy, generalized Kullback-Leibler and Jensen-Shannon divergences, the latter allowing us to define a natural distance measure between complex networks. First, we show that by minimizing the Kullback-Leibler divergence between an observed network and a parametric network model, inference of model parameter(s) by means of maximum-likelihood estimation can be achieved and model selection can be performed with appropriate information criteria. Second, we show that the information-theoretic metric quantifies the distance between pairs of networks and we can use it, for instance, to cluster the layers of a multilayer system. By applying this framework to networks corresponding to sites of the human microbiome, we perform hierarchical cluster analysis and recover with high accuracy existing community-based associations. Our results imply that spectral-based statistical inference in complex networks results in demonstrably superior performance as well as a conceptual backbone, filling a gap towards a network information theory.

Original languageEnglish (US)
Article number041062
JournalPhysical Review X
Volume6
Issue number4
DOIs
StatePublished - Dec 21 2016

Fingerprint

entropy
inference
divergence
cluster analysis
information theory
statistical mechanics

All Science Journal Classification (ASJC) codes

  • Physics and Astronomy(all)

Cite this

De Domenico, Manlio ; Biamonte, Jacob. / Spectral Entropies as Information-Theoretic Tools for Complex Network Comparison. In: Physical Review X. 2016 ; Vol. 6, No. 4.
@article{bd3adcd2674e4b0e901e3ec141eb36bd,
title = "Spectral Entropies as Information-Theoretic Tools for Complex Network Comparison",
abstract = "Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we use techniques inspired by quantum statistical mechanics to define an entropy measure for complex networks and to develop a set of information-theoretic tools, based on network spectral properties, such as R{\'e}nyi q entropy, generalized Kullback-Leibler and Jensen-Shannon divergences, the latter allowing us to define a natural distance measure between complex networks. First, we show that by minimizing the Kullback-Leibler divergence between an observed network and a parametric network model, inference of model parameter(s) by means of maximum-likelihood estimation can be achieved and model selection can be performed with appropriate information criteria. Second, we show that the information-theoretic metric quantifies the distance between pairs of networks and we can use it, for instance, to cluster the layers of a multilayer system. By applying this framework to networks corresponding to sites of the human microbiome, we perform hierarchical cluster analysis and recover with high accuracy existing community-based associations. Our results imply that spectral-based statistical inference in complex networks results in demonstrably superior performance as well as a conceptual backbone, filling a gap towards a network information theory.",
author = "{De Domenico}, Manlio and Jacob Biamonte",
year = "2016",
month = "12",
day = "21",
doi = "10.1103/PhysRevX.6.041062",
language = "English (US)",
volume = "6",
journal = "Physical Review X",
issn = "2160-3308",
publisher = "American Physical Society",
number = "4",

}

Spectral Entropies as Information-Theoretic Tools for Complex Network Comparison. / De Domenico, Manlio; Biamonte, Jacob.

In: Physical Review X, Vol. 6, No. 4, 041062, 21.12.2016.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Spectral Entropies as Information-Theoretic Tools for Complex Network Comparison

AU - De Domenico, Manlio

AU - Biamonte, Jacob

PY - 2016/12/21

Y1 - 2016/12/21

N2 - Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we use techniques inspired by quantum statistical mechanics to define an entropy measure for complex networks and to develop a set of information-theoretic tools, based on network spectral properties, such as Rényi q entropy, generalized Kullback-Leibler and Jensen-Shannon divergences, the latter allowing us to define a natural distance measure between complex networks. First, we show that by minimizing the Kullback-Leibler divergence between an observed network and a parametric network model, inference of model parameter(s) by means of maximum-likelihood estimation can be achieved and model selection can be performed with appropriate information criteria. Second, we show that the information-theoretic metric quantifies the distance between pairs of networks and we can use it, for instance, to cluster the layers of a multilayer system. By applying this framework to networks corresponding to sites of the human microbiome, we perform hierarchical cluster analysis and recover with high accuracy existing community-based associations. Our results imply that spectral-based statistical inference in complex networks results in demonstrably superior performance as well as a conceptual backbone, filling a gap towards a network information theory.

AB - Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we use techniques inspired by quantum statistical mechanics to define an entropy measure for complex networks and to develop a set of information-theoretic tools, based on network spectral properties, such as Rényi q entropy, generalized Kullback-Leibler and Jensen-Shannon divergences, the latter allowing us to define a natural distance measure between complex networks. First, we show that by minimizing the Kullback-Leibler divergence between an observed network and a parametric network model, inference of model parameter(s) by means of maximum-likelihood estimation can be achieved and model selection can be performed with appropriate information criteria. Second, we show that the information-theoretic metric quantifies the distance between pairs of networks and we can use it, for instance, to cluster the layers of a multilayer system. By applying this framework to networks corresponding to sites of the human microbiome, we perform hierarchical cluster analysis and recover with high accuracy existing community-based associations. Our results imply that spectral-based statistical inference in complex networks results in demonstrably superior performance as well as a conceptual backbone, filling a gap towards a network information theory.

UR - http://www.scopus.com/inward/record.url?scp=85013649861&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85013649861&partnerID=8YFLogxK

U2 - 10.1103/PhysRevX.6.041062

DO - 10.1103/PhysRevX.6.041062

M3 - Article

VL - 6

JO - Physical Review X

JF - Physical Review X

SN - 2160-3308

IS - 4

M1 - 041062

ER -