Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis

Kamesh Madduri, David A. Bader

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Citations (Scopus)

Abstract

Graph-theoretic abstractions are extensively used to analyze massive data sets. Temporal data streams from socioeconomic interactions, social networking web sites, communication traffic, and scientific computing can be intuitively modeled as graphs. We present the first study of novel highperformance combinatorial techniques for analyzing largescale information networks, encapsulating dynamic interaction data in the order of billions of entities. We present new data structures to represent dynamic interaction networks, and discuss algorithms for processing parallel insertions and deletions of edges in small-world networks. With these new approaches, we achieve an average performance rate of 25 million structural updates per second and a parallel speedup of nearly 28 on a 64-way Sun UltraSPARC T2 multicore processor, for insertions and deletions to a small-world network of 33.5 million vertices and 268 million edges. We also design parallel implementations of fundamental dynamic graph kernels related to connectivity and centrality queries. Our implementations are freely distributed as part of the open-source SNAP (Small-world Network Analysis and Partitioning) complex network analysis framework.

Original languageEnglish (US)
Title of host publicationIPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium
DOIs
StatePublished - Nov 25 2009
Event23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009 - Rome, Italy
Duration: May 23 2009May 29 2009

Publication series

NameIPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium

Other

Other23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009
CountryItaly
CityRome
Period5/23/095/29/09

Fingerprint

Small-world networks
Electric network analysis
Parallel algorithms
Natural sciences computing
Complex networks
Telecommunication traffic
Sun
Data structures
Websites
Processing

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Hardware and Architecture
  • Software

Cite this

Madduri, K., & Bader, D. A. (2009). Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis. In IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium [5161060] (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium). https://doi.org/10.1109/IPDPS.2009.5161060
Madduri, Kamesh ; Bader, David A. / Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis. IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium. 2009. (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium).
@inproceedings{c673915d6c324158a0411a790bb41515,
title = "Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis",
abstract = "Graph-theoretic abstractions are extensively used to analyze massive data sets. Temporal data streams from socioeconomic interactions, social networking web sites, communication traffic, and scientific computing can be intuitively modeled as graphs. We present the first study of novel highperformance combinatorial techniques for analyzing largescale information networks, encapsulating dynamic interaction data in the order of billions of entities. We present new data structures to represent dynamic interaction networks, and discuss algorithms for processing parallel insertions and deletions of edges in small-world networks. With these new approaches, we achieve an average performance rate of 25 million structural updates per second and a parallel speedup of nearly 28 on a 64-way Sun UltraSPARC T2 multicore processor, for insertions and deletions to a small-world network of 33.5 million vertices and 268 million edges. We also design parallel implementations of fundamental dynamic graph kernels related to connectivity and centrality queries. Our implementations are freely distributed as part of the open-source SNAP (Small-world Network Analysis and Partitioning) complex network analysis framework.",
author = "Kamesh Madduri and Bader, {David A.}",
year = "2009",
month = "11",
day = "25",
doi = "10.1109/IPDPS.2009.5161060",
language = "English (US)",
isbn = "9781424437504",
series = "IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium",
booktitle = "IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium",

}

Madduri, K & Bader, DA 2009, Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis. in IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium., 5161060, IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium, 23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009, Rome, Italy, 5/23/09. https://doi.org/10.1109/IPDPS.2009.5161060

Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis. / Madduri, Kamesh; Bader, David A.

IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium. 2009. 5161060 (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis

AU - Madduri, Kamesh

AU - Bader, David A.

PY - 2009/11/25

Y1 - 2009/11/25

N2 - Graph-theoretic abstractions are extensively used to analyze massive data sets. Temporal data streams from socioeconomic interactions, social networking web sites, communication traffic, and scientific computing can be intuitively modeled as graphs. We present the first study of novel highperformance combinatorial techniques for analyzing largescale information networks, encapsulating dynamic interaction data in the order of billions of entities. We present new data structures to represent dynamic interaction networks, and discuss algorithms for processing parallel insertions and deletions of edges in small-world networks. With these new approaches, we achieve an average performance rate of 25 million structural updates per second and a parallel speedup of nearly 28 on a 64-way Sun UltraSPARC T2 multicore processor, for insertions and deletions to a small-world network of 33.5 million vertices and 268 million edges. We also design parallel implementations of fundamental dynamic graph kernels related to connectivity and centrality queries. Our implementations are freely distributed as part of the open-source SNAP (Small-world Network Analysis and Partitioning) complex network analysis framework.

AB - Graph-theoretic abstractions are extensively used to analyze massive data sets. Temporal data streams from socioeconomic interactions, social networking web sites, communication traffic, and scientific computing can be intuitively modeled as graphs. We present the first study of novel highperformance combinatorial techniques for analyzing largescale information networks, encapsulating dynamic interaction data in the order of billions of entities. We present new data structures to represent dynamic interaction networks, and discuss algorithms for processing parallel insertions and deletions of edges in small-world networks. With these new approaches, we achieve an average performance rate of 25 million structural updates per second and a parallel speedup of nearly 28 on a 64-way Sun UltraSPARC T2 multicore processor, for insertions and deletions to a small-world network of 33.5 million vertices and 268 million edges. We also design parallel implementations of fundamental dynamic graph kernels related to connectivity and centrality queries. Our implementations are freely distributed as part of the open-source SNAP (Small-world Network Analysis and Partitioning) complex network analysis framework.

UR - http://www.scopus.com/inward/record.url?scp=70449844302&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70449844302&partnerID=8YFLogxK

U2 - 10.1109/IPDPS.2009.5161060

DO - 10.1109/IPDPS.2009.5161060

M3 - Conference contribution

AN - SCOPUS:70449844302

SN - 9781424437504

T3 - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium

BT - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium

ER -

Madduri K, Bader DA. Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis. In IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium. 2009. 5161060. (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium). https://doi.org/10.1109/IPDPS.2009.5161060