A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets

Kamesh Madduri, David Ediger, Karl Jiang, David A. Bader, Daniel Chavarría-Miranda

Research output: Chapter in Book/Report/Conference proceedingConference contribution

87 Citations (Scopus)

Abstract

We present a new lock-free parallel algorithm for computing betweenness centrality of massive complex networks that achieves better spatial locality compared with previous approaches. Betweenness centrality is a key kernel in analyzing the importance of vertices (or edges) in applications ranging from social networks, to power grids, to the influence of jazz musicians, and is also incorporated into the DARPA HPCS SSCA#2, a benchmark extensively used to evaluate the performance of emerging high-performance computing architectures for graph analytics. We design an optimized implementation of betweenness centrality for the massively multithreaded Cray XMT system with the Threadstorm processor. For a small-world network of 268 million vertices and 2.147 billion edges, the 16-processor XMT system achieves a TEPS rate (an algorithmic performance count for the number of edges traversed per second) of 160 million per second, which corresponds to more than a 2× performance improvement over the previous parallel implementation. We demonstrate the applicability of our implementation to analyze massive real-world datasets by computing approximate betweenness centrality for the large IMDb movie-actor network.

Original languageEnglish (US)
Title of host publicationIPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium
DOIs
StatePublished - Nov 25 2009
Event23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009 - Rome, Italy
Duration: May 23 2009May 29 2009

Publication series

NameIPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium

Other

Other23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009
CountryItaly
CityRome
Period5/23/095/29/09

Fingerprint

Small-world networks
Complex networks
Parallel algorithms

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Hardware and Architecture
  • Software

Cite this

Madduri, K., Ediger, D., Jiang, K., Bader, D. A., & Chavarría-Miranda, D. (2009). A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets. In IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium [5161100] (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium). https://doi.org/10.1109/IPDPS.2009.5161100
Madduri, Kamesh ; Ediger, David ; Jiang, Karl ; Bader, David A. ; Chavarría-Miranda, Daniel. / A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets. IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium. 2009. (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium).
@inproceedings{d3e65722239f4cc68fc5f5117344eb96,
title = "A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets",
abstract = "We present a new lock-free parallel algorithm for computing betweenness centrality of massive complex networks that achieves better spatial locality compared with previous approaches. Betweenness centrality is a key kernel in analyzing the importance of vertices (or edges) in applications ranging from social networks, to power grids, to the influence of jazz musicians, and is also incorporated into the DARPA HPCS SSCA#2, a benchmark extensively used to evaluate the performance of emerging high-performance computing architectures for graph analytics. We design an optimized implementation of betweenness centrality for the massively multithreaded Cray XMT system with the Threadstorm processor. For a small-world network of 268 million vertices and 2.147 billion edges, the 16-processor XMT system achieves a TEPS rate (an algorithmic performance count for the number of edges traversed per second) of 160 million per second, which corresponds to more than a 2× performance improvement over the previous parallel implementation. We demonstrate the applicability of our implementation to analyze massive real-world datasets by computing approximate betweenness centrality for the large IMDb movie-actor network.",
author = "Kamesh Madduri and David Ediger and Karl Jiang and Bader, {David A.} and Daniel Chavarr{\'i}a-Miranda",
year = "2009",
month = "11",
day = "25",
doi = "10.1109/IPDPS.2009.5161100",
language = "English (US)",
isbn = "9781424437504",
series = "IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium",
booktitle = "IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium",

}

Madduri, K, Ediger, D, Jiang, K, Bader, DA & Chavarría-Miranda, D 2009, A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets. in IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium., 5161100, IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium, 23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009, Rome, Italy, 5/23/09. https://doi.org/10.1109/IPDPS.2009.5161100

A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets. / Madduri, Kamesh; Ediger, David; Jiang, Karl; Bader, David A.; Chavarría-Miranda, Daniel.

IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium. 2009. 5161100 (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets

AU - Madduri, Kamesh

AU - Ediger, David

AU - Jiang, Karl

AU - Bader, David A.

AU - Chavarría-Miranda, Daniel

PY - 2009/11/25

Y1 - 2009/11/25

N2 - We present a new lock-free parallel algorithm for computing betweenness centrality of massive complex networks that achieves better spatial locality compared with previous approaches. Betweenness centrality is a key kernel in analyzing the importance of vertices (or edges) in applications ranging from social networks, to power grids, to the influence of jazz musicians, and is also incorporated into the DARPA HPCS SSCA#2, a benchmark extensively used to evaluate the performance of emerging high-performance computing architectures for graph analytics. We design an optimized implementation of betweenness centrality for the massively multithreaded Cray XMT system with the Threadstorm processor. For a small-world network of 268 million vertices and 2.147 billion edges, the 16-processor XMT system achieves a TEPS rate (an algorithmic performance count for the number of edges traversed per second) of 160 million per second, which corresponds to more than a 2× performance improvement over the previous parallel implementation. We demonstrate the applicability of our implementation to analyze massive real-world datasets by computing approximate betweenness centrality for the large IMDb movie-actor network.

AB - We present a new lock-free parallel algorithm for computing betweenness centrality of massive complex networks that achieves better spatial locality compared with previous approaches. Betweenness centrality is a key kernel in analyzing the importance of vertices (or edges) in applications ranging from social networks, to power grids, to the influence of jazz musicians, and is also incorporated into the DARPA HPCS SSCA#2, a benchmark extensively used to evaluate the performance of emerging high-performance computing architectures for graph analytics. We design an optimized implementation of betweenness centrality for the massively multithreaded Cray XMT system with the Threadstorm processor. For a small-world network of 268 million vertices and 2.147 billion edges, the 16-processor XMT system achieves a TEPS rate (an algorithmic performance count for the number of edges traversed per second) of 160 million per second, which corresponds to more than a 2× performance improvement over the previous parallel implementation. We demonstrate the applicability of our implementation to analyze massive real-world datasets by computing approximate betweenness centrality for the large IMDb movie-actor network.

UR - http://www.scopus.com/inward/record.url?scp=70449792770&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70449792770&partnerID=8YFLogxK

U2 - 10.1109/IPDPS.2009.5161100

DO - 10.1109/IPDPS.2009.5161100

M3 - Conference contribution

SN - 9781424437504

T3 - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium

BT - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium

ER -

Madduri K, Ediger D, Jiang K, Bader DA, Chavarría-Miranda D. A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets. In IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium. 2009. 5161100. (IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium). https://doi.org/10.1109/IPDPS.2009.5161100