Ranking causal anomalies by modeling local propagations on networked systems

Jingchao Ni, Wei Cheng, Kai Zhang, Dongjin Song, Tan Yan, Haifeng Chen, Xiang Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Complex systems are prevalent in many fields such as finance, security and industry. A fundamental problem in system management is to perform diagnosis in case of system failure such that the causal anomalies, i.e., root causes, can be identified for system debugging and repair. Recently, invariant network has proven a powerful tool in characterizing complex system behaviors. In an invariant network, a node represents a system component, and an edge indicates a stable interaction between two components. Recent approaches have shown that by modeling fault propagation in the invariant network, causal anomalies can be effectively discovered. Despite their success, the existing methods have a major limitation: they typically assume there is only a single and global fault propagation in the entire network. However, in real-world large-scale complex systems, it's more common for multiple fault propagations to grow simultaneously and locally within different node clusters and jointly define the system failure status. Inspired by this key observation, we propose a two-phase framework to identify and rank causal anomalies. In the first phase, a probabilistic clustering is performed to uncover impaired node clusters in the invariant network. Then, in the second phase, a low-rank network diffusion model is designed to backtrack causal anomalies in different impaired clusters. Extensive experimental results on real-life datasets demonstrate the effectiveness of our method.

Original languageEnglish (US)
Title of host publicationProceedings - 17th IEEE International Conference on Data Mining, ICDM 2017
EditorsGeorge Karypis, Srinivas Alu, Vijay Raghavan, Xindong Wu, Lucio Miele
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1003-1008
Number of pages6
ISBN (Electronic)9781538638347
DOIs
StatePublished - Dec 15 2017
Event17th IEEE International Conference on Data Mining, ICDM 2017 - New Orleans, United States
Duration: Nov 18 2017Nov 21 2017

Publication series

NameProceedings - IEEE International Conference on Data Mining, ICDM
Volume2017-November
ISSN (Print)1550-4786

Other

Other17th IEEE International Conference on Data Mining, ICDM 2017
CountryUnited States
CityNew Orleans
Period11/18/1711/21/17

Fingerprint

Large scale systems
Finance
Repair
Industry

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Ni, J., Cheng, W., Zhang, K., Song, D., Yan, T., Chen, H., & Zhang, X. (2017). Ranking causal anomalies by modeling local propagations on networked systems. In G. Karypis, S. Alu, V. Raghavan, X. Wu, & L. Miele (Eds.), Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017 (pp. 1003-1008). (Proceedings - IEEE International Conference on Data Mining, ICDM; Vol. 2017-November). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2017.129
Ni, Jingchao ; Cheng, Wei ; Zhang, Kai ; Song, Dongjin ; Yan, Tan ; Chen, Haifeng ; Zhang, Xiang. / Ranking causal anomalies by modeling local propagations on networked systems. Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. editor / George Karypis ; Srinivas Alu ; Vijay Raghavan ; Xindong Wu ; Lucio Miele. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 1003-1008 (Proceedings - IEEE International Conference on Data Mining, ICDM).
@inproceedings{c39434d368dd4698a03a96dd26c76d96,
title = "Ranking causal anomalies by modeling local propagations on networked systems",
abstract = "Complex systems are prevalent in many fields such as finance, security and industry. A fundamental problem in system management is to perform diagnosis in case of system failure such that the causal anomalies, i.e., root causes, can be identified for system debugging and repair. Recently, invariant network has proven a powerful tool in characterizing complex system behaviors. In an invariant network, a node represents a system component, and an edge indicates a stable interaction between two components. Recent approaches have shown that by modeling fault propagation in the invariant network, causal anomalies can be effectively discovered. Despite their success, the existing methods have a major limitation: they typically assume there is only a single and global fault propagation in the entire network. However, in real-world large-scale complex systems, it's more common for multiple fault propagations to grow simultaneously and locally within different node clusters and jointly define the system failure status. Inspired by this key observation, we propose a two-phase framework to identify and rank causal anomalies. In the first phase, a probabilistic clustering is performed to uncover impaired node clusters in the invariant network. Then, in the second phase, a low-rank network diffusion model is designed to backtrack causal anomalies in different impaired clusters. Extensive experimental results on real-life datasets demonstrate the effectiveness of our method.",
author = "Jingchao Ni and Wei Cheng and Kai Zhang and Dongjin Song and Tan Yan and Haifeng Chen and Xiang Zhang",
year = "2017",
month = "12",
day = "15",
doi = "10.1109/ICDM.2017.129",
language = "English (US)",
series = "Proceedings - IEEE International Conference on Data Mining, ICDM",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1003--1008",
editor = "George Karypis and Srinivas Alu and Vijay Raghavan and Xindong Wu and Lucio Miele",
booktitle = "Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017",
address = "United States",

}

Ni, J, Cheng, W, Zhang, K, Song, D, Yan, T, Chen, H & Zhang, X 2017, Ranking causal anomalies by modeling local propagations on networked systems. in G Karypis, S Alu, V Raghavan, X Wu & L Miele (eds), Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. Proceedings - IEEE International Conference on Data Mining, ICDM, vol. 2017-November, Institute of Electrical and Electronics Engineers Inc., pp. 1003-1008, 17th IEEE International Conference on Data Mining, ICDM 2017, New Orleans, United States, 11/18/17. https://doi.org/10.1109/ICDM.2017.129

Ranking causal anomalies by modeling local propagations on networked systems. / Ni, Jingchao; Cheng, Wei; Zhang, Kai; Song, Dongjin; Yan, Tan; Chen, Haifeng; Zhang, Xiang.

Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. ed. / George Karypis; Srinivas Alu; Vijay Raghavan; Xindong Wu; Lucio Miele. Institute of Electrical and Electronics Engineers Inc., 2017. p. 1003-1008 (Proceedings - IEEE International Conference on Data Mining, ICDM; Vol. 2017-November).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Ranking causal anomalies by modeling local propagations on networked systems

AU - Ni, Jingchao

AU - Cheng, Wei

AU - Zhang, Kai

AU - Song, Dongjin

AU - Yan, Tan

AU - Chen, Haifeng

AU - Zhang, Xiang

PY - 2017/12/15

Y1 - 2017/12/15

N2 - Complex systems are prevalent in many fields such as finance, security and industry. A fundamental problem in system management is to perform diagnosis in case of system failure such that the causal anomalies, i.e., root causes, can be identified for system debugging and repair. Recently, invariant network has proven a powerful tool in characterizing complex system behaviors. In an invariant network, a node represents a system component, and an edge indicates a stable interaction between two components. Recent approaches have shown that by modeling fault propagation in the invariant network, causal anomalies can be effectively discovered. Despite their success, the existing methods have a major limitation: they typically assume there is only a single and global fault propagation in the entire network. However, in real-world large-scale complex systems, it's more common for multiple fault propagations to grow simultaneously and locally within different node clusters and jointly define the system failure status. Inspired by this key observation, we propose a two-phase framework to identify and rank causal anomalies. In the first phase, a probabilistic clustering is performed to uncover impaired node clusters in the invariant network. Then, in the second phase, a low-rank network diffusion model is designed to backtrack causal anomalies in different impaired clusters. Extensive experimental results on real-life datasets demonstrate the effectiveness of our method.

AB - Complex systems are prevalent in many fields such as finance, security and industry. A fundamental problem in system management is to perform diagnosis in case of system failure such that the causal anomalies, i.e., root causes, can be identified for system debugging and repair. Recently, invariant network has proven a powerful tool in characterizing complex system behaviors. In an invariant network, a node represents a system component, and an edge indicates a stable interaction between two components. Recent approaches have shown that by modeling fault propagation in the invariant network, causal anomalies can be effectively discovered. Despite their success, the existing methods have a major limitation: they typically assume there is only a single and global fault propagation in the entire network. However, in real-world large-scale complex systems, it's more common for multiple fault propagations to grow simultaneously and locally within different node clusters and jointly define the system failure status. Inspired by this key observation, we propose a two-phase framework to identify and rank causal anomalies. In the first phase, a probabilistic clustering is performed to uncover impaired node clusters in the invariant network. Then, in the second phase, a low-rank network diffusion model is designed to backtrack causal anomalies in different impaired clusters. Extensive experimental results on real-life datasets demonstrate the effectiveness of our method.

UR - http://www.scopus.com/inward/record.url?scp=85043976130&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85043976130&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2017.129

DO - 10.1109/ICDM.2017.129

M3 - Conference contribution

AN - SCOPUS:85043976130

T3 - Proceedings - IEEE International Conference on Data Mining, ICDM

SP - 1003

EP - 1008

BT - Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017

A2 - Karypis, George

A2 - Alu, Srinivas

A2 - Raghavan, Vijay

A2 - Wu, Xindong

A2 - Miele, Lucio

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Ni J, Cheng W, Zhang K, Song D, Yan T, Chen H et al. Ranking causal anomalies by modeling local propagations on networked systems. In Karypis G, Alu S, Raghavan V, Wu X, Miele L, editors, Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 1003-1008. (Proceedings - IEEE International Conference on Data Mining, ICDM). https://doi.org/10.1109/ICDM.2017.129