A self-organizing method for predictive modeling with highly-redundant variables

Gang Liu, Hui Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Rapid advancement of sensing and information technology brings the big data, which presents a gold mine of the 21st century. However, big data also brings significant challenges for data-driven decision making. In particular, it is not uncommon that a large number of variables (or features) underlie the big data. Complex interdependence structures among variables challenge the traditional framework of predictive modeling. This paper presents a new methodology of self-organizing network for variable clustering and predictive modeling. Specifically, we developed a new approach, namely nonlinear coupling analysis to measure nonlinear interdependence structures among variables. Further, all the variables are embedded as nodes in a complex network. Nonlinear-coupling forces move these nodes to derive a self-organizing topology of network. As such, variables are clustered as sub-network communities in the space. Experimental results demonstrated that the proposed methodology not only outperforms traditional variable clustering algorithms such as hierarchical clustering and oblique principal component analysis, but also effectively identify interdependent structures among variables and further improves the performance of predictive modeling. The proposed new idea of self-organizing network is generally applicable for predictive modeling in many disciplines that involve a large number of highly-redundant variables.

Original languageEnglish (US)
Title of host publication2015 IEEE Conference on Automation Science and Engineering
Subtitle of host publicationAutomation for a Sustainable Future, CASE 2015
PublisherIEEE Computer Society
Pages1084-1089
Number of pages6
ISBN (Electronic)9781467381833
DOIs
StatePublished - Oct 7 2015
Event11th IEEE International Conference on Automation Science and Engineering, CASE 2015 - Gothenburg, Sweden
Duration: Aug 24 2015Aug 28 2015

Publication series

NameIEEE International Conference on Automation Science and Engineering
Volume2015-October
ISSN (Print)2161-8070
ISSN (Electronic)2161-8089

Other

Other11th IEEE International Conference on Automation Science and Engineering, CASE 2015
CountrySweden
CityGothenburg
Period8/24/158/28/15

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'A self-organizing method for predictive modeling with highly-redundant variables'. Together they form a unique fingerprint.

Cite this