REDUS: Finding reducible subspaces in high dimensional data

Xiang Zhang, Feng Pan, Wei Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Finding latent patterns in high dimensional data is an important research problem with numerous applications. The most well known approaches for high dimensional data analysis are feature selection and dimensionality reduction. Being widely used in many applications, these methods aim to capture global patterns and are typically performed in the full feature space. In many emerging applications, however, scientists are interested in the local latent patterns held by feature subspaces, which may be invisible via any global transformation. In this paper, we investigate the problem of finding strong linear and nonlinear correlations hidden in feature subspaces of high dimensional data. We formalize this problem as identifying reducible subspaces in the full dimensional space. Intuitively, a reducible subspace is a feature subspace whose intrinsic dimensionality is smaller than the number of features. We present an efective algorithm, REDUS, for finding the reducible subspaces. Two key components of our algorithm are finding the overall reducible subspace, and uncovering the individual reducible subspaces from the overall reducible subspace. A broad experimental evaluation demonstrates the efectiveness of our algorithm.

Original languageEnglish (US)
Title of host publicationProceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08
Pages961-970
Number of pages10
DOIs
StatePublished - Dec 1 2008
Event17th ACM Conference on Information and Knowledge Management, CIKM'08 - Napa Valley, CA, United States
Duration: Oct 26 2008Oct 30 2008

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings

Other

Other17th ACM Conference on Information and Knowledge Management, CIKM'08
CountryUnited States
CityNapa Valley, CA
Period10/26/0810/30/08

Fingerprint

Feature selection
Dimensionality reduction
Evaluation
Intrinsic
Dimensionality

All Science Journal Classification (ASJC) codes

  • Decision Sciences(all)
  • Business, Management and Accounting(all)

Cite this

Zhang, X., Pan, F., & Wang, W. (2008). REDUS: Finding reducible subspaces in high dimensional data. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08 (pp. 961-970). (International Conference on Information and Knowledge Management, Proceedings). https://doi.org/10.1145/1458082.1458209
Zhang, Xiang ; Pan, Feng ; Wang, Wei. / REDUS : Finding reducible subspaces in high dimensional data. Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08. 2008. pp. 961-970 (International Conference on Information and Knowledge Management, Proceedings).
@inproceedings{911675d0f74a4a44bd5c44f0f7e6cd22,
title = "REDUS: Finding reducible subspaces in high dimensional data",
abstract = "Finding latent patterns in high dimensional data is an important research problem with numerous applications. The most well known approaches for high dimensional data analysis are feature selection and dimensionality reduction. Being widely used in many applications, these methods aim to capture global patterns and are typically performed in the full feature space. In many emerging applications, however, scientists are interested in the local latent patterns held by feature subspaces, which may be invisible via any global transformation. In this paper, we investigate the problem of finding strong linear and nonlinear correlations hidden in feature subspaces of high dimensional data. We formalize this problem as identifying reducible subspaces in the full dimensional space. Intuitively, a reducible subspace is a feature subspace whose intrinsic dimensionality is smaller than the number of features. We present an efective algorithm, REDUS, for finding the reducible subspaces. Two key components of our algorithm are finding the overall reducible subspace, and uncovering the individual reducible subspaces from the overall reducible subspace. A broad experimental evaluation demonstrates the efectiveness of our algorithm.",
author = "Xiang Zhang and Feng Pan and Wei Wang",
year = "2008",
month = "12",
day = "1",
doi = "10.1145/1458082.1458209",
language = "English (US)",
isbn = "9781595939913",
series = "International Conference on Information and Knowledge Management, Proceedings",
pages = "961--970",
booktitle = "Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08",

}

Zhang, X, Pan, F & Wang, W 2008, REDUS: Finding reducible subspaces in high dimensional data. in Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08. International Conference on Information and Knowledge Management, Proceedings, pp. 961-970, 17th ACM Conference on Information and Knowledge Management, CIKM'08, Napa Valley, CA, United States, 10/26/08. https://doi.org/10.1145/1458082.1458209

REDUS : Finding reducible subspaces in high dimensional data. / Zhang, Xiang; Pan, Feng; Wang, Wei.

Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08. 2008. p. 961-970 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - REDUS

T2 - Finding reducible subspaces in high dimensional data

AU - Zhang, Xiang

AU - Pan, Feng

AU - Wang, Wei

PY - 2008/12/1

Y1 - 2008/12/1

N2 - Finding latent patterns in high dimensional data is an important research problem with numerous applications. The most well known approaches for high dimensional data analysis are feature selection and dimensionality reduction. Being widely used in many applications, these methods aim to capture global patterns and are typically performed in the full feature space. In many emerging applications, however, scientists are interested in the local latent patterns held by feature subspaces, which may be invisible via any global transformation. In this paper, we investigate the problem of finding strong linear and nonlinear correlations hidden in feature subspaces of high dimensional data. We formalize this problem as identifying reducible subspaces in the full dimensional space. Intuitively, a reducible subspace is a feature subspace whose intrinsic dimensionality is smaller than the number of features. We present an efective algorithm, REDUS, for finding the reducible subspaces. Two key components of our algorithm are finding the overall reducible subspace, and uncovering the individual reducible subspaces from the overall reducible subspace. A broad experimental evaluation demonstrates the efectiveness of our algorithm.

AB - Finding latent patterns in high dimensional data is an important research problem with numerous applications. The most well known approaches for high dimensional data analysis are feature selection and dimensionality reduction. Being widely used in many applications, these methods aim to capture global patterns and are typically performed in the full feature space. In many emerging applications, however, scientists are interested in the local latent patterns held by feature subspaces, which may be invisible via any global transformation. In this paper, we investigate the problem of finding strong linear and nonlinear correlations hidden in feature subspaces of high dimensional data. We formalize this problem as identifying reducible subspaces in the full dimensional space. Intuitively, a reducible subspace is a feature subspace whose intrinsic dimensionality is smaller than the number of features. We present an efective algorithm, REDUS, for finding the reducible subspaces. Two key components of our algorithm are finding the overall reducible subspace, and uncovering the individual reducible subspaces from the overall reducible subspace. A broad experimental evaluation demonstrates the efectiveness of our algorithm.

UR - http://www.scopus.com/inward/record.url?scp=70349254659&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70349254659&partnerID=8YFLogxK

U2 - 10.1145/1458082.1458209

DO - 10.1145/1458082.1458209

M3 - Conference contribution

AN - SCOPUS:70349254659

SN - 9781595939913

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 961

EP - 970

BT - Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08

ER -

Zhang X, Pan F, Wang W. REDUS: Finding reducible subspaces in high dimensional data. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM'08. 2008. p. 961-970. (International Conference on Information and Knowledge Management, Proceedings). https://doi.org/10.1145/1458082.1458209