Locality preserving feature learning

Quanquan Gu, Marina Danilevsky, Zhenhui Li, Jiawei Han

Research output: Contribution to journalConference article

5 Citations (Scopus)

Abstract

Locality Preserving Indexing (LPI) has been quite successful in tackling document analysis problems, such as clustering or classification. The approach relies on the Locality Preserving Criterion, which preserves the locality of the data points. However, LPI takes every word in a data corpus into account, even though many words may not be useful for document clustering. To overcome this problem, we propose an approach called Locality Preserving Feature Learning (LPFL), which incorporates feature selection into LPI. Specifically, we aim to find a subset of features, and learn a linear transformation to optimize the Locality Preserving Criterion based on these features. The resulting optimization problem is a mixed integer programming problem, which we relax into a constrained Frobenius norm minimization problem, and solve using a variation of Alternating Direction Method (ADM). ADM, which iteratively updates the linear transformation matrix, the residue matrix and the Lagrangian multiplier, is theoretically guaranteed to converge at the rate O( 1/t ). Experiments on benchmark document datasets show that our proposed method outperforms LPI, as well as other state-of-the-art document analysis approaches.

Original languageEnglish (US)
Pages (from-to)477-485
Number of pages9
JournalJournal of Machine Learning Research
Volume22
StatePublished - Jan 1 2012
Event15th International Conference on Artificial Intelligence and Statistics, AISTATS 2012 - La Palma, Spain
Duration: Apr 21 2012Apr 23 2012

Fingerprint

Linear transformations
Locality
Integer programming
Set theory
Indexing
Feature extraction
Document Analysis
Alternating Direction Method
Linear transformation
Experiments
Lagrangian multiplier
Document Clustering
Frobenius norm
Transformation Matrix
Learning
Mixed Integer Programming
Feature Selection
Minimization Problem
Update
Optimise

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

Cite this

Gu, Q., Danilevsky, M., Li, Z., & Han, J. (2012). Locality preserving feature learning. Journal of Machine Learning Research, 22, 477-485.
Gu, Quanquan ; Danilevsky, Marina ; Li, Zhenhui ; Han, Jiawei. / Locality preserving feature learning. In: Journal of Machine Learning Research. 2012 ; Vol. 22. pp. 477-485.
@article{0455ec5611314daa83184c4f7016a3c4,
title = "Locality preserving feature learning",
abstract = "Locality Preserving Indexing (LPI) has been quite successful in tackling document analysis problems, such as clustering or classification. The approach relies on the Locality Preserving Criterion, which preserves the locality of the data points. However, LPI takes every word in a data corpus into account, even though many words may not be useful for document clustering. To overcome this problem, we propose an approach called Locality Preserving Feature Learning (LPFL), which incorporates feature selection into LPI. Specifically, we aim to find a subset of features, and learn a linear transformation to optimize the Locality Preserving Criterion based on these features. The resulting optimization problem is a mixed integer programming problem, which we relax into a constrained Frobenius norm minimization problem, and solve using a variation of Alternating Direction Method (ADM). ADM, which iteratively updates the linear transformation matrix, the residue matrix and the Lagrangian multiplier, is theoretically guaranteed to converge at the rate O( 1/t ). Experiments on benchmark document datasets show that our proposed method outperforms LPI, as well as other state-of-the-art document analysis approaches.",
author = "Quanquan Gu and Marina Danilevsky and Zhenhui Li and Jiawei Han",
year = "2012",
month = "1",
day = "1",
language = "English (US)",
volume = "22",
pages = "477--485",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

Gu, Q, Danilevsky, M, Li, Z & Han, J 2012, 'Locality preserving feature learning', Journal of Machine Learning Research, vol. 22, pp. 477-485.

Locality preserving feature learning. / Gu, Quanquan; Danilevsky, Marina; Li, Zhenhui; Han, Jiawei.

In: Journal of Machine Learning Research, Vol. 22, 01.01.2012, p. 477-485.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Locality preserving feature learning

AU - Gu, Quanquan

AU - Danilevsky, Marina

AU - Li, Zhenhui

AU - Han, Jiawei

PY - 2012/1/1

Y1 - 2012/1/1

N2 - Locality Preserving Indexing (LPI) has been quite successful in tackling document analysis problems, such as clustering or classification. The approach relies on the Locality Preserving Criterion, which preserves the locality of the data points. However, LPI takes every word in a data corpus into account, even though many words may not be useful for document clustering. To overcome this problem, we propose an approach called Locality Preserving Feature Learning (LPFL), which incorporates feature selection into LPI. Specifically, we aim to find a subset of features, and learn a linear transformation to optimize the Locality Preserving Criterion based on these features. The resulting optimization problem is a mixed integer programming problem, which we relax into a constrained Frobenius norm minimization problem, and solve using a variation of Alternating Direction Method (ADM). ADM, which iteratively updates the linear transformation matrix, the residue matrix and the Lagrangian multiplier, is theoretically guaranteed to converge at the rate O( 1/t ). Experiments on benchmark document datasets show that our proposed method outperforms LPI, as well as other state-of-the-art document analysis approaches.

AB - Locality Preserving Indexing (LPI) has been quite successful in tackling document analysis problems, such as clustering or classification. The approach relies on the Locality Preserving Criterion, which preserves the locality of the data points. However, LPI takes every word in a data corpus into account, even though many words may not be useful for document clustering. To overcome this problem, we propose an approach called Locality Preserving Feature Learning (LPFL), which incorporates feature selection into LPI. Specifically, we aim to find a subset of features, and learn a linear transformation to optimize the Locality Preserving Criterion based on these features. The resulting optimization problem is a mixed integer programming problem, which we relax into a constrained Frobenius norm minimization problem, and solve using a variation of Alternating Direction Method (ADM). ADM, which iteratively updates the linear transformation matrix, the residue matrix and the Lagrangian multiplier, is theoretically guaranteed to converge at the rate O( 1/t ). Experiments on benchmark document datasets show that our proposed method outperforms LPI, as well as other state-of-the-art document analysis approaches.

UR - http://www.scopus.com/inward/record.url?scp=84908161652&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84908161652&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:84908161652

VL - 22

SP - 477

EP - 485

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -

Gu Q, Danilevsky M, Li Z, Han J. Locality preserving feature learning. Journal of Machine Learning Research. 2012 Jan 1;22:477-485.