Clustering via finite nonparametric ICA mixture models

Xiaotian Zhu, David Russell Hunter

Research output: Contribution to journalArticle

Abstract

We propose a novel extension of nonparametric multivariate finite mixture models by dropping the standard conditional independence assumption and incorporating the independent component analysis (ICA) structure instead. This innovation extends nonparametric mixture model estimation methods to situations in which conditional independence, a necessary assumption for the unique identifiability of the parameters in such models, is clearly violated. We formulate an objective function in terms of penalized smoothed Kullback–Leibler distance and introduce the nonlinear smoothed majorization-minimization independent component analysis algorithm for optimizing this function and estimating the model parameters. Our algorithm does not require any labeled observations a priori; it may be used for fully unsupervised clustering problems in a multivariate setting. We have implemented a practical version of this algorithm, which utilizes the FastICA algorithm, in the R package icamix. We illustrate this new methodology using several applications in unsupervised learning and image processing.

Original languageEnglish (US)
Pages (from-to)65-87
Number of pages23
JournalAdvances in Data Analysis and Classification
Volume13
Issue number1
DOIs
StatePublished - Mar 8 2019

Fingerprint

Independent component analysis
Independent Component Analysis
Mixture Model
Clustering
Conditional Independence
Finite Mixture Models
Unsupervised Clustering
Unsupervised learning
Majorization
Unsupervised Learning
Nonparametric Model
Identifiability
Image Processing
Image processing
Objective function
Innovation
Necessary
Methodology
Model

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Applied Mathematics

Cite this

@article{c1f3200f1c7646b28a2d617d806dcfaa,
title = "Clustering via finite nonparametric ICA mixture models",
abstract = "We propose a novel extension of nonparametric multivariate finite mixture models by dropping the standard conditional independence assumption and incorporating the independent component analysis (ICA) structure instead. This innovation extends nonparametric mixture model estimation methods to situations in which conditional independence, a necessary assumption for the unique identifiability of the parameters in such models, is clearly violated. We formulate an objective function in terms of penalized smoothed Kullback–Leibler distance and introduce the nonlinear smoothed majorization-minimization independent component analysis algorithm for optimizing this function and estimating the model parameters. Our algorithm does not require any labeled observations a priori; it may be used for fully unsupervised clustering problems in a multivariate setting. We have implemented a practical version of this algorithm, which utilizes the FastICA algorithm, in the R package icamix. We illustrate this new methodology using several applications in unsupervised learning and image processing.",
author = "Xiaotian Zhu and Hunter, {David Russell}",
year = "2019",
month = "3",
day = "8",
doi = "10.1007/s11634-018-0338-x",
language = "English (US)",
volume = "13",
pages = "65--87",
journal = "Advances in Data Analysis and Classification",
issn = "1862-5347",
publisher = "Springer Verlag",
number = "1",

}

Clustering via finite nonparametric ICA mixture models. / Zhu, Xiaotian; Hunter, David Russell.

In: Advances in Data Analysis and Classification, Vol. 13, No. 1, 08.03.2019, p. 65-87.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Clustering via finite nonparametric ICA mixture models

AU - Zhu, Xiaotian

AU - Hunter, David Russell

PY - 2019/3/8

Y1 - 2019/3/8

N2 - We propose a novel extension of nonparametric multivariate finite mixture models by dropping the standard conditional independence assumption and incorporating the independent component analysis (ICA) structure instead. This innovation extends nonparametric mixture model estimation methods to situations in which conditional independence, a necessary assumption for the unique identifiability of the parameters in such models, is clearly violated. We formulate an objective function in terms of penalized smoothed Kullback–Leibler distance and introduce the nonlinear smoothed majorization-minimization independent component analysis algorithm for optimizing this function and estimating the model parameters. Our algorithm does not require any labeled observations a priori; it may be used for fully unsupervised clustering problems in a multivariate setting. We have implemented a practical version of this algorithm, which utilizes the FastICA algorithm, in the R package icamix. We illustrate this new methodology using several applications in unsupervised learning and image processing.

AB - We propose a novel extension of nonparametric multivariate finite mixture models by dropping the standard conditional independence assumption and incorporating the independent component analysis (ICA) structure instead. This innovation extends nonparametric mixture model estimation methods to situations in which conditional independence, a necessary assumption for the unique identifiability of the parameters in such models, is clearly violated. We formulate an objective function in terms of penalized smoothed Kullback–Leibler distance and introduce the nonlinear smoothed majorization-minimization independent component analysis algorithm for optimizing this function and estimating the model parameters. Our algorithm does not require any labeled observations a priori; it may be used for fully unsupervised clustering problems in a multivariate setting. We have implemented a practical version of this algorithm, which utilizes the FastICA algorithm, in the R package icamix. We illustrate this new methodology using several applications in unsupervised learning and image processing.

UR - http://www.scopus.com/inward/record.url?scp=85064011403&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064011403&partnerID=8YFLogxK

U2 - 10.1007/s11634-018-0338-x

DO - 10.1007/s11634-018-0338-x

M3 - Article

AN - SCOPUS:85064011403

VL - 13

SP - 65

EP - 87

JO - Advances in Data Analysis and Classification

JF - Advances in Data Analysis and Classification

SN - 1862-5347

IS - 1

ER -