Two-way Poisson mixture models for simultaneous document classification and word clustering

Jia Li, Hongyuan Zha

Research output: Contribution to journalArticlepeer-review

40 Scopus citations

Abstract

An approach to simultaneous document classification and word clustering is developed using a two-way mixture model of Poisson distributions. Each document is represented by a vector with each dimension specifying the number of occurrences of a particular word in the document in question. As a collection of documents across several classes usually makes use of a large number of words, the document vectors are of high dimension. On the other hand, the number of distinct words in any single document is usually substantially smaller than the size of the vocabulary, leading to sparse document vectors. A mixture of Poisson distributions is used to model the multivariate distribution of the word counts in the documents within each class. To address the issues of high dimensionality and sparsity, the parameters in the mixture model are regularized by imposing a clustering structure on the set of words. An EM-style algorithm for the two-way mixture model will be derived for parameter estimation with the clustering of words part of the estimation process. The connection of the two-way mixture model with dimension reduction will also be elucidated. Experiments on the newsgroup data have demonstrated promising results.

Original languageEnglish (US)
Pages (from-to)163-180
Number of pages18
JournalComputational Statistics and Data Analysis
Volume50
Issue number1 SPEC. ISS.
DOIs
StatePublished - Jan 10 2006

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Computational Mathematics
  • Computational Theory and Mathematics
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Two-way Poisson mixture models for simultaneous document classification and word clustering'. Together they form a unique fingerprint.

Cite this