A mixture model and EM-based algorithm for class discovery, robust classification, and outlier rejection in mixed labeled/unlabeled data sets

David J. Miller, John Browning

    Research output: Contribution to journalArticlepeer-review

    77 Scopus citations

    Abstract

    Several authors have shown that, when labeled data are scarce, improved classifiers can be built by augmenting the training set with a large set of unlabeled examples and then performing suitable learning. These works assume each unlabeled sample originates from one of the (known) classes. Here, we assume each unlabeled sample comes from either a known or from a heretofore undiscovered class. We propose a novel mixture model which treats as observed data not only the feature vector and the class label, but also the fact of label presence/absence for each sample. Two types of mixture components are posited. "Predefined" components generate data from known classes and assume class labels are missing at random. "Nonpredefined" components only generate unlabeled data - i.e., they capture exclusively unlabeled subsets, consistent with an outlier distribution or new classes. The predefined/nonpredefined natures are data-driven, learned along with the other parameters via an extension of the EM algorithm. Our modeling framework addresses problems involving both the known and unknown classes: 1) robust classifier design, 2) classification with rejections, and 3) identification of the unlabeled samples (and their components) from unknown classes. Case 3 is a step toward new class discovery. Experiments are reported for each application, including topic discovery for the Reuters domain. Experiments also demonstrate the value of label presence/absence data in learning accurate mixtures.

    Original languageEnglish (US)
    Pages (from-to)1468-1483
    Number of pages16
    JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
    Volume25
    Issue number11
    DOIs
    StatePublished - Nov 2003

    All Science Journal Classification (ASJC) codes

    • Software
    • Computer Vision and Pattern Recognition
    • Computational Theory and Mathematics
    • Artificial Intelligence
    • Applied Mathematics

    Fingerprint

    Dive into the research topics of 'A mixture model and EM-based algorithm for class discovery, robust classification, and outlier rejection in mixed labeled/unlabeled data sets'. Together they form a unique fingerprint.

    Cite this