Critic-driven ensemble classification

David J. Miller, Lian Yan

    Research output: Contribution to journalArticle

    27 Scopus citations

    Abstract

    We develop new rules for combining the estimates obtained from each classifier in an ensemble, in order to address problems involving multiple (>2) classes. A variety of techniques have been previously suggested, including averaging probability estimates from each classifier, as well as hard (0-1) voting schemes. In this work, we introduce the notion of a critic associated with each classifier, whose objective is to predict the classifier's errors. Since the critic only tackles a two-class problem, its predictions are generally more reliable than those of the classifier and, thus, can be used as the basis for improved combination rules. Several such rules are suggested here. While previous techniques are only effective when the individual classifier error rate is p<0.5, the new approach is successful, as proved under an independence assumption, even when this condition is violated - in particular, so long as p+q<1, with q the critic's error rate. More generally, critic-driven combining is found to achieve significant performance gains over alternative methods on a number of benchmark data sets. We also propose a new analytical tool for modeling ensemble performance, based on dependence between experts. This approach is substantially more accurate than the analysis based on independence that is often used to justify ensemble methods.

    Original languageEnglish (US)
    Pages (from-to)2833-2844
    Number of pages12
    JournalIEEE Transactions on Signal Processing
    Volume47
    Issue number10
    DOIs
    Publication statusPublished - Dec 1 1999

      Fingerprint

    All Science Journal Classification (ASJC) codes

    • Signal Processing
    • Electrical and Electronic Engineering

    Cite this