Critic-driven ensemble classification

    Research output: Contribution to journalArticle

    27 Citations (Scopus)

    Abstract

    We develop new rules for combining the estimates obtained from each classifier in an ensemble, in order to address problems involving multiple (>2) classes. A variety of techniques have been previously suggested, including averaging probability estimates from each classifier, as well as hard (0-1) voting schemes. In this work, we introduce the notion of a critic associated with each classifier, whose objective is to predict the classifier's errors. Since the critic only tackles a two-class problem, its predictions are generally more reliable than those of the classifier and, thus, can be used as the basis for improved combination rules. Several such rules are suggested here. While previous techniques are only effective when the individual classifier error rate is p<0.5, the new approach is successful, as proved under an independence assumption, even when this condition is violated - in particular, so long as p+q<1, with q the critic's error rate. More generally, critic-driven combining is found to achieve significant performance gains over alternative methods on a number of benchmark data sets. We also propose a new analytical tool for modeling ensemble performance, based on dependence between experts. This approach is substantially more accurate than the analysis based on independence that is often used to justify ensemble methods.

    Original languageEnglish (US)
    Pages (from-to)2833-2844
    Number of pages12
    JournalIEEE Transactions on Signal Processing
    Volume47
    Issue number10
    DOIs
    StatePublished - Dec 1 1999

    Fingerprint

    Classifiers

    All Science Journal Classification (ASJC) codes

    • Signal Processing
    • Electrical and Electronic Engineering

    Cite this

    @article{05ba6e44eb194ff49aac20aa970baef3,
    title = "Critic-driven ensemble classification",
    abstract = "We develop new rules for combining the estimates obtained from each classifier in an ensemble, in order to address problems involving multiple (>2) classes. A variety of techniques have been previously suggested, including averaging probability estimates from each classifier, as well as hard (0-1) voting schemes. In this work, we introduce the notion of a critic associated with each classifier, whose objective is to predict the classifier's errors. Since the critic only tackles a two-class problem, its predictions are generally more reliable than those of the classifier and, thus, can be used as the basis for improved combination rules. Several such rules are suggested here. While previous techniques are only effective when the individual classifier error rate is p<0.5, the new approach is successful, as proved under an independence assumption, even when this condition is violated - in particular, so long as p+q<1, with q the critic's error rate. More generally, critic-driven combining is found to achieve significant performance gains over alternative methods on a number of benchmark data sets. We also propose a new analytical tool for modeling ensemble performance, based on dependence between experts. This approach is substantially more accurate than the analysis based on independence that is often used to justify ensemble methods.",
    author = "Miller, {David Jonathan} and Lian Yan",
    year = "1999",
    month = "12",
    day = "1",
    doi = "10.1109/78.790663",
    language = "English (US)",
    volume = "47",
    pages = "2833--2844",
    journal = "IEEE Transactions on Signal Processing",
    issn = "1053-587X",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "10",

    }

    Critic-driven ensemble classification. / Miller, David Jonathan; Yan, Lian.

    In: IEEE Transactions on Signal Processing, Vol. 47, No. 10, 01.12.1999, p. 2833-2844.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Critic-driven ensemble classification

    AU - Miller, David Jonathan

    AU - Yan, Lian

    PY - 1999/12/1

    Y1 - 1999/12/1

    N2 - We develop new rules for combining the estimates obtained from each classifier in an ensemble, in order to address problems involving multiple (>2) classes. A variety of techniques have been previously suggested, including averaging probability estimates from each classifier, as well as hard (0-1) voting schemes. In this work, we introduce the notion of a critic associated with each classifier, whose objective is to predict the classifier's errors. Since the critic only tackles a two-class problem, its predictions are generally more reliable than those of the classifier and, thus, can be used as the basis for improved combination rules. Several such rules are suggested here. While previous techniques are only effective when the individual classifier error rate is p<0.5, the new approach is successful, as proved under an independence assumption, even when this condition is violated - in particular, so long as p+q<1, with q the critic's error rate. More generally, critic-driven combining is found to achieve significant performance gains over alternative methods on a number of benchmark data sets. We also propose a new analytical tool for modeling ensemble performance, based on dependence between experts. This approach is substantially more accurate than the analysis based on independence that is often used to justify ensemble methods.

    AB - We develop new rules for combining the estimates obtained from each classifier in an ensemble, in order to address problems involving multiple (>2) classes. A variety of techniques have been previously suggested, including averaging probability estimates from each classifier, as well as hard (0-1) voting schemes. In this work, we introduce the notion of a critic associated with each classifier, whose objective is to predict the classifier's errors. Since the critic only tackles a two-class problem, its predictions are generally more reliable than those of the classifier and, thus, can be used as the basis for improved combination rules. Several such rules are suggested here. While previous techniques are only effective when the individual classifier error rate is p<0.5, the new approach is successful, as proved under an independence assumption, even when this condition is violated - in particular, so long as p+q<1, with q the critic's error rate. More generally, critic-driven combining is found to achieve significant performance gains over alternative methods on a number of benchmark data sets. We also propose a new analytical tool for modeling ensemble performance, based on dependence between experts. This approach is substantially more accurate than the analysis based on independence that is often used to justify ensemble methods.

    UR - http://www.scopus.com/inward/record.url?scp=0033327709&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=0033327709&partnerID=8YFLogxK

    U2 - 10.1109/78.790663

    DO - 10.1109/78.790663

    M3 - Article

    VL - 47

    SP - 2833

    EP - 2844

    JO - IEEE Transactions on Signal Processing

    JF - IEEE Transactions on Signal Processing

    SN - 1053-587X

    IS - 10

    ER -