Deterministically annealed mixture of experts models for statistical regression

Ajit Rao, David Jonathan Miller, Kenneth Rose, Allen Gersho

    Research output: Contribution to journalConference articlepeer-review

    Abstract

    A new and effective design method is presented for statistical regression functions that belong to the class of mixture models. The class includes the hierarchical mixture of experts (HME) and the normalized radial basis functions (NRBF). Design algorithms based on the maximum likelihood (ML) approach, which emphasize a probabilistic description of the model, have attracted much interest in HME and NRBF models. However, their design objective is mismatched to the original squared-error regression cost and the algorithms are easily trapped by poor local minima on the cost surface. In this paper, we propose an extension of the deterministic annealing (DA) method for the design of mixture-based regression models. We construct a probabilistic framework, but unlike the ML method, we directly optimize the squared-error regression cost, while avoiding poor local minima. Experimental results show that the DA method outperforms standard design methods for both HME and NRBF regression models.

    Original languageEnglish (US)
    Pages (from-to)3201-3204
    Number of pages4
    JournalICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
    Volume4
    StatePublished - Jan 1 1997
    EventProceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP. Part 1 (of 5) - Munich, Ger
    Duration: Apr 21 1997Apr 24 1997

    All Science Journal Classification (ASJC) codes

    • Software
    • Signal Processing
    • Electrical and Electronic Engineering

    Fingerprint Dive into the research topics of 'Deterministically annealed mixture of experts models for statistical regression'. Together they form a unique fingerprint.

    Cite this