General statistical inference for discrete and mixed spaces by an approximate application of the maximum entropy principle

Lian Yan, David J. Miller

    Research output: Contribution to journalArticle

    18 Scopus citations

    Abstract

    We propose a new method for learning a general statistical inference engine, operating on discrete and mixed discrete/continuous feature spaces. Such a model allows inference on any of the discrete features, given values for the remaining features. Applications are, e.g., to medical diagnosis with multiple possible diseases, fault diagnosis, information retrieval, and imputation in databases. Bayesian networks (BN's) are versatile tools that possess this inference capability. However, BN's require explicit specification of conditional independences, which may be difficult to assess given limited data. Alternatively, Cheeseman proposed finding the maximum entropy (ME) joint probability mass function (pmf) consistent with arbitrary lower order probability constraints. This approach is in principle powerful and does not require explicit expression of conditional independence. However, until now, the huge learning complexity has severely limited the use of this approach. Here we propose an approximate ME method, which also encodes arbitrary low-order constraints but while retaining quite tractable learning. Our method uses a restriction of joint pmf support (during learning) to a subset of the feature space. Results on the University of California-Irvine repository reveal performance gains over several BN approaches and over multilayer perceptrons.

    Original languageEnglish (US)
    Pages (from-to)558-573
    Number of pages16
    JournalIEEE Transactions on Neural Networks
    Volume11
    Issue number3
    DOIs
    StatePublished - May 1 2000

    All Science Journal Classification (ASJC) codes

    • Software
    • Computer Science Applications
    • Computer Networks and Communications
    • Artificial Intelligence

    Fingerprint Dive into the research topics of 'General statistical inference for discrete and mixed spaces by an approximate application of the maximum entropy principle'. Together they form a unique fingerprint.

  • Cite this