Deterministic annealing for trellis quantizer and HMM design using Baum-Welch re-estimation

David Jonathan Miller, K. Rose, P. A. Chou

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

The deterministic annealing algorithm for data clustering is extended to address the trellis quantizer design problem. The approach is derived within information theory and probability theory, using the principle of maximum entropy to induce a distribution over all possible path encodings of the training set. The resulting method is intimately connected to estimation procedures on Markov chains. Performance gains over known methods are obtained for memoryless, multimodal scalar sources as well as for the vector Gaussian and Laplacian sources. The method is also suggested for an estimation problem in hidden Markov models. For a Gaussian mixture state example, this approach achieves a greater likelihood value than the best result of standard Baum-Welch re-estimation, based on numerous initializations within the data.

Original languageEnglish (US)
Article number389482
JournalICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume5
StatePublished - Jan 1 1994

Fingerprint

Annealing
Information theory
Hidden Markov models
Markov processes
Entropy

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

@article{d5e7099f73d440a3b76593859f8e8846,
title = "Deterministic annealing for trellis quantizer and HMM design using Baum-Welch re-estimation",
abstract = "The deterministic annealing algorithm for data clustering is extended to address the trellis quantizer design problem. The approach is derived within information theory and probability theory, using the principle of maximum entropy to induce a distribution over all possible path encodings of the training set. The resulting method is intimately connected to estimation procedures on Markov chains. Performance gains over known methods are obtained for memoryless, multimodal scalar sources as well as for the vector Gaussian and Laplacian sources. The method is also suggested for an estimation problem in hidden Markov models. For a Gaussian mixture state example, this approach achieves a greater likelihood value than the best result of standard Baum-Welch re-estimation, based on numerous initializations within the data.",
author = "Miller, {David Jonathan} and K. Rose and Chou, {P. A.}",
year = "1994",
month = "1",
day = "1",
language = "English (US)",
volume = "5",
journal = "Proceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing",
issn = "0736-7791",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Deterministic annealing for trellis quantizer and HMM design using Baum-Welch re-estimation

AU - Miller, David Jonathan

AU - Rose, K.

AU - Chou, P. A.

PY - 1994/1/1

Y1 - 1994/1/1

N2 - The deterministic annealing algorithm for data clustering is extended to address the trellis quantizer design problem. The approach is derived within information theory and probability theory, using the principle of maximum entropy to induce a distribution over all possible path encodings of the training set. The resulting method is intimately connected to estimation procedures on Markov chains. Performance gains over known methods are obtained for memoryless, multimodal scalar sources as well as for the vector Gaussian and Laplacian sources. The method is also suggested for an estimation problem in hidden Markov models. For a Gaussian mixture state example, this approach achieves a greater likelihood value than the best result of standard Baum-Welch re-estimation, based on numerous initializations within the data.

AB - The deterministic annealing algorithm for data clustering is extended to address the trellis quantizer design problem. The approach is derived within information theory and probability theory, using the principle of maximum entropy to induce a distribution over all possible path encodings of the training set. The resulting method is intimately connected to estimation procedures on Markov chains. Performance gains over known methods are obtained for memoryless, multimodal scalar sources as well as for the vector Gaussian and Laplacian sources. The method is also suggested for an estimation problem in hidden Markov models. For a Gaussian mixture state example, this approach achieves a greater likelihood value than the best result of standard Baum-Welch re-estimation, based on numerous initializations within the data.

UR - http://www.scopus.com/inward/record.url?scp=0005236957&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0005236957&partnerID=8YFLogxK

M3 - Article

VL - 5

JO - Proceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing

JF - Proceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing

SN - 0736-7791

M1 - 389482

ER -