Nonparametric entropy estimation for stationary processesand random fields, with applications to english text

I. Kontoyiannis, P. H. Algoet, Yu M. Suhov, A. J. Wyner

Research output: Contribution to journalArticlepeer-review

128 Scopus citations

Abstract

We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesàro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker. We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields.

Original languageEnglish (US)
Pages (from-to)1319-1327
Number of pages9
JournalIEEE Transactions on Information Theory
Volume44
Issue number3
DOIs
StatePublished - 1998

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Fingerprint Dive into the research topics of 'Nonparametric entropy estimation for stationary processesand random fields, with applications to english text'. Together they form a unique fingerprint.

Cite this