### Abstract

An algorithm for supervised classification using vector quantization and entropy coding is presented. The classification rule is formed from a set of training data {(X_{i}, Y_{i})}_{i=1}^{n}, which are independent samples from a joint distribution P_{XY}. Based on the principle of minimum description length (MDL), a statistical model that approximates the distribution P_{XY} ought to enable efficient coding of X and Y. On the other hand, we expect a system that encodes (X, Y) efficiently to provide ample information on the distribution P_{XY}. This information can then be used to classify X, i.e., to predict the corresponding Y based on X. To encode both X and Y, a two-stage vector quantizer is applied to X and a Huffman code is formed for Y conditioned on each quantized value of X. The optimization of the encoder is equivalent to the design of a vector quantizer with an objective function reflecting the joint penalty of quantization error and misclassification rate. This vector quantizer provides an estimation of the conditional distribution of Y given X, which in turn yields an approximation to the Bayes classification rule. This algorithm, namely discriminant vector quantization (DVQ), is compared with learning vector quantization (LVQ) and CART^{R} on a number of data sets. DVQ outperforms the other two on several data sets. The relation between DVQ, density estimation, and regression is also discussed.

Original language | English (US) |
---|---|

Title of host publication | Proceedings - DCC 2002 |

Subtitle of host publication | Data Compression Conference |

Editors | James A. Storer, Martin Cohn |

Publisher | Institute of Electrical and Electronics Engineers Inc. |

Pages | 382-391 |

Number of pages | 10 |

ISBN (Electronic) | 0769514774 |

DOIs | |

State | Published - Jan 1 2002 |

Event | Data Compression Conference, DCC 2002 - Snowbird, United States Duration: Apr 2 2002 → Apr 4 2002 |

### Publication series

Name | Data Compression Conference Proceedings |
---|---|

Volume | 2002-January |

ISSN (Print) | 1068-0314 |

### Other

Other | Data Compression Conference, DCC 2002 |
---|---|

Country | United States |

City | Snowbird |

Period | 4/2/02 → 4/4/02 |

### All Science Journal Classification (ASJC) codes

- Computer Networks and Communications

## Fingerprint Dive into the research topics of 'A source coding approach to classification by vector quantization and the principle of minimum description length'. Together they form a unique fingerprint.

## Cite this

*Proceedings - DCC 2002: Data Compression Conference*(pp. 382-391). [999978] (Data Compression Conference Proceedings; Vol. 2002-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DCC.2002.999978