Improving part-of-speech guessing of Chinese unknown words using hybrid models

Research output: Contribution to journalArticle

2 Scopus citations

Abstract

This paper presents a hybrid model for part-of-speech (POS) guessing of Chinese unknown words. Most previous studies on this task have developed a unified statistical model for all Chinese unknown words and have rejected rulebased models without testing. We argue that models that use different sources of information about unknown words, both structural and contextual, can be effective for handling different types of unknown words. We propose a rulebased model that uses information about the type, length, and internal structure of unknown words and combine it with two existing statistical models that use information about the POS context and component characters of unknown words respectively for this task. By combining the complementary strengths of the three models that use different sources of information, the hybrid model achieves an accuracy of 89%, a significant improvement over the best result reported in previous studies.

Original languageEnglish (US)
Pages (from-to)169-193
Number of pages25
JournalInternational Journal of Corpus Linguistics
Volume13
Issue number2
DOIs
StatePublished - Dec 1 2008

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Linguistics and Language

Fingerprint Dive into the research topics of 'Improving part-of-speech guessing of Chinese unknown words using hybrid models'. Together they form a unique fingerprint.

  • Cite this