Separating the wheat from the chaff: Applications of automated document classification using support vector machines

Vito D'Orazio, Steven T. Landis, Glenn Palmer, Philip Schrodt

Research output: Contribution to journalArticle

25 Scopus citations

Abstract

Due in large part to the proliferation of digitized text, much of it available for little or no cost from the Internet, political science research has experienced a substantial increase in the number of data sets and large-n research initiatives. As the ability to collect detailed information on events of interest expands, so does the need to efficiently sort through the volumes of available information. Automated document classification presents a particularly attractive methodology for accomplishing this task. It is efficient, widely applicable to a variety of data collection efforts, and considerably flexible in tailoring its application for specific research needs. This article offers a holistic review of the application of automated document classification for data collection in political science research by discussing the process in its entirety. We argue that the application of a two-stage support vector machine (SVM) classification process offers advantages over other wellknown alternatives, due to the nature of SVMs being a discriminative classifier and having the ability to effectively address two primary attributes of textual data: high dimensionality and extreme sparseness. Evidence for this claim is presented through a discussion of the efficiency gains derived from using automated document classification on the Militarized Interstate Dispute 4 (MID4) data collection project.

Original languageEnglish (US)
Article numbermpt030
Pages (from-to)224-242
Number of pages19
JournalPolitical Analysis
Volume22
Issue number2
DOIs
StatePublished - Apr 2014

All Science Journal Classification (ASJC) codes

  • Sociology and Political Science
  • Political Science and International Relations

Fingerprint Dive into the research topics of 'Separating the wheat from the chaff: Applications of automated document classification using support vector machines'. Together they form a unique fingerprint.

  • Cite this