Crowdsourcing the measurement of interstate conflict

Vito D'Orazio, Michael Kenwick, Matthew Lane, Glenn Palmer, David Reitter

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Much of the data used to measure conflict is extracted from news reports. This is typically accomplished using either expert coders to quantify the relevant information or machine coders to automatically extract data from documents. Although expert coding is costly, it produces quality data. Machine coding is fast and inexpensive, but the data are noisy. To diminish the severity of this tradeoff, we introduce a method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches. The new method is tested on documents about Militarized Interstate Disputes, and its accuracy ranges between about 68 and 76 percent. This is shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.

Original languageEnglish (US)
Article numbere0156527
JournalPloS one
Volume11
Issue number6
DOIs
StatePublished - Jun 1 2016

Fingerprint

Crowdsourcing
Dissent and Disputes
Costs
Costs and Cost Analysis
methodology
Conflict (Psychology)

All Science Journal Classification (ASJC) codes

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)

Cite this

D'Orazio, Vito ; Kenwick, Michael ; Lane, Matthew ; Palmer, Glenn ; Reitter, David. / Crowdsourcing the measurement of interstate conflict. In: PloS one. 2016 ; Vol. 11, No. 6.
@article{1cbdd7a9c6d7472294c258fa58587564,
title = "Crowdsourcing the measurement of interstate conflict",
abstract = "Much of the data used to measure conflict is extracted from news reports. This is typically accomplished using either expert coders to quantify the relevant information or machine coders to automatically extract data from documents. Although expert coding is costly, it produces quality data. Machine coding is fast and inexpensive, but the data are noisy. To diminish the severity of this tradeoff, we introduce a method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches. The new method is tested on documents about Militarized Interstate Disputes, and its accuracy ranges between about 68 and 76 percent. This is shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.",
author = "Vito D'Orazio and Michael Kenwick and Matthew Lane and Glenn Palmer and David Reitter",
year = "2016",
month = "6",
day = "1",
doi = "10.1371/journal.pone.0156527",
language = "English (US)",
volume = "11",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "6",

}

Crowdsourcing the measurement of interstate conflict. / D'Orazio, Vito; Kenwick, Michael; Lane, Matthew; Palmer, Glenn; Reitter, David.

In: PloS one, Vol. 11, No. 6, e0156527, 01.06.2016.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Crowdsourcing the measurement of interstate conflict

AU - D'Orazio, Vito

AU - Kenwick, Michael

AU - Lane, Matthew

AU - Palmer, Glenn

AU - Reitter, David

PY - 2016/6/1

Y1 - 2016/6/1

N2 - Much of the data used to measure conflict is extracted from news reports. This is typically accomplished using either expert coders to quantify the relevant information or machine coders to automatically extract data from documents. Although expert coding is costly, it produces quality data. Machine coding is fast and inexpensive, but the data are noisy. To diminish the severity of this tradeoff, we introduce a method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches. The new method is tested on documents about Militarized Interstate Disputes, and its accuracy ranges between about 68 and 76 percent. This is shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.

AB - Much of the data used to measure conflict is extracted from news reports. This is typically accomplished using either expert coders to quantify the relevant information or machine coders to automatically extract data from documents. Although expert coding is costly, it produces quality data. Machine coding is fast and inexpensive, but the data are noisy. To diminish the severity of this tradeoff, we introduce a method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches. The new method is tested on documents about Militarized Interstate Disputes, and its accuracy ranges between about 68 and 76 percent. This is shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.

UR - http://www.scopus.com/inward/record.url?scp=84976271826&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84976271826&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0156527

DO - 10.1371/journal.pone.0156527

M3 - Article

VL - 11

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 6

M1 - e0156527

ER -