Information markets vs. opinion pools

An empirical comparison

Yiling Chen, Chao Hsien Chu, Tracy Mullen, David M. Pennock

Research output: Chapter in Book/Report/Conference proceedingConference contribution

43 Citations (Scopus)

Abstract

In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people's subjective probability judgments on 2003 US National Football League games and compare with the "market probabilities" given by two different information markets on exactly the same events, We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.

Original languageEnglish (US)
Title of host publicationEC'05: Proceedings of the 6th ACM Conference on Electronic Commerce
Pages58-67
Number of pages10
StatePublished - 2005
EventEC'05: 6th ACM Conference on Electronic Commerce - Vancouver, Canada
Duration: Jun 5 2005Jun 8 2005

Other

OtherEC'05: 6th ACM Conference on Electronic Commerce
CountryCanada
CityVancouver
Period6/5/056/8/05

Fingerprint

Agglomeration
Prediction
Information market
Screening
Pooling
Prediction markets
Subjective probability
National Football League
Probability judgment
Forecast accuracy
Weighting
Data sources
Leverage

All Science Journal Classification (ASJC) codes

  • Computer Science (miscellaneous)
  • Business, Management and Accounting (miscellaneous)

Cite this

Chen, Y., Chu, C. H., Mullen, T., & Pennock, D. M. (2005). Information markets vs. opinion pools: An empirical comparison. In EC'05: Proceedings of the 6th ACM Conference on Electronic Commerce (pp. 58-67)
Chen, Yiling ; Chu, Chao Hsien ; Mullen, Tracy ; Pennock, David M. / Information markets vs. opinion pools : An empirical comparison. EC'05: Proceedings of the 6th ACM Conference on Electronic Commerce. 2005. pp. 58-67
@inproceedings{1c486ea7aa4944d29b3233b9094c40e6,
title = "Information markets vs. opinion pools: An empirical comparison",
abstract = "In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people's subjective probability judgments on 2003 US National Football League games and compare with the {"}market probabilities{"} given by two different information markets on exactly the same events, We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.",
author = "Yiling Chen and Chu, {Chao Hsien} and Tracy Mullen and Pennock, {David M.}",
year = "2005",
language = "English (US)",
pages = "58--67",
booktitle = "EC'05: Proceedings of the 6th ACM Conference on Electronic Commerce",

}

Chen, Y, Chu, CH, Mullen, T & Pennock, DM 2005, Information markets vs. opinion pools: An empirical comparison. in EC'05: Proceedings of the 6th ACM Conference on Electronic Commerce. pp. 58-67, EC'05: 6th ACM Conference on Electronic Commerce, Vancouver, Canada, 6/5/05.

Information markets vs. opinion pools : An empirical comparison. / Chen, Yiling; Chu, Chao Hsien; Mullen, Tracy; Pennock, David M.

EC'05: Proceedings of the 6th ACM Conference on Electronic Commerce. 2005. p. 58-67.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Information markets vs. opinion pools

T2 - An empirical comparison

AU - Chen, Yiling

AU - Chu, Chao Hsien

AU - Mullen, Tracy

AU - Pennock, David M.

PY - 2005

Y1 - 2005

N2 - In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people's subjective probability judgments on 2003 US National Football League games and compare with the "market probabilities" given by two different information markets on exactly the same events, We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.

AB - In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people's subjective probability judgments on 2003 US National Football League games and compare with the "market probabilities" given by two different information markets on exactly the same events, We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.

UR - http://www.scopus.com/inward/record.url?scp=30044438408&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=30044438408&partnerID=8YFLogxK

M3 - Conference contribution

SP - 58

EP - 67

BT - EC'05: Proceedings of the 6th ACM Conference on Electronic Commerce

ER -

Chen Y, Chu CH, Mullen T, Pennock DM. Information markets vs. opinion pools: An empirical comparison. In EC'05: Proceedings of the 6th ACM Conference on Electronic Commerce. 2005. p. 58-67