Investigating the Amazon Mechanical Turk Market Through Tool Design

Benjamin Vincent Hanrahan, David Martin, Jutta Willamowski, John Carroll

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

We developed TurkBench to better understand the work of crowdworkers on the Amazon Mechanical Turk (AMT) marketplace. While we aimed to reduce the amount of invisible, unpaid work that these crowdworkers performed, we also probed the day-to-day practices of crowdworkers. Through this probe we encountered a number of previously unreported difficulties that are representative of the difficulties that crowdworkers face in both building their own tools and working on AMT. In this article, our contributions are insights into 1) a number of breakdowns that are occurring on AMT and 2) how the AMT platform is being appropriated in ways that, at the same time, mitigate some breakdowns while exacerbating others. The breakdowns that we specifically discuss in this paper, are the increasing velocity of the market (good HITs are grabbed within seconds), the high amount of flexibility that requesters can and do exercise in specifying their HITs, and the difficulty crowdworkers had in navigating the market due to the large amount of variation in how HITs were constructed by requesters. When the velocity of the market is combined with a poor search interface, a large amount in variation in how HITs are constructed, and little infrastructural support for workers, the resulting work environment can be frustrating and difficult to thrive in.

Original languageEnglish (US)
Pages (from-to)1255-1274
Number of pages20
JournalComputer Supported Cooperative Work: CSCW: An International Journal
Volume27
Issue number3-6
DOIs
StatePublished - Dec 1 2018

All Science Journal Classification (ASJC) codes

  • Computer Science(all)

Cite this

@article{7a39c415a91c4d1a8202c6a989fddd99,
title = "Investigating the Amazon Mechanical Turk Market Through Tool Design",
abstract = "We developed TurkBench to better understand the work of crowdworkers on the Amazon Mechanical Turk (AMT) marketplace. While we aimed to reduce the amount of invisible, unpaid work that these crowdworkers performed, we also probed the day-to-day practices of crowdworkers. Through this probe we encountered a number of previously unreported difficulties that are representative of the difficulties that crowdworkers face in both building their own tools and working on AMT. In this article, our contributions are insights into 1) a number of breakdowns that are occurring on AMT and 2) how the AMT platform is being appropriated in ways that, at the same time, mitigate some breakdowns while exacerbating others. The breakdowns that we specifically discuss in this paper, are the increasing velocity of the market (good HITs are grabbed within seconds), the high amount of flexibility that requesters can and do exercise in specifying their HITs, and the difficulty crowdworkers had in navigating the market due to the large amount of variation in how HITs were constructed by requesters. When the velocity of the market is combined with a poor search interface, a large amount in variation in how HITs are constructed, and little infrastructural support for workers, the resulting work environment can be frustrating and difficult to thrive in.",
author = "Hanrahan, {Benjamin Vincent} and David Martin and Jutta Willamowski and John Carroll",
year = "2018",
month = "12",
day = "1",
doi = "10.1007/s10606-018-9312-6",
language = "English (US)",
volume = "27",
pages = "1255--1274",
journal = "Computer Supported Cooperative Work",
issn = "0925-9724",
publisher = "Springer Netherlands",
number = "3-6",

}

Investigating the Amazon Mechanical Turk Market Through Tool Design. / Hanrahan, Benjamin Vincent; Martin, David; Willamowski, Jutta; Carroll, John.

In: Computer Supported Cooperative Work: CSCW: An International Journal, Vol. 27, No. 3-6, 01.12.2018, p. 1255-1274.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Investigating the Amazon Mechanical Turk Market Through Tool Design

AU - Hanrahan, Benjamin Vincent

AU - Martin, David

AU - Willamowski, Jutta

AU - Carroll, John

PY - 2018/12/1

Y1 - 2018/12/1

N2 - We developed TurkBench to better understand the work of crowdworkers on the Amazon Mechanical Turk (AMT) marketplace. While we aimed to reduce the amount of invisible, unpaid work that these crowdworkers performed, we also probed the day-to-day practices of crowdworkers. Through this probe we encountered a number of previously unreported difficulties that are representative of the difficulties that crowdworkers face in both building their own tools and working on AMT. In this article, our contributions are insights into 1) a number of breakdowns that are occurring on AMT and 2) how the AMT platform is being appropriated in ways that, at the same time, mitigate some breakdowns while exacerbating others. The breakdowns that we specifically discuss in this paper, are the increasing velocity of the market (good HITs are grabbed within seconds), the high amount of flexibility that requesters can and do exercise in specifying their HITs, and the difficulty crowdworkers had in navigating the market due to the large amount of variation in how HITs were constructed by requesters. When the velocity of the market is combined with a poor search interface, a large amount in variation in how HITs are constructed, and little infrastructural support for workers, the resulting work environment can be frustrating and difficult to thrive in.

AB - We developed TurkBench to better understand the work of crowdworkers on the Amazon Mechanical Turk (AMT) marketplace. While we aimed to reduce the amount of invisible, unpaid work that these crowdworkers performed, we also probed the day-to-day practices of crowdworkers. Through this probe we encountered a number of previously unreported difficulties that are representative of the difficulties that crowdworkers face in both building their own tools and working on AMT. In this article, our contributions are insights into 1) a number of breakdowns that are occurring on AMT and 2) how the AMT platform is being appropriated in ways that, at the same time, mitigate some breakdowns while exacerbating others. The breakdowns that we specifically discuss in this paper, are the increasing velocity of the market (good HITs are grabbed within seconds), the high amount of flexibility that requesters can and do exercise in specifying their HITs, and the difficulty crowdworkers had in navigating the market due to the large amount of variation in how HITs were constructed by requesters. When the velocity of the market is combined with a poor search interface, a large amount in variation in how HITs are constructed, and little infrastructural support for workers, the resulting work environment can be frustrating and difficult to thrive in.

UR - http://www.scopus.com/inward/record.url?scp=85046656808&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046656808&partnerID=8YFLogxK

U2 - 10.1007/s10606-018-9312-6

DO - 10.1007/s10606-018-9312-6

M3 - Article

AN - SCOPUS:85046656808

VL - 27

SP - 1255

EP - 1274

JO - Computer Supported Cooperative Work

JF - Computer Supported Cooperative Work

SN - 0925-9724

IS - 3-6

ER -