Task relevance and diversity as worker motivation in crowdsourcing

Julien Pilourdault, Sihem Amer-Yahia, Senjuti Basu Roy, Dongwon Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Task assignment is a central component in crowdsourcing. Organizational studies have shown that worker motivation in completing tasks has a direct impact on the quality of individual contributions. In this work, we examine motivation-Aware task assignment in the presence of a set of workers. We propose to model motivation as a balance between task relevance and task diversity and argue that an adaptive approach to task assignment can best capture the evolving nature of motivation. Worker motivation is observed and task assignment is revisited appropriately across iterations. We prove the problem to be NP-hard as well as MaxSNP-Hard and develop efficient approximation algorithms with provable guarantees. Our experiments with synthetic data examine the scalability of our algorithms, and our live real data experiments show that capturing motivation using relevance and diversity leads to high crowdwork quality.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages365-376
Number of pages12
ISBN (Electronic)9781538655207
DOIs
StatePublished - Oct 24 2018
Event34th IEEE International Conference on Data Engineering, ICDE 2018 - Paris, France
Duration: Apr 16 2018Apr 19 2018

Publication series

NameProceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

Other

Other34th IEEE International Conference on Data Engineering, ICDE 2018
CountryFrance
CityParis
Period4/16/184/19/18

Fingerprint

Approximation algorithms
Workers
Task assignment
Scalability
Experiments
Experiment
Guarantee
NP-hard
Organizational studies

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Information Systems and Management
  • Hardware and Architecture

Cite this

Pilourdault, J., Amer-Yahia, S., Basu Roy, S., & Lee, D. (2018). Task relevance and diversity as worker motivation in crowdsourcing. In Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018 (pp. 365-376). [8509262] (Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDE.2018.00041
Pilourdault, Julien ; Amer-Yahia, Sihem ; Basu Roy, Senjuti ; Lee, Dongwon. / Task relevance and diversity as worker motivation in crowdsourcing. Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 365-376 (Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018).
@inproceedings{2a4dd382bac3486bbb8e4193f9e40a42,
title = "Task relevance and diversity as worker motivation in crowdsourcing",
abstract = "Task assignment is a central component in crowdsourcing. Organizational studies have shown that worker motivation in completing tasks has a direct impact on the quality of individual contributions. In this work, we examine motivation-Aware task assignment in the presence of a set of workers. We propose to model motivation as a balance between task relevance and task diversity and argue that an adaptive approach to task assignment can best capture the evolving nature of motivation. Worker motivation is observed and task assignment is revisited appropriately across iterations. We prove the problem to be NP-hard as well as MaxSNP-Hard and develop efficient approximation algorithms with provable guarantees. Our experiments with synthetic data examine the scalability of our algorithms, and our live real data experiments show that capturing motivation using relevance and diversity leads to high crowdwork quality.",
author = "Julien Pilourdault and Sihem Amer-Yahia and {Basu Roy}, Senjuti and Dongwon Lee",
year = "2018",
month = "10",
day = "24",
doi = "10.1109/ICDE.2018.00041",
language = "English (US)",
series = "Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "365--376",
booktitle = "Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018",
address = "United States",

}

Pilourdault, J, Amer-Yahia, S, Basu Roy, S & Lee, D 2018, Task relevance and diversity as worker motivation in crowdsourcing. in Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018., 8509262, Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018, Institute of Electrical and Electronics Engineers Inc., pp. 365-376, 34th IEEE International Conference on Data Engineering, ICDE 2018, Paris, France, 4/16/18. https://doi.org/10.1109/ICDE.2018.00041

Task relevance and diversity as worker motivation in crowdsourcing. / Pilourdault, Julien; Amer-Yahia, Sihem; Basu Roy, Senjuti; Lee, Dongwon.

Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 365-376 8509262 (Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Task relevance and diversity as worker motivation in crowdsourcing

AU - Pilourdault, Julien

AU - Amer-Yahia, Sihem

AU - Basu Roy, Senjuti

AU - Lee, Dongwon

PY - 2018/10/24

Y1 - 2018/10/24

N2 - Task assignment is a central component in crowdsourcing. Organizational studies have shown that worker motivation in completing tasks has a direct impact on the quality of individual contributions. In this work, we examine motivation-Aware task assignment in the presence of a set of workers. We propose to model motivation as a balance between task relevance and task diversity and argue that an adaptive approach to task assignment can best capture the evolving nature of motivation. Worker motivation is observed and task assignment is revisited appropriately across iterations. We prove the problem to be NP-hard as well as MaxSNP-Hard and develop efficient approximation algorithms with provable guarantees. Our experiments with synthetic data examine the scalability of our algorithms, and our live real data experiments show that capturing motivation using relevance and diversity leads to high crowdwork quality.

AB - Task assignment is a central component in crowdsourcing. Organizational studies have shown that worker motivation in completing tasks has a direct impact on the quality of individual contributions. In this work, we examine motivation-Aware task assignment in the presence of a set of workers. We propose to model motivation as a balance between task relevance and task diversity and argue that an adaptive approach to task assignment can best capture the evolving nature of motivation. Worker motivation is observed and task assignment is revisited appropriately across iterations. We prove the problem to be NP-hard as well as MaxSNP-Hard and develop efficient approximation algorithms with provable guarantees. Our experiments with synthetic data examine the scalability of our algorithms, and our live real data experiments show that capturing motivation using relevance and diversity leads to high crowdwork quality.

UR - http://www.scopus.com/inward/record.url?scp=85057124544&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057124544&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2018.00041

DO - 10.1109/ICDE.2018.00041

M3 - Conference contribution

AN - SCOPUS:85057124544

T3 - Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

SP - 365

EP - 376

BT - Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Pilourdault J, Amer-Yahia S, Basu Roy S, Lee D. Task relevance and diversity as worker motivation in crowdsourcing. In Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 365-376. 8509262. (Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018). https://doi.org/10.1109/ICDE.2018.00041