Evorus: A Crowd-powered conversational assistant built to automate itself over time

Ting Hao Huang, Joseph Chee Chang, Jeffrey P. Bigham

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

Crowd-powered conversational assistants have been shown to be more robust than automated systems, but do so at the cost of higher response latency and monetary costs. A promising direction is to combine the two approaches for high quality, low latency, and low cost solutions. In this paper, we introduce Evorus, a crowd-powered conversational assistant built to automate itself over time by (i) allowing new chatbots to be easily integrated to automate more scenarios, (ii) reusing prior crowd answers, and (iii) learning to automatically approve response candidates. Our 5-month-long deployment with 80 participants and 281 conversations shows that Evorus can automate itself without compromising conversation quality. Crowd-AI architectures have long been proposed as a way to reduce cost and latency for crowd-powered systems; Evorus demonstrates how automation can be introduced successfully in a deployed system. Its architecture allows future researchers to make further innovation on the underlying automated components in the context of a deployed open domain dialog system.

Original languageEnglish (US)
Title of host publicationCHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
Subtitle of host publicationEngage with CHI
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450356206, 9781450356213
DOIs
StatePublished - Apr 20 2018
Event2018 CHI Conference on Human Factors in Computing Systems, CHI 2018 - Montreal, Canada
Duration: Apr 21 2018Apr 26 2018

Publication series

NameConference on Human Factors in Computing Systems - Proceedings
Volume2018-April

Other

Other2018 CHI Conference on Human Factors in Computing Systems, CHI 2018
CountryCanada
CityMontreal
Period4/21/184/26/18

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Evorus: A Crowd-powered conversational assistant built to automate itself over time'. Together they form a unique fingerprint.

  • Cite this

    Huang, T. H., Chang, J. C., & Bigham, J. P. (2018). Evorus: A Crowd-powered conversational assistant built to automate itself over time. In CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI (Conference on Human Factors in Computing Systems - Proceedings; Vol. 2018-April). Association for Computing Machinery. https://doi.org/10.1145/3173574.3173869