The influence of agent reliability on trust in human-agent collaboration

Xiaocong Fan, Sooyoung Oh, Michael McNeese, John Yen, Haydee Cuevas, Laura Strater, Mica R. Endsley

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Motivation - To investigate ways to support human-automation teams with real-world, imperfect automation where many system failures are the result of systematic failure. Research approach - An experimental approach was used to investigate how variance in agent reliability may influence human's trust and subsequent reliance on agent's decision aids. Sixty command and control (C2) teams, each consisting of a human operator and two cognitive agents, were asked to detect and respond to battlefield threats in six ten-minute scenarios. At the end of each scenario, participants completed the SAGAT queries, followed by the NASA TLX queries. Findings/Design - Results revealed that teams with experienced human operators accepted significantly less inappropriate recommendations from agents than teams with inexperienced operators. More importantly, the knowledge of agent's reliability and the ratio of unreliable tasks have significant effects on human's trust, as manifested in both team performance and human operators' rectification of inappropriate recommendations from agents. Originality/Value - It represents an important step toward uncovering the nature of human trust in human-agent collaboration. Take away message - This research has shown that given even minimal basis for understanding when the operator should and should not trust the agent recommendations allows operators to make better AUDs, to have better situation awareness on the critical issues associated with automation error, and to establish better trust in intelligent agents.

Original languageEnglish (US)
Title of host publicationProceedings of the ECCE 2008
Subtitle of host publicationEuropean Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction
DOIs
StatePublished - Dec 1 2008
Event15th European Conference on Cognitive Ergonomics the Ergonomics of Cool Interaction, ECCE 2008 - Funchal, Portugal
Duration: Sep 16 2008Sep 19 2008

Publication series

NameACM International Conference Proceeding Series
Volume369

Other

Other15th European Conference on Cognitive Ergonomics the Ergonomics of Cool Interaction, ECCE 2008
CountryPortugal
CityFunchal
Period9/16/089/19/08

Fingerprint

Automation
Intelligent agents
NASA

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Cite this

Fan, X., Oh, S., McNeese, M., Yen, J., Cuevas, H., Strater, L., & Endsley, M. R. (2008). The influence of agent reliability on trust in human-agent collaboration. In Proceedings of the ECCE 2008: European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction [1473028] (ACM International Conference Proceeding Series; Vol. 369). https://doi.org/10.1145/1473018.1473028
Fan, Xiaocong ; Oh, Sooyoung ; McNeese, Michael ; Yen, John ; Cuevas, Haydee ; Strater, Laura ; Endsley, Mica R. / The influence of agent reliability on trust in human-agent collaboration. Proceedings of the ECCE 2008: European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction. 2008. (ACM International Conference Proceeding Series).
@inproceedings{3d1b9b20058c4353a2112d1be4a296e8,
title = "The influence of agent reliability on trust in human-agent collaboration",
abstract = "Motivation - To investigate ways to support human-automation teams with real-world, imperfect automation where many system failures are the result of systematic failure. Research approach - An experimental approach was used to investigate how variance in agent reliability may influence human's trust and subsequent reliance on agent's decision aids. Sixty command and control (C2) teams, each consisting of a human operator and two cognitive agents, were asked to detect and respond to battlefield threats in six ten-minute scenarios. At the end of each scenario, participants completed the SAGAT queries, followed by the NASA TLX queries. Findings/Design - Results revealed that teams with experienced human operators accepted significantly less inappropriate recommendations from agents than teams with inexperienced operators. More importantly, the knowledge of agent's reliability and the ratio of unreliable tasks have significant effects on human's trust, as manifested in both team performance and human operators' rectification of inappropriate recommendations from agents. Originality/Value - It represents an important step toward uncovering the nature of human trust in human-agent collaboration. Take away message - This research has shown that given even minimal basis for understanding when the operator should and should not trust the agent recommendations allows operators to make better AUDs, to have better situation awareness on the critical issues associated with automation error, and to establish better trust in intelligent agents.",
author = "Xiaocong Fan and Sooyoung Oh and Michael McNeese and John Yen and Haydee Cuevas and Laura Strater and Endsley, {Mica R.}",
year = "2008",
month = "12",
day = "1",
doi = "10.1145/1473018.1473028",
language = "English (US)",
isbn = "9781605583990",
series = "ACM International Conference Proceeding Series",
booktitle = "Proceedings of the ECCE 2008",

}

Fan, X, Oh, S, McNeese, M, Yen, J, Cuevas, H, Strater, L & Endsley, MR 2008, The influence of agent reliability on trust in human-agent collaboration. in Proceedings of the ECCE 2008: European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction., 1473028, ACM International Conference Proceeding Series, vol. 369, 15th European Conference on Cognitive Ergonomics the Ergonomics of Cool Interaction, ECCE 2008, Funchal, Portugal, 9/16/08. https://doi.org/10.1145/1473018.1473028

The influence of agent reliability on trust in human-agent collaboration. / Fan, Xiaocong; Oh, Sooyoung; McNeese, Michael; Yen, John; Cuevas, Haydee; Strater, Laura; Endsley, Mica R.

Proceedings of the ECCE 2008: European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction. 2008. 1473028 (ACM International Conference Proceeding Series; Vol. 369).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - The influence of agent reliability on trust in human-agent collaboration

AU - Fan, Xiaocong

AU - Oh, Sooyoung

AU - McNeese, Michael

AU - Yen, John

AU - Cuevas, Haydee

AU - Strater, Laura

AU - Endsley, Mica R.

PY - 2008/12/1

Y1 - 2008/12/1

N2 - Motivation - To investigate ways to support human-automation teams with real-world, imperfect automation where many system failures are the result of systematic failure. Research approach - An experimental approach was used to investigate how variance in agent reliability may influence human's trust and subsequent reliance on agent's decision aids. Sixty command and control (C2) teams, each consisting of a human operator and two cognitive agents, were asked to detect and respond to battlefield threats in six ten-minute scenarios. At the end of each scenario, participants completed the SAGAT queries, followed by the NASA TLX queries. Findings/Design - Results revealed that teams with experienced human operators accepted significantly less inappropriate recommendations from agents than teams with inexperienced operators. More importantly, the knowledge of agent's reliability and the ratio of unreliable tasks have significant effects on human's trust, as manifested in both team performance and human operators' rectification of inappropriate recommendations from agents. Originality/Value - It represents an important step toward uncovering the nature of human trust in human-agent collaboration. Take away message - This research has shown that given even minimal basis for understanding when the operator should and should not trust the agent recommendations allows operators to make better AUDs, to have better situation awareness on the critical issues associated with automation error, and to establish better trust in intelligent agents.

AB - Motivation - To investigate ways to support human-automation teams with real-world, imperfect automation where many system failures are the result of systematic failure. Research approach - An experimental approach was used to investigate how variance in agent reliability may influence human's trust and subsequent reliance on agent's decision aids. Sixty command and control (C2) teams, each consisting of a human operator and two cognitive agents, were asked to detect and respond to battlefield threats in six ten-minute scenarios. At the end of each scenario, participants completed the SAGAT queries, followed by the NASA TLX queries. Findings/Design - Results revealed that teams with experienced human operators accepted significantly less inappropriate recommendations from agents than teams with inexperienced operators. More importantly, the knowledge of agent's reliability and the ratio of unreliable tasks have significant effects on human's trust, as manifested in both team performance and human operators' rectification of inappropriate recommendations from agents. Originality/Value - It represents an important step toward uncovering the nature of human trust in human-agent collaboration. Take away message - This research has shown that given even minimal basis for understanding when the operator should and should not trust the agent recommendations allows operators to make better AUDs, to have better situation awareness on the critical issues associated with automation error, and to establish better trust in intelligent agents.

UR - http://www.scopus.com/inward/record.url?scp=77953729494&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77953729494&partnerID=8YFLogxK

U2 - 10.1145/1473018.1473028

DO - 10.1145/1473018.1473028

M3 - Conference contribution

AN - SCOPUS:77953729494

SN - 9781605583990

T3 - ACM International Conference Proceeding Series

BT - Proceedings of the ECCE 2008

ER -

Fan X, Oh S, McNeese M, Yen J, Cuevas H, Strater L et al. The influence of agent reliability on trust in human-agent collaboration. In Proceedings of the ECCE 2008: European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction. 2008. 1473028. (ACM International Conference Proceeding Series). https://doi.org/10.1145/1473018.1473028