Human-automated judge learning: A methodology for examining human interaction with information analysis automation

Ellen J. Bass, Amy R. Pritchett

Research output: Contribution to journalArticle

23 Citations (Scopus)

Abstract

Human-automated judge learning (HAJL) is a methodology providing a three-phase process, quantitative measures, and analytical methods to support design of information analysis automation. HAJL's measures capture the human and automation's judgment processes, relevant features of the environment, and the relationships between each. Specific measures include achievement of the human and the automation, conflict between them, compromise and adaptation by the human toward the automation, and the human's ability to predict the automation. HAJL's utility is demonstrated herein using a simplified air traffic conflict prediction task. HAJL was able to capture patterns of behavior within and across the three phases with measures of individual judgments and human-automation interaction. Its measures were also used for statistical tests of aggregate effects across human judges. Two between-subject manipulations were crossed to investigate HAJL's sensitivity to interventions in the human's training (sensor noise during training) and in display design (information from the automation about its judgment strategy). HAJL identified that the design intervention impacted conflict and compromise with the automation, participants learned from the automation over time, and those with higher individual judgment achievement were also better able to predict the automation.

Original languageEnglish (US)
Pages (from-to)759-776
Number of pages18
JournalIEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans
Volume38
Issue number4
DOIs
StatePublished - Jul 1 2008

Fingerprint

Information analysis
Automation
Statistical tests
Display devices

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

@article{9912e6af26a14191afe3ac3aa43dcf34,
title = "Human-automated judge learning: A methodology for examining human interaction with information analysis automation",
abstract = "Human-automated judge learning (HAJL) is a methodology providing a three-phase process, quantitative measures, and analytical methods to support design of information analysis automation. HAJL's measures capture the human and automation's judgment processes, relevant features of the environment, and the relationships between each. Specific measures include achievement of the human and the automation, conflict between them, compromise and adaptation by the human toward the automation, and the human's ability to predict the automation. HAJL's utility is demonstrated herein using a simplified air traffic conflict prediction task. HAJL was able to capture patterns of behavior within and across the three phases with measures of individual judgments and human-automation interaction. Its measures were also used for statistical tests of aggregate effects across human judges. Two between-subject manipulations were crossed to investigate HAJL's sensitivity to interventions in the human's training (sensor noise during training) and in display design (information from the automation about its judgment strategy). HAJL identified that the design intervention impacted conflict and compromise with the automation, participants learned from the automation over time, and those with higher individual judgment achievement were also better able to predict the automation.",
author = "Bass, {Ellen J.} and Pritchett, {Amy R.}",
year = "2008",
month = "7",
day = "1",
doi = "10.1109/TSMCA.2008.923068",
language = "English (US)",
volume = "38",
pages = "759--776",
journal = "IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans",
issn = "1083-4427",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "4",

}

TY - JOUR

T1 - Human-automated judge learning

T2 - A methodology for examining human interaction with information analysis automation

AU - Bass, Ellen J.

AU - Pritchett, Amy R.

PY - 2008/7/1

Y1 - 2008/7/1

N2 - Human-automated judge learning (HAJL) is a methodology providing a three-phase process, quantitative measures, and analytical methods to support design of information analysis automation. HAJL's measures capture the human and automation's judgment processes, relevant features of the environment, and the relationships between each. Specific measures include achievement of the human and the automation, conflict between them, compromise and adaptation by the human toward the automation, and the human's ability to predict the automation. HAJL's utility is demonstrated herein using a simplified air traffic conflict prediction task. HAJL was able to capture patterns of behavior within and across the three phases with measures of individual judgments and human-automation interaction. Its measures were also used for statistical tests of aggregate effects across human judges. Two between-subject manipulations were crossed to investigate HAJL's sensitivity to interventions in the human's training (sensor noise during training) and in display design (information from the automation about its judgment strategy). HAJL identified that the design intervention impacted conflict and compromise with the automation, participants learned from the automation over time, and those with higher individual judgment achievement were also better able to predict the automation.

AB - Human-automated judge learning (HAJL) is a methodology providing a three-phase process, quantitative measures, and analytical methods to support design of information analysis automation. HAJL's measures capture the human and automation's judgment processes, relevant features of the environment, and the relationships between each. Specific measures include achievement of the human and the automation, conflict between them, compromise and adaptation by the human toward the automation, and the human's ability to predict the automation. HAJL's utility is demonstrated herein using a simplified air traffic conflict prediction task. HAJL was able to capture patterns of behavior within and across the three phases with measures of individual judgments and human-automation interaction. Its measures were also used for statistical tests of aggregate effects across human judges. Two between-subject manipulations were crossed to investigate HAJL's sensitivity to interventions in the human's training (sensor noise during training) and in display design (information from the automation about its judgment strategy). HAJL identified that the design intervention impacted conflict and compromise with the automation, participants learned from the automation over time, and those with higher individual judgment achievement were also better able to predict the automation.

UR - http://www.scopus.com/inward/record.url?scp=46649096011&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=46649096011&partnerID=8YFLogxK

U2 - 10.1109/TSMCA.2008.923068

DO - 10.1109/TSMCA.2008.923068

M3 - Article

AN - SCOPUS:46649096011

VL - 38

SP - 759

EP - 776

JO - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

JF - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

SN - 1083-4427

IS - 4

ER -