DARPA communicator: Cross-system results for the 2001 evaluation

Marilyn A. Walker, Alex Rudnicky, Rashmi Prasad, John Aberdeen, Elizabeth Owen Bratt, John Garofolo, Helen Hastie, Audrey Le, Bryan Pellom, Alex Potamianos, Rebecca Passonneau, Salim Roukos, Greg Sanders, Stephanie Seneff, Dave Stallard

Research output: Contribution to conferencePaper

57 Citations (Scopus)

Abstract

This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.

Original languageEnglish (US)
Pages269-272
Number of pages4
StatePublished - Jan 1 2002
Event7th International Conference on Spoken Language Processing, ICSLP 2002 - Denver, United States
Duration: Sep 16 2002Sep 20 2002

Other

Other7th International Conference on Spoken Language Processing, ICSLP 2002
CountryUnited States
CityDenver
Period9/16/029/20/02

Fingerprint

communicator
evaluation
travel
dialogue
planning
experiment
methodology
Evaluation
Communicators

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Linguistics and Language

Cite this

Walker, M. A., Rudnicky, A., Prasad, R., Aberdeen, J., Bratt, E. O., Garofolo, J., ... Stallard, D. (2002). DARPA communicator: Cross-system results for the 2001 evaluation. 269-272. Paper presented at 7th International Conference on Spoken Language Processing, ICSLP 2002, Denver, United States.
Walker, Marilyn A. ; Rudnicky, Alex ; Prasad, Rashmi ; Aberdeen, John ; Bratt, Elizabeth Owen ; Garofolo, John ; Hastie, Helen ; Le, Audrey ; Pellom, Bryan ; Potamianos, Alex ; Passonneau, Rebecca ; Roukos, Salim ; Sanders, Greg ; Seneff, Stephanie ; Stallard, Dave. / DARPA communicator : Cross-system results for the 2001 evaluation. Paper presented at 7th International Conference on Spoken Language Processing, ICSLP 2002, Denver, United States.4 p.
@conference{98081b6bb17343d1a92c122a1cae5cc0,
title = "DARPA communicator: Cross-system results for the 2001 evaluation",
abstract = "This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.",
author = "Walker, {Marilyn A.} and Alex Rudnicky and Rashmi Prasad and John Aberdeen and Bratt, {Elizabeth Owen} and John Garofolo and Helen Hastie and Audrey Le and Bryan Pellom and Alex Potamianos and Rebecca Passonneau and Salim Roukos and Greg Sanders and Stephanie Seneff and Dave Stallard",
year = "2002",
month = "1",
day = "1",
language = "English (US)",
pages = "269--272",
note = "7th International Conference on Spoken Language Processing, ICSLP 2002 ; Conference date: 16-09-2002 Through 20-09-2002",

}

Walker, MA, Rudnicky, A, Prasad, R, Aberdeen, J, Bratt, EO, Garofolo, J, Hastie, H, Le, A, Pellom, B, Potamianos, A, Passonneau, R, Roukos, S, Sanders, G, Seneff, S & Stallard, D 2002, 'DARPA communicator: Cross-system results for the 2001 evaluation', Paper presented at 7th International Conference on Spoken Language Processing, ICSLP 2002, Denver, United States, 9/16/02 - 9/20/02 pp. 269-272.

DARPA communicator : Cross-system results for the 2001 evaluation. / Walker, Marilyn A.; Rudnicky, Alex; Prasad, Rashmi; Aberdeen, John; Bratt, Elizabeth Owen; Garofolo, John; Hastie, Helen; Le, Audrey; Pellom, Bryan; Potamianos, Alex; Passonneau, Rebecca; Roukos, Salim; Sanders, Greg; Seneff, Stephanie; Stallard, Dave.

2002. 269-272 Paper presented at 7th International Conference on Spoken Language Processing, ICSLP 2002, Denver, United States.

Research output: Contribution to conferencePaper

TY - CONF

T1 - DARPA communicator

T2 - Cross-system results for the 2001 evaluation

AU - Walker, Marilyn A.

AU - Rudnicky, Alex

AU - Prasad, Rashmi

AU - Aberdeen, John

AU - Bratt, Elizabeth Owen

AU - Garofolo, John

AU - Hastie, Helen

AU - Le, Audrey

AU - Pellom, Bryan

AU - Potamianos, Alex

AU - Passonneau, Rebecca

AU - Roukos, Salim

AU - Sanders, Greg

AU - Seneff, Stephanie

AU - Stallard, Dave

PY - 2002/1/1

Y1 - 2002/1/1

N2 - This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.

AB - This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.

UR - http://www.scopus.com/inward/record.url?scp=85009257788&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85009257788&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85009257788

SP - 269

EP - 272

ER -

Walker MA, Rudnicky A, Prasad R, Aberdeen J, Bratt EO, Garofolo J et al. DARPA communicator: Cross-system results for the 2001 evaluation. 2002. Paper presented at 7th International Conference on Spoken Language Processing, ICSLP 2002, Denver, United States.