A survey of current datasets for vision and language research

Francis Ferraro; Nasrin Mostafazadeh; Ting Hao Huang; Lucy Vanderwende; Jacob Devlin; Michel Galley; Margaret Mitchell

doi:10.18653/v1/d15-1021

A survey of current datasets for vision and language research

Francis Ferraro, Nasrin Mostafazadeh, Ting Hao Huang, Lucy Vanderwende, Jacob Devlin, Michel Galley, Margaret Mitchell

College of Information Sciences and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

34 Scopus citations

Abstract

Integrating vision and language has long been a dream in work on artificial intelligence (AI). In the past two years, we have witnessed an explosion of work that brings together vision and language from images to videos and beyond. The available corpora have played a crucial role in advancing this area of research. In this paper, we propose a set of quality metrics for evaluating and analyzing the vision & language datasets and categorize them accordingly. Our analyses show that the most recent datasets have been using more complex language and more abstract concepts, however, there are different strengths and weaknesses in each.

Original language	English (US)
Title of host publication	Conference Proceedings - EMNLP 2015
Subtitle of host publication	Conference on Empirical Methods in Natural Language Processing
Publisher	Association for Computational Linguistics (ACL)
Pages	207-213
Number of pages	7
ISBN (Electronic)	9781941643327
DOIs	https://doi.org/10.18653/v1/d15-1021
State	Published - 2015
Event	Conference on Empirical Methods in Natural Language Processing, EMNLP 2015 - Lisbon, Portugal Duration: Sep 17 2015 → Sep 21 2015

Publication series

Name	Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing

Other

Other	Conference on Empirical Methods in Natural Language Processing, EMNLP 2015
Country/Territory	Portugal
City	Lisbon
Period	9/17/15 → 9/21/15

All Science Journal Classification (ASJC) codes

Computational Theory and Mathematics
Computer Science Applications
Information Systems

Access to Document

10.18653/v1/d15-1021

Cite this

Ferraro, F., Mostafazadeh, N., Huang, T. H., Vanderwende, L., Devlin, J., Galley, M., & Mitchell, M. (2015). A survey of current datasets for vision and language research. In Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing (pp. 207-213). (Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/d15-1021

Ferraro, Francis ; Mostafazadeh, Nasrin ; Huang, Ting Hao ; Vanderwende, Lucy ; Devlin, Jacob ; Galley, Michel ; Mitchell, Margaret. / A survey of current datasets for vision and language research. Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2015. pp. 207-213 (Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing).

@inproceedings{a38bc565b53c49c4b229bf4019b29dd6,

title = "A survey of current datasets for vision and language research",

abstract = "Integrating vision and language has long been a dream in work on artificial intelligence (AI). In the past two years, we have witnessed an explosion of work that brings together vision and language from images to videos and beyond. The available corpora have played a crucial role in advancing this area of research. In this paper, we propose a set of quality metrics for evaluating and analyzing the vision & language datasets and categorize them accordingly. Our analyses show that the most recent datasets have been using more complex language and more abstract concepts, however, there are different strengths and weaknesses in each.",

author = "Francis Ferraro and Nasrin Mostafazadeh and Huang, {Ting Hao} and Lucy Vanderwende and Jacob Devlin and Michel Galley and Margaret Mitchell",

note = "Publisher Copyright: {\textcopyright} 2015 Association for Computational Linguistics.; Conference on Empirical Methods in Natural Language Processing, EMNLP 2015 ; Conference date: 17-09-2015 Through 21-09-2015",

year = "2015",

doi = "10.18653/v1/d15-1021",

language = "English (US)",

series = "Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing",

publisher = "Association for Computational Linguistics (ACL)",

pages = "207--213",

booktitle = "Conference Proceedings - EMNLP 2015",

}

Ferraro, F, Mostafazadeh, N, Huang, TH, Vanderwende, L, Devlin, J, Galley, M & Mitchell, M 2015, A survey of current datasets for vision and language research. in Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing. Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics (ACL), pp. 207-213, Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, 9/17/15. https://doi.org/10.18653/v1/d15-1021

A survey of current datasets for vision and language research. / Ferraro, Francis; Mostafazadeh, Nasrin; Huang, Ting Hao; Vanderwende, Lucy; Devlin, Jacob; Galley, Michel; Mitchell, Margaret.

Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2015. p. 207-213 (Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A survey of current datasets for vision and language research

AU - Ferraro, Francis

AU - Mostafazadeh, Nasrin

AU - Huang, Ting Hao

AU - Vanderwende, Lucy

AU - Devlin, Jacob

AU - Galley, Michel

AU - Mitchell, Margaret

PY - 2015

Y1 - 2015

N2 - Integrating vision and language has long been a dream in work on artificial intelligence (AI). In the past two years, we have witnessed an explosion of work that brings together vision and language from images to videos and beyond. The available corpora have played a crucial role in advancing this area of research. In this paper, we propose a set of quality metrics for evaluating and analyzing the vision & language datasets and categorize them accordingly. Our analyses show that the most recent datasets have been using more complex language and more abstract concepts, however, there are different strengths and weaknesses in each.

AB - Integrating vision and language has long been a dream in work on artificial intelligence (AI). In the past two years, we have witnessed an explosion of work that brings together vision and language from images to videos and beyond. The available corpora have played a crucial role in advancing this area of research. In this paper, we propose a set of quality metrics for evaluating and analyzing the vision & language datasets and categorize them accordingly. Our analyses show that the most recent datasets have been using more complex language and more abstract concepts, however, there are different strengths and weaknesses in each.

UR - http://www.scopus.com/inward/record.url?scp=84959904882&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84959904882&partnerID=8YFLogxK

U2 - 10.18653/v1/d15-1021

DO - 10.18653/v1/d15-1021

M3 - Conference contribution

AN - SCOPUS:84959904882

T3 - Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing

SP - 207

EP - 213

BT - Conference Proceedings - EMNLP 2015

PB - Association for Computational Linguistics (ACL)

T2 - Conference on Empirical Methods in Natural Language Processing, EMNLP 2015

Y2 - 17 September 2015 through 21 September 2015

ER -

Ferraro F, Mostafazadeh N, Huang TH, Vanderwende L, Devlin J, Galley M et al. A survey of current datasets for vision and language research. In Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL). 2015. p. 207-213. (Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing). https://doi.org/10.18653/v1/d15-1021

A survey of current datasets for vision and language research

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this