Functional semantic categories for art history text - Human labeling and preliminary machine learning

Rebecca Jane Passonneau, Tae Yano, Tom Lippincott, Judith Klavans

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

The CLiMB project investigates semi-automatic methods to extract descriptive metadata from texts for indexing digital image collections. We developed a set of functional semantic categories to classify text extracts that describe images. Each semantic category names a functional relation between an image depicting a work of art historical significance, and expository text associated with the image. This includes description of the image, discussion of the historical context in which the work was created, and so on. We present interannotator agreement results on human classification of text extracts, and accuracy results from initial machine learning experiments. In our pilot studies, human agreement varied widely, depending the labeler's expertise, the image-text pair under consideration, the number of labels that could be assigned to one text, and the type of training, if any, we gave labelers. Initial machine learning results indicate the three most relevant categories are machine learnable. Based on our pilot work, we implemented a labeling interface that we are currently using to collect a large dataset of text that will be used in training and testing machine classifiers.

Original languageEnglish (US)
Title of host publicationProceedings of the 1st International Workshop on Metadata Mining for Image Understanding, MMIU 2008 - In Conjunction with VISIGRAPP 2008
Pages13-22
Number of pages10
StatePublished - Dec 15 2008
Event1st International Workshop on Metadata Mining for Image Understanding, MMIU 2008 - In Conjunction with VISIGRAPP 2008 - Funchal, Madeira, Portugal
Duration: Jan 22 2008Jan 25 2008

Publication series

NameProceedings of the 1st International Workshop on Metadata Mining for Image Understanding, MMIU 2008 - In Conjunction with VISIGRAPP 2008

Other

Other1st International Workshop on Metadata Mining for Image Understanding, MMIU 2008 - In Conjunction with VISIGRAPP 2008
CountryPortugal
CityFunchal, Madeira
Period1/22/081/25/08

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint Dive into the research topics of 'Functional semantic categories for art history text - Human labeling and preliminary machine learning'. Together they form a unique fingerprint.

  • Cite this

    Passonneau, R. J., Yano, T., Lippincott, T., & Klavans, J. (2008). Functional semantic categories for art history text - Human labeling and preliminary machine learning. In Proceedings of the 1st International Workshop on Metadata Mining for Image Understanding, MMIU 2008 - In Conjunction with VISIGRAPP 2008 (pp. 13-22). (Proceedings of the 1st International Workshop on Metadata Mining for Image Understanding, MMIU 2008 - In Conjunction with VISIGRAPP 2008).