Automatic image semantic interpretation using social action and tagging data

Neela Sawant, Jia Li, James Z. Wang

Research output: Contribution to journalArticlepeer-review

35 Scopus citations


The plethora of social actions and annotations (tags, comments, ratings) from online media sharing Websites and collaborative games have induced a paradigm shift in the research on image semantic interpretation. Social inputs with their added context represent a strong substitute for expert annotations. Novel algorithms have been designed to fuse visual features with noisy social labels and behavioral signals. In this survey, we review nearly 200 representative papers to identify the current trends, challenges as well as opportunities presented by social inputs for research on image semantics. Our study builds on an interdisciplinary confluence of insights from image processing, data mining, human computer interaction, and sociology to describe the folksonomic features of users, annotations and images. Applications are categorized into four types: concept semantics, person identification, location semantics and event semantics. The survey concludes with a summary of principle research directions for the present and the future.

Original languageEnglish (US)
Pages (from-to)213-246
Number of pages34
JournalMultimedia Tools and Applications
Issue number1
StatePublished - Jan 2011

All Science Journal Classification (ASJC) codes

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Automatic image semantic interpretation using social action and tagging data'. Together they form a unique fingerprint.

Cite this