The story picturing engine: Finding elite images to illustrate a story using mutual reinforcement

Dhiraj Joshi, James Z. Wang, Jia Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

28 Scopus citations

Abstract

In this paper, we present an approach towards automated story picturing based on mutual reinforcement principle. Story picturing refers to the process of illustrating a story with suitable pictures. In our approach, semantic keywords are extracted from the story text and an annotated image database is searched to form an initial picture pool. Thereafter, a novel image ranking scheme automatically determines the importance of each image. Both lexical annotations and visual content of an image play a role in determining its rank. Annotations are processed using the Wordnet to derive a lexical signature for each image. An integrated region based similarity is also calculated between each pair of images. An overall similarity measure is formed using lexical and visual features. In the end, a mutual reinforcement based rank is calculated for each image using the image similarity matrix. We also present a human behavior model based on a discrete state Markov process which captures the intuition for our technique. Experimental results have demonstrated the effectiveness of our scheme.

Original languageEnglish (US)
Title of host publicationMIR'04 - Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval
Pages119-126
Number of pages8
StatePublished - Dec 1 2004
EventMIR'04 - Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval - New York, NY, United States
Duration: Oct 15 2004Oct 16 2004

Publication series

NameMIR'04 - Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval

Other

OtherMIR'04 - Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval
CountryUnited States
CityNew York, NY
Period10/15/0410/16/04

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Fingerprint Dive into the research topics of 'The story picturing engine: Finding elite images to illustrate a story using mutual reinforcement'. Together they form a unique fingerprint.

  • Cite this

    Joshi, D., Wang, J. Z., & Li, J. (2004). The story picturing engine: Finding elite images to illustrate a story using mutual reinforcement. In MIR'04 - Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval (pp. 119-126). (MIR'04 - Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval).