Detecting arbitrary oriented text in the wild with a visual attention model

Wenyi Huang; Dafang He; Xiao Yang; Zihan Zhou; Daniel Kifer; C. Lee Giles

doi:10.1145/2964284.2967282

Detecting arbitrary oriented text in the wild with a visual attention model

Wenyi Huang, Dafang He, Xiao Yang, Zihan Zhou, Daniel Kifer, C. Lee Giles

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Scopus citations

Abstract

Text embedded in images provides important semantic information about a scene and its content. Detecting text in an unconstrained environment is a challenging task because of the many fonts, sizes, backgrounds, and alignments of the characters. We present a novel attention model for detecting arbitrary oriented and curved scene text. Inspired by the attention mechanisms in the human visual system, our model utilizes a spatial glimpse network to processes the attended area and deploys a recurrent neural network that aggregates the information over time to determine the attention movement. Combining this with an off-the-shelf region proposal method, the model achieves the state-of-the-art performance on the highly cited ICDAR2013 dataset, and the MSRA-TD500 dataset which contains arbitrary oriented text.

Original language	English (US)
Title of host publication	MM 2016 - Proceedings of the 2016 ACM Multimedia Conference
Publisher	Association for Computing Machinery, Inc
Pages	551-555
Number of pages	5
ISBN (Electronic)	9781450336031
DOIs	https://doi.org/10.1145/2964284.2967282
State	Published - Oct 1 2016
Event	24th ACM Multimedia Conference, MM 2016 - Amsterdam, United Kingdom Duration: Oct 15 2016 → Oct 19 2016

Publication series

Name	MM 2016 - Proceedings of the 2016 ACM Multimedia Conference

Other

Other	24th ACM Multimedia Conference, MM 2016
Country	United Kingdom
City	Amsterdam
Period	10/15/16 → 10/19/16

Fingerprint

All Science Journal Classification (ASJC) codes

Computer Graphics and Computer-Aided Design
Human-Computer Interaction
Computer Vision and Pattern Recognition
Software

Cite this

Huang, W., He, D., Yang, X., Zhou, Z., Kifer, D., & Giles, C. L. (2016). Detecting arbitrary oriented text in the wild with a visual attention model. In MM 2016 - Proceedings of the 2016 ACM Multimedia Conference (pp. 551-555). (MM 2016 - Proceedings of the 2016 ACM Multimedia Conference). Association for Computing Machinery, Inc. https://doi.org/10.1145/2964284.2967282

@inproceedings{3db959e7696345a0a640b076f1ddb150,

title = "Detecting arbitrary oriented text in the wild with a visual attention model",

abstract = "Text embedded in images provides important semantic information about a scene and its content. Detecting text in an unconstrained environment is a challenging task because of the many fonts, sizes, backgrounds, and alignments of the characters. We present a novel attention model for detecting arbitrary oriented and curved scene text. Inspired by the attention mechanisms in the human visual system, our model utilizes a spatial glimpse network to processes the attended area and deploys a recurrent neural network that aggregates the information over time to determine the attention movement. Combining this with an off-the-shelf region proposal method, the model achieves the state-of-the-art performance on the highly cited ICDAR2013 dataset, and the MSRA-TD500 dataset which contains arbitrary oriented text.",

author = "Wenyi Huang and Dafang He and Xiao Yang and Zihan Zhou and Daniel Kifer and Giles, {C. Lee}",

year = "2016",

month = oct,

day = "1",

doi = "10.1145/2964284.2967282",

language = "English (US)",

series = "MM 2016 - Proceedings of the 2016 ACM Multimedia Conference",

publisher = "Association for Computing Machinery, Inc",

pages = "551--555",

booktitle = "MM 2016 - Proceedings of the 2016 ACM Multimedia Conference",

note = "24th ACM Multimedia Conference, MM 2016 ; Conference date: 15-10-2016 Through 19-10-2016",

}

Huang, W, He, D, Yang, X, Zhou, Z , Kifer, D & Giles, CL 2016, Detecting arbitrary oriented text in the wild with a visual attention model. in MM 2016 - Proceedings of the 2016 ACM Multimedia Conference. MM 2016 - Proceedings of the 2016 ACM Multimedia Conference, Association for Computing Machinery, Inc, pp. 551-555, 24th ACM Multimedia Conference, MM 2016, Amsterdam, United Kingdom, 10/15/16. https://doi.org/10.1145/2964284.2967282

Detecting arbitrary oriented text in the wild with a visual attention model. / Huang, Wenyi; He, Dafang; Yang, Xiao; Zhou, Zihan ; Kifer, Daniel ; Giles, C. Lee.

MM 2016 - Proceedings of the 2016 ACM Multimedia Conference. Association for Computing Machinery, Inc, 2016. p. 551-555 (MM 2016 - Proceedings of the 2016 ACM Multimedia Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Detecting arbitrary oriented text in the wild with a visual attention model

AU - Huang, Wenyi

AU - He, Dafang

AU - Yang, Xiao

AU - Zhou, Zihan

AU - Kifer, Daniel

AU - Giles, C. Lee

PY - 2016/10/1

Y1 - 2016/10/1

N2 - Text embedded in images provides important semantic information about a scene and its content. Detecting text in an unconstrained environment is a challenging task because of the many fonts, sizes, backgrounds, and alignments of the characters. We present a novel attention model for detecting arbitrary oriented and curved scene text. Inspired by the attention mechanisms in the human visual system, our model utilizes a spatial glimpse network to processes the attended area and deploys a recurrent neural network that aggregates the information over time to determine the attention movement. Combining this with an off-the-shelf region proposal method, the model achieves the state-of-the-art performance on the highly cited ICDAR2013 dataset, and the MSRA-TD500 dataset which contains arbitrary oriented text.

AB - Text embedded in images provides important semantic information about a scene and its content. Detecting text in an unconstrained environment is a challenging task because of the many fonts, sizes, backgrounds, and alignments of the characters. We present a novel attention model for detecting arbitrary oriented and curved scene text. Inspired by the attention mechanisms in the human visual system, our model utilizes a spatial glimpse network to processes the attended area and deploys a recurrent neural network that aggregates the information over time to determine the attention movement. Combining this with an off-the-shelf region proposal method, the model achieves the state-of-the-art performance on the highly cited ICDAR2013 dataset, and the MSRA-TD500 dataset which contains arbitrary oriented text.

UR - http://www.scopus.com/inward/record.url?scp=84994626543&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994626543&partnerID=8YFLogxK

U2 - 10.1145/2964284.2967282

DO - 10.1145/2964284.2967282

M3 - Conference contribution

AN - SCOPUS:84994626543

T3 - MM 2016 - Proceedings of the 2016 ACM Multimedia Conference

SP - 551

EP - 555

BT - MM 2016 - Proceedings of the 2016 ACM Multimedia Conference

PB - Association for Computing Machinery, Inc

T2 - 24th ACM Multimedia Conference, MM 2016

Y2 - 15 October 2016 through 19 October 2016

ER -

Access to Document

10.1145/2964284.2967282