An FPGA implementation of information theoretic visual-saliency system and its optimization

Sungmin Bae, Yong Cheol Peter Cho, Sungho Park, Kevin M. Irick, Yongseok Jin, Vijaykrishnan Narayanan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

25 Citations (Scopus)

Abstract

Biological vision systems use saliency-based visual attention mechanisms to limit higher-level vision processing on the most visually-salient subsets of an input image. Among several computational models that capture the visual-saliency in biological system, an information theoretic AIM(Attention based on Information Maximization) algorithm has been demonstrated to predict human gaze patterns better than other existing models. We present an FPGA based implementation of this computationally intensive AIM algorithm to support embedded vision applications. Our implementation provides performance of processing about 4M pixels/sec for 25 basis functions with a convolution kernel size of 21 by 21 for each of the R, G, and B color-channels, when implemented on a Virtex-6 LX240T. We also provide an optimization aimed at controlling the trade-off between power consumption and latency, and performance comparisons with a GPU implementation.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011
Pages41-48
Number of pages8
DOIs
StatePublished - Jun 17 2011
Event19th IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011 - Salt Lake City, UT, United States
Duration: May 1 2011May 3 2011

Publication series

NameProceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011

Other

Other19th IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011
CountryUnited States
CitySalt Lake City, UT
Period5/1/115/3/11

Fingerprint

Field programmable gate arrays (FPGA)
Biological systems
Processing
Convolution
Electric power utilization
Pixels
Color
Graphics processing unit

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

Cite this

Bae, S., Cho, Y. C. P., Park, S., Irick, K. M., Jin, Y., & Narayanan, V. (2011). An FPGA implementation of information theoretic visual-saliency system and its optimization. In Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011 (pp. 41-48). [5771246] (Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011). https://doi.org/10.1109/FCCM.2011.41
Bae, Sungmin ; Cho, Yong Cheol Peter ; Park, Sungho ; Irick, Kevin M. ; Jin, Yongseok ; Narayanan, Vijaykrishnan. / An FPGA implementation of information theoretic visual-saliency system and its optimization. Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011. 2011. pp. 41-48 (Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011).
@inproceedings{4a75aa96b06e42fcb4e4f0fc9dbd52bd,
title = "An FPGA implementation of information theoretic visual-saliency system and its optimization",
abstract = "Biological vision systems use saliency-based visual attention mechanisms to limit higher-level vision processing on the most visually-salient subsets of an input image. Among several computational models that capture the visual-saliency in biological system, an information theoretic AIM(Attention based on Information Maximization) algorithm has been demonstrated to predict human gaze patterns better than other existing models. We present an FPGA based implementation of this computationally intensive AIM algorithm to support embedded vision applications. Our implementation provides performance of processing about 4M pixels/sec for 25 basis functions with a convolution kernel size of 21 by 21 for each of the R, G, and B color-channels, when implemented on a Virtex-6 LX240T. We also provide an optimization aimed at controlling the trade-off between power consumption and latency, and performance comparisons with a GPU implementation.",
author = "Sungmin Bae and Cho, {Yong Cheol Peter} and Sungho Park and Irick, {Kevin M.} and Yongseok Jin and Vijaykrishnan Narayanan",
year = "2011",
month = "6",
day = "17",
doi = "10.1109/FCCM.2011.41",
language = "English (US)",
isbn = "9780769543017",
series = "Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011",
pages = "41--48",
booktitle = "Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011",

}

Bae, S, Cho, YCP, Park, S, Irick, KM, Jin, Y & Narayanan, V 2011, An FPGA implementation of information theoretic visual-saliency system and its optimization. in Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011., 5771246, Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011, pp. 41-48, 19th IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011, Salt Lake City, UT, United States, 5/1/11. https://doi.org/10.1109/FCCM.2011.41

An FPGA implementation of information theoretic visual-saliency system and its optimization. / Bae, Sungmin; Cho, Yong Cheol Peter; Park, Sungho; Irick, Kevin M.; Jin, Yongseok; Narayanan, Vijaykrishnan.

Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011. 2011. p. 41-48 5771246 (Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - An FPGA implementation of information theoretic visual-saliency system and its optimization

AU - Bae, Sungmin

AU - Cho, Yong Cheol Peter

AU - Park, Sungho

AU - Irick, Kevin M.

AU - Jin, Yongseok

AU - Narayanan, Vijaykrishnan

PY - 2011/6/17

Y1 - 2011/6/17

N2 - Biological vision systems use saliency-based visual attention mechanisms to limit higher-level vision processing on the most visually-salient subsets of an input image. Among several computational models that capture the visual-saliency in biological system, an information theoretic AIM(Attention based on Information Maximization) algorithm has been demonstrated to predict human gaze patterns better than other existing models. We present an FPGA based implementation of this computationally intensive AIM algorithm to support embedded vision applications. Our implementation provides performance of processing about 4M pixels/sec for 25 basis functions with a convolution kernel size of 21 by 21 for each of the R, G, and B color-channels, when implemented on a Virtex-6 LX240T. We also provide an optimization aimed at controlling the trade-off between power consumption and latency, and performance comparisons with a GPU implementation.

AB - Biological vision systems use saliency-based visual attention mechanisms to limit higher-level vision processing on the most visually-salient subsets of an input image. Among several computational models that capture the visual-saliency in biological system, an information theoretic AIM(Attention based on Information Maximization) algorithm has been demonstrated to predict human gaze patterns better than other existing models. We present an FPGA based implementation of this computationally intensive AIM algorithm to support embedded vision applications. Our implementation provides performance of processing about 4M pixels/sec for 25 basis functions with a convolution kernel size of 21 by 21 for each of the R, G, and B color-channels, when implemented on a Virtex-6 LX240T. We also provide an optimization aimed at controlling the trade-off between power consumption and latency, and performance comparisons with a GPU implementation.

UR - http://www.scopus.com/inward/record.url?scp=79958701449&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79958701449&partnerID=8YFLogxK

U2 - 10.1109/FCCM.2011.41

DO - 10.1109/FCCM.2011.41

M3 - Conference contribution

AN - SCOPUS:79958701449

SN - 9780769543017

T3 - Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011

SP - 41

EP - 48

BT - Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011

ER -

Bae S, Cho YCP, Park S, Irick KM, Jin Y, Narayanan V. An FPGA implementation of information theoretic visual-saliency system and its optimization. In Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011. 2011. p. 41-48. 5771246. (Proceedings - IEEE International Symposium on Field-Programmable Custom Computing Machines, FCCM 2011). https://doi.org/10.1109/FCCM.2011.41