Heuristic Approximation of Early-Stage CNN Data Representation for Vision Intelligence Systems

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Massive memory requirements and disruptive I/O traffic in Convolutional Neural Networks (CNNs) restrict hardware/software optimization for training and inference tasks using larger networks despite their success in improving the intelligence of systems into which they are integrated. Interestingly, it has been observed that the early CNN layers in vision tasks consistently produce edge-like feature representations that can be mimicked by traditional vision algorithms. In this work, we demonstrate how to exploit this to create common edge-like features for sharing among multiple CNNs, and how to heuristically approximate them in a reduced dimensionality. In our proposed approximation, the feature space of three representative CNNs decreases by 1.6×-5.1×, and the size of training dataset is halved. As a result, we enhance both inference throughput and training speed by 2×, while providing accuracies that are still close to the original versions. We anticipate that this approach will lead to new opportunities in distributed intelligence systems, and the technique for redesigning CNN models based on dimensional reduction of feature space is orthogonal to and compatible with many other existing hardware/software optimizations.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE 36th International Conference on Computer Design, ICCD 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages218-225
Number of pages8
ISBN (Electronic)9781538684771
DOIs
StatePublished - Jan 16 2019
Event36th International Conference on Computer Design, ICCD 2018 - Orlando, United States
Duration: Oct 7 2018Oct 10 2018

Publication series

NameProceedings - 2018 IEEE 36th International Conference on Computer Design, ICCD 2018

Conference

Conference36th International Conference on Computer Design, ICCD 2018
CountryUnited States
CityOrlando
Period10/7/1810/10/18

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Hardware and Architecture
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'Heuristic Approximation of Early-Stage CNN Data Representation for Vision Intelligence Systems'. Together they form a unique fingerprint.

Cite this