DoubtNet: Using semantic context to enable adaptive inference for the IoT

Eric Homan, Chonghan Lee, Jack Sampson, John Sustersic, Vijaykrishnan Narayanan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While techniques such as model pruning and precision reduction have improved the efficiency of Deep Neural Network (DNN) inference, DNN-based object detection still involves substantial computation, thereby limiting the scope of DNN inference on compute-limited platforms, such as many IoT devices. Proposed methods for differentiating easy versus difficult classification cases, such as early exit, can reduce the average amount of compute incurred but are limited by the accuracy to which they can correctly predict the level of effort needed for classification based only upon early stage features. In this work, we propose an alternative approach to predicting necessary effort that leverages semantic context. Namely, in any image there will usually be several objects present that can provide context for the plausibility of the set of classifications as a whole, with greater effort being applied only to outliers. Rather than relying on co-location of objects within training data, we derive our plausibility model from WordNet, a large pre-existing database of semantic relationships among objects, allowing classifier training to be performed in a traditional, relationship-agnostic fashion. We demonstrate the effectiveness of our approach, DoubtNet, using MobileNet as the initial low-power classifier, ResNet as the high-powered classifier, and developing an outlier detection module that is independent of both networks. DoubtNet increases the mAP of our base classifier by as much as 22% and consistently outperforms prior, training-set-based context models.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE Computer Society Annual Symposium on VLSI, ISVLSI 2020
PublisherIEEE Computer Society
Pages586-591
Number of pages6
ISBN (Electronic)9781728157757
DOIs
StatePublished - Jul 2020
Event19th IEEE Computer Society Annual Symposium on VLSI, ISVLSI 2020 - Limassol, Cyprus
Duration: Jul 6 2020Jul 8 2020

Publication series

NameProceedings of IEEE Computer Society Annual Symposium on VLSI, ISVLSI
Volume2020-July
ISSN (Print)2159-3469
ISSN (Electronic)2159-3477

Conference

Conference19th IEEE Computer Society Annual Symposium on VLSI, ISVLSI 2020
CountryCyprus
CityLimassol
Period7/6/207/8/20

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'DoubtNet: Using semantic context to enable adaptive inference for the IoT'. Together they form a unique fingerprint.

Cite this