Sensitivity-Based Error Resilient Techniques with Heterogeneous Multiply-Accumulate Unit for Voltage Scalable Deep Neural Network Accelerators

Dongyeob Shin, Wonseok Choi, Jongsun Park, Swaroop Ghosh

Research output: Contribution to journalArticle

1 Scopus citations

Abstract

With inherent algorithmic error resilience of deep neural networks (DNNs), supply voltage scaling could be a promising technique for energy efficient DNN accelerator design. In this paper, we present an error resilient technique to enable aggressive voltage scaling by exploiting the asymmetric error resilience (sensitivity) with respect to DNN layers, filters, and channels. First-order Taylor expansion is used to evaluate the filter/channel-level weight sensitivities of large scale DNNs which accurately approximates weight sensitivities from actual error injection simulations. We also present the heterogeneous multiply-accumulate (MAC) unit based design approach where some of the MAC units are designed larger with shorter critical path delays for robustness to aggressive voltage scaling while other MAC units are designed relatively smaller. The sensitivity variations among filter weights can be leveraged to design DNN accelerator such that the computations with more sensitive weights are assigned to more robust (larger) MAC units while the computations with less sensitive weights are assigned to less robust (smaller) MAC units. Using dynamic programming, the sizes of MAC units are selected to achieve best DNN accuracy under ISO area constraint. As a result, the proposed voltage scalable DNN accelerator can achieve 34% energy savings in post layout simulations using 65 nm CMOS process with ImageNet dataset using ResNet-18 compared to state-of-the-art timing error recovery technique.

Original languageEnglish (US)
Article number8792195
Pages (from-to)520-531
Number of pages12
JournalIEEE Journal on Emerging and Selected Topics in Circuits and Systems
Volume9
Issue number3
DOIs
StatePublished - Sep 2019

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Sensitivity-Based Error Resilient Techniques with Heterogeneous Multiply-Accumulate Unit for Voltage Scalable Deep Neural Network Accelerators'. Together they form a unique fingerprint.

  • Cite this