RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars

Shubham Jain, Abhronil Sengupta, Kaushik Roy, Anand Raghunathan

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Resistive crossbars designed with nonvolatile memory devices have emerged as promising building blocks for deep neural network (DNN) hardware, due to their ability to compactly and efficiently realize vector-matrix multiplication (VMM), the dominant computational kernel in DNNs. However, a key challenge with resistive crossbars is that they suffer from a range of device and circuit level nonidealities, such as driver resistance, sensing resistance, sneak paths, interconnect parasitics, nonlinearities in the peripheral circuits, stochastic write operations, and process variations. These nonidealities can lead to errors in VMMs, eventually degrading the DNN's accuracy. It is therefore critical to study the impact of crossbar nonidealities on the accuracy of large-scale DNNs (with millions of neurons and billions of synaptic connections). However, this is challenging because the existing device and circuit models are too slow to use in application-level evaluations. We present RxNN, a fast and accurate simulation framework to evaluate large-scale DNNs on resistive crossbar systems. RxNN splits and maps the computations involved in each DNN layer into crossbar operations, and evaluates them using a fast crossbar model (FCM) that accurately captures the errors arising due to crossbar nonidealities while being four-to-five orders of magnitude faster than circuit simulation. FCM models a crossbar-based VMM operation using three stages - nonlinear models for the input and output peripheral circuits (digital-to-analog and analog-to-digital converters), and an equivalent nonideal conductance matrix for the core crossbar array. We implement RxNN by extending the Caffe machine learning framework and use it to evaluate a suite of six large-scale DNNs developed for the ImageNet Challenge (ILSVRC). Our experiments reveal that resistive crossbar nonidealities can lead to significant accuracy degradations (9.6%-32%) for these large-scale DNNs. To the best of our knowledge, this article is the first quantitative evaluation of the accuracy of large-scale DNNs on resistive crossbar-based hardware. We also demonstrate that RxNN enables fast model-in-the-loop retraining of DNNs to partially mitigate the accuracy degradation.

Original languageEnglish (US)
Article number9108292
Pages (from-to)326-338
Number of pages13
JournalIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume40
Issue number2
DOIs
StatePublished - Feb 2021

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars'. Together they form a unique fingerprint.

Cite this