PSB-RNN: A Processing-in-Memory Systolic Array Architecture using Block Circulant Matrices for Recurrent Neural Networks

Nagadastagiri Challapalle, Sahithi Rampalli, Makesh Chandran, Gurpreet Kalsi, Sreenivas Subramoney, John Sampson, Vijaykrishnan Narayanan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recurrent Neural Networks (RNNs) are widely used in Natural Language Processing (NLP) applications as they inherently capture contextual information across spatial and temporal dimensions. Compared to other classes of neural networks, RNNs have more weight parameters as they primarily consist of fully connected layers. Recently, several techniques such as weight pruning, zero-skipping, and block circulant compression have been introduced to reduce the storage and access requirements of RNN weight parameters. In this work, we present a ReRAM crossbar based processing-in-memory (PIM) architecture with systolic dataflow incorporating block circulant compression for RNNs. The block circulant compression decomposes the operations in a fully connected layer into a series of Fourier transforms and point-wise operations resulting in reduced space and computational complexity. We formulate the Fourier transform and point-wise operations into in-situ multiply-and-accumulate (MAC) operations mapped to ReRAM crossbars for high energy efficiency and throughput. We also incorporate systolic dataflow for communication within the crossbar arrays, in contrast to broadcast and multicast communications, to further improve energy efficiency. The proposed architecture achieves average improvements in compute efficiency of 44x and 17x over a custom FPGA architecture and conventional crossbar based architecture implementations, respectively.

Original languageEnglish (US)
Title of host publicationProceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
EditorsGiorgio Di Natale, Cristiana Bolchini, Elena-Ioana Vatajelu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages180-185
Number of pages6
ISBN (Electronic)9783981926347
DOIs
StatePublished - Mar 2020
Event2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020 - Grenoble, France
Duration: Mar 9 2020Mar 13 2020

Publication series

NameProceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020

Conference

Conference2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
CountryFrance
CityGrenoble
Period3/9/203/13/20

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Safety, Risk, Reliability and Quality
  • Modeling and Simulation
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'PSB-RNN: A Processing-in-Memory Systolic Array Architecture using Block Circulant Matrices for Recurrent Neural Networks'. Together they form a unique fingerprint.

Cite this