Addressing Unreliability in Emerging Devices and Non-von Neumann Architectures Using Coded Computing

Sanghamitra Dutta, Haewon Jeong, Yaoqing Yang, Viveck Cadambe, Tze Meng Low, Pulkit Grover

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Computing systems are evolving rapidly. At the device level, emerging devices are beginning to compete with traditional CMOS systems. At the architecture level, novel architectures are successfully avoiding the communication bottleneck that is a central feature, and a central limitation, of the von Neumann architecture. Furthermore, such systems are increasingly plagued by unreliability. This unreliability arises at device or gate-level in emerging devices, and can percolate up to processor or system-level if left unchecked. The goal of this article is to survey recent advances in reliable computing using unreliable elements, with an eye on nonsilicon and non-von Neumann architectures. We first observe that instead of aiming for generic computing problems, the community could use 'dwarfs of modern computing,' first noted in the high-performance computing (HPC) community, as a starting point. These computing problems are the basic building blocks of almost all scientific computing, machine learning, and data analytics today. Next, we survey the state of the art in 'coded computing,' which is an emerging area that advances on classical algorithm-based fault-tolerance (ABFT) and brings a fundamental information-theoretic perspective. By weaving error-correcting codes into a computing algorithm, coded computing provides dramatic improvements on solutions, as well as obtains novel fundamental limits, for problems that have been open for more than 30 years. We introduce existing and novel coded computing techniques in the context of 'coded dwarfs,' where a specific dwarf's computation is made resilient by applying coding. We discuss how, for the same redundancy, 'coded dwarfs' are significantly more resilient compared to classical techniques such as replication. Furthermore, by examining a widely popular computation task - training large neural networks - we demonstrate how coded dwarfs can be applied to address this fundamentally nonlinear problem. Finally, we discuss practical challenges and future directions in implementing coded computing techniques on emerging and existing nonsilicon and/or non-von Neumann architectures.

Original languageEnglish (US)
Article number9093912
Pages (from-to)1219-1234
Number of pages16
JournalProceedings of the IEEE
Volume108
Issue number8
DOIs
StatePublished - Aug 2020

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Addressing Unreliability in Emerging Devices and Non-von Neumann Architectures Using Coded Computing'. Together they form a unique fingerprint.

Cite this