Codes for Distributed Finite Alphabet Matrix-Vector Multiplication

Farzin Haddadpour, Viveck Ramesh Cadambe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Recent work has developed coding theoretic approaches to add redundancy to distributed matrix-vector multiplications with the goal of speeding up the computation by mitigating the straggler effect in distributed computing. In this paper, we consider the case where the matrix comes from a small (e.g., binary) alphabet, where a variant of a popular method called the 'Four-Russians method' is known to have significantly lower computational complexity as compared with the usual matrix-vector multiplication algorithm. We develop novel code constructions that are applicable to binary matrix-vector multiplication via a variant of the Four-Russians method called the Mailman algorithm. Specifically, in our constructions, the encoded matrices have a low alphabet that ensures lower computational complexity, as well as good straggler tolerance. We also present a trade-off between the communication and computation cost of distributed coded matrix-vector multiplication for general, possibly non-binary, matrices.

Original languageEnglish (US)
Title of host publication2018 IEEE International Symposium on Information Theory, ISIT 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1625-1629
Number of pages5
ISBN (Print)9781538647806
DOIs
StatePublished - Aug 15 2018
Event2018 IEEE International Symposium on Information Theory, ISIT 2018 - Vail, United States
Duration: Jun 17 2018Jun 22 2018

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2018-June
ISSN (Print)2157-8095

Other

Other2018 IEEE International Symposium on Information Theory, ISIT 2018
CountryUnited States
CityVail
Period6/17/186/22/18

Fingerprint

Matrix-vector multiplication
Low Complexity
Computational Complexity
Binary
Distributed Computing
Computational complexity
Redundancy
Tolerance
Coding
Trade-offs
Distributed computer systems
Costs
Communication

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Cite this

Haddadpour, F., & Cadambe, V. R. (2018). Codes for Distributed Finite Alphabet Matrix-Vector Multiplication. In 2018 IEEE International Symposium on Information Theory, ISIT 2018 (pp. 1625-1629). [8437542] (IEEE International Symposium on Information Theory - Proceedings; Vol. 2018-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2018.8437542
Haddadpour, Farzin ; Cadambe, Viveck Ramesh. / Codes for Distributed Finite Alphabet Matrix-Vector Multiplication. 2018 IEEE International Symposium on Information Theory, ISIT 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1625-1629 (IEEE International Symposium on Information Theory - Proceedings).
@inproceedings{0ceec6eb62c04703b4724c79b27bc9f3,
title = "Codes for Distributed Finite Alphabet Matrix-Vector Multiplication",
abstract = "Recent work has developed coding theoretic approaches to add redundancy to distributed matrix-vector multiplications with the goal of speeding up the computation by mitigating the straggler effect in distributed computing. In this paper, we consider the case where the matrix comes from a small (e.g., binary) alphabet, where a variant of a popular method called the 'Four-Russians method' is known to have significantly lower computational complexity as compared with the usual matrix-vector multiplication algorithm. We develop novel code constructions that are applicable to binary matrix-vector multiplication via a variant of the Four-Russians method called the Mailman algorithm. Specifically, in our constructions, the encoded matrices have a low alphabet that ensures lower computational complexity, as well as good straggler tolerance. We also present a trade-off between the communication and computation cost of distributed coded matrix-vector multiplication for general, possibly non-binary, matrices.",
author = "Farzin Haddadpour and Cadambe, {Viveck Ramesh}",
year = "2018",
month = "8",
day = "15",
doi = "10.1109/ISIT.2018.8437542",
language = "English (US)",
isbn = "9781538647806",
series = "IEEE International Symposium on Information Theory - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1625--1629",
booktitle = "2018 IEEE International Symposium on Information Theory, ISIT 2018",
address = "United States",

}

Haddadpour, F & Cadambe, VR 2018, Codes for Distributed Finite Alphabet Matrix-Vector Multiplication. in 2018 IEEE International Symposium on Information Theory, ISIT 2018., 8437542, IEEE International Symposium on Information Theory - Proceedings, vol. 2018-June, Institute of Electrical and Electronics Engineers Inc., pp. 1625-1629, 2018 IEEE International Symposium on Information Theory, ISIT 2018, Vail, United States, 6/17/18. https://doi.org/10.1109/ISIT.2018.8437542

Codes for Distributed Finite Alphabet Matrix-Vector Multiplication. / Haddadpour, Farzin; Cadambe, Viveck Ramesh.

2018 IEEE International Symposium on Information Theory, ISIT 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1625-1629 8437542 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2018-June).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Codes for Distributed Finite Alphabet Matrix-Vector Multiplication

AU - Haddadpour, Farzin

AU - Cadambe, Viveck Ramesh

PY - 2018/8/15

Y1 - 2018/8/15

N2 - Recent work has developed coding theoretic approaches to add redundancy to distributed matrix-vector multiplications with the goal of speeding up the computation by mitigating the straggler effect in distributed computing. In this paper, we consider the case where the matrix comes from a small (e.g., binary) alphabet, where a variant of a popular method called the 'Four-Russians method' is known to have significantly lower computational complexity as compared with the usual matrix-vector multiplication algorithm. We develop novel code constructions that are applicable to binary matrix-vector multiplication via a variant of the Four-Russians method called the Mailman algorithm. Specifically, in our constructions, the encoded matrices have a low alphabet that ensures lower computational complexity, as well as good straggler tolerance. We also present a trade-off between the communication and computation cost of distributed coded matrix-vector multiplication for general, possibly non-binary, matrices.

AB - Recent work has developed coding theoretic approaches to add redundancy to distributed matrix-vector multiplications with the goal of speeding up the computation by mitigating the straggler effect in distributed computing. In this paper, we consider the case where the matrix comes from a small (e.g., binary) alphabet, where a variant of a popular method called the 'Four-Russians method' is known to have significantly lower computational complexity as compared with the usual matrix-vector multiplication algorithm. We develop novel code constructions that are applicable to binary matrix-vector multiplication via a variant of the Four-Russians method called the Mailman algorithm. Specifically, in our constructions, the encoded matrices have a low alphabet that ensures lower computational complexity, as well as good straggler tolerance. We also present a trade-off between the communication and computation cost of distributed coded matrix-vector multiplication for general, possibly non-binary, matrices.

UR - http://www.scopus.com/inward/record.url?scp=85052482857&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052482857&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2018.8437542

DO - 10.1109/ISIT.2018.8437542

M3 - Conference contribution

AN - SCOPUS:85052482857

SN - 9781538647806

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 1625

EP - 1629

BT - 2018 IEEE International Symposium on Information Theory, ISIT 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Haddadpour F, Cadambe VR. Codes for Distributed Finite Alphabet Matrix-Vector Multiplication. In 2018 IEEE International Symposium on Information Theory, ISIT 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1625-1629. 8437542. (IEEE International Symposium on Information Theory - Proceedings). https://doi.org/10.1109/ISIT.2018.8437542