Joint Coreset Construction and Quantization for Distributed Machine Learning

Hanlin Lu, Changchang Liu, Shiqiang Wang, Ting He, Vijaykrishnan Narayanan, Kevin S. Chan, Stephen Pasteris

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Coresets are small, weighted summaries of larger datasets, aiming at providing provable error bounds for machine learning (ML) tasks while significantly reducing the communication and computation costs. To achieve a better trade-off between ML error bounds and costs, we propose the first framework to incorporate quantization techniques into the process of coreset construction. Specifically, we theoretically analyze the ML error bounds caused by a combination of coreset construction and quantization. Based on that, we formulate an optimization problem to minimize the ML error under a fixed budget of communication cost. To improve the scalability for large datasets, we identify two proxies of the original objective function, for which efficient algorithms are developed. For the case of data on multiple nodes, we further design a novel algorithm to allocate the communication budget to the nodes while minimizing the overall ML error. Through extensive experiments on multiple real-world datasets, we demonstrate the effectiveness and efficiency of our proposed algorithms for a variety of ML tasks. In particular, our algorithms have achieved more than 90% data reduction with less than 10% degradation in ML performance in most cases.

Original languageEnglish (US)
Title of host publicationIFIP Networking 2020 Conference and Workshops, Networking 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages172-180
Number of pages9
ISBN (Electronic)9783903176287
StatePublished - Jun 2020
Event2020 IFIP Networking Conference and Workshops, Networking 2020 - Paris, France
Duration: Jun 22 2020Jun 25 2020

Publication series

NameIFIP Networking 2020 Conference and Workshops, Networking 2020

Conference

Conference2020 IFIP Networking Conference and Workshops, Networking 2020
CountryFrance
CityParis
Period6/22/206/25/20

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'Joint Coreset Construction and Quantization for Distributed Machine Learning'. Together they form a unique fingerprint.

Cite this