Adaptive Federated Learning in Resource Constrained Edge Computing Systems

Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, Kevin Chan

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Emerging technologies and applications including Internet of Things, social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent-based approaches. We analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions.

Original languageEnglish (US)
Article number8664630
Pages (from-to)1205-1221
Number of pages17
JournalIEEE Journal on Selected Areas in Communications
Volume37
Issue number6
DOIs
StatePublished - Jun 2019

Fingerprint

Learning systems
Agglomeration
Bandwidth
Experiments
Internet of things

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Cite this

Wang, Shiqiang ; Tuor, Tiffany ; Salonidis, Theodoros ; Leung, Kin K. ; Makaya, Christian ; He, Ting ; Chan, Kevin. / Adaptive Federated Learning in Resource Constrained Edge Computing Systems. In: IEEE Journal on Selected Areas in Communications. 2019 ; Vol. 37, No. 6. pp. 1205-1221.
@article{4b1e9be9311947c8858cd303ba88fad2,
title = "Adaptive Federated Learning in Resource Constrained Edge Computing Systems",
abstract = "Emerging technologies and applications including Internet of Things, social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent-based approaches. We analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions.",
author = "Shiqiang Wang and Tiffany Tuor and Theodoros Salonidis and Leung, {Kin K.} and Christian Makaya and Ting He and Kevin Chan",
year = "2019",
month = "6",
doi = "10.1109/JSAC.2019.2904348",
language = "English (US)",
volume = "37",
pages = "1205--1221",
journal = "IEEE Journal on Selected Areas in Communications",
issn = "0733-8716",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

Wang, S, Tuor, T, Salonidis, T, Leung, KK, Makaya, C, He, T & Chan, K 2019, 'Adaptive Federated Learning in Resource Constrained Edge Computing Systems', IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, 8664630, pp. 1205-1221. https://doi.org/10.1109/JSAC.2019.2904348

Adaptive Federated Learning in Resource Constrained Edge Computing Systems. / Wang, Shiqiang; Tuor, Tiffany; Salonidis, Theodoros; Leung, Kin K.; Makaya, Christian; He, Ting; Chan, Kevin.

In: IEEE Journal on Selected Areas in Communications, Vol. 37, No. 6, 8664630, 06.2019, p. 1205-1221.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Adaptive Federated Learning in Resource Constrained Edge Computing Systems

AU - Wang, Shiqiang

AU - Tuor, Tiffany

AU - Salonidis, Theodoros

AU - Leung, Kin K.

AU - Makaya, Christian

AU - He, Ting

AU - Chan, Kevin

PY - 2019/6

Y1 - 2019/6

N2 - Emerging technologies and applications including Internet of Things, social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent-based approaches. We analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions.

AB - Emerging technologies and applications including Internet of Things, social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent-based approaches. We analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions.

UR - http://www.scopus.com/inward/record.url?scp=85065907659&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065907659&partnerID=8YFLogxK

U2 - 10.1109/JSAC.2019.2904348

DO - 10.1109/JSAC.2019.2904348

M3 - Article

AN - SCOPUS:85065907659

VL - 37

SP - 1205

EP - 1221

JO - IEEE Journal on Selected Areas in Communications

JF - IEEE Journal on Selected Areas in Communications

SN - 0733-8716

IS - 6

M1 - 8664630

ER -