TY - JOUR
T1 - Distributed Big-Data Optimization via Blockwise Gradient Tracking
AU - Notarnicola, Ivano
AU - Sun, Ying
AU - Scutari, Gesualdo
AU - Notarstefano, Giuseppe
N1 - Funding Information:
Manuscript received August 7, 2019; revised February 6, 2020; accepted June 14, 2020. Date of publication July 13, 2020; date of current version April 26, 2021. The work of Ivano Notarnicola and Giuseppe No-tarstefano was supported by the European Research Council under the European Union’s Horizon 2020 Research and Innovation Programme under Grant 638992-OPT4SMART. The work of Ying Sun and Gesualdo Scutari was supported in part by the USA National Science Foundation under Grant CIF 1564044, Grant CIF 1719205, Grant CMMI 1832688, and in part by the Army Research Office under Grant W911NF1810238. This paper was presented in part at the IEEE 56th Annual Conference on Decision and Control, Melbourne, VIC, Australia, December 2017 and in part at the IEEE 7th International Workshop on Computational Advances in Multisensor Adaptive Processing, Curacao, December 2017. Recommended by Associate Editor F. Wirth. (Ivano Notarnicola and Ying Sun contributed equally to this work.) (Corresponding author: Ivano Notarnicola.) Ivano Notarnicola and Giuseppe Notarstefano are with the Department of Electrical, Electronic and Information Engineering, University of Bologna, 40126 Bologna, Italy (e-mail: ivano.notarnicola@unibo.it; giuseppe.notarstefano@unibo.it).
Publisher Copyright:
© 1963-2012 IEEE.
PY - 2021/5
Y1 - 2021/5
N2 - We study distributed big-data nonconvex optimization in multiagent networks. We consider the (constrained) minimization of the sum of a smooth (possibly) nonconvex function, i.e., the agents' sum-utility, plus a convex (possibly) nonsmooth regularizer. Our interest is on big-data problems in which there is a large number of variables to optimize. If treated by means of standard distributed optimization algorithms, these large-scale problems may be intractable due to the prohibitive local computation and communication burden at each node. We propose a novel distributed solution method where, at each iteration, agents update in an uncoordinated fashion only one block of the entire decision vector. To deal with the nonconvexity of the cost function, the novel scheme hinges on successive convex approximation techniques combined with a novel blockwise perturbed push-sum consensus protocol, which is instrumental to perform local block-averaging operations and tracking of gradient averages. Asymptotic convergence to stationary solutions of the nonconvex problem is established. Finally, numerical results show the effectiveness of the proposed algorithm and highlight how the block dimension impacts on the communication overhead and practical convergence speed.
AB - We study distributed big-data nonconvex optimization in multiagent networks. We consider the (constrained) minimization of the sum of a smooth (possibly) nonconvex function, i.e., the agents' sum-utility, plus a convex (possibly) nonsmooth regularizer. Our interest is on big-data problems in which there is a large number of variables to optimize. If treated by means of standard distributed optimization algorithms, these large-scale problems may be intractable due to the prohibitive local computation and communication burden at each node. We propose a novel distributed solution method where, at each iteration, agents update in an uncoordinated fashion only one block of the entire decision vector. To deal with the nonconvexity of the cost function, the novel scheme hinges on successive convex approximation techniques combined with a novel blockwise perturbed push-sum consensus protocol, which is instrumental to perform local block-averaging operations and tracking of gradient averages. Asymptotic convergence to stationary solutions of the nonconvex problem is established. Finally, numerical results show the effectiveness of the proposed algorithm and highlight how the block dimension impacts on the communication overhead and practical convergence speed.
UR - http://www.scopus.com/inward/record.url?scp=85104895582&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85104895582&partnerID=8YFLogxK
U2 - 10.1109/TAC.2020.3008713
DO - 10.1109/TAC.2020.3008713
M3 - Article
AN - SCOPUS:85104895582
VL - 66
SP - 2045
EP - 2060
JO - IRE Transactions on Automatic Control
JF - IRE Transactions on Automatic Control
SN - 0018-9286
IS - 5
M1 - 9139372
ER -