### Abstract

We propose a distributed first-order augmented Lagrangian (DFAL) algorithm to minimize the sum of composite convex functions, where each term in the sum is a private cost function belonging to a node, and only nodes connected by an edge can directly communicate with each other. This optimization model abstracts a number of applications in distributed sensing and machine learning. We show that any limit point of DFAL iterates is optimal; and for any ∈ > 0, an ∈-optimal and e-feasible solution can be computed within O(log(∈^{-1})) DFAL iterations, which require O(ψ^{1.5}_{max}/d_{m} ∈^{-1}) proximal gradient computations and communications per node in total, where ψ_{max} denotes the largest eigenvalue of the graph Laplacian, and d_{min} is the minimum degree of the graph. We also propose an asynchronous version of DFAL by incorporating randomized block coordinate descent methods; and demonstrate the efficiency of DFAL on large scale sparse-group LASSO problems.

Original language | English (US) |
---|---|

Title of host publication | 32nd International Conference on Machine Learning, ICML 2015 |

Editors | Francis Bach, David Blei |

Publisher | International Machine Learning Society (IMLS) |

Pages | 2444-2452 |

Number of pages | 9 |

Volume | 3 |

ISBN (Electronic) | 9781510810587 |

State | Published - Jan 1 2015 |

Event | 32nd International Conference on Machine Learning, ICML 2015 - Lile, France Duration: Jul 6 2015 → Jul 11 2015 |

### Other

Other | 32nd International Conference on Machine Learning, ICML 2015 |
---|---|

Country | France |

City | Lile |

Period | 7/6/15 → 7/11/15 |

### Fingerprint

### All Science Journal Classification (ASJC) codes

- Human-Computer Interaction
- Computer Science Applications

### Cite this

*32nd International Conference on Machine Learning, ICML 2015*(Vol. 3, pp. 2444-2452). International Machine Learning Society (IMLS).

}

*32nd International Conference on Machine Learning, ICML 2015.*vol. 3, International Machine Learning Society (IMLS), pp. 2444-2452, 32nd International Conference on Machine Learning, ICML 2015, Lile, France, 7/6/15.

**An asynchronous distributed proximal gradient method for composite convex optimization.** / Aybat, Necdet S.; Wang, Z.; Iyengar, G.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - An asynchronous distributed proximal gradient method for composite convex optimization

AU - Aybat, Necdet S.

AU - Wang, Z.

AU - Iyengar, G.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - We propose a distributed first-order augmented Lagrangian (DFAL) algorithm to minimize the sum of composite convex functions, where each term in the sum is a private cost function belonging to a node, and only nodes connected by an edge can directly communicate with each other. This optimization model abstracts a number of applications in distributed sensing and machine learning. We show that any limit point of DFAL iterates is optimal; and for any ∈ > 0, an ∈-optimal and e-feasible solution can be computed within O(log(∈-1)) DFAL iterations, which require O(ψ1.5max/dm ∈-1) proximal gradient computations and communications per node in total, where ψmax denotes the largest eigenvalue of the graph Laplacian, and dmin is the minimum degree of the graph. We also propose an asynchronous version of DFAL by incorporating randomized block coordinate descent methods; and demonstrate the efficiency of DFAL on large scale sparse-group LASSO problems.

AB - We propose a distributed first-order augmented Lagrangian (DFAL) algorithm to minimize the sum of composite convex functions, where each term in the sum is a private cost function belonging to a node, and only nodes connected by an edge can directly communicate with each other. This optimization model abstracts a number of applications in distributed sensing and machine learning. We show that any limit point of DFAL iterates is optimal; and for any ∈ > 0, an ∈-optimal and e-feasible solution can be computed within O(log(∈-1)) DFAL iterations, which require O(ψ1.5max/dm ∈-1) proximal gradient computations and communications per node in total, where ψmax denotes the largest eigenvalue of the graph Laplacian, and dmin is the minimum degree of the graph. We also propose an asynchronous version of DFAL by incorporating randomized block coordinate descent methods; and demonstrate the efficiency of DFAL on large scale sparse-group LASSO problems.

UR - http://www.scopus.com/inward/record.url?scp=84970028395&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84970028395&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84970028395

VL - 3

SP - 2444

EP - 2452

BT - 32nd International Conference on Machine Learning, ICML 2015

A2 - Bach, Francis

A2 - Blei, David

PB - International Machine Learning Society (IMLS)

ER -