Scaling up discrete distribution clustering using ADMM

Jianbo Ye, Jia Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

The discrete distribution as a sparse representation, equipped with the Kantorovich-Wasserstein metric, has been proven effective in learning tasks on imagery data. However, clustering based on the Kantorovich metric under a principled optimization criterion is computationally challenging, and has not been adequately explored. In this paper, we focus on the scalability issue and develop a new algorithm for clustering distributions. An optimal centroid or representative distribution in the sense of the Kantorovich metric is solved for each cluster. The key idea is to adapt the state-of-the-art distributed optimization approach called alternating direction method of multipliers (ADMM). The new algorithm achieves linear complexity in the update of each centroid and can be easily parallelizable, improving significantly over the existing method. It is also observed that in practice, satisfactory results can be obtained after a few tens of iterations. We conduct experiments on both synthetic and real data to demonstrate the computational efficiency and accuracy of the new algorithm.

Original languageEnglish (US)
Title of host publication2014 IEEE International Conference on Image Processing, ICIP 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages5267-5271
Number of pages5
ISBN (Electronic)9781479957514
DOIs
StatePublished - Jan 28 2014

Publication series

Name2014 IEEE International Conference on Image Processing, ICIP 2014

Fingerprint

Computational efficiency
Scalability
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition

Cite this

Ye, J., & Li, J. (2014). Scaling up discrete distribution clustering using ADMM. In 2014 IEEE International Conference on Image Processing, ICIP 2014 (pp. 5267-5271). [7026066] (2014 IEEE International Conference on Image Processing, ICIP 2014). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICIP.2014.7026066
Ye, Jianbo ; Li, Jia. / Scaling up discrete distribution clustering using ADMM. 2014 IEEE International Conference on Image Processing, ICIP 2014. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 5267-5271 (2014 IEEE International Conference on Image Processing, ICIP 2014).
@inproceedings{87ed0269256b42caafc3767b62e224aa,
title = "Scaling up discrete distribution clustering using ADMM",
abstract = "The discrete distribution as a sparse representation, equipped with the Kantorovich-Wasserstein metric, has been proven effective in learning tasks on imagery data. However, clustering based on the Kantorovich metric under a principled optimization criterion is computationally challenging, and has not been adequately explored. In this paper, we focus on the scalability issue and develop a new algorithm for clustering distributions. An optimal centroid or representative distribution in the sense of the Kantorovich metric is solved for each cluster. The key idea is to adapt the state-of-the-art distributed optimization approach called alternating direction method of multipliers (ADMM). The new algorithm achieves linear complexity in the update of each centroid and can be easily parallelizable, improving significantly over the existing method. It is also observed that in practice, satisfactory results can be obtained after a few tens of iterations. We conduct experiments on both synthetic and real data to demonstrate the computational efficiency and accuracy of the new algorithm.",
author = "Jianbo Ye and Jia Li",
year = "2014",
month = "1",
day = "28",
doi = "10.1109/ICIP.2014.7026066",
language = "English (US)",
series = "2014 IEEE International Conference on Image Processing, ICIP 2014",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "5267--5271",
booktitle = "2014 IEEE International Conference on Image Processing, ICIP 2014",
address = "United States",

}

Ye, J & Li, J 2014, Scaling up discrete distribution clustering using ADMM. in 2014 IEEE International Conference on Image Processing, ICIP 2014., 7026066, 2014 IEEE International Conference on Image Processing, ICIP 2014, Institute of Electrical and Electronics Engineers Inc., pp. 5267-5271. https://doi.org/10.1109/ICIP.2014.7026066

Scaling up discrete distribution clustering using ADMM. / Ye, Jianbo; Li, Jia.

2014 IEEE International Conference on Image Processing, ICIP 2014. Institute of Electrical and Electronics Engineers Inc., 2014. p. 5267-5271 7026066 (2014 IEEE International Conference on Image Processing, ICIP 2014).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Scaling up discrete distribution clustering using ADMM

AU - Ye, Jianbo

AU - Li, Jia

PY - 2014/1/28

Y1 - 2014/1/28

N2 - The discrete distribution as a sparse representation, equipped with the Kantorovich-Wasserstein metric, has been proven effective in learning tasks on imagery data. However, clustering based on the Kantorovich metric under a principled optimization criterion is computationally challenging, and has not been adequately explored. In this paper, we focus on the scalability issue and develop a new algorithm for clustering distributions. An optimal centroid or representative distribution in the sense of the Kantorovich metric is solved for each cluster. The key idea is to adapt the state-of-the-art distributed optimization approach called alternating direction method of multipliers (ADMM). The new algorithm achieves linear complexity in the update of each centroid and can be easily parallelizable, improving significantly over the existing method. It is also observed that in practice, satisfactory results can be obtained after a few tens of iterations. We conduct experiments on both synthetic and real data to demonstrate the computational efficiency and accuracy of the new algorithm.

AB - The discrete distribution as a sparse representation, equipped with the Kantorovich-Wasserstein metric, has been proven effective in learning tasks on imagery data. However, clustering based on the Kantorovich metric under a principled optimization criterion is computationally challenging, and has not been adequately explored. In this paper, we focus on the scalability issue and develop a new algorithm for clustering distributions. An optimal centroid or representative distribution in the sense of the Kantorovich metric is solved for each cluster. The key idea is to adapt the state-of-the-art distributed optimization approach called alternating direction method of multipliers (ADMM). The new algorithm achieves linear complexity in the update of each centroid and can be easily parallelizable, improving significantly over the existing method. It is also observed that in practice, satisfactory results can be obtained after a few tens of iterations. We conduct experiments on both synthetic and real data to demonstrate the computational efficiency and accuracy of the new algorithm.

UR - http://www.scopus.com/inward/record.url?scp=84949927003&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84949927003&partnerID=8YFLogxK

U2 - 10.1109/ICIP.2014.7026066

DO - 10.1109/ICIP.2014.7026066

M3 - Conference contribution

AN - SCOPUS:84949927003

T3 - 2014 IEEE International Conference on Image Processing, ICIP 2014

SP - 5267

EP - 5271

BT - 2014 IEEE International Conference on Image Processing, ICIP 2014

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Ye J, Li J. Scaling up discrete distribution clustering using ADMM. In 2014 IEEE International Conference on Image Processing, ICIP 2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 5267-5271. 7026066. (2014 IEEE International Conference on Image Processing, ICIP 2014). https://doi.org/10.1109/ICIP.2014.7026066