Adaptive lightweight regularization tool for complex analytics

Zhaojing Luo, Shaofeng Cai, Jinyang Gao, Meihui Zhang, Kee Yuan Ngiam, Gang Chen, Wang Chien Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

Deep Learning and Machine Learning models have recently been shown to be effective in many real world applications. While these models achieve increasingly better predictive performance, their structures have also become much more complex. A common and difficult problem for complex models is overfitting. Regularization is used to penalize the complexity of the model in order to avoid overfitting. However, in most learning frameworks, regularization function is usually set as some hyper parameters, and therefore the best setting is difficult to find. In this paper, we propose an adaptive regularization method, as part of a large end-To-end healthcare data analytics software stack, which effectively addresses the above difficulty. First, we propose a general adaptive regularization method based on Gaussian Mixture (GM) to learn the best regularization function according to the observed parameters. Second, we develop an effective update algorithm which integrates Expectation Maximization (EM) with Stochastic Gradient Descent (SGD). Third, we design a lazy update algorithm to reduce the computational cost by 4x. The overall regularization framework is fast, adaptive and easy-To-use. We validate the effectiveness of our regularization method through an extensive experimental study over 13 standard benchmark datasets and three kinds of deep learning/machine learning models. The results illustrate that our proposed adaptive regularization method achieves significant improvement over state-of-The-Art regularization methods.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages485-496
Number of pages12
ISBN (Electronic)9781538655207
DOIs
StatePublished - Oct 24 2018
Event34th IEEE International Conference on Data Engineering, ICDE 2018 - Paris, France
Duration: Apr 16 2018Apr 19 2018

Publication series

NameProceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

Other

Other34th IEEE International Conference on Data Engineering, ICDE 2018
CountryFrance
CityParis
Period4/16/184/19/18

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Information Systems and Management
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Adaptive lightweight regularization tool for complex analytics'. Together they form a unique fingerprint.

Cite this