Regularization parameter selections via generalized information criterion

Yiyun Zhang, Runze Li, Chih Ling Tsai

Research output: Contribution to journalArticlepeer-review

146 Scopus citations

Abstract

We apply the nonconcave penalized likelihood approach to obtain variable selections as well as shrinkage estimators. This approach relies heavily on the choice of regularization parameter, which controls the model complexity. In this paper, we propose employing the generalized information criterion, encompassing the commonly used Akaike information criterion (AIC) and Bayesian information criterion (BIC), for selecting the regularization parameter. Our proposal makes a connection between the classical variable selection criteria and the regularization parameter selections for the nonconcave penalized likelihood approaches. We show that the BIC-type selector enables identification of the true model consistently, and the resulting estimator possesses the oracle property in the terminology of Fan and Li (2001). In contrast, however, the AIC-type selector tends to overfit with positive probability. We further show that the AIC-type selector is asymptotically loss efficient, while the BIC-type selector is not. Our simulation results confirm these theoretical findings, and an empirical example is presented. Some technical proofs are given in the online supplementary material.

Original languageEnglish (US)
Pages (from-to)312-323
Number of pages12
JournalJournal of the American Statistical Association
Volume105
Issue number489
DOIs
StatePublished - Mar 2010

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Regularization parameter selections via generalized information criterion'. Together they form a unique fingerprint.

Cite this