TY - JOUR
T1 - Convergence de classifieurs par réseaux de neurones formellement divergents
AU - Berlyand, Leonid
AU - Jabin, Pierre Emmanuel
N1 - Funding Information:
LB was partially supported by NSF DMREF grant DMS-1628411 ; PEJ was partially supported by NSF Grant DMS-161453 , NSF Grant RNMS (Ki-Net) DMS-1107444 and by LTS grants DO 0048-0049-0050 and 0052 .
Funding Information:
LB was partially supported by NSF DMREF grant DMS-1628411; PEJ was partially supported by NSF Grant DMS-161453, NSF Grant RNMS (Ki-Net) DMS-1107444 and by LTS grants DO 0048-0049-0050 and 0052.
Publisher Copyright:
© 2018 Académie des sciences
PY - 2018/4
Y1 - 2018/4
N2 - We present an analytical study of gradient descent algorithms applied to a classification problem in machine learning based on artificial neural networks. Our approach is based on entropy–entropy dissipation estimates that yield explicit rates. Specifically, as long as the neural nets remain within a set of “good classifiers” we establish a striking feature of the algorithm: it mathematically diverges as the number of gradient descent iterations (“time”) goes to infinity but this divergence is only logarithmic, while the loss function vanishes polynomially. As a consequence, this algorithm still yields a classifier that exhibits good numerical performance and may even appear to converge.
AB - We present an analytical study of gradient descent algorithms applied to a classification problem in machine learning based on artificial neural networks. Our approach is based on entropy–entropy dissipation estimates that yield explicit rates. Specifically, as long as the neural nets remain within a set of “good classifiers” we establish a striking feature of the algorithm: it mathematically diverges as the number of gradient descent iterations (“time”) goes to infinity but this divergence is only logarithmic, while the loss function vanishes polynomially. As a consequence, this algorithm still yields a classifier that exhibits good numerical performance and may even appear to converge.
UR - http://www.scopus.com/inward/record.url?scp=85043773021&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85043773021&partnerID=8YFLogxK
U2 - 10.1016/j.crma.2018.03.003
DO - 10.1016/j.crma.2018.03.003
M3 - Article
AN - SCOPUS:85043773021
VL - 356
SP - 395
EP - 405
JO - Comptes Rendus Mathematique
JF - Comptes Rendus Mathematique
SN - 1631-073X
IS - 4
ER -