MgNet: A unified framework of multigrid and convolutional neural network

Juncai He, Jinchao Xu

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

We develop a unified model, known as MgNet, that simultaneously recovers some convolutional neural networks (CNN) for image classification and multigrid (MG) methods for solving discretized partial differential equations (PDEs). This model is based on close connections that we have observed and uncovered between the CNN and MG methodologies. For example, pooling operation and feature extraction in CNN correspond directly to restriction operation and iterative smoothers in MG, respectively. As the solution space is often the dual of the data space in PDEs, the analogous concept of feature space and data space (which are dual to each other) is introduced in CNN. With such connections and new concept in the unified model, the function of various convolution operations and pooling used in CNN can be better understood. As a result, modified CNN models (with fewer weights and hyperparameters) are developed that exhibit competitive and sometimes better performance in comparison with existing CNN models when applied to both CIFAR-10 and CIFAR-100 data sets.

Original languageEnglish (US)
Pages (from-to)1331-1354
Number of pages24
JournalScience China Mathematics
Volume62
Issue number7
DOIs
StatePublished - Jul 1 2019

Fingerprint

Neural Networks
Pooling
Neural Network Model
Partial differential equation
Hyperparameters
Image Classification
Multigrid Method
Feature Space
Feature Extraction
Convolution
Framework
Model
Restriction
Methodology
Concepts

All Science Journal Classification (ASJC) codes

  • Mathematics(all)

Cite this

@article{da3d224adb5d46769b1053e5f2cb90e3,
title = "MgNet: A unified framework of multigrid and convolutional neural network",
abstract = "We develop a unified model, known as MgNet, that simultaneously recovers some convolutional neural networks (CNN) for image classification and multigrid (MG) methods for solving discretized partial differential equations (PDEs). This model is based on close connections that we have observed and uncovered between the CNN and MG methodologies. For example, pooling operation and feature extraction in CNN correspond directly to restriction operation and iterative smoothers in MG, respectively. As the solution space is often the dual of the data space in PDEs, the analogous concept of feature space and data space (which are dual to each other) is introduced in CNN. With such connections and new concept in the unified model, the function of various convolution operations and pooling used in CNN can be better understood. As a result, modified CNN models (with fewer weights and hyperparameters) are developed that exhibit competitive and sometimes better performance in comparison with existing CNN models when applied to both CIFAR-10 and CIFAR-100 data sets.",
author = "Juncai He and Jinchao Xu",
year = "2019",
month = "7",
day = "1",
doi = "10.1007/s11425-019-9547-2",
language = "English (US)",
volume = "62",
pages = "1331--1354",
journal = "Science China Mathematics",
issn = "1674-7283",
publisher = "Science in China Press",
number = "7",

}

MgNet : A unified framework of multigrid and convolutional neural network. / He, Juncai; Xu, Jinchao.

In: Science China Mathematics, Vol. 62, No. 7, 01.07.2019, p. 1331-1354.

Research output: Contribution to journalArticle

TY - JOUR

T1 - MgNet

T2 - A unified framework of multigrid and convolutional neural network

AU - He, Juncai

AU - Xu, Jinchao

PY - 2019/7/1

Y1 - 2019/7/1

N2 - We develop a unified model, known as MgNet, that simultaneously recovers some convolutional neural networks (CNN) for image classification and multigrid (MG) methods for solving discretized partial differential equations (PDEs). This model is based on close connections that we have observed and uncovered between the CNN and MG methodologies. For example, pooling operation and feature extraction in CNN correspond directly to restriction operation and iterative smoothers in MG, respectively. As the solution space is often the dual of the data space in PDEs, the analogous concept of feature space and data space (which are dual to each other) is introduced in CNN. With such connections and new concept in the unified model, the function of various convolution operations and pooling used in CNN can be better understood. As a result, modified CNN models (with fewer weights and hyperparameters) are developed that exhibit competitive and sometimes better performance in comparison with existing CNN models when applied to both CIFAR-10 and CIFAR-100 data sets.

AB - We develop a unified model, known as MgNet, that simultaneously recovers some convolutional neural networks (CNN) for image classification and multigrid (MG) methods for solving discretized partial differential equations (PDEs). This model is based on close connections that we have observed and uncovered between the CNN and MG methodologies. For example, pooling operation and feature extraction in CNN correspond directly to restriction operation and iterative smoothers in MG, respectively. As the solution space is often the dual of the data space in PDEs, the analogous concept of feature space and data space (which are dual to each other) is introduced in CNN. With such connections and new concept in the unified model, the function of various convolution operations and pooling used in CNN can be better understood. As a result, modified CNN models (with fewer weights and hyperparameters) are developed that exhibit competitive and sometimes better performance in comparison with existing CNN models when applied to both CIFAR-10 and CIFAR-100 data sets.

UR - http://www.scopus.com/inward/record.url?scp=85066889939&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066889939&partnerID=8YFLogxK

U2 - 10.1007/s11425-019-9547-2

DO - 10.1007/s11425-019-9547-2

M3 - Article

AN - SCOPUS:85066889939

VL - 62

SP - 1331

EP - 1354

JO - Science China Mathematics

JF - Science China Mathematics

SN - 1674-7283

IS - 7

ER -