Deep learning based image super-resolution with coupled backpropagation

Tiantong Guo, Hojjat S. Mousavi, Vishal Monga

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    6 Citations (Scopus)

    Abstract

    Recently deep learning methods have been applied to image super-resolution (SR). Typically, these approaches involve training a single convolutional neural network that is trained to perform resolution enhancement. We propose a new low-complexity but effective algorithm called Superresolution with Coupled Backpropagation (SR-CBP) which builds two Coupled Auto-encoder Networks (CAN), resp. the high-resolution (HR) and low-resolution (LR) networks, that capture the features of both high and low resolution images. The two networks in CAN have the ability to self-reconstruct its own input. Specifically, SR-CBP allows joint training of the LR and HR networks to have middle layer representations that agree for a pair of images (high-resolution image and its corresponding low-resolution version). For an LR input image, its middle layer representation obtained via the trained LR network can be fed into the HR network to generate the SR result. Preliminary experiments show that SR-CBP can produce better results than state of the art single image superresolution methods based on sparse representations. The memory and storage requirements of CAN are lesser than existing deep learning based SR methods.

    Original languageEnglish (US)
    Title of host publication2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages237-241
    Number of pages5
    ISBN (Electronic)9781509045457
    DOIs
    StatePublished - Apr 19 2017
    Event2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Washington, United States
    Duration: Dec 7 2016Dec 9 2016

    Publication series

    Name2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings

    Other

    Other2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016
    CountryUnited States
    CityWashington
    Period12/7/1612/9/16

    Fingerprint

    Image resolution
    Backpropagation
    Optical resolving power
    Neural networks
    Data storage equipment
    Experiments
    Deep learning

    All Science Journal Classification (ASJC) codes

    • Signal Processing
    • Computer Networks and Communications

    Cite this

    Guo, T., Mousavi, H. S., & Monga, V. (2017). Deep learning based image super-resolution with coupled backpropagation. In 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings (pp. 237-241). [7905839] (2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/GlobalSIP.2016.7905839
    Guo, Tiantong ; Mousavi, Hojjat S. ; Monga, Vishal. / Deep learning based image super-resolution with coupled backpropagation. 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 237-241 (2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings).
    @inproceedings{33a661ef8ef24490ac8d5bb2aa8d54ed,
    title = "Deep learning based image super-resolution with coupled backpropagation",
    abstract = "Recently deep learning methods have been applied to image super-resolution (SR). Typically, these approaches involve training a single convolutional neural network that is trained to perform resolution enhancement. We propose a new low-complexity but effective algorithm called Superresolution with Coupled Backpropagation (SR-CBP) which builds two Coupled Auto-encoder Networks (CAN), resp. the high-resolution (HR) and low-resolution (LR) networks, that capture the features of both high and low resolution images. The two networks in CAN have the ability to self-reconstruct its own input. Specifically, SR-CBP allows joint training of the LR and HR networks to have middle layer representations that agree for a pair of images (high-resolution image and its corresponding low-resolution version). For an LR input image, its middle layer representation obtained via the trained LR network can be fed into the HR network to generate the SR result. Preliminary experiments show that SR-CBP can produce better results than state of the art single image superresolution methods based on sparse representations. The memory and storage requirements of CAN are lesser than existing deep learning based SR methods.",
    author = "Tiantong Guo and Mousavi, {Hojjat S.} and Vishal Monga",
    year = "2017",
    month = "4",
    day = "19",
    doi = "10.1109/GlobalSIP.2016.7905839",
    language = "English (US)",
    series = "2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    pages = "237--241",
    booktitle = "2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings",
    address = "United States",

    }

    Guo, T, Mousavi, HS & Monga, V 2017, Deep learning based image super-resolution with coupled backpropagation. in 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings., 7905839, 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 237-241, 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016, Washington, United States, 12/7/16. https://doi.org/10.1109/GlobalSIP.2016.7905839

    Deep learning based image super-resolution with coupled backpropagation. / Guo, Tiantong; Mousavi, Hojjat S.; Monga, Vishal.

    2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. p. 237-241 7905839 (2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    TY - GEN

    T1 - Deep learning based image super-resolution with coupled backpropagation

    AU - Guo, Tiantong

    AU - Mousavi, Hojjat S.

    AU - Monga, Vishal

    PY - 2017/4/19

    Y1 - 2017/4/19

    N2 - Recently deep learning methods have been applied to image super-resolution (SR). Typically, these approaches involve training a single convolutional neural network that is trained to perform resolution enhancement. We propose a new low-complexity but effective algorithm called Superresolution with Coupled Backpropagation (SR-CBP) which builds two Coupled Auto-encoder Networks (CAN), resp. the high-resolution (HR) and low-resolution (LR) networks, that capture the features of both high and low resolution images. The two networks in CAN have the ability to self-reconstruct its own input. Specifically, SR-CBP allows joint training of the LR and HR networks to have middle layer representations that agree for a pair of images (high-resolution image and its corresponding low-resolution version). For an LR input image, its middle layer representation obtained via the trained LR network can be fed into the HR network to generate the SR result. Preliminary experiments show that SR-CBP can produce better results than state of the art single image superresolution methods based on sparse representations. The memory and storage requirements of CAN are lesser than existing deep learning based SR methods.

    AB - Recently deep learning methods have been applied to image super-resolution (SR). Typically, these approaches involve training a single convolutional neural network that is trained to perform resolution enhancement. We propose a new low-complexity but effective algorithm called Superresolution with Coupled Backpropagation (SR-CBP) which builds two Coupled Auto-encoder Networks (CAN), resp. the high-resolution (HR) and low-resolution (LR) networks, that capture the features of both high and low resolution images. The two networks in CAN have the ability to self-reconstruct its own input. Specifically, SR-CBP allows joint training of the LR and HR networks to have middle layer representations that agree for a pair of images (high-resolution image and its corresponding low-resolution version). For an LR input image, its middle layer representation obtained via the trained LR network can be fed into the HR network to generate the SR result. Preliminary experiments show that SR-CBP can produce better results than state of the art single image superresolution methods based on sparse representations. The memory and storage requirements of CAN are lesser than existing deep learning based SR methods.

    UR - http://www.scopus.com/inward/record.url?scp=85019245288&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85019245288&partnerID=8YFLogxK

    U2 - 10.1109/GlobalSIP.2016.7905839

    DO - 10.1109/GlobalSIP.2016.7905839

    M3 - Conference contribution

    T3 - 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings

    SP - 237

    EP - 241

    BT - 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -

    Guo T, Mousavi HS, Monga V. Deep learning based image super-resolution with coupled backpropagation. In 2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. p. 237-241. 7905839. (2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings). https://doi.org/10.1109/GlobalSIP.2016.7905839