TY - JOUR
T1 - Deep MR Brain Image Super-Resolution Using Spatio-Structural Priors
AU - Cherukuri, Venkateswararao
AU - Guo, Tiantong
AU - Schiff, Steven J.
AU - Monga, Vishal
N1 - Funding Information:
Manuscript received November 3, 2018; revised June 16, 2019 and July 29, 2019; accepted September 2, 2019. Date of publication September 25, 2019; date of current version November 4, 2019. This work was supported by NIH under Grant 7R01HD085853. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Hitoshi Kiya. (Corresponding author: Venkateswararao Cherukuri.) V. Cherukuri is with the Department of Electrical Engineering, The Pennsylvania State University, University Park, PA 16801 USA, and also with the Center for Neural Engineering, The Pennsylvania State University, University Park, PA 16801 USA (e-mail: vmc5164@psu.edu).
Publisher Copyright:
© 1992-2012 IEEE.
PY - 2020
Y1 - 2020
N2 - High resolution Magnetic Resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware and processing constraints. Recently, deep learning methods have been shown to produce compelling state-of-the-art results for image enhancement/super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image super-resolution (SR). Our contributions are then incorporating these priors in an analytically tractable fashion as well as towards a novel prior guided network architecture that accomplishes the super-resolution task. This is particularly challenging for the low rank prior since the rank is not a differentiable function of the image matrix (and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. As a key extension, we modify the fixed feedback (Laplacian) layer by learning a new set of training data driven filters that are optimized for enhanced sharpness. Experiments performed on publicly available MR brain image databases and comparisons against existing state-of-the-art methods show that the proposed prior guided network offers significant practical gains in terms of improved SNR/image quality measures. Because our priors are on output images, the proposed method is versatile and can be combined with a wide variety of existing network architectures to further enhance their performance.
AB - High resolution Magnetic Resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware and processing constraints. Recently, deep learning methods have been shown to produce compelling state-of-the-art results for image enhancement/super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image super-resolution (SR). Our contributions are then incorporating these priors in an analytically tractable fashion as well as towards a novel prior guided network architecture that accomplishes the super-resolution task. This is particularly challenging for the low rank prior since the rank is not a differentiable function of the image matrix (and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. As a key extension, we modify the fixed feedback (Laplacian) layer by learning a new set of training data driven filters that are optimized for enhanced sharpness. Experiments performed on publicly available MR brain image databases and comparisons against existing state-of-the-art methods show that the proposed prior guided network offers significant practical gains in terms of improved SNR/image quality measures. Because our priors are on output images, the proposed method is versatile and can be combined with a wide variety of existing network architectures to further enhance their performance.
UR - http://www.scopus.com/inward/record.url?scp=85077495396&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85077495396&partnerID=8YFLogxK
U2 - 10.1109/TIP.2019.2942510
DO - 10.1109/TIP.2019.2942510
M3 - Article
C2 - 31562091
AN - SCOPUS:85077495396
VL - 29
SP - 1368
EP - 1383
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
SN - 1057-7149
M1 - 8848868
ER -