Multi-scale spatial fusion and regularization induced unsupervised auxiliary task CNN model for deep super-resolution of hyperspectral image.

Ha, Viet Khanh; Ren, Jinchang; Wang, Zheng; Genyun, Sun; Zhao, Huimin; Marshall, Stephen

Abstract

Hyperspectral images (HSI) features rich spectral information in many narrow bands but at a cost of a relatively low spatial resolution. As such, various methods have been developed for enhancing the spatial resolution of the low-resolution HSI (Lr-HSI) by fusing it with high-resolution multispectral images (Hr-MSI). The difference in spectrum range and spatial dimensions between the Lr-HSI and Hr-SI have been fundamental but challenging for multispectral/hyperspectral (MS/HS) fusion. In this paper, a multi-scale spatial fusion and regularization induced auxiliary task (MSAT) based CNN model is proposed for deep super-resolution of HSI, where a Lr-HSI is fused with a Hr-MSI to reconstruct a high-resolution HSI (Hr-HSI) counterpart. The multi-scale fusion is used to efficiently address the discrepancy in spatial resolutions between two inputs. Based on the general assumption that the acquired Hr-MSI and the reconstructed Hr-HSI share similar underlying characteristics, the auxiliary task is proposed to learn a representation for improved generality of the model and reduced overfitting. Experimental results on three public datasets have validated the effectiveness of our approach in comparison with several state-of-the-art methods.

Citation

HA, V.K., REN, J., WANG, Z., SUN, G., ZHAO, H. and MARSHALL, S. 2022. Multi-scale spatial fusion and regularization induced unsupervised auxiliary task CNN model for deep super-resolution of hyperspectral image. IEEE journal of selected topics in applied earth observations and remote sensing [online], 15, pages 4583-4598. Available from: https://doi.org/10.1109/JSTARS.2022.3176969

• Journal Article Type: Article
• Acceptance Date: May 11, 2022
• Online Publication Date: May 23, 2022
• Publication Date: Dec 31, 2022
• Deposit Date: May 26, 2022
• Publicly Available Date: May 26, 2022
• Journal: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
• Print ISSN: 1939-1404
• Electronic ISSN: 2151-1535
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• Peer Reviewed: Peer Reviewed
• Volume: 15
• Pages: 4583-4598
• DOI: https://doi.org/10.1109/jstars.2022.3176969
• Keywords: Hyperspectral image (HSI); Super-resolution (SR); Multi-scale spatial fusion; Auxiliary task; Convolutional neural networks (CNN)
• Public URL: https://rgu-repository.worktribe.com/output/1674656