Skip to main content

Research Repository

Advanced Search

ICSF: integrating inter-modal and cross-modal learning framework for self-supervised heterogeneous change detection.

Zhang, Erlei; Zong, He; Li, Xinyu; Feng, Mingchen; Ren, Jinchang

Authors

Erlei Zhang

He Zong

Xinyu Li

Mingchen Feng



Abstract

Heterogeneous change detection (HCD) is a process to determine the change information by analyzing heterogeneous images of the same geographic location taken at different times, which plays an important role in remote sensing applications such as disaster response and environmental monitoring. However, the different imaging mechanisms result in different visual appearances in heterogeneous images, making it difficult to accurately detect changes through direct comparison. To address this problem, we propose a novel self-supervised dual-branch framework (ICSF) for HCD that incorporates inter-modal and cross-modal learning. First, in the inter-modal branch, we perform the contrastive learning on heterogeneous images within their respective modalities to learn the robust and discriminative features, rather than relying on the raw spectral or spatial information from these images. Second, in the cross-modal branch, we perform cross-modal reconstruction to ensure the obtained features exhibit consistent comparability, thereby facilitating the extraction of rich information of the real changes within the images. Next, the difference images (DIs) computed from both branches are further refined using a superpixel segmentation strategy to preserve the consistency of differences within the same ground object. Experimental results on five public datasets with different modality combinations and change events demonstrate the effectiveness of the proposed approach in comparison to ten state-of-the-art methods, achieving the best performance with an average overall accuracy of 95.88% and an average Kappa coefficient of 74.20%.

Citation

ZHANG, E., ZONG, H., LI, X., FENG, M. and REN, J. 2025. ICSF: integrating inter-modal and cross-modal learning framework for self-supervised heterogeneous change detection. IEEE transactions on geoscience and remote sensing [online], 63, 501516. Available from: https://doi.org/10.1109/TGRS.2024.3519195

Journal Article Type Article
Acceptance Date Dec 10, 2024
Online Publication Date Dec 19, 2024
Publication Date Dec 31, 2025
Deposit Date Jan 7, 2025
Publicly Available Date Jan 7, 2025
Journal IEEE transactions on geoscience and remote sensing
Print ISSN 0196-2892
Electronic ISSN 1558-0644
Publisher Institute of Electrical and Electronics Engineers (IEEE)
Peer Reviewed Peer Reviewed
Volume 63
Article Number 501516
DOI https://doi.org/10.1109/TGRS.2024.3519195
Keywords Contrastive learning; Dual-branch; Heterogeneous change detection (HCD); Self-supervised learning; Structural relationship
Public URL https://rgu-repository.worktribe.com/output/2626172

Files

ZHANG 2025 ICSF (AAM) (13 Mb)
PDF

Publisher Licence URL
https://creativecommons.org/licenses/by/4.0/

Copyright Statement
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.




You might also like



Downloadable Citations