Awesome

Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity

PyTorch implementation for "Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity"

• [03 August 2021] Release the code of MFPNet model.
• [28 June 2022] Release the processed datasets, training/evaluation codes.
• [08 JUly 2022] Release model weights for Season-varying/LEVIR-CD/Google datasets.

Introduction

Remote sensing change detection (RSCD) is an important yet challenging task in Earth observation. The booming development of convolutional neural networks (CNNs) in computer vision raises new possibilities for RSCD, and many recent RSCD methods have introduced CNNs to achieve promising improvements in performance. This paper proposes a novel multidirectional fusion and perception network for change detection in bi-temporal very-high-resolution remote sensing images. First, we propose an elaborate feature fusion module consisting of a multidirectional fusion pathway (MFP) and an adaptive weighted fusion (AWF) strategy for RSCD to boost the way that information propagates in the network. The MFP enhances the flexibility and diversity of information paths by creating extra top-down and shortcut-connection paths. The AWF strategy conducts weight recalibration for every fusion node to highlight salient feature maps and overcome semantic gaps between different features. Second, a novel perceptual similarity module is designed to introduce perceptual loss into the RSCD task, which adds the perceptual information, such as structure and semantic, for high-quality change maps generation. Extensive experiments on four challenging benchmark datasets demonstrate the superiority of the proposed network comparing with eight state-of-the-art methods in terms of F1, Kappa, and visual qualities.

Content

Architecture

Fig.1 Overall architecture of the proposed multidirectional fusion and perception network (MFPNet). <br> Note that the process with the dashed line only participates in model training.

Datasets

The processed and original datasets can be downloaded from the table below, we recommended downloading the processed one directly to get a quick start on our codes:

Setup & Usage for the Code

1. Check the structure of data folders:

(root folder)
├── dataset1
|  ├── train
|  |  ├── A
|  |  ├── B
|  |  ├── label
|  ├── val
|  |  ├── A
|  |  ├── B
|  |  ├── label
|  ├── test
|  |  ├── A
|  |  ├── B
|  |  ├── label
├── ...

2. Check dependencies:

- Python 3.6+
- PyTorch 1.7.0+
- scikit-learn
- cudatoolkit
- cudnn
- OpenCV-Python

3. Change paths:

- Change the 'metadata_json_path' in 'train.py' to your 'metadata.json' path.
- Change the 'dataset_dir' and 'weight_dir' in 'metadata.json' to your own path.

4. Train the MFPNet:

python train.py

5. Evaluate the MFPNet:

- Download model weights (optional).
- Change the 'weight_path' in 'eval.py' to your model weight path.
- python eval.py

Model Weights

Model weights for Season-varying/LEVIR-CD/Google datasets are available via Google Drive or Baidu Drive. Note that the training/dataloader codes are rewritten and improved so the performance is a little different from the paper.

Reference

Appreciate the work from the following repositories:

• likyoo/Siam-NestedUNet
• zylo117/Yet-Another-EfficientDet-Pytorch

Cite

If this repository is useful for your research, please cite:

@Article{rs13153053,
AUTHOR = {Xu, Jialang and Luo, Chunbo and Chen, Xinyue and Wei, Shicai and Luo, Yang},
TITLE = {Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity},
JOURNAL = {Remote Sensing},
VOLUME = {13},
YEAR = {2021},
NUMBER = {15},
ARTICLE-NUMBER = {3053},
URL = {https://www.mdpi.com/2072-4292/13/15/3053},
ISSN = {2072-4292},
DOI = {10.3390/rs13153053}
}