Awesome

Learning An Effective Transformer for Remote Sensing Satellite Image Dehazing

Abstract: The existing remote sensing (RS) image dehazing methods based on deep learning have sought help from the convolutional frameworks. Nevertheless, the inherent limitations of convolution, {\em i.e.,} local receptive fields and independent input elements, curtail the network to learn the long-range dependencies and non-uniform distributions. To this end, we design an effective RS image dehazing Transformer architecture, denoted as RSDformer. Firstly, given the irregular shapes and non-uniform distributions of haze in RS images, capturing both local and non-local features is crucial for RS image dehazing models. Hence, we propose a detail-compensated transposed attention to extract the global and local dependencies by across channels. Secondly, for enhancing the ability to learn degraded features and better guide the restoration process, we develop a dual-frequency adaptive block with dynamic filters. Finally, a dynamic gated fusion block is designed to achieve fuse and exchange features across different scales effectively. In this way, the model exhibits robust capabilities to capture dependencies from both global and local areas, resulting in improving image content recovery. Extensive experiments prove that the proposed method obtains more appealing performances against other competitive methods. The source codes are available at https://github.com/MingTian99/RSDformer

News

• July 4, 2023: Paper submitted.
• Sep 13, 2023: The basic version is released, including codes, pre-trained models on the Sate 1k dataset, and the used dataset.
• Sep 14, 2023: RICE dataset updated.

Preparation

Datasets

Install

We test the code on PyTorch 1.9.1 + CUDA 11.1 + cuDNN 8.0.5.

1. Create a new conda environment

conda create -n RSDformer python=3.8
conda activate RSDformer 

2. Install dependencies

pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

pip install matplotlib scikit-image opencv-python numpy einops math natsort tqdm lpips time tensorboardX

Download

You can download the pre-trained models and datasets on Google Drive or BaiduPan.

Currently, we only provide the pre-trained model trained on the Sate 1K dataset and the used dataset (Sate 1K, RICE and RRSD300).

The pre-trained models trained on RICE will be updated as quickly as possible.

The final file path should be the same as the following:

┬─ pretrained_models
│   ├─ thin_haze.pth
│   ├─ moderate_haze.pth
│   ├─ ... (model name)
│   └─ ... (exp name)
└─ data
    ├─ Sate_1K
    │├─ Haze1k_thick
    ││   ├─ train
    ││   │   ├─ input
    ││   │   │   └─ ... (image filename)
    ││   │   └─ target
    ││   │       └─ ... (corresponds to the former)
    ││   └─ test
    ││       └─ ...
    │└────  ... (dataset name)
    │
    │
    └─ ... (dataset name)

Training, Testing and Evaluation

Train

The training code will be released after the paper is accepted. You should change the path to yours in the Train.py file. Then run the following script to test the trained model:

python Train.py

Test

You should change the path to yours in the Test.py file. Then run the following script to test the trained model:

python Test.py

Notes

1. Send e-mail to songtienyu@163.com if you have critical issues to be addressed.
2. Please note that there exists the slight gap in the final version due to errors caused by different testing devices and environments.
3. Because the synthetic dataset is not realistic enough, the trained models may not work well on real hazy images.

Acknowledgment

This code is based on the Restormer. The real-world dataset RRSD300 is collected from RSHazeNet. Thanks for their awesome work.