Awesome

Generic Knowledge Boosted Pre-training For Remote Sensing Images

This is the official PyTorch implementation of the GeRSP:

@ARTICLE{10400411,
  author={Huang, Ziyue and Zhang, Mingming and Gong, Yuan and Liu, Qingjie and Wang, Yunhong},
  journal={IEEE Transactions on Geoscience and Remote Sensing}, 
  title={Generic Knowledge Boosted Pre-training For Remote Sensing Images}, 
  year={2024},
  volume={},
  number={},
  pages={1-1},
  doi={10.1109/TGRS.2024.3354031}
  }

Preparation

Install Python dependencies by running:

#----- Create conda environment
conda create --name gersp python=3.9 -y
conda activate gersp

#----- Install pytorch. We use PyTorch: 1.11.0 or 1.12.1 in our experiments.
#----- You could also use pip to install pytorch following https://pytorch.org/get-started/previous-versions/
conda install pytorch torchvision -c pytorch

#----- Install others. We use mmcv-full: 1.7.1 or 1.6.2 in our experiments.
pip install -U openmim
pip install -v -e .
pip install mmcv-full==1.6.2
pip install mmengine==0.9.0
pip install yapf==0.40.1

#----- MMDetection need these
pip install terminaltables
pip install pycocotools

Datasets

First, download Million-AID dataset by following https://captain-whu.github.io/DiRS/. Afterward, unzip all the files in each folder.

Then, download ImageNet dataset and also unzip all files.

Next, create a 'data' directory and place the ImageNet and Million-AID datasets into the 'data' directory according to the following directory format.

data
|---million_aid
|   |---test
|   |   |---P0000000.jpg
|   |   |---P0000001.jpg
|   |   |...
|   |...
|---ImageNet
|   |---train
|   |   |---n01440764
|   |   |---n01443537
|   |   |---n01484850
|   |   |...
|   |...

Pre-training

Use the following command for pre-training ResNet-50:

bash ./tools/dist_train_GeRSP.sh

GeRSP can be pre-trained using eight 2080Ti (10GB). You can reduce the number of GPUs by increasing the batch size and using GPUs with larger memory, but aligning performance is not guaranteed.

Model Convert

Use the following commands to convert the pre-trained model into usable weights:

python ./M_Convert_Checkpoints/convert_gersp.py

We also provide model conversion code for CMID and GeoAware, allowing to download their pre-trained weights and perform the conversion. Other methods we compared were also converted in a similar manner.

Downstream tasks

To train a Faster R-CNN on DIOR from a pre-trained GeRSP model, run:

python ./train_mmdet.py ./configs_mmdet/faster_rcnn_GeRSP.py  --work-dir ./results/faster_rcnn_GeRSP

Similar approaches can be applied to train RetinaNet and Dynamic R-CNN, the results can be found in our paper. For semantic segmentation and classification, we conducted experiments using MMSegmentation and MMClassification, with experimental configurations detailed in the paper.

Pre-trained Models

Our pre-trained GeRSP models can be downloaded as following:

Results on EuroSAT and NWPU-RESISC45

Results on DIOR and DOTA

Results on LoveDA