Home

Awesome

<h1 align="center">Changes to Captions: An Attentive Network for Remote Sensing Change Captioning</h1> <h3 align="center"> Shizhen Chang and <a href="https://www.ai4rs.com/">Pedram Ghamisi</a></h3> <br

This is the official PyTorch implementation of Changes to Captions: An Attentive Network for Remote Sensing Change Captioning, a project conducted at the Institute of Advanced Research in Artificial Intelligence (IARAI).

Preparation

├─/root/Data/LEVIR_CC/
        ├─LevirCCcaptions.json
        ├─images
             ├─train
             │  ├─A
             │  ├─B
             ├─val
             │  ├─A
             │  ├─B
             ├─test
             │  ├─A
             │  ├─B

where folder A contains images of pre-phase, folder B contains images of post-phase.

$ python preprocess_data.py

!NOTE: When preparing the text token files, we suggest setting the word count threshold of LEVIR-CC to 5 and Dubai_CC to 0 for fair comparisons.

Training

$ python train.py

!NOTE: If the program encounters the error: "'Meteor' object has no attribute 'lock'," we recommend installing it with sudo apt install openjdk-11-jdk to resolve this issue.

Alternatively, you can obtain our pretrained models from Google Drive.

Caption Generation

$ python test.py

Quantitative Evaluation and Visual Examples

Here are some visualized examples of the generated captions in LEVIR-CC:

Paper

Changes to Captions: An Attentive Network for Remote Sensing Change Captioning

Please cite the following paper if you find it useful for your research:

@article{chg2cap,
  title={Changes to Captions: An Attentive Network for Remote Sensing Change Captioning},
  author={Chang, Shizhen and Ghamisi, Pedram},
  journal={IEEE Trans. Image Process.}, 
  doi={10.1109/TIP.2023.3328224},
  year={2023}
}

Acknowledgement

The authors would like to thank the contributors to the LEVIR-CC and Dubai-CC datasets.

License

This repo is distributed under MIT License. The code can be used for academic purposes only.