Home

Awesome

3DCD: Inferring 3D change detection from bitemporal optical images

<div align="center"> <img src="images/logo_3d_cd.png", width="250"> </div>

ISPRS J. Photogramm paper | arXiv paper

3DCD project has the goal of automatically inferring 2D and 3D change detection (elevation change) maps of human artefacts from bitemporal optical images, without the need of relying on DSMs.

The project is carried out by Virginia Coletta, Valerio Marsocci and Dr. Roberta Ravanelli, under the supervision of professors Simone Scardapane and Mattia Crespi.

For more information click on the site 3DCD or frame the QR code.

<div align="center"> <img src="images/qr.png", width="250"> </div>

Contents

  1. Features
  2. Dataset
  3. Installation
  4. Usage
  5. Results
  6. Related resources
  7. Citation
  8. License

Features

Change detection is one of the most active research areas in Remote Sensing (RS). Most of the recently developed change detection methods are based on deep learning (DL) algorithms. This kind of algorithms is generally focused on generating two-dimensional (2D) change maps, thus only identifying planimetric changes in land use/land cover (LULC) and not considering nor returning any information on the corresponding elevation changes.

Our work goes one step further, proposing two novel networks, able to solve simultaneously the 2D and 3D CD tasks, and the 3DCD dataset, a novel and freely available dataset precisely designed for this multitask. Particularly, the aim of this work is to lay the foundations for the development of DL algorithms able to automatically infer an elevation (3D) CD map - together with a standard 2D CD map, starting only from a pair of bitemporal optical images.

The proposed architecture consists of a transformer-based network, the MultiTask Bitemporal Images Transformer (MTBIT). Particularly, MTBIT is a transformer-based architecture, that makes use of a semantic tokenizer. This model is, thus, able to obtain 3D CD maps from two optical images taken at different time instants, without the need to rely directly on elevation data during the inference step. Another model capable to solve this problem effectively is SUNet, a modified version of FC-Siam-conc.

Dataset

At this link, the dataset is available.

<div align="center"><img src="images/ex1.jpg", width="500"></div>

Main features of the data contained in the proposed dataset in the table below.

ImageNumber of pixelsGSD
Orthophotos from 2010400x4000.50 m
Orthophotos from 2017400x4000.50 m
DSMs from 2010200x2001.0 m
DSMs from 2017200x2001.0 m
2D CD maps400x4000.50 m
3D CD maps200x2001.0 m

To use your custom dataset, please respect this structure (and remember to rename the subfolder "2010" and "2017" in the dataloader):

Custom dataset
├─train
   ├─t1
   ├─t2
   ├─2dcd
   └─3dcd
├─val
   ├─t1
   ├─t2
   ├─2dcd
   └─3dcd
└─test
   ├─t1
   ├─t2
   ├─2dcd
   └─3dcd

Installation

pip install -r requirements.txt

Usage

To train the model, prepare a .yaml file and put in the config directory (where you can find an example) and then run the following command:

python train.py --config="your_config_file"

To test your model, follow the same steps for training and then run the following command:

python test.py --config="your_config_file"

Results

In this section we briefly show the results obtained. We compared our models with other models available in the literature.

WARNING: if you use this code te reproduce MTBIT results, please consider these metrics w.r.t. the one in the paper, due to un update of the way of computing the cRMSE.

ArchitectureF1 (%)IoU (%)RMSE (m)cRMSE (m)
ChangeFormer42.8927.301.316.24
FC-EF46.3230.141.416.74
FC-Siam-conc38.3023.691.426.18
SUNet59.7242.571.245.86
ResNet187.874.821.567.26
IM2HEIGHT--1.576.71
MTBIT (ours)62.1545.091.205.76
<div align="center"><img src="images/comp.jpg", width="500"></div> <div align="center"><img src="images/isto.png", width="500"></div>

Related resources

In this section, we point out some useful repositories, resources and connected projects.

Reach us out if you would like to be inserted in this list.

Citation

If you found our work useful, consider citing these works:

-Inferring 3D change detection from bitemporal optical images

@article{MARSOCCI2023325,
title = {Inferring 3D change detection from bitemporal optical images},
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
volume = {196},
pages = {325-339},
year = {2023},
issn = {0924-2716},
doi = {https://doi.org/10.1016/j.isprsjprs.2022.12.009},
url = {https://www.sciencedirect.com/science/article/pii/S0924271622003240},
author = {Valerio Marsocci and Virginia Coletta and Roberta Ravanelli and Simone Scardapane and Mattia Crespi},
keywords = {3D change detection, Remote sensing, Deep learning, Elevation change detection, Dataset},
}

-3DCD: a new dataset for 2D and 3D change detection using deep learning techniques

@Article{isprs-archives-XLIII-B3-2022-1349-2022,
AUTHOR = {Coletta, V. and Marsocci, V. and Ravanelli, R.},
TITLE = {3DCD: A NEW DATASET FOR 2D AND 3D CHANGE DETECTION USING DEEP LEARNING TECHNIQUES},
JOURNAL = {The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
VOLUME = {XLIII-B3-2022},
YEAR = {2022},
PAGES = {1349--1354},
URL = {https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B3-2022/1349/2022/},
DOI = {10.5194/isprs-archives-XLIII-B3-2022-1349-2022}
}

License

Code is released for non-commercial and research purposes only. For commercial purposes, please contact the authors.