Home

Awesome

Table of Contents:

Monitoring Urban Changes in Mariupol/Ukraine

This repository demonstrates the transferred ERCNN-DRS to monitor urban changes in Mariupol/Ukraine in 2022/23.

Prediction values (probabilities) over all detected urban changes with a half-year sliding window, on a static background (for reference only) and superimposed in colors according to the legend (Nov. 2021 - Nov. 2023):

Watch the video

This video is located in this repository here, without YouTube compression artifacts.

We continue to update the visualizations as time progresses. The ones from the paper (see below) are available as well.

Maximum values over all detected urban changes with a sliding window, superimposed in red (Nov. 2021 - Dec. 2023):

<p align="center"> <img src="./images/s12_grid_Mariupol_comb_pred_11_2021-12_2023.png" /> </p>

The GeoTIFF file is here.

Prediction values (probabilities) over all detected urban changes with a half-year sliding window (Nov. 2021 - Dec. 2023):

<p align="center"> <img src="./images/Mariupol_11_2021-12_2023.gif" /> </p>

Video file is located here.

The older counterparts from the paper and following months can be found here.

Note: These visualizations were created with 93x93 tiles and an 8 pixel overlap (dead zone of 4 in rsdtlib) to provide a seamless presentation without the issues of tile borders.

The urban changes were detected and monitored with a transferred version of the pre-trained ERCNN-DRS model for Sentinel 1 & 2 missions.

The windowed observations were pre-processed with rsdtlib. This library downloads all Sentinel 1 & 2 observations from Sentinel Hub and pre-processes the observations for time series analysis (windowing). This library was used for training and inference. We used as time frame November 2021 up to today (December 2023), with sliding windows of six month duration.

Pre-processing with rsdtlib

The scripts to download and process the Sentinel 1 and Sentinel 2 observations to windowed time series are available at the rsdtlib repository in the subdirectory urban_change_monitoring_mariupol_ua. See below for the already processed and ready-to-go training/validation datasets.

Training/Validation Datasets

Thanks to the data providers, we can make available the training/validation datasets on Google Drive.

Note: The training/validation datasets are TFRecord files, with one file for each tile and each tile containing all windows from 2017-2020. Two features are availble, with one describing the time series of observations for each window and a label. The label is the synthetic ground truth which is not used for transfer learning! Instead labels need to be loaded separately from folder training/numpy_ground_truth.

ATTENTION, these files are large!

Extract the tar balls V[1-4].tar in the respective subdirectories ./training/V1/, ./training/V2/, ./training/V3/, and ./training/V4/.

Versions V[1-4] are using different subsets of tiles for training, with valiation tiles being disjunct.

Training

Execute the training script training/train.py. It is recommended to use the NVIDIA GPU Cloud Tensorflow container docker://nvcr.io/nvidia/tensorflow:22.02-tf2-py3 and at least eight GPUs with a total of 320 GB of memory (8x40 GB).

Change the variable exp to the version to train, e.g. exp = "V1".

Trained Models

We provide all trained models:

Integration into Floreon

The data is also available in an experimental layer in Floreon: here

Set the time units to Days to scroll faster through time. Please note that the window with the nearest start time is shown in the timeline and that windows are only available in the ranges mentioned above.

Other Use Cases

Paper and Citation

The full paper can be found at IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

@ARTICLE{10423108,
      author={Zitzlsberger, Georg and Podhoranyi, Michal},
      journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
      title={Monitoring of Urban Changes With Multimodal Sentinel 1 and 2 Data in Mariupol, Ukraine, in 2022/23},
      year={2024},
      volume={17},
      number={},
      pages={5245-5265},
      doi={10.1109/JSTARS.2024.3362688},
      url={https://doi.org/10.1109/JSTARS.2024.3362688}
}

Contact

Should you have any feedback or questions, please contact the main author: Georg Zitzlsberger (georg.zitzlsberger(a)vsb.cz).

Acknowledgments

This research was funded by the Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPS II) project “IT4Innovations excellence in science - LQ1602” and by the IT4Innovations Infrastructure, which is supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90140), and via the Open Access Grant Competition (OPEN-25-24 and OPEN-27-1). This work was also supported by ESA Network of Resources Initiative (ID:2923ca) to provide access to Sentinel Hub, and Airbus Pléiades.

We also would like to thank CESNET Meta Centrum for providing us access to a DGX H100 node.

License

This project is made available under the GNU General Public License, version 3 (GPLv3).

Disclaimer

This work and results is active research and subject of change. They hence are provided "as is", without guarantee of correctness or liability. Use at your own risk.

Ethical Statement

Due to the ongoing Russian-Ukrainian war, the selection of locations of visual samples was done with care to minimize risks of influence and harm. To the best of our knowledge, we only selected locations and data that did not give direct insight to the ongoing war, but only documented the resulting (urban) changes. We also would like to underline that our monitoring methods used six-month windows and hence did not and shall not provide real-time information that could be used for military purposes. Our methods are optimized for inertial urban changes that manifest over longer periods.