Home

Awesome

SEN12MS Toolbox

This repository is supposed to collect tools and utilities for working with the SEN12MS dataset.

Dataset Link

The dataset itself can be downloaded here: https://mediatum.ub.tum.de/1474000

Publications

Information about the dataset can be found in two related publications:

The original description of the dataset and its generation is found in

Schmitt M, Hughes LH, Qiu C, Zhu XX (2019) SEN12MS - a curated dataset of georeferenced multi-spectral Sentinel-1/2 imagery for deep learning and data fusion. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2/W7: 153-160

Link: https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2-W7/153/2019/

@inproceedings{Schmitt2019,
    author = {Michael Schmitt and Lloyd Haydn Hughes and Chunping Qiu and Xiao Xiang Zhu},
    title = {SEN12MS -- a curated dataset of georeferenced multi-spectral Sentinel-1/2 imagery for deep learning and data fusion},
    booktitle={ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences}, 
    volume={IV-2/W7},
    year = {2019},
    pages = {153-160},
    doi={10.5194/isprs-annals-IV-2-W7-153-2019}
}

There is also a second paper describing a small re-design of the SEN12MS dataset for single-label and multi-label scene classification tasks, including benchmark results on state-of-the-art CNN architectures:

Schmitt M, Wu Y-L (2021) Remote sensing image classification with the SEN12MS dataset. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences V-2-2021: 101–106

Link: https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2021/101/2021/

@inproceedings{Schmitt2021,
      title={Remote sensing image classification with the SEN12MS dataset}, 
      author={Michael Schmitt and Yu-Lun Wu},
      year={2021},
      booktitle={ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
      volume={V-2-2021},
      pages={101-106},
      doi={10.5194/isprs-annals-V-2-2021-101-2021}
}

If you use the dataset in the context of scientific publications, please cite the appropriate reference in your paper!

Contents

The repository contains the following folders:

labels

In this folder, text and pickle files containing single labels for every scene (patch) of SEN12MS are provided, following the IGBP land cover scheme. They can be used to train scene classification instead of semantic segmentation models. The class numbers of both the original IGBP scheme and the simplified IGBP scheme can be found in (Schmitt & Wu, 2021).

In addition, there is a list of multi-class labels for every scene:

Please note: The python scripts provided in the classification folder of this repository only read probability labels in the original IGBP scheme (i.e. IGBP_probability_labels.pkl), and convert them into single-label/ multi-label in the simplified IGBP land cover scheme on the fly. The other files are intended for sake of convenience to be used with other frameworks.

splits

In this folder, text files containing suggestions for splits are stored, pointing either at complete folders or individual files. Due to the folder structure and naming convention of SEN12MS, such file/folder list files should only point to Sentinel-1 or Sentinel-2 data (i.e. with the identifier _s1_ or _s2_in folder and/or file name. After reading in such a file, the identifier can easily be replaced to _s2_, _s1_ or _lc_, respectively, to address the corresponding Sentinel-2, Sentinel-1 or land cover data.
Current split suggestions:

The file lists are available both in plain ASCII (.txt) format, as well as .pkl format, with the .pkl files being the ones that area read by the scripts provided in the classification folder of this repository.

utils

In this folder, other utilities that can help to load, process, or analyze the data can be stored.

classification

In this folder, you can find codes for image classification CNNs (e.g. ResNet and DenseNet models) aiming at single-label and multi-label scene classification. They were developed using Python 3.7.7 and using several packages (NumPy, Rasterio, Scikit-Learn, TensorboardX, Torch, TorchVision, TQDM). To install the packages run pip install requirements.txt with your development environment activated from the classification folder.

The files needed for training and evaluating SEN12MS-based classification models are described as follows:

Pre-trained weights and optimization parameters for these models can be downloaded from here: https://syncandshare.lrz.de/getlink/fiCDbqiiSFSNwot5exvUcW1y/trained_models.

The models' respective input modalities are specified by their suffixes:

Additional Resources

Semantic Segmentation

The following repository created by Lukas Liebel contains DeepLabv3 and Unet models adapted to the peculiarities of SEN12MS, so that they can be directly trained and evaluated on SEN12MS (and DFC2020 data, see below) without much further ado: https://github.com/lukasliebel/dfc2020_baseline.

DFC2020

SEN12MS is used as backbone dataset of the 2020 IEEE-GRSS Data Fusion Contest (DFC2020). In the frame of the contest, high-resolution (GSD: 10m) validation and test data is released. The data and more information can be retrieved via the following links:

Papers working with SEN12MS Data