Home

Awesome

batchgenerators by MIC@DKFZ

Copyright German Cancer Research Center (DKFZ) and contributors. Please make sure that your usage of this code is in compliance with its license.

batchgenerators is a python package for data augmentation. It is developed jointly between the Division of Medical Image Computing at the German Cancer Research Center (DKFZ) and the Applied Computer Vision Lab of the Helmholtz Imaging Platform.

It is not (yet) perfect, but we feel it is good enough to be shared with the community. If you encounter bug, feel free to contact us or open a github issue.

If you use it please cite the following work:

Isensee Fabian, Jäger Paul, Wasserthal Jakob, Zimmerer David, Petersen Jens, Kohl Simon, 
Schock Justus, Klein Andre, Roß Tobias, Wirkert Sebastian, Neher Peter, Dinkelacker Stefan, 
Köhler Gregor, Maier-Hein Klaus (2020). batchgenerators - a python framework for data 
augmentation. doi:10.5281/zenodo.3632567

batchgenerators also contains the following application-specific augmentations:

If you use these augmentations please cite them too.

Build Status

Supported Augmentations

We supports a variety of augmentations, all of which are compatible with 2D and 3D input data! (This is something that was missing in most other frameworks).

Note: Stack transforms by using batchgenerators.transforms.abstract_transforms.Compose. Finish it up by plugging the composed transform into our multithreader: batchgenerators.dataloading.multi_threaded_augmenter.MultiThreadedAugmenter

How to use it

The working principle is simple: Derive from DataLoaderBase class, reimplement generate_train_batch member function and use it to stack your augmentations! For simple example see batchgenerators/examples/example_ipynb.ipynb

A heavily commented example for using SlimDataLoaderBase and MultithreadedAugmentor is available at: batchgenerators/examples/multithreaded_with_batches.ipynb. It gives an idea of the interplay between the SlimDataLoaderBase and the MultiThreadedAugmentor. The example uses the MultiThreadedAugmentor for loading and augmentation on mutiple processes, while covering the entire dataset only once per epoch (basically sampling without replacement).

We also now have an extensive example for BraTS2017/2018 with both 2D and 3D DataLoader and augmentations: batchgenerators/examples/brats2017/

There are also CIFAR10/100 datasets and DataLoader available at batchgenerators/datasets/cifar.py

Data Structure

The data structure that is used internally (and with which you have to comply when implementing generate_train_batch) is kept simple as well: It is just a regular python dictionary! We did this to allow maximum flexibility in the kind of data that is passed along through the pipeline. The dictionary must have a 'data' key:value pair. It optionally can handle a 'seg' key:vlaue pair to hold a segmentation. If a 'seg' key:value pair is present all spatial transformations will also be applied to the segmentation! A part from 'data' and 'seg' you are free to do whatever you want (your image classification/regression target for example). All key:value pairs other than 'data' and 'seg' will be passed through the pipeline unmodified.

'data' value must have shape (b, c, x, y) for 2D or shape (b, c, x, y, z) for 3D! 'seg' value must have shape (b, c, x, y) for 2D or shape (b, c, x, y, z) for 3D! Color channel may be used here to allow for several segmentation maps. If you have only one segmentation, make sure to have shape (b, 1, x, y (, z))

How to install locally

Install batchgenerators

pip install --upgrade batchgenerators

Import as follows

from batchgenerators.transforms.color_transforms import ContrastAugmentationTransform

Windows Support is very experimental!

Batchgenerators makes heavy use of python multiprocessing and python multiprocessing on windows is different from linux. To prevent the workers from freezing in windows, you have to guard your code with if __name__ == '__main__' and use multiprocessing's freeze_support. The executed script may then look like this:

# some imports and functions here

def main():
    # do some stuff

if __name__ == '__main__':
    from multiprocessing import freeze_support
    freeze_support()
    main()

This is not required on Linux.

Release Notes

(only highlights, not an exhaustive list)

<img src="DKFZ_Logo.png" width="512px" /> <img src="HIP_Logo.png" width="512px" />

batchgenerators is developed by the Division of Medical Image Computing of the German Cancer Research Center (DKFZ) and the Applied Computer Vision Lab (ACVL) of the Helmholtz Imaging Platform.