Awesome

Adversarial attacks on segmentation models

Implementation of membership inference attacks 🗡 on (poisoned 🧪) binary and multi-class segmentation models. Running the experiments requires Citycapes dataset [1], Medical Segmentation Decathlon dataset (its subset of liver images) [2] and Kvasir-SEG dataset [3]. The implementation assumes attacks on a victim model with the use of a single shadow model.

🎓 The code was developed for my master's thesis in data engineering and analytics at Technische Universität München (TUM).

Prerequisites

Python 3.7 or higher
PyTorch 1.12.1 or higher
Torchvision 0.13 or higher
Opacus 1.2 or higher
SimpleITK 2.1 or higher
segmentation_models_pytorch 0.3.0 or higher

Data

Cityscapes data need to be organised as follows:

seg-mia
└───mia-cityscapes
│   └───cityscapes
│   │   └───leftImg8bit
│   │   │   └───train
│   │   └───gtFine
│   │   │   └───train
...

Medical Segmentation Decathlon data need to be organised as follows:

seg-mia
└───mia-liver(-backdoor)
│   └───liver
│   │   └───imgs
│   │   └───lbls
...

Kvasir-SEG data need to be organised as follows:

seg-mia
└───mia-kvasir
│   └───Kvasir-SEG
│   │   └───images
│   │   └───masks
...

Evaluation

Each attack can be evaluated by running python main.py and controlled with the following command-line options.

General settings:

Arguments --victim and --shadow define the encoder architectures for victim and shadow U-Net segmentation models, respectively (see segmentation_models_pytorch)
--defensetype sets the defense methods applied to a victim model
--trainsize defines the size of the training dataset for the victim and shadow models

Settings for MIAs on poisoned models:

--attacktype defines the attack type
--triggertype controls the trigger shape (either 'line' or 'square')
--triggersize sets the height for a line trigger or the edge size of a square trigger (in pixels)
--triggerval sets the trigger values
--poison defines the poisoning probability for each training data sample; in the range [0,1]

Attack types:

Attack type	Description
1	Type-I
2	Type-II
3	Global loss-based

Defense types:

Defense type	Description	Implementation
1	No defense	`mia-liver`, `mia-kvasir`, `mia-cityscapes`, `mia-liver-backdoor`
2	Argmax	`mia-liver`, `mia-kvasir`, `mia-cityscapes`
3	Crop training	`mia-liver`, `mia-kvasir`, `mia-cityscapes`
4	Mix-up	`mia-liver`, `mia-kvasir`, `mia-cityscapes`
5	Min-max	`mia-liver`, `mia-kvasir`
6	DP	`mia-liver`
7	Knowledge distiallation	`mia-liver`

References

[1] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. “The cityscapes dataset for semantic urban scene understanding.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 3213–3223.

[2] A. L. Simpson, M. Antonelli, S. Bakas, M. Bilello, K. Farahani, B. Van Ginneken, A. Kopp-Schneider, B. A. Landman, G. Litjens, B. Menze, et al. “A large annotated medical image dataset for the development and evaluation of segmentation algorithms.” In: arXiv preprint arXiv:1902.09063 (2019).

[3] K. Pogorelov, K. R. Randel, C. Griwodz, S. L. Eskeland, T. de Lange, D. Johansen, C. Spampinato, D.-T. Dang-Nguyen, M. Lux, P. T. Schmidt, M. Riegler, and P. Halvorsen. “KVASIR: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection.” In: Proceedings of the 8th ACM on Multimedia Systems Conference. MMSys’17. Taipei, Taiwan: ACM, 2017, pp. 164–169. isbn: 978-1-4503-5002-0. doi: 10.1145/3083187. 3083212.