Home

Awesome

VA-GAN Code

Method overview

Public tensorflow implementation for the Visual Feature Attribution using Wasserstein GANs method, which was accepted for presentation at CVPR 2018.

If you find this code helpful in your research please cite the following paper:

@InProceedings{baumgartner2018visual,
author = {Baumgartner, Christian F. and Koch, Lisa M. and Tezcan, Kerem Can and Ang, Jia Xi and Konukoglu, Ender},
title = {Visual Feature Attribution Using Wasserstein {GAN}s},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}

The paper can be found here.

There is also a pytorch version of the code implemented by Orobix, which can be found here. However, we would like to note that this third-party implementation has some issues, and we do not currently fully endorse using it.

Abstract: Attributing the pixels of an input image to a certain category is an important and well-studied problem in computer vision, with applications ranging from weakly supervised localisation to understanding hidden effects in the data. In recent years, approaches based on interpreting a previously trained neural network classifier have become the de facto state-of-the-art and are commonly used on medical as well as natural image datasets. In this paper, we discuss a limitation of these approaches which may lead to only a subset of the category specific features being detected. To address this problem we develop a novel feature attribution technique based on Wasserstein Generative Adversarial Networks (WGAN), which does not suffer from this limitation. We show that our proposed method performs substantially better than the state-of-the-art for visual attribution on a synthetic dataset and on real 3D neuroimaging data from patients with mild cognitive impairment (MCI) and Alzheimer’s disease (AD). For AD patients the method produces compellingly realistic disease effect maps which are very close to the observed effects.

Author of this code:

How to get the code to run:

You can also train a classifier (classifier_train.py) and then generate all the types of saliency maps discussed in the paper using the classifier_test_saliencies.py script.

How is the code structured:

Tricks for training and tuning

Bugs

Please use the issue tracking tool to report bugs or email directly to baumgartner@vision.ee.ethz.ch.

Results

Synthetic data

Synthetic Results

ADNI brain data

ADNI Results

Close up of observed disease effects vs VA-GAN prediction in 3 planes.

ADNI 3D closeup