Home

Awesome

FourierHeatmap (latest release: v0.2.0)

CI codecov MIT License

This is an unofficial pytorch implementation of Fourier Heat Map which is proposed in the paper, A Fourier Perspective on Model Robustness in Computer Vision [Yin+, NeurIPS2019].

Fourier Heat Map allows to investigate the sensitivity of CNNs to high and low frequency corruptions via a perturbation analysis in the Fourier domain.

<img src="samples/FourierHeatmap-Teaser.png" height="300px">

News

Requirements

This library requires following as a pre-requisite.

Note that I run the code with Ubuntu 20, Pytorch 1.8.1, CUDA 11.0.

Installation

This repo uses poetry as a package manager. The following code will install all necessary libraries under .venv/.

$ git clone git@github.com:gatheluck/FourierHeatmap.git
$ cd FourierHeatmap
$ pip install poetry  # If you haven't installed poetry yet.
$ poetry install

Setup

Dataset

This codes expect datasets exist under data/. For example, if you want to evaluate Fourier Heat Map for ImageNet, please set up like follows:

FourierHeatmap
├── data
│	└── imagenet
│		├── train/
│		└── val/

Usage

Visualizing Fourier basis

The script fhmap/fourier/basis.py generates Fourier base functions. For example:

$ poetry run python fhmap/fourier/basis.py

will generate 31x31 2D Fourier basis and save as an image under outputs/basis.png. The generated image should be like follows.

<img src="samples/basis_31x31.png" height="300px">

Evaluating Fourier Heat Map

The script fhmap/apps/eval_fhmap.py eveluate Fourier Heat Map for a model. For example:

$ poetry run python fhmap/apps/eval_fhmap.py dataset=cifar10 arch=resnet56 weightpath=[PYTORCH_MODEL_WEIGHT_PATH] eps=4.0

will generate 31x31 Fourier Heat Map for ResNet56 on CIFAR-10 dataset and save as an image under outputs/eval_fhmap/. The generated image should be like follows.

<img src="samples/cifar10_resnet56_natural.png" height="300px">

Note that the L2 norm size (=eps) of Fourier basis use in original paper is following:

dataseteps
CIFAR-104.0
ImageNet15.7

Evaluating custom dataset and model

Evaluating your custom dataset

If you want to evaluate Fourier Heat Map on your custom dataset, please refer follwing instraction.

Now, you will be able to call your custom dataset like following.

$ poetry run python fhmap/apps/eval_fhmap.py dataset=yourcustomdataset arch=resnet50 weightpath=[PYTORCH_MODEL_WEIGHT_PATH] eps=4.0

Evaluating your custom architecture (model)

If you want to evaluate Fourier Heat Map on your custom architecture (model), please refer follwing instraction.

Now, you will be able to call your custom arch like following.

$ poetry run python fhmap/apps/eval_fhmap.py dataset=cifar10 arch=yourcustomarch weightpath=[PYTORCH_MODEL_WEIGHT_PATH] eps=4.0

Evaluating Fourier Heat Map through Docker

In order to use FourierHeatmap throgh docker, please install Docker with NVIDIA Container Toolkit beforehand. For detail, please refere official installation guide.

If nvidia-smi is able to run through docker like following, it is successfully installed.

$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

Tue Apr 27 06:46:09 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.102.04   Driver Version: 450.102.04   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 1080    Off  | 00000000:01:00.0  On |                  N/A |
| N/A   56C    P0    42W /  N/A |   1809MiB /  8114MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

We use environmental variables to specify the arguments. The variables that can be specified and their meanings are as follows:

nameoptionaldefaultdescription
HOST_DATADIRFalsePath to the directory where the dataset is located in the host.
HOST_OUTPUTSDIRFalsePath to the directory where the output will be located in the host.
HOST_WEIGHTDIRFalsePath to the directory where the pretrained wight is located in the host.
WEIGHTFILEFalseFile name of the pretrained wight.
ARCHTrueresnet56Name of the architecture.
BATCH_SIZETrue512Size of batch.
DATASETTruecifar10Name of dataset.
EPSTrue4.0L2 norm size of Fourier basis.
IGNORE_EDGE_SIZETrue0Size of the edge to ignore.
NUM_SAMPLESTrue-1Number of samples used from dataset. If -1, use all samples.
NVIDIA_VISIBLE_DEVICESTrue0Device number (or list of number) visible from CUDA.

For example:

$ export HOST_DATADIR=[DATASET_DIRECTORY_PATH]
$ export HOST_OUTPUTSDIR=[OUTPUTS_DIRECTORY_PATH]
$ export HOST_WEIGHTDIR=[WEIGHT_DIRECTORY_PATH]
$ export WEIGHTFILE=[PYTORCH_MODEL_FILE]
$ cd provision/docker
$ sudo -E docker-compose up  # -E option is needed to inherit environment variables.

will generate 31x31 Fourier Heat Map for ResNet56 on CIFAR-10 dataset and save as an image under OUTPUTS_DIRECTORY_PATH/eval_fhmap/.

References