Awesome
Consistency Regularization for Certified Robustness of Smoothed Classifiers (NeurIPS2020)
This repository contains code for the paper "Consistency Regularization for Certified Robustness of Smoothed Classifiers" by Jongheon Jeong and Jinwoo Shin.
Dependencies
conda create -n smoothing-consistency python=3
conda activate smoothing-consistency
# IMPORTANT: Please make sure `pytorch != 1.4.0`
# Currently, our code is not compatible to `pytorch == 1.4.0`;
# See more details at `https://github.com/pytorch/pytorch/issues/32395`.
# Below is for linux, with CUDA 10; see https://pytorch.org/ for the correct command for your system
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
conda install scipy pandas statsmodels matplotlib seaborn
pip install setGPU tensorboardX
Scripts
Training Scripts
The main script is train_consistency.py
; We also provide training scripts
to reproduce other baseline methods in train_*.py
, as listed in what follows:
File | Description |
---|---|
train_consistency.py | The main script; Consistency regularization |
train_cohen.py | Gaussian augmentation (Cohen et al., 2019) |
train_salman.py | SmoothAdv (Salman et al., 2019) |
train_stab.py | Stability training (Li et al., 2019) |
train_macer.py | MACER (Zhai et al., 2020) |
The sample scripts below demonstrate how to run train.py
with Gaussian training and SmoothAdv.
Notice that SmoothAdv training is enabled by simply passing --adv-training
option to the script.
One can modify CUDA_VISIBLE_DEVICES
to further specify GPU number(s) to work on.
# Consistency regularization (lbd=5) with Gaussian augmentation (Cohen et al., 2019)
CUDA_VISIBLE_DEVICES=0 python code/train_consistency.py mnist lenet --lr 0.01 --lr_step_size 30 --epochs 90 --noise 1.00 \
--num-noise-vec 2 --lbd 5
# Consistency regularization (lbd=1) with SmoothAdv (Salman et al., 2019)
CUDA_VISIBLE_DEVICES=0 python code/train_consistency.py mnist lenet --lr 0.01 --lr_step_size 30 --epochs 90 --noise 1.00 \
--num-noise-vec 2 --lbd 1 --adv-training --epsilon 255 --num-steps 2 --warmup 10
For a more detailed instruction to reproduce our experiments, see EXPERIMENTS.MD
.
Testing Scripts
All the testing scripts is originally from https://github.com/locuslab/smoothing:
- The script certify.py certifies the robustness of a smoothed classifier. For example,
python code/certify.py mnist model_output_dir/checkpoint.pth.tar 0.50 certification_output --alpha 0.001 --N0 100 --N 100000
will load the base classifier saved at model_output_dir/checkpoint.pth.tar
, smooth it using noise level σ=0.50,
and certify the MNIST test set with parameters N0=100
, N=100000
, and alpha=0.001
.
- The script predict.py makes predictions using a smoothed classifier. For example,
python code/predict.py mnist model_output_dir/checkpoint.pth.tar 0.50 prediction_outupt --alpha 0.001 --N 1000
will load the base classifier saved at model_output_dir/checkpoint.pth.tar
, smooth it using noise level σ=0.50,
and classify the MNIST test set with parameters N=1000
and alpha=0.001
.
- The script analyze.py contains some useful classes and functions to analyze the result data from certify.py or predict.py.