Home

Awesome

Diffusion Models for Adversarial Purification

<p align="center"> <img width="460" height="300" src="./assets/teaser_v7.jpeg"> </p>

Official PyTorch implementation of the ICML 2022 paper:<br> Diffusion Models for Adversarial Purification <br> Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, Anima Anandkumar<br> https://diffpure.github.io <br>

Abstract: Adversarial purification refers to a class of defense methods that remove adversarial perturbations using a generative model. These methods do not make assumptions on the form of attack and the classification model, and thus can defend pre-existing classifiers against unseen threats. However, their performance currently falls behind adversarial training methods. In this work, we propose <i>DiffPure</i> that uses diffusion models for adversarial purification: Given an adversarial example, we first diffuse it with a small amount of noise following a forward diffusion process, and then recover the clean image through a reverse generative process. To evaluate our method against strong adaptive attacks in an efficient and scalable way, we propose to use the adjoint method to compute full gradients of the reverse generative process. Extensive experiments on three image datasets including CIFAR-10, ImageNet and CelebA-HQ with three classifier architectures including ResNet, WideResNet and ViT demonstrate that our method achieves the state-of-the-art results, outperforming current adversarial training and adversarial purification methods, often by a large margin.

Requirements

Data and pre-trained models

Before running our code on ImageNet and CelebA-HQ, you have to first download these two datasets. For example, you can follow the instructions to download CelebA-HQ. Note that we use the LMDB format for ImageNet, so you may need to convert the ImageNet dataset to LMDB. There is no need to download CIFAR-10 separately.

Note that you have to put all the datasets in the datasest directory.

For the pre-trained diffusion models, you need to first download them from the following links:

For the pre-trained classifiers, most of them do not need to be downloaded separately, except for

Note that you have to put all the pretrained models in the pretrained directory.

Run experiments on CIFAR-10

AutoAttack Linf

cd run_scripts/cifar10
bash run_cifar_rand_inf.sh [seed_id] [data_id]  # WideResNet-28-10
bash run_cifar_rand_inf_70-16-dp.sh [seed_id] [data_id]  # WideResNet-70-16
bash run_cifar_rand_inf_rn50.sh [seed_id] [data_id]  # ResNet-50
cd run_scripts/cifar10
bash run_cifar_stand_inf.sh [seed_id] [data_id]  # WideResNet-28-10
bash run_cifar_stand_inf_70-16-dp.sh [seed_id] [data_id]  # WideResNet-70-16
bash run_cifar_stand_inf_rn50.sh [seed_id] [data_id]  # ResNet-50

Note that [seed_id] is used for getting error bars, and [data_id] is used for sampling a fixed set of images.

To reproduce the numbers in the paper, we recommend using three seeds (e.g., 121..123) for [seed_id] and eight seeds (e.g., 0..7) for [data_id], and averaging all the results across [seed_id] and [data_id], accordingly. To measure the worse-case defense performance of our method, the reported robust accuracy is the minimum robust accuracy of these two versions: Rand and Standard.

AutoAttack L2

cd run_scripts/cifar10
bash run_cifar_rand_L2.sh [seed_id] [data_id]  # WideResNet-28-10
bash run_cifar_rand_L2_70-16-dp.sh [seed_id] [data_id]  # WideResNet-70-16
bash run_cifar_rand_L2_rn50.sh [seed_id] [data_id]  # ResNet-50
cd run_scripts/cifar10
bash run_cifar_stand_L2.sh [seed_id] [data_id]  # WideResNet-28-10
bash run_cifar_stand_L2_70-16-dp.sh [seed_id] [data_id]  # WideResNet-70-16
bash run_cifar_stand_L2_rn50.sh [seed_id] [data_id]  # ResNet-50

Note that [seed_id] is used for getting error bars, and [data_id] is used for sampling a fixed set of images.

To reproduce the numbers in the paper, we recommend using three seeds (e.g., 121..123) for [seed_id] and eight seeds (e.g., 0..7) for [data_id], and averaging all the results across [seed_id] and [data_id], accordingly. To measure the worse-case defense performance of our method, the reported robust accuracy is the minimum robust accuracy of these two versions: Rand and Standard.

StAdv

cd run_scripts/cifar10
bash run_cifar_stadv_rn50.sh [seed_id] [data_id]  # ResNet-50

Note that [seed_id] is used for getting error bars, and [data_id] is used for sampling a fixed set of images.

To reproduce the numbers in the paper, we recommend using three seeds (e.g., 121..123) for [seed_id] and eight seeds (e.g., 0..7) for [data_id], and averaging all the results across [seed_id] and [data_id], accordingly.

BPDA+EOT

cd run_scripts/cifar10
bash run_cifar_bpda_eot.sh [seed_id] [data_id]  # WideResNet-28-10

Note that [seed_id] is used for getting error bars, and [data_id] is used for sampling a fixed set of images.

To reproduce the numbers in the paper, we recommend using three seeds (e.g., 121..123) for [seed_id] and five seeds (e.g., 0..4) for [data_id], and averaging all the results across [seed_id] and [data_id], accordingly.

Run experiments on ImageNet

AutoAttack Linf

cd run_scripts/imagenet
bash run_in_rand_inf.sh [seed_id] [data_id]  # ResNet-50
bash run_in_rand_inf_50-2.sh [seed_id] [data_id]  # WideResNet-50-2
bash run_in_rand_inf_deits.sh [seed_id] [data_id]  # DeiT-S
cd run_scripts/imagenet
bash run_in_stand_inf.sh [seed_id] [data_id]  # ResNet-50
bash run_in_stand_inf_50-2.sh [seed_id] [data_id]  # WideResNet-50-2
bash run_in_stand_inf_deits.sh [seed_id] [data_id]  # DeiT-S

Note that [seed_id] is used for getting error bars, and [data_id] is used for sampling a fixed set of images.

To reproduce the numbers in the paper, we recommend using three seeds (e.g., 121..123) for [seed_id] and 32 seeds (e.g., 0..31) for [data_id], and averaging all the results across [seed_id] and [data_id], accordingly. To measure the worse-case defense performance of our method, the reported robust accuracy is the minimum robust accuracy of these two versions: Rand and Standard.

Run experiments on CelebA-HQ

BPDA+EOT

cd run_scripts/celebahq
bash run_celebahq_bpda_glasses.sh [seed_id] [data_id]  # the glasses attribute
bash run_celebahq_bpda_smiling.sh [seed_id] [data_id]  # the smiling attribute

Note that [seed_id] is used for getting error bars, and [data_id] is used for sampling a fixed set of images.

To reproduce the numbers in the paper, we recommend using three seeds (e.g., 121..123) for [seed_id] and 64 seeds (e.g., 0..63) for [data_id], and averaging all the results across [seed_id] and [data_id], accordingly.

License

Please check the LICENSE file. This work may be used non-commercially, meaning for research or evaluation purposes only. For business inquiries, please contact researchinquiries@nvidia.com.

Citation

Please cite our paper, if you happen to use this codebase:

@inproceedings{nie2022DiffPure,
  title={Diffusion Models for Adversarial Purification},
  author={Nie, Weili and Guo, Brandon and Huang, Yujia and Xiao, Chaowei and Vahdat, Arash and Anandkumar, Anima},
  booktitle = {International Conference on Machine Learning (ICML)},
  year={2022}
}