Awesome

Latent HSJA

Unrestricted Black-box Adversarial Attack Using GAN with Limited Queries

• This repository provides official PyTorch implementations for <b>Latent-HSJA</b>.
• This work is presented at ECCV 2022 Workshop on Adversarial Robustness in the Real World.

Authors

• Dongbin Na, Sangwoo Ji, Jong Kim
• Pohang University of Science and Technology (POSTECH), Pohang, South Korea

Abstract

Adversarial examples are inputs intentionally generated for fooling a deep neural network. Recent studies have proposed unrestricted adversarial attacks that are not norm-constrained. However, the previous unrestricted attack methods still have limitations to fool real-world applications in a black-box setting. In this paper, we present a novel method for generating unrestricted adversarial examples using GAN where an attacker can only access the top-1 final decision of a classification model. Our method, Latent-HSJA, efficiently leverages the advantages of a decision-based attack in the latent space and successfully manipulates the latent vectors for fooling the classification model.

Demonstration

Source Codes

• HSJA
• LatentHSJA (Ours)

Datasets

• All datasets are based on the CelebAMask-HQ dataset.
  • The original dataset contains 6,217 identities.
  • The original dataset contains 30,000 face images.

1. Celeb-HQ Facial Identity Recognition Dataset

• This dataset is <b>curated</b> for the facial identity classification task.
• There are <b>307 identities</b> (celebrities).
  • Each identity has <b>15 or more</b> images.
• The dataset contains 5,478 images.
  • There are 4,263 train images.
  • There are 1,215 test images.
• Dataset download link (527MB)

2. Celeb-HQ Face Gender Recognition Dataset

• This dataset is <b>curated</b> for the face gender classification task.
• The dataset contains 30,000 images.
  • There are 23,999 train images.
  • There are 6,001 test images.
• The whole face images are divided into two classes.
  • There are 11,057 <b>male</b> images.
  • There are 18,943 <b>female</b> images.
• Dataset download link (2.54GB)
• Dataset (1,000 test images version) download link (97MB)

Classification Models to Attack

• The size of input images is <b>256 X 256</b> resolution (normalization 0.5).

Generate Datasets for Experiments with Encoding Networks

• For this work, we utilize the pixel2style2pixel (pSp) encoder network.
  • E(ꞏ) denotes the pSp encoding method that maps an image into a latent vector.
  • G(ꞏ) is the StyleGAN2 model that generates an image given a latent vector.
  • F(ꞏ) is a classification model that returns a predicted label.
• We can validate the performance of a encoding method given a dataset that contains (image x, label y) pairs.
  • <b>Consistency accuracy</b> denotes the ratio of test images such that F(G(E(x))) = F(x) over all test images.
  • <b>Correctly consistency accuracy</b> denotes the ratio of test images such that F(G(E(x))) = F(x) and F(x) = y over all test images.
• We can generate a dataset in which images are <b>correctly consistent</b>.

Training the Original Classification Models

• CelebA HQ Face Identity and Attributes Recognition using PyTorch

Citation

If this work can be useful for your research, please cite our paper: