Awesome

ODSCAN: Backdoor Scanning for Object Detection Models

Table of Contents

• Table of Contents
  • Overview
  • Code Architecutre
  • Environments
  • Requirement
  • Train an Object Detection Model with Backdoor
    • Data-poisoning
    • Training
    • Evaluation
  • Backdoor Scanning by ODSCAN
  • Citation
  • Acknowledgement

Overview

• This is the official implementation for IEEE S&P 2024 paper "ODSCAN: Backdoor Scanning for Object Detection Models".
• [video] | [slides] | [poster]

Code Architecture

.
├── adaptived_nc_pixel        # Example baselines on TrojAI dataset
├── ckpt                      # Model checkpoints
├── data                      # Utilized data
│   ├── backgrounds           # Background images
│   ├── forgrounds            # Foreground images
│   ├── test                  # Test set of synthesis dataset
│   ├── train                 # Train set of synthesis dataset
│   ├── triggers              # Trigger patterns
│   └── fg_class_translation.json  # Image to class translation
├── dataset.py                # Dataset functions for training
├── poison_data.py            # Data-poisoning functions
├── scan_appearing            # Scanner against object appearing attacks
├── scan_misclassification    # Scanner against object misclassification attacks
├── train.py                  # Model training functions
└── utils.py                  # Utility functions

Environments

# Create python environment (optional)
conda env create -f environment.yml
source activate odscan

Requirement

• Please download the required data from the following link: Download Data
• Once the download is complete, unzip the file in the same directory.

Train an Object Detection Model with Backdoor

• We use a simplified TrojAI synthesis dataset as an illustrative example for examining backdoor attacks in object detection models.
• This dataset is located in the ./data/train and ./data/test folders, which contain five different traffic signs (./data/foregrounds) as five objects. The images are created by overlaying traffic signs onto street images (./data/backgrounds).
• We employ the SSD300 model as an example model architecture for object detection.
• The code currently supports object misclassification and object appearing attacks.

Data-poisoning

• Use the following command to generate a poisoned dataset for object misclassification attacks

# Stamp the trigger on images and modify their annotations
CUDA_VISIBLE_DEVICES="0" python train.py --phase data_poison --data_folder data_poison --trigger_filepath data/triggers/0.png --victim_class 0 --target_class 3 --trig_effect misclassification --location foreground

• After the data-poisoning process, the directory ./data_poison will include a new subfolder ./data_poison/misclassification_foreground_0_3 containing train and test subdirectories. These specify the poisoned samples for training and testing respectively.
• To generate a poisoned dataset for object appearing attacks, use the following command

# Stamp the trigger on images and modify their annotations
CUDA_VISIBLE_DEVICES="1" python train.py --trig_effect appearing --location background

Training

• Use the following command to train a poisoned model under object misclassification attacks

# Train a poisoned model
CUDA_VISIBLE_DEVICES="1" python train.py --phase train

• After training, the model will be saved in the ./ckpt folder under the filename ./ckpt/ssd_poison_misclassification_foreground_0_3.pt.
• You can also train a clean model using the following command and the model will be saved as ./ckpt/ssd_clean.pt.

# Train a clean model
CUDA_VISIBLE_DEVICES="0" python train.py --phase poison

Evaluation

• Use the following command to evaluate the trained model, calculating both the clean Mean Average Precision (mAP) and Attack Success Rate (ASR)

# Evaluate the model
CUDA_VISIBLE_DEVICES="0" python train.py --phase test

• You can also view visualizations of some model predictions in the ./visualize folder by the following command

# Visualization of predictions
CUDA_VISIBLE_DEVICES="0" python train.py --phase visual

Backdoor Scanning by ODSCAN

• Scan the model to detect object misclassification or appearing backdoor

# Detect object misclassification backdoor
CUDA_VISIBLE_DEVICES="0" python scan_misclassification.py --model_filepath ckpt/ssd_poison_misclassification_foreground_0_3.pt

# Detect object appearing backdoor
CUDA_VISIBLE_DEVICES="1" python scan_appearing.py --model_filepath ckpt/ssd_poison_appearing_background_0_3.pt

• The decision result will be displayed in your command line.
• You can also view the inverted triggers and predictions in the ./invert_misclassification and ./invert_appearing directories if you set verbose to 1.

Citation

Please cite our paper if you find it useful for your research.😀

@inproceedings{cheng2024odscan,
    title={ODSCAN: Backdoor Scanning for Object Detection Models},
    author={Cheng, Siyuan and Shen, Guangyu and Tao, Guanhong and Zhang, Kaiyuan and Zhang, Zhuo and An, Shengwei and Xu, Xiangzhe and Liu, Yingqi and Ma, Shiqing and Zhang, Xiangyu},
    booktitle={2024 IEEE Symposium on Security and Privacy (SP)},
    pages={119--119},
    year={2024},
    organization={IEEE Computer Society}
}

Acknowledgement

• TrojAI Round-10
• TrojAI Round-13
• Django