Awesome

Out-of-Bounding-Box Triggers: A Stealthy Approach to Cheat Object Detector

Official code release for Out-of-Bounding-Box Triggers: A Stealthy Approach to Cheat Object Detector.

<a href="https://github.com/linToTao/Out-of-bbox-attack">Paper</a> | <a href="https://github.com/linToTao/Out-of-bbox-attack">Code</a> <img src='static/out-of-bbox-attack.png' width='1000'/>

Abstract: In recent years, the study of adversarial robustness in object detection systems, particularly those based on deep neural networks (DNNs), has become a pivotal area of research. Traditional physical attacks targeting object detectors, such as adversarial patches and texture manipulations, directly manipulate the surface of the object. While these methods are effective, their overt manipulation of objects may draw attention in real-world applications. To address this, this paper introduces a more subtle approach: an inconspicuous adversarial trigger that operates outside the bounding boxes, rendering the object undetectable to the model. We further enhance this approach by proposing the Feature Guidance (FG) technique and the Universal Auto-PGD (UAPGD) optimization strategy for crafting high-quality triggers. The effectiveness of our method is validated through extensive empirical testing, demonstrating its high performance in both digital and physical environments.

Getting Started

Environmental Setups

git clone https://github.com/linToTao/Out-of-bbox-attack
cd Out-of-bbox-attack
conda env create -f environment.yml -n OOBA
conda activate OOBA

Model and Dataset Download

We can download 2017 train images, 2017 val images and their annotations of COCO2017 from the official website. Please create a new folder named "dataset", download and unzip the two datasets into the folder. And then we can process the data.
```
mkdir dataset/COCO
cd dataset/COCO
# download and move the zip files into the folder
```

download YOLOv5m weight

mkdir ./yolov5/weight/
cd ./yolov5/weight/
wget https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5m.pt

download YOLOv3

bash ./PyTorchYOLOv3/weights/download_weights.sh

Preparation before training

process data
```
python ./dataProcessing.py
```

save feature maps that FG method need

python ./saveFeatureMap.py --model_name yolov3
python ./saveFeatureMap.py --model_name yolov5

Finally the dateset file structure is organized as:

Out-of-bbox-attack
├── dataset
│   ├── COCO
│   |   ├── train2017
│   |   ├── val2017
│   |   ├── annotations
│   ├── coco
│   |   ├── train_stop_images
│   |   ├── train_stop_labels
│   |   ├── test_stop_images
│   |   ├── test_stop_labels
│   |   ├── train_stop_images_withGrayMask 
│   |   ├── train_stop_feature-yolov3
│   |   ├── train_stop_feature-yolov5
└── other codes...

Train

python ./train.py

evaluation

python ./evaluation.py

The parser is not used yet, and it will be added soon.