Awesome

T-SEA: Transfer-based Self-Ensemble Attack on Object Detection(CVPR 2023)

English | 中文

Paper | Hao Huang*, Ziyan Chen*, Huanran Chen*, Yongtao Wang, Kevin Zhang

(*Equal contribution)

An official implementation of T-SEA, and also a framework provided to achieve universal (cross model&instance) patch-based adversarial attack.

If T-SEA is helpful for your work, please help star this repo~ Thanks! :-D

Update

• 2023.02.02 - v1.2 An anchor-free detector, CenterNet, is now supported.
• 2022.11.22 - v1.1 Fix bugs for running train & eval scripts.
• 2022.11.18 - v1.0 This repo is created.

Install

Environment

conda create -n tsea python=3.7
conda activate tsea
pip install -r requirements.txt

Please refer to PyTorch Docs to install torch and torchvision for better compatibility.

Data

See more details in Docs.

Run

Patch Zoo - See more results in GoogleDrive | BaiduCloud.

Faster RCNN	SSD	Yolo V5	Yolo V3	CenterNet

Evaluation

The evaluation metrics of the Mean Average Precision(mAP) is provided.

# You can run the demo script directly:
bash ./scripts/eval.sh 0 # gpu id

# To run the full command in the root proj dir:
python evaluate.py \
-p ./results/v5-demo.png \
-cfg ./configs/eval/coco80.yaml \
-lp ./data/INRIAPerson/Test/labels \
-dr ./data/INRIAPerson/Test/pos \
-s ./data/test \
-e 0 # attack class id

# for torch-models(coco91): replace -cfg with ./configs/eval/coco91.yaml

# For detailed supports of the arguments:
python evaluate.py -h

Training

# You can run the demo script directly:
bash ./scripts/train.sh 0 -np
# args: 0 gpu-id, -np new tensorboard process

# Or run the full command:
python train_optim.py -np \
-cfg=demo.yaml \
-s=./results/demo \
-n=v5-combine-demo # patch name & tensorboard name

# For detailed supports of the arguments:
python train_optim.py -h

The default save path of tensorboard logs is runs/.

Modify the config .yaml files for custom settings, see details in README.

Framework Overview

We provide a main pipeline to craft a universal adversarial patch to achieve cross-model & cross-instance attack on detectors, and support evaluations on given data & models. See more details in README.

Acknowledgements

• AdvPatch - Paper | Source Code

Citation

@inproceedings{huang2023t,
  title={T-sea: Transfer-based self-ensemble attack on object detection},
  author={Huang, Hao and Chen, Ziyan and Chen, Huanran and Wang, Yongtao and Zhang, Kevin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={20514--20523},
  year={2023}
}

Contact Us

If you have any problem about this work, please feel free to reach us out at huanghao@stu.pku.edu.cn.

License

The project is only free for academic research purposes, but needs authorization forcommerce. For commerce permission, please contact wyt@pku.edu.cn.