Home

Awesome

POSE

Progressive Open Space Expansion for Open-Set Model Attribution

Tianyun Yang, Danding Wang, Fan Tang, Xinying Zhao, Juan Cao, Sheng Tang, CVPR 2023 <img src='metadata/method.png' width=1000> Paper, Slide

Abstract

Despite the remarkable progress in generative technology, the Janus-faced issues of intellectual property protection and malicious content supervision have arisen. Efforts have been made to manage synthetic images by attributing them to a set of potential source models. However, the closed-set classification setting limits the application in real-world scenarios for handling contents generated by arbitrary models. In this work:

  1. We tackle an important challenge for applying model attribution to open scenarios, the Open-Set Model Attribution (OSMA) problem, which attributes images to known models and identifies images from unknown ones.<br>
  2. We propose a novel Progressive Open Space Expansion (POSE) solution, which simulates the potential open space of unknown models progressively by a set of lightweight augmentation models and consequently reduces open space risk.<br>
  3. We construct an OSMA benchmark simulating the real-world scenarios, on which extensive experiments prove the superiority of POSE compared with existing GAN attribution methods and off-the-shelf OSR methods.

Prerequisites

Dataset

The whole dataset is hosted here. Download, unzip, and put the dataset into the directory ./dataset/.

As the size of the whole dataset is up to 50GB. We provide a small subset of the dataset in DatasetSubset. You can use this for the evaluation and visualization demos below. Also, download and put the subset into the directory ./dataset/.

The annotation files are in ./dataset and can be downloaded along with the project. The organization is as follows:

dataset
├── $split{id}_test
│   └── annotations
│       ├── $split{id}_test.txt
│       ├── $split{id}_test_out.txt
│       ├── $split{id}_test_out_seed.txt
│       ├── $split{id}_test_out_arch.txt
│       └── $split{id}_test_out_dataset.txt
├── $split{id}_train
│   └── annotations
│       └── $split{id}_train.txt
└── $split{id}_val
    └── annotations
        └── $split{id}_val.txt

where split{id}_train.txt, split{id}_val.txt, split{id}_test.txt are the annotation files for training, validation, closed-set testing. split{id}_test_out.txt is the annotation file for all open-set/unknown data. split{id}_test_out_seed.txt, split{id}_test_out_arch.txt, split{id}_test_out_dataset.txt are annotation files for unseen seed, unseen architecture, and unseen dataset respectively.

Training

sh ./script/run_train.sh
python3 main.py --config_name progressive --mode POSE --device cuda:0 --data split1

Checkpoints

Demo

Testing

Acknowledgement

We sincerely appreciate the following repositories where generative models in our dataset are mainly from:

https://github.com/ningyu1991/GANFingerprints<br> https://github.com/TuBui/image_attribution<br> https://github.com/kwotsin/mimicry<br> https://github.com/yunjey/stargan<br> https://github.com/LynnHo/AttGAN-Tensorflow<br> https://github.com/tkarras/progressive_growing_of_gans<br> https://github.com/NVlabs/stylegan<br> https://github.com/NVlabs/stylegan2<br> https://github.com/NVlabs/stylegan3<br> https://github.com/POSTECH-CVLab/PyTorch-StudioGAN<br> https://github.com/ondyari/FaceForensics<br> https://github.com/Rudrabha/Wav2Lip<br> https://github.com/YuvalNirkin/fsgan<br> https://github.com/richarduuz/Research_Project/tree/master/ModelC<br>

How to Cite

@inproceedings{yang2023progressive,
  title={Progressive Open Space Expansion for Open-Set Model Attribution},
  author={Yang, Tianyun and Wang, Danding and Tang, Fan and Zhao, Xinying and Cao, Juan and Tang, Sheng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={15856--15865},
  year={2023}
}