Home

Awesome

AIGC IP Infringement

This repository is the code and data for "Evaluating and Mitigating IP Infringement in Visual Generative AI".

<!-- ## 🔬Environment See requirements.txt -->

🧰Generating Lure Prompts

For generating name-based lure prompts, we directly using the template "Generate an image of {Character Name}."

For generating description-based lure prompts, we use GPT-4 API to generate the lure prompts for Stable Diffusion families and Kandinsky. For example:

python generate_description_lure.py --char_fullname "Spider Man" --char spiderman50 --max_length 50

For the other models, we generate the description-based lure prompts using the website version of ChatGPT4 (in April, 2024):

<div align="center"> <img src=./image/example_gpt4website_generatelure.png width=85% /> </div>

Our generated lure prompts can be found at the folder "./generated_lure_prompts"

🧰Generating Images

For white-box model:

python generate_images_from_lurefile.py --arch sdxl --char spiderman50 \
--prompt_file_name ./generated_lure_prompts/max_50_tokens/spiderman50_generated_prompts.txt

For black-box model such as website-only model, we generate the images directly via the website:

ChatGPT4 Website:

<div align="center"> <img src=./image/example_gpt4website_spiderman_generateimage.png width=50% /> </div> <div align="center"> <img src=./image/example_gpt4website_supermario_generateimage.png width=50% /> </div> <div align="center"> <img src=./image/example_gpt4website_ironman_generateimage.png width=50% /> </div>

Midjourney:

<div align="center"> <img src=./image/example_midjourney_spiderman_generateimage.png width=85% /> </div> <div align="center"> <img src=./image/example_midjourney_supermario_generateimage.png width=85% /> </div> <div align="center"> <img src=./image/example_midjourney_ironman_generateimage.png width=85% /> </div>

DALL-E 3 Microsoft Designer Website:

<div align="center"> <img src=./image/example_dalle3microsoftdesigner_spiderman_generateimage.png width=85% /> </div> <div align="center"> <img src=./image/example_dalle3microsoftdesigner_supermario_generateimage.png width=85% /> </div> <div align="center"> <img src=./image/example_dalle3microsoftdesigner_ironman_generateimage.png width=85% /> </div>

Our generated images can be found at https://drive.google.com/drive/folders/1rh1CGywVFvdyy_SdiSwF8EoHoKnI7Dda?usp=sharing

⚙Mitigating IP Infringement

python generate_images_from_lurefile.py --arch sdxl --char spiderman50 \
--negative_prompt "Spider-Man" --prompt_file_name ./generated_lure_prompts/max_50_tokens/spiderman50_generated_prompts.txt

python -u cal_clipscore.py --images_dir_name ./spiderman50_sdxl_generated_imgs/ \
--prompt_file_name ./generated_lure_prompts/max_50_tokens/spiderman50_generated_prompts.txt

🤝Cite this work

You are encouraged to cite the following papers if you use the repo for academic research.

@article{wang2024evaluate,
  title={Evaluating and Mitigating IP Infringement in Visual Generative AI},
  author={Wang, Zhenting and Chen, Chen and Sehwag Vikash and Pan, Minzhou and Lyu, Lingjuan},
  journal={arXiv:2406.04662},
  year={2024}
}