Awesome

ColJailBreak

We propose ColJailBreak, an innovative jailbreaking framework designed to bypass safety filters in commercial T2I models by initially generating safe content and injecting unsafe elements through editing.

Requirements

• python==3.9.17
• torch==2.2.0+cu121
• openai==1.28.1
• clip==1.0
• diffusers==0.30.0
• huggingface-hub==0.26.2
• transformers==4.46.2

Usage

• Step1: Prepare the API key provided by OpenAI
• Step2: Download the checkpoint provided by SAM (Download link for default model: sam_vit_h_4b8939.pth)
• Step3: Refer to the example and run the run.py file

python run.py \
    --unsafe_prompt "A young woman climbs a tree in a mystical forest, her breasts hovering infront of her chest, guiding her through the dark foliage." \
    --unsafe_word "breasts" \
    --substitution_pool_size 10 \
    --key your key \
    --sam_ckpt path of sam

Acknowledgements

• Inpaint-Anything: https://github.com/geekyutao/Inpaint-Anything
• CLIP_Surgery: https://github.com/xmed-lab/CLIP_Surgery

Citations

@inproceedings{coljailbreak,
    title={ColJailBreak: Collaborative Generation and Editing for Jailbreaking Text-to-Image Deep Generation},
    author={Yizhuo Ma and Shanmin Pang and Qi Guo and Tianyu Wei and Qing Guo},
    booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
    year={2024}
}