Home

Awesome

<p align="center"> <img src= "scripts/icon.png" height=100> </p> <!-- ## <div align="center"><b>ConsistentID</b></div> --> <div align="center">

Theatergen: Character Management with LLM for Consistent Multi-turn Image Generation

[πŸ“„Paper]   [🚩Project Page] <br>

Teaser figure

</div>

Model Architecture

Teaser figure

Introduction

We propose Theatergen, a tuning-free method for consistent multi-turn image generation. The key idea is to utilize LLM for character management with layout and id and customize each character to avoid attention leakage. We further propose the CMIGBench for evaluating the consistency in multi-turn image generation.

TODO

:fire: News

Setup

πŸ”§ Requirements

To install requirements:

pip install -r requirements.txt

πŸš€ Generate

Generate with CMIGBench or replace with your own demo

python generate.py --task story --sd_version '1.5' --dataset_path CMIGBench

πŸ§ͺ Evaluate

Prepare the output in the following format

β”œβ”€β”€ output_dir
|   β”œβ”€β”€ dialogue 1
|      β”œβ”€β”€ turn1.png 
|      β”œβ”€β”€ turn2.png 
|      β”œβ”€β”€ turn3.png 
|      └── turn4.png 
|   β”œβ”€β”€ dialogue 2
|      ...

Evalutate the generated results of CMIGBench

python CMIGBench/eval/eval.py 
python CMIGBench/eval/eval_extra.py 

πŸ‘€ Contact Us

If you have any questions, please feel free to email us at howe4884@outlook.com.

🌟🌟🌟(I am an undergraduate student actively seeking opportunities for a Ph.D. program in 25 fall.)🌟🌟🌟

πŸ’‘Acknowledgement

Our work is based on stable diffusion, Grounded-SAM, T2I-Adapter, and IP-Adapter. We appreciate their outstanding contributions.

Citation

If you found this code helpful, please consider citing:

@article{cheng2024theatergen,
  title={TheaterGen: Character Management with LLM for Consistent Multi-turn Image Generation},
  author={Cheng, Junhao and Yin, Baiqiao and Cai, Kaixin and Huang, Minbin and Li, Hanhui and He, Yuxin and Lu, Xi and Li, Yue and Li, Yifei and Cheng, Yuhao and others},
  journal={arXiv preprint arXiv:2404.18919},
  year={2024}
}