Home

Awesome

πŸ–ΌοΈ ImagenHub

arXiv

Docs

contributors license GitHub Hits

ImagenHub: Standardizing the evaluation of conditional image generation models <br> ICLR 2024

<div align="center"> <img src="https://github.com/TIGER-AI-Lab/ImagenHub/blob/gh-pages/static/images/banner.png" width="40%"> </div>

ImagenHub is a one-stop library to standardize the inference and evaluation of all the conditional image generation models.

<div align="center"> <a href = "https://tiger-ai-lab.github.io/ImagenHub/">[🌐 Project Page]</a> <a href = "https://imagenhub.readthedocs.io/en/latest/index.html">[πŸ“˜ Documentation]</a> <a href = "https://arxiv.org/abs/2310.01596">[πŸ“„ Arxiv]</a> <a href = "https://huggingface.co/ImagenHub">[πŸ’Ύ Datasets]</a> <a href = "https://chromaica.github.io/#imagen-museum">[πŸ›οΈ ImagenMuseum]</a> <a href = "https://huggingface.co/spaces/TIGER-Lab/GenAI-Arena">[βš”οΈ GenAI-Arena]</a> </div>

πŸ“° News

πŸ“„ Table of Contents

πŸ› οΈ Installation πŸ”

Install from PyPI:

pip install imagen-hub

Or build from source:

git clone https://github.com/TIGER-AI-Lab/ImagenHub.git
cd ImagenHub
conda env create -f env_cfg/imagen_environment.yml
conda activate imagen
pip install -e .

For models like Dall-E, DreamEdit, and BLIPDiffusion, please see Extra Setup

For some models (Stable Diffusion, SDXL, CosXL, etc.), you need to login through huggingface-cli.

huggingface-cli login

πŸ‘¨β€πŸ« Get Started πŸ”

Benchmarking

To reproduce our experiment reported in the paper:

Example for text-guided image generation:

python3 benchmarking.py -cfg benchmark_cfg/ih_t2i.yml

Note that the expected output structure would be:

result_root_folder
└── experiment_basename_folder
    β”œβ”€β”€ input (If applicable)
    β”‚   └── image_1.jpg ...
    β”œβ”€β”€ model1
    β”‚   └── image_1.jpg ...
    β”œβ”€β”€ model2
    β”‚   └── image_1.jpg ...
    β”œβ”€β”€ ...

Then after running the experiment, you can run

python3 visualize.py --cfg benchmark_cfg/ih_t2i.yml

to produce a index.html file for visualization.

The file would look like something like this. We hosted our experiment results on Imagen Museum. <img src="https://i.imgur.com/0uOMhtT.png" width="512" />

Infering one model

import imagen_hub

model = imagen_hub.load("SDXL")
image = model.infer_one_image(prompt="people reading pictures in a museum, watercolor", seed=1)
image
<img src="https://i.imgur.com/ruU0BJ0.jpg" width="256" />

Running Metrics

from imagen_hub.metrics import MetricLPIPS
from imagen_hub.utils import load_image, save_pil_image, get_concat_pil_images

def evaluate_one(model, real_image, generated_image):
  score = model.evaluate(real_image, generated_image)
  print("====> Score : ", score)

image_I = load_image("https://chromaica.github.io/Museum/ImagenHub_Text-Guided_IE/input/sample_102724_1.jpg")
image_O = load_image("https://chromaica.github.io/Museum/ImagenHub_Text-Guided_IE/DiffEdit/sample_102724_1.jpg")
show_image = get_concat_pil_images([image_I, image_O], 'h')

model = MetricLPIPS()
evaluate_one(model, image_I, image_O) # ====> Score :  0.11225218325853348

show_image
<img src="https://i.imgur.com/af8CB4c.jpg" width="512" />

πŸ“˜ Documentation πŸ”

The tutorials and API documentation are hosted on imagenhub.readthedocs.io.

🧠 Philosophy πŸ”

By streamlining research and collaboration, ImageHub plays a pivotal role in propelling the field of Image Generation and Editing.

Implemented Models

We included more than 30 Models in image synthesis. See the full list here:

MethodVenueType
Stable Diffusion-Text-To-Image Generation
Stable Diffusion XLarXiv'23Text-To-Image Generation
DeepFloyd-IF-Text-To-Image Generation
OpenJourney-Text-To-Image Generation
Dall-E-Text-To-Image Generation
Kandinsky-Text-To-Image Generation
MagicBrusharXiv'23Text-guided Image Editing
InstructPix2PixCVPR'23Text-guided Image Editing
DiffEditICLR'23Text-guided Image Editing
ImagicCVPR'23Text-guided Image Editing
CycleDiffusionICCV'23Text-guided Image Editing
SDEditICLR'22Text-guided Image Editing
Prompt-to-PromptICLR'23Text-guided Image Editing
Text2LiveECCV'22Text-guided Image Editing
Pix2PixZeroSIGGRAPH'23Text-guided Image Editing
GLIDEICML'22Mask-guided Image Editing
Blended DiffusionCVPR'22Mask-guided Image Editing
Stable Diffusion Inpainting-Mask-guided Image Editing
Stable Diffusion XL Inpainting-Mask-guided Image Editing
TextualInversionICLR'23Subject-driven Image Generation
BLIP-DiffusionarXiv'23Subject-Driven Image Generation
DreamBooth(+ LoRA)CVPR'23Subject-Driven Image Generation
PhotoswaparXiv'23Subject-Driven Image Editing
DreamEditarXiv'23Subject-Driven Image Editing
Custom DiffusionCVPR'23Multi-Subject-Driven Generation
ControlNetarXiv'23Control-guided Image Generation
UniControlarXiv'23Control-guided Image Generation

Comprehensive Functionality

High quality software engineering standard.

πŸ™Œ Contributing πŸ”

For the Community

Community contributions are encouraged!

ImagenHub is still under development. More models and features are going to be added and we always welcome contributions to help make ImagenHub better. If you would like to contribute, please check out CONTRIBUTING.md.

We believe that everyone can contribute and make a difference. Whether it's writing code πŸ’», fixing bugs πŸ›, or simply sharing feedback πŸ’¬, your contributions are definitely welcome and appreciated πŸ™Œ

And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about:

For the Researchers:

πŸ–ŠοΈ Citation πŸ”

Please kindly cite our paper if you use our code, data, models or results:

@inproceedings{
ku2024imagenhub,
title={ImagenHub: Standardizing the evaluation of conditional image generation models},
author={Max Ku and Tianle Li and Kai Zhang and Yujie Lu and Xingyu Fu and Wenwen Zhuang and Wenhu Chen},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=OuV9ZrkQlc}
}
@article{ku2023imagenhub,
  title={ImagenHub: Standardizing the evaluation of conditional image generation models},
  author={Max Ku and Tianle Li and Kai Zhang and Yujie Lu and Xingyu Fu and Wenwen Zhuang and Wenhu Chen},
  journal={arXiv preprint arXiv:2310.01596},
  year={2023}
}

🀝 Acknowledgement πŸ”

Please refer to ACKNOWLEDGEMENTS.md

🎫 License πŸ”

This project is released under the License.

⭐ Star History πŸ”

Star History Chart