Home

Awesome

InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser (ECCV2024)

Xing Cui, Zekun Li, Peipei Li, Huaibo Huang, Xuannan Liu, Zhaofeng He

Project page paper arXiv

🚩 New Features/Updates

Introduction

InstaStyle is a powerful method for stylized image generation. The core idea of InstaStyle is based on the finding that the inversion noise from a stylized reference image inherently carries the style signal. It can perform stylized image generation given only one reference image. Besides, InstaStyle can generate images in a combined sytle and supports adjusting the degree of two styles during combination, demonstrating its flexibility.

🔥🔥🔥 Main Features

Stylized image generation with a single reference image

InstaStyle excels at capturing style details including colors, textures, and brush strokes.

<p align="center"> <img src="./assets/teaser.png"> </p>

Combination of two styles

InstaStyle supports adjusting the degree of two styles during combination, dynamically ranging from one style to another.

<p align="center"> <img src="assets/style_combine.png"> </p>

🔧 Dependencies and Installation

# create an environment
conda create -n instastyle python==3.11.4
# activate the environment
conda activate instastyle
# install pytorch using pip
# for example: for Linux with CUDA 11.7
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2
# install other dependencies
pip install -r requirements.txt
# diffusers
pip install diffusers==0.21.0
# install xformers 
pip install -U xformers==0.0.21

⏬ Download Diffuser Models from Hugging Face (Optional)

The diffuser model can be downloaded automatically when the path is specified as "CompVis/stable-diffusion-v1-4", but we recommend that users download the model locally.

By running download.py, the stable diffusion model will be saved to "./stable-diffusion-v1-4"

python download.py

💻 Quick run

The experiment can be carried out on a NVIDIA GeForce RTX 3090 GPU with 24g memory.

We provide a quick start on gradio demo.

python app.py

Related Works

[1] <a href="https://arxiv.org/abs/2306.00983"> StyleDrop: Text-to-Image Generation in Any Style</a>

</p> <p> [2] <a href="https://arxiv.org/abs/2306.00763">Learning disentangled prompts for compositional image synthesis</a> </p>

🤗 Acknowledgements

We appreciate the foundational work done by Null-Text Inversion and CustomDiffusion. This readme file is modified from Dragon Diffusion and we thank them for their work.

BibTeX

@inproceedings{cui2024instastyle,
  title={InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser},
  author={Cui, Xing and Li, Zekun and Li, Pei Pei and Huang, Huaibo and Liu, Xuannan and He, Zhaofeng},
  booktitle={ECCV},
  year={2024}