Awesome
Free-Bloom
This repository is the official implementation of Free-Bloom.
Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator
Hanzhuo Huang∗ , Yufan Feng∗, Cheng Shi, Lan Xu, Jingyi Yu, Sibei Yang†
*Equal contribution; †Corresponding Author
Setup
Requirements
conda create -n fb python=3.10
conda activate fb
pip install -r requirements.txt
Installing xformers is highly recommended for more efficiency and speed on GPUs.
To enable xformers, set enable_xformers_memory_efficient_attention=True (default).
Usage
Generate
python main.py --config configs/flowers.yaml
Change the path of diffusion models to your own for the pretrained_model_path key in config yaml file.
Results
A Flower is blooming
<table class="center"> <tr> <td><img src="__assets__/flower_bloom.gif"></td> <td><img src="__assets__/Van_Gogh_flower.gif"></td> <td><img src="__assets__/yellow_flower.gif"></td> <td><img src="__assets__/long_video.gif"></td> </tr> </table>Volcano eruption
<table class="center"> <tr> <td><img src="__assets__/volcano_eruption1.gif"></td> <td><img src="__assets__/volcano_eruption2.gif"></td> <td><img src="__assets__/volcano_eruption3.gif"></td> <td><img src="__assets__/volcano_eruption4.gif"></td> </tr> </table>A rainbow is forming
<table class="center"> <tr> <td><img src="__assets__/rainbow_forming1.gif"></td> <td><img src="__assets__/rainbow_forming2.gif"></td> <td><img src="__assets__/rainbow_forming3.gif"></td> <td><img src="__assets__/rainbow_forming4.gif"></td> </tr> </table>Citation
@article{freebloom,
title={Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator},
author={Huang, Hanzhuo and Feng, Yufan and Shi, Cheng and Xu, Lan and Yu, Jingyi and Yang, Sibei},
journal={arXiv preprint arXiv:2309.14494},
year={2023}
}