Home

Awesome

šŸ£ Please follow me for new updates https://twitter.com/camenduru <br /> šŸ”„ Please join our discord server https://discord.gg/k5BwmmvJJU <br /> šŸ„³ Please join my patreon community https://patreon.com/camenduru <br />

šŸ¦’ Colab

ColabType
Open In Colablongscope_text_to_video_colab polyware-ai/longscope <br /> (Thanks to @polyware_ai ā¤ for the model)
Open In Colabzeroscope_v2_xl_text_to_video_colab cerspense/zeroscope_v2_XL <br /> (Thanks to @cerspense ā¤ for the model)
Open In Colabzeroscope_v2_576w_text_to_video_colab cerspense/zeroscope_v2_576w <br /> (Thanks to @cerspense ā¤ for the model)
Open In Colabpotat1_exp1_text_to_video_colab (Thanks to lambda labs ā¤ for the gpu)
Open In Colabzeroscope_v2_dark_text_to_video_colab cerspense/zeroscope_v2_dark_30x448x256 <br /> (Thanks to @cerspense ā¤ for the model)
Open In Colabzeroscope_v2_30x448x256_text_to_video_colab cerspense/zeroscope_v2_30x448x256 <br /> (Thanks to @cerspense ā¤ for the model)
Open In Colabzeroscope_v1_1_320s_text_to_video_colab cerspense/zeroscope_v1-1_320s <br /> (Thanks to @cerspense ā¤ for the model)
Open In Colabpotat1_text_to_video_colab camenduru/potat1 (Thanks to lambda labs ā¤ for the gpu)
Open In Colabzeroscope_v1_320s_text_to_video_colab cerspense/zeroscope_v1_320s <br /> (Thanks to @cerspense ā¤ for the model)
Open In Colabms_1_7b_text_to_video_colab damo-vilab/text-to-video-ms-1.7b <br /> (Thanks to damo ā¤ damo-vilab ā¤ for the model)
Open In Colabanimov_512x_text_to_video_colab strangeman3107/animov-512x <br /> (Thanks to @bRuefiRe ā¤ for the model)
Open In Colabanimov_0_1_1_text_to_video_colab strangeman3107/animov-0.1.1 <br /> (Thanks to @bRuefiRe ā¤ for the model)
Open In Colabanimov_0_1_text_to_video_colab strangeman3107/animov-0.1.1 <br /> (Thanks to @bRuefiRe ā¤ for the model)
Open In ColabWatermark Remover
Open In Colabtext_to_video_synthesis <br /> (Thanks to damo ā¤ damo-vilab ā¤ for the model)
Open In Colabtext_to_video_synthesis_diffusers <br /> (Thanks to damo ā¤ damo-vilab ā¤ for the model)
<!-- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/camenduru/text-to-video-synthesis-colab/blob/main/zeroscope_video_webui_colab.ipynb) | zeroscope_video_webui_colab [cerspense/zeroscope_v2_576w](https://huggingface.co/cerspense/zeroscope_v2_576w) (576x320 txt2vid) <br /> [cerspense/zeroscope_v2_30x448x256](https://huggingface.co/cerspense/zeroscope_v2_30x448x256) (448x256 txt2vid) <br /> [cerspense/zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) <br /> (1024x576 vid2vid denoise strength between 0.66 and 0.85) <br /> (Thanks to [@cerspense](https://twitter.com/cerspense) ā¤ for the model) <br /> (Thanks to [@kabachuha](https://github.com/kabachuha) ā¤ for the [text2video webui extension](https://github.com/deforum-art/sd-webui-text2video)) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/camenduru/text-to-video-synthesis-colab/blob/main/zeroscope_v2_576w_video_webui_colab.ipynb) | zeroscope_v2_576w_video_webui_colab [cerspense/zeroscope_v2_576w](https://huggingface.co/cerspense/zeroscope_v2_576w) <br /> (576x320 txt2vid) <br /> (Thanks to [@cerspense](https://twitter.com/cerspense) ā¤ for the model) <br /> (Thanks to [@kabachuha](https://github.com/kabachuha) ā¤ for the [text2video webui extension](https://github.com/deforum-art/sd-webui-text2video)) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/camenduru/text-to-video-synthesis-colab/blob/main/zeroscope_upscale_video_webui_colab.ipynb) | zeroscope_upscale_video_webui_colab [cerspense/zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) <br /> (1024x576 vid2vid denoise strength between 0.66 and 0.85) <br /> (Thanks to [@cerspense](https://twitter.com/cerspense) ā¤ for the model) <br /> (Thanks to [@kabachuha](https://github.com/kabachuha) ā¤ for the [text2video webui extension](https://github.com/deforum-art/sd-webui-text2video)) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/camenduru/text-to-video-synthesis-colab/blob/main/potat1_video_webui_colab.ipynb) | potat1_video_webui_colab (1024x576 txt2vid) (Thanks to [lambda labs](https://lambdalabs.com/) ā¤ for the gpu) -->

šŸ¦’ Community Colab

ColabFunctionAuthorReadme
Open In Colabpotat1_text_to_video_colab_ALM (Thanks to snydjos ā¤)snydjos AILostMediaReadme

Tutorials

https://www.youtube.com/watch?v=WHcjD-uraUs (for zeroscope_video_webui_colab and zeroscope_upscale_video_webui_colab) <br /> https://www.youtube.com/watch?v=E45AO1w9gRU (for Potat 1ļøāƒ£ and Zeroscope) <br /> https://www.youtube.com/watch?v=b8D4am73e6I <br />

Main Repo

https://www.modelscope.cn/models/damo/text-to-video-synthesis/summary <br /> https://github.com/modelscope/modelscope <br /> https://github.com/huggingface/diffusers <br />

Paper

https://arxiv.org/abs/2308.06571

Models License

Apache License 2.0

Examples

Potat 1ļøāƒ£

https://github.com/camenduru/text-to-video-synthesis-colab/assets/54370274/604a8817-5ad3-421a-9240-e5d3b195074d

https://github.com/camenduru/text-to-video-synthesis-colab/assets/54370274/842da028-df43-4dba-bb14-a41d66edeb4e

https://github.com/camenduru/text-to-video-synthesis-colab/assets/54370274/c5201c8a-2815-4533-9474-1e312c564f4e

https://github.com/camenduru/text-to-video-synthesis-colab/assets/54370274/281d1a9e-b930-4e8a-9554-5203e15b4425

https://github.com/camenduru/text-to-video-synthesis-colab/assets/54370274/0bb74810-13bf-4e20-950f-b569a60ea5c8

https://github.com/camenduru/text-to-video-synthesis-colab/assets/54370274/1025f302-9fa1-4bd3-8996-1da94697151e

Base Model

<table><tbody><tr><td><center> A giraffe underneath a microwave. <br> <img src="https://user-images.githubusercontent.com/54370274/226195676-edd1b5da-906c-445e-b6a5-612a4dbfb1fe.gif" alt="A giraffe underneath a microwave." style="width: 300px;"> </center></td><td><center> A goldendoodle playing in a park by a lake. <br> <img src="https://user-images.githubusercontent.com/54370274/226195681-f54e38c2-1936-4153-b145-f238853a4df0.gif" alt="A goldendoodle playing in a park by a lake." style="width: 300px;"> </center></td><td><center> A panda bear driving a car. <br> <img src="https://user-images.githubusercontent.com/54370274/226195685-e188e342-5c6d-4e68-ab3f-32e2d7d30e34.gif" alt="A panda bear driving a car." style="width: 300px;"> </center></td></tr><tr><td><center> A teddy bear running in New York City. <br> <img src="https://user-images.githubusercontent.com/54370274/226195689-318e0e5e-ee14-4443-84a0-a3c8b07b8aed.gif" alt="A teddy bear running in New York City." style="width: 300px;"> </center></td><td><center> Drone flythrough of a fast food restaurant <br>on a dystopian alien planet. <br> <img src="https://user-images.githubusercontent.com/54370274/226195692-0853b49a-9cd5-4f9b-84e6-2288632ca2f7.gif" alt="Drone flythrough of a fast food restaurant on a dystopian alien planet." style="width: 300px;"> </center></td><td><center> A dog wearing a Superhero outfit with red cape <br>flying through the sky. <br> <img src="https://user-images.githubusercontent.com/54370274/226195699-14b16290-15e7-4577-aaae-ea16c15f44c3.gif" alt="A dog wearing a Superhero outfit with red cape flying through the sky." style="width: 300px;"> </center></td></tr><tr><td><center> Monkey learning to play the piano. <br> <img src="https://user-images.githubusercontent.com/54370274/226195867-f6b079ff-ee1a-4dea-928c-dbf28d4a656e.gif" alt="Monkey learning to play the piano." style="width: 300px;"> </center></td><td><center> A litter of puppies running through the yard. <br> <img src="https://user-images.githubusercontent.com/54370274/226195930-1fd957df-f403-4ae3-9a85-f4f954c82f5a.gif" alt="A litter of puppies running through the yard." style="width: 300px;"> </center></td><td><center> Robot dancing in times square. <br> <img src="https://user-images.githubusercontent.com/54370274/226209983-eae320fc-078e-4e62-9989-d97beb9477eb.gif" alt="Robot dancing in times square." style="width: 300px;"> </center></td></tr></tbody></table>

Related Colab

https://github.com/camenduru/text2video-zero-colab