Home

Awesome

Disco Diffusion

<a href="https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab"/></a>

A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations.

[to be updated with further info soon]

Contributing

This project uses a special conversion tool to convert the python files into notebooks for easier development.

What this means is you do not have to touch the notebook directly to make changes to it

the tool being used is called Colab-Convert

Changelog

v1 Oct 29th 2021 - Somnai

v1.1 Nov 13th 2021 - Somnai

v2 Update: Nov 22nd 2021 - Somnai

v3 Update: Dec 24th 2021 - Somnai

v4 Update: Jan 2022 - Somnai

v4.1 Update: Jan 14th 2022 - Somnai

v5 Update: Feb 20th 2022 - gandamu / Adam Letts

v5.1 Update: Mar 30th 2022 - zippy / Chris Allen and gandamu / Adam Letts

v5.1 Update: Apr 4th 2022 - MSFTserver aka HostsServer

v5.2 Update: Apr 10th 2022 - nin_artificial / Tom Mason

v5.3 Update: Jun 10th 2022 - nshepperd, huemin, cut_pow

v5.4 Update: Jun 14th 2022 - devdef / Alex Spirin, integrated into DD main by gandamu / Adam Letts

v5.5 Update: Jul 11th 2022 - Palmweaver / Chris Scalf, KaliYuga_ai, further integration by gandamu / Adam Letts

v5.6 Update: Jul 13th 2022 - Felipe3DArtist, integration by gandamu / Adam Letts

v5.7 Update: Dec 31st 2022 - Steffen Moelter (with minor colab-convert integration by gandamu)

Notebook Provenance

Original notebook by Katherine Crowson (https://github.com/crowsonkb, https://twitter.com/RiversHaveWings). It uses either OpenAI's 256x256 unconditional ImageNet or Katherine Crowson's fine-tuned 512x512 diffusion model (https://github.com/openai/guided-diffusion), together with CLIP (https://github.com/openai/CLIP) to connect text prompts with images.

Modified by Daniel Russell (https://github.com/russelldc, https://twitter.com/danielrussruss) to include (hopefully) optimal params for quick generations in 15-100 timesteps rather than 1000, as well as more robust augmentations.

Further improvements from Dango233 and nshepperd helped improve the quality of diffusion in general, and especially so for shorter runs like this notebook aims to achieve.

Vark added code to load in multiple Clip models at once, which all prompts are evaluated against, which may greatly improve accuracy.

The latest zoom, pan, rotation, and keyframes features were taken from Chigozie Nri's VQGAN Zoom Notebook (https://github.com/chigozienri, https://twitter.com/chigozienri)

Advanced DangoCutn Cutout method is also from Dango223.

--

Somnai (https://twitter.com/Somnai_dreams) added 2D Diffusion animation techniques, QoL improvements and various implementations of tech and techniques, mostly listed in the changelog below.

3D animation implementation added by Adam Letts (https://twitter.com/gandamu_ml) in collaboration with Somnai.

Turbo feature by Chris Allen (https://twitter.com/zippy731)

Improvements to ability to run on local systems, Windows support, and dependency installation by HostsServer (https://twitter.com/HostsServer)

VR Mode by Tom Mason (https://twitter.com/nin_artificial)

Horizontal and Vertical symmetry functionality by nshepperd. Symmetry transformation_steps by huemin (https://twitter.com/huemin_art). Symmetry integration into Disco Diffusion by Dmitrii Tochilkin (https://twitter.com/cut_pow).

Warp and custom model support by Alex Spirin (https://twitter.com/devdef).

Pixel Art Diffusion, Watercolor Diffusion, and Pulp SciFi Diffusion models from KaliYuga (https://twitter.com/KaliYuga_ai). Follow KaliYuga's Twitter for the latest models and for notebooks with specialized settings.

Integration of OpenCLIP models and initiation of integration of KaliYuga models by Palmweaver / Chris Scalf (https://twitter.com/ChrisScalf11)

Integrated portrait_generator_v001 from Felipe3DArtist (https://twitter.com/Felipe3DArtist)

MiDaS version tag fix by Steffen Moelter (a necessary fix for 3D animation mode after MiDaS v3.1 was introduced).