Home

Awesome

stable-diffusion-webui-prompt-travel

Travel between prompts in the latent space to make pseudo-animation, extension script for AUTOMATIC1111/stable-diffusion-webui.
<p align="left"> <a href="https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel/commits"><img alt="Last Commit" src="https://img.shields.io/github/last-commit/Kahsolt/stable-diffusion-webui-prompt-travel"></a> <a href="https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel/issues"><img alt="GitHub issues" src="https://img.shields.io/github/issues/Kahsolt/stable-diffusion-webui-prompt-travel"></a> <a href="https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel/stargazers"><img alt="GitHub stars" src="https://img.shields.io/github/stars/Kahsolt/stable-diffusion-webui-prompt-travel"></a> <a href="https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel/network"><img alt="GitHub forks" src="https://img.shields.io/github/forks/Kahsolt/stable-diffusion-webui-prompt-travel"></a> <img alt="Language" src="https://img.shields.io/github/languages/top/Kahsolt/stable-diffusion-webui-prompt-travel"> <img alt="License" src="https://img.shields.io/github/license/Kahsolt/stable-diffusion-webui-prompt-travel"> <br/> </p>

:stable-diffusion-webui-prompt-travel

Try interpolating on the hidden vectors of conditioning prompt to make seemingly-continuous image sequence, or let's say a pseudo-animation. 😀
Not only prompts! We also support non-prompt conditions, read => README_ext.md ~

⚠ 我们成立了插件反馈 QQ 群: 616795645 (赤狐屿)，欢迎出建议、意见、报告bug等 (w
⚠ We have a QQ chat group (616795645) now, any suggestions, discussions and bug reports are highly wellllcome!!

ℹ 实话不说，我想有可能通过这个来做ppt童话绘本<del>甚至本子</del>……
ℹ 聪明的用法：先手工盲搜两张好看的图 (只有prompt差异)，然后再尝试在其间 travel :lolipop:

⚠ Remeber to check "Always save all generated images" on in the settings tab, otherwise "upscaling" and "saving intermediate images" would not work. ⚠ 记得在设置页勾选 “总是保存所有生成的图片”，否则 上采样 与 保存中间图片 将无法工作。

Change Log

⚪ Compatibility

The latest version v3.1 is synced & tested with:

⚪ Features

⚪ Fixups

⚠ this script will NOT probably support the schedule syntax (i.e.: [prompt:prompt:number]), because interpolate on mutable conditions requires sampler level tracing which is hard to maintain :(
⚠ this script will NOT probably work together with hires.fix due to some inner conceptual/logical conflict of denoising_strength, you can alternatively perform batch-upscale then batch-img2img.

How it works?

⚪ Txt2Img

sampler \ genesisfixedsuccessiveembryo
Eular at2i-f-euler_at2i-s-euler_at2i-e-euler_a
DDIMt2i-f-ddimt2i-s-ddimt2i-e-ddim

⚪ Img2Img

sampler \ genesisfixedsuccessiveembryo
Eular ai2i-f-euler_ai2i-s-euler_ai2i-e-euler_a
DDIMi2i-f-ddimi2i-s-ddimi2i-e-ddim

post-processing pipeline (case i2i-f-ddim):

w/o. post-processingw/. post-processing
i2i-f-ddimi2i-f-ddim-pp

other stuff:

reference image for img2imgembryo image decoded <br/> case i2i-e-euler_a with embryo_step=8
i2i-refembryo

⚪ ControlNet support

prompt-travel with ControlNet (depth)controlnet-travel (depth)
ctrlnet-refctrlnet-depth

Example above run configure:

Prompt:
(((masterpiece))), highres, ((boy)), child, cat ears, white hair, red eyes, yellow bell, red cloak, barefoot, angel, [flying], egyptian
((masterpiece)), highres, ((girl)), loli, cat ears, light blue hair, red eyes, magical wand, barefoot, [running]

Negative prompt:
(((nsfw))), ugly,duplicate,morbid,mutilated,tranny,trans,trannsexual,mutation,deformed,long neck,bad anatomy,bad proportions,extra arms,extra legs, disfigured,more than 2 nipples,malformed,mutated,hermaphrodite,out of frame,extra limbs,missing arms,missing legs,poorly drawn hands,poorty drawn face,mutation,poorly drawn,long body,multiple breasts,cloned face,gross proportions, mutated hands,bad hands,bad feet,long neck,missing limb,malformed limbs,malformed hands,fused fingers,too many fingers,extra fingers,missing fingers,extra digit,fewer digits,mutated hands and fingers,lowres,text,error,cropped,worst quality,low quality,normal quality,jpeg artifacts,signature,watermark,username,blurry,text font ufemale focus, poorly drawn, deformed, poorly drawn face, (extra leg:1.3), (extra fingers:1.2),out of frame

Steps: 15
CFG scale: 7
Clip skip: 1
Seed: 114514
Size: 512 x 512
Model hash: animefull-final-pruned.ckpt
Hypernet: (this is my secret :)

Options

Installation

Easiest way to install it is to:

  1. Go to the "Extensions" tab in the webui, switch to the "Install from URL" tab
  2. Paste https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel.git into "URL for extension's git repository" and click install
  3. (Optional) You will need to restart the webui for dependencies to be installed or you won't be able to generate video files

Manual install:

  1. Copy this repo folder to the 'extensions' folder of https://github.com/AUTOMATIC1111/stable-diffusion-webui
  2. (Optional) Restart the webui

Post-processing pipeline

There are still two steps away from a really smooth and high resolution animation, namely image super-resolution & video frame interpolation (see third-party tools below).
⚠ Media data processing is intrinsic resource-exhausting, and it's also not webui's work or duty, hence we separated it out. 😃

setup once

⚪ auto install (Windows)

⚪ manually install (Windows/Linux/Mac)

ℹ Understand the tools folder layout first => tools/README.txt
ℹ If you indeed wanna put the tools elsewhere, modify paths in tools/link.cmd and run cd tools & link.cmd 😉

For Windows:

For Linux/Mac:

run each time

⚪ tkinter GUI (Windows/Linux/Mac)

manager

For Windows:

For Linux/Mac:

ℹ find usage help message in right click pop menu~

⚪ <del> cmd script (Windows) - deprecated </del>

Related Projects

⚪ extensions that inspired this repo

⚪ third-party tools

⚪ my other experimental toy extensions

by Armit 2022/11/10