Home

Awesome

Prompt Fusion

Prompt Fusion is an auto1111 webui extension that adds more possibilities to the native prompt syntax. Among other additions, it allows to interpolate between the embeddings of different prompts, continuously:

# linear prompt interpolation
[night light:magical forest: 5, 15]

# catmull-rom curve prompt interpolation
[night light:magical forest:norvegian territory: 5, 15, 25:catmull]

# alternation interpolation
[ufo|a strange sight:0.5]

# linear attention interpolation
(fire extinguisher: 1.0, 2.0)

# prompt-editing-aware attention interpolation
[(fire extinguisher: 1.0, 2.0)::5]

# weighted sum of conditions
[space station : kitchen mixer :: mean]

# define functions and variables to simplify repeating patterns and use a consistent structure
$prompt($style, $quality, $character, $background) = (
    A detailed picture in the style of $style,
    $quality,
    $character lying back,
    $background in the background
:1)

Features

The prompt interpolation feature is similar to Prompt Travel, which allows to create videos of images generated by navigating the latent space iteratively. Unlike Prompt Travel however, instead of generating multiple images, Prompt Fusion allows you to travel during the sampling process of a single image. Also, instead of interpolating the latent space, it uses the embedding space to determine intermediate embeddings.

Prompt interpolation is also similar to Prompt Blending. The main difference is that this extension calculates a new embedding for every step, as opposed to calculating it once and using that same one embedding for all the steps.

The attention interpolation feature is similar to Shift Attention, which allows to generate multiple images with slight variations in the attention given to certain parts of the prompt. Unlike Shift Attention, instead of generating multiple images, Prompt Fusion allows to shift the attention of certain parts of a prompt during the sampling process of a single image.

Usage

Examples

1. Influencing the beginning of the sampling process

Interpolate linearly (by default) from lion (step 0) to bird (step 8) to girl (step 11), and stay at girl for the rest of the sampling steps:

[lion:bird:girl: , 7, 10]

curve1

2. Influencing the middle of the sampling process

Interpolate using a bezier curve from fireball monster (step 0) to dragon monster (step 12, because 30 steps * 0.4 = step 12), while using seawater monster as an intermediate control point to steer the curve away during interpolation and to get creative results:

[fireball:seawater:dragon: , .1, .4:bezier] monster

curve2

Webui supported releases

The following webui releases are officially supported:

Installation

  1. Visit the Extensions tab of Automatic's WebUI.
  2. Visit the Available subtab.
  3. Look for Prompt Fusion.
  4. Press the Install button.
  5. Wait for the webui to finish downloading the extension.
  6. Visit the Installed subtab.
  7. click on Apply and restart UI.

Alternatively, instead of steps 6 and 7, you can restart the webui completely.

Related Projects