Home

Awesome

Overly Complicated Sampling

Experimental and mathematically unsound (but fun!) sampling for ComfyUI.

Status: In flux, may be useful but likely to change/break workflows frequently. Mainly for advanced users.

Feel free create a question in Discussions for usage help: OCS Q&A Discussion

Note for Flux users: Set cfg1_uncond_optimization: true in the model block for the main OCS Sampler as Flux does not use CFG. CFG++ and alt CFG++ features do not work with Flux.

Features

Credits

I can move code around but sampling math and creating samplers is far beyond my ability. I didn't write any of the original samplers:

This repo wouldn't be possible without building on the work of others. Thanks!

Usage

First, a note on the basic structure:

Basic nodes example

The sampler node connects to a group node. You can chain group nodes, however only one can match per step. Checking for a match starts at the group furthest from the sampler node. So if you have Group1 -> Group2 -> Group3 -> Sampler, the groups will be tried starting from Group1. Currently matching groups is only time based.

You will then connect a substeps node to the group. These can also be chained and like groups, execution starts with the node furthest from the group. I.E.: Substeps1 -> Substeps2 -> Substeps3 -> Group will start with Substeps1.

Most of these nodes have a text parameter input (YAML format - JSON is also valid YAML so you can use that instead if you prefer) and a parameter input. The parameter input can be used to specify stuff like custom noise types.

You may use filters and expressions in the text parameter input. See:

Integration

If you're going to use OCS, I strongly recommend also installing ComfyUI-bleh and ComfyUI-sonar as they increase the functionality a lot.

Nodes

OCS Sampler

The main sampler node, with an output suitable for connecting to a SamplerCustom. This node has builtin support for Restart sampling, if you are using Restart don't use the RestartSampler node.

You can connect a chain of OCS Group nodes to it and it will choose one per step (based on conditions like time).

Input Parameters

Text Parameters

Shown in YAML with default values.

<details> <summary>★★ Expand ★★</summary> 
# Noise scale. May not do anything currently.
s_noise: 1.0

# ETA (basically ancestralness). May not do anything currently.
eta: 1.0

# Reversible ETA (used for reversible samplers). May not do anything currently.
reta: 1.0

# Parameters related to restart sampling.
restart:
    # Scales the noise added by restart sampling.
    s_noise: 1.0
    # Immiscible block same as described below.
    immiscible:
        size: 0


# The noise block allows defining global noise sampling parameters.
noise:
    # You can disable this to allow GPU noise generation. I believe it only makes a difference for Brownian.
    cpu_noise: true

    # ComfyUI has a bug where if you disable add_noise in the sampler, no seed gets set. If you
    # are manually noising a sample and have add_noise turned off then you should enable this if
    # you want reproducible generations.
    set_seed: false

    # Global scale scale for generated noise
    scale: 1.0

    # Whether the generated noise should be normalized before use. Generally a good idea to leave enabled.
    normalize_noise: true

    # Dimensions to normalize over (when normalization is enabled). Negative values mean starting
    # from the end (i.e. -1 means the last dimension, -2 means the penultimate dimension).
    # Latents generally have these dimensions: batch, channels, height, width
    # The default of [-3, -2, -1] normalizes noise over the batch. You can try something like
    # [-2, -1] to normalize over the batch and channels. See: https://pytorch.org/docs/stable/generated/torch.std.html
    normalize_dims: [-3, -2, -1]

    # When caching, the batch size for chunks of noise to generate in advance. Generating a batch of noise
    # can be more efficient than generating on demand when using a high number of substeps (>10) per step.
    batch_size: 1

    # Whether to cache noise.
    caching: true

    # Interval (in full steps) to reset the cache. Brownian noise takes time into account so
    # if using Brownian with caching enabled you will generally want to reset each step.
    cache_reset_interval: 9999

    # Immiscible noise processing, see: https://arxiv.org/abs/2406.12303
    immiscible:
        # Batch size, 0 disables.
        size: 0

        # Reference mode, values can be one of:
        #   x: Uses the current latent as a reference.
        #   noise: Uses the current noise as a reference (x - denoised)
        #   denoised: Uses the model image prediction as a reference (factors in positive and negative prompts).
        #   uncond: The model unconditional prediction (negative prompt)
        #   cond: The model conditional prediction (positive prompt)
        # Advanced feature: Additionally you may enter a string of operations in the format:
        #   "x - denoised * 2 + cond" (just an example, not a recommended setting)
        # Possible operations: + - / * min max add sub div mul
        # Note: Each value and operation must be space delimited (i.e. "x-1" will not work).
        #       Also normal operator precedence does not apply here.
        ref: default

        # Batching mode, one of:
        #   batch: Matches vs batches. Immiscible mode is disabled if size < 2
        #   channel: Splits the batch into a list of channels and matches against those.
        #   row: Splits the batch into a list of rows and matches against those.
        #   column: Splits the batch into a list of columns and matches against those.
        #           Note: Requires reshaping both the noise and x, may be slow and consume
        #                 a lot of VRAM.
        batching: channel

        # Scale for reference latent. Can be negative.
        scale_ref: 1.0

        # Allows normalizing the reference. If this is a list, you can specify the dimensions to
        # normalize. See normalize_dims above and https://pytorch.org/docs/stable/generated/torch.std.html
        normalize_ref: false

        # The proportion of immiscible-ized noise.
        # You get (immiscible_noise * strength) + ((1.0 - strength) * normal_noise) - LERP.
        strength: 1.0

        # See: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html#scipy.optimize.linear_sum_assignment
        maximize: false

    filter: null


# Model calls can be cached. This is very experimental: I don't recommend using it
# unless you know what you're doing.
model:
    # When enabled, skips generating uncond when you have CFG set to 1. Disabled by
    # default as stuff like CFG++ won't work without uncond. Useful to enable for
    # models like Flux that don't actually use CFG.
    cfg1_uncond_optimization: false

    cache:
        # The cache size.
        size: 0

        # Threshold for model call caching. For example if you have size=3 and threshold=1
        # then model calls 1 through 3 will be cached, but model call 0 will not be (the first one).
        # Additional explanation: Some samplers call the model multiple times per step. For example,
        # Bogacki uses three model calls: 0, 1, 2
        threshold: 1

        # Maximum use count for cache items.
        max_use: 1000000

    filter:
        input: null
        denoised: null
        jdenoised: null
        cond: null
        uncond: null
</details><br/>

Any parameters you don't specify will use the defaults. For example if your text parameter block is:

noise:
    cpu_noise: false

Then the rest of the parameters will use the defaults shown above.

OCS Group

Defines a group of substeps.

Merging

When running multiple substeps per step, the results will combined based on the merge strategy. Possible strategies (in order of least weird to most weird):

Dynamic Groups: When merge_method is set to dynamic you must specify a dynamic block in the text parameters. The dynamic block may be either a string with the expression or a list of objects with an (optional) when key and a (required) expression key. The expression should return a dictionary of parameters you can set in the node (including both keys/values from the text parameters and widgets in the node). The first matching item will be used. Example:

# Use the simple merge method when step < 3, otherwise use overshoot
dynamic:
    - when: step < 3
      expression: dict(merge_method :> 'simple)
    - expression: dict(merge_method :> 'overshoot)

# You could also write it like this:
dynamic: |
    dict(merge_method :> (step < 3 ? 'simple : 'overshoot)

Node Parameters

Example:

Group time filter example

The left side group matches steps 0, 1, 2. The right side group matches all steps. This setup will use whatever substeps are connected to the first group for the first three steps and the second group will handle the rest.

Input Parameters

<!-- * `merge_sampler`: Value type: `OCS_SUBSTEPS`. Only used when `merge_method` is `sample` or `sample_uncached`. Allows defining the sampler used for merging substeps. -->

Text Parameters

Shown in YAML with default values.

<details> <summary>★★ Expand ★★</summary> 
# Noise scale. May not do anything currently.
s_noise: 1.0

# ETA (basically ancestralness). May not do anything currently.
eta: 1.0

# Reversible ETA (used for reversible samplers). May not do anything currently.
reta: 1.0

# Sets the type of preview used for sampling in this group. One of:
#   denoised: The default, shows the model prediction (takes positive and negative prompt into account).
#   cond: Shows the model cond prediction (basically the positive prompt).
#   uncond: Shows the model uncond prediction (basically the negative prompt).
#   raw: Shows the raw noisy latent input.
#   noisy: 10% of the noise + denoised.
#   diff: Multiplies the difference between cond and uncond.
preview_mode: denoised

# Expression.
when: null

# Interpolate the schedule by the specified factor. Only used by the overshoot merge method.
#: Example if factor 2 and steps [0,1,2] you'd get [0, 0.5, 1.0, 1.5, 2]
overshoot_expand_steps: 1

# Only used by the overshoot merge method currently.
restart:
    # Scales the noise added by restart sampling.
    s_noise: 1.0
    # Immiscible block same as described above.
    immiscible:
        size: 0

# Only used by the lookahead merge method currently.
lookahead:
    # Works like normal samplers, essentially. Disabled by default.
    eta: 0.0

    # Scales the noise added by lookahead sampling.
    s_noise: 1.0

    # Controls how much noise to remove in the prediction phase. Higher values will remove more noise.
    dt_factor: 1.0

    # Immiscible block same as described above.
    immiscible:
        size: 0

pre_filter: null

post_filter: null
</details>

OCS Substeps

Step Methods (Samplers)

In alphabetical order.

Sampler Feature Support

NameCostHistoryOrderReversibleCFG++
adapter?????
bogacki2
deis11-3 (1)
distancevariable
dpmpp_2m_sde11
dpmpp_2m11
dpmpp_2s2
dpmpp_3m_sde11-2 (2)
dpmpp_sde2
dynamic?????
euler_cycle1X
euler_dancing1
euler1X
heun2X
heun_1s11X
heunpp1-3X
ipndm_v11-3 (1)
ipndm11-3 (1)
res2
reversible_bogacki2X
reversible_heun2X
reversible_heun_1s11X
rk44
rkf455
rk41-4
solver_diffraxvariable
solver_torchdiffeqvariable
solver_torchodevariable
solver_torchsdevariable
trapezoidal2
trapezoidal_cycle2
ttm_jvp2

deis, ipndm* do not seem to work well with ancestralness, I recommend eta: 0.25 or disable it completely.

Solver Backend Samplers:

You will need to have the relevant Python package installed in your venv to use these. TDE cannot handle batches and each batch item will be evaluated separately. Using tode may be faster for batch sizes over 1.

ode_solver types for TDE: adaptive: dopri8, dopri5, bosh3, fehlberg2, adaptive_heun, fixed step: euler, midpoint, rk4, explicit_adams, implicit_adams

ode_solver types for TODE: adaptive only: dopri5, tsit5, heun. I haven't much luck with anything other than dopri5.

Note that adaptive solvers may be very slow. Think along the lines of 20-100 model calls per substep (or in other words, the equivalent for running that many euler steps). Tolerances only apply to adaptive solvers.

Cycle Samplers (euler_cycle, trapezoidal_cycle)

Basically a different approach to ancestral sampling. First a crash course on how sampling works:

Each step has an expected noise level, with the first step generally being pure noise and the end of the last step aiming to end with no noise remaining. Let's say the image on the current step is called x, calling the model with x gives us a prediction of what the image looks like with all noise removed (denoised), however the model is not capable of just removing all the noise in a single step: its prediction will be imprecise. x - denoised leaves us with just the noise (we subtract the prediction which theoretically has no noise from the noisy sample). This is a very simplified, but the idea is basically to add the noise back into denoised, but scaled so that it matches the amount of noise expected on the next step. denoised + noise * expected_noise_at_next_step.

When doing ancestral sampling, we actually overshoot expected noise for the next step and add less than that amount back to denoised. Then we generate some of our own noise and add it, scaled so that the result matches expected_noise_at_next_step. eta controls how the scale of the overshoot.

The difference with cycle is that instead of adding noise * expected_noise_at_next_step, we instead first add noise * (expected_noise_at_next_step * (1.0 - cycle_pct)) and then we generate noise and scale it to cycle_pct and add it too. Just for example, suppose cycle_pct is 0.2: we'll add 80% of the expected noise at the next step (1.0 - 0.2 == 0.8) and then generate the remaining 20% and add it in to meet the expected amount. I don't recommend setting cycle_pct to values over 0.5, especially if using "weird" noise types.

Dynamic Step Method: When step_method is set to dynamic you must specify a dynamic block in the text parameters. The dynamic block may be either a string with the expression or a list of objects with an (optional) when key and a (required) expression key. The first matching item will be used. The expression should return a dictionary of parameters you can set in the node (including both keys/values from the text parameters and widgets in the node). Example:

# Use the rk4 merge method when step < 3, otherwise use euler
dynamic:
    - when: step < 3
      expression: dict(step_method :> 'rk4)
    - expression: dict(step_method :> 'euler)

# You could also write it like this:
dynamic: |
    dict(step_method :> (step < 3 ? 'rk4 : 'euler)

Note: You can set all sampler parameters except substeps this way. Sampling will use the substeps value from the OCS Substeps node.

Node Parameters

Input Parameters

Text Parameters

Shown in YAML with default values.

<details> <summary>★★ Expand ★★</summary> 
# Scale for added noise.
s_noise: 1.0

# ETA (basically ancestralness).
eta: 1.0

# If the ETA calculation fails, it will retry with eta - eta_retry_increment until it either
# succeeds or eta becomes <= 0 (in which case ancestralness just gets disabled).
# In other words, you can set ETA as high as you want, set eta_retry_increment to something like 0.1 and
# it will just do whatever it takes to find an ETA that works.
eta_retry_increment: 0

# No effect unless both start and end are set. Will scale the eta value based on the
# percentage of sampling. In other words, eta*dyn_eta_start at the beginning,
# eta*dyn_eta_end at the end.
dyn_eta_start: null
dyn_eta_end: null

# alt CFG++ scale (see https://cfgpp-diffusion.github.io/)
# Based on the initial incorrect ComfyUI implementation, but it seems to
# produce decent results sometimes.
# Can also be set to a negative value (I don't recommend going lower than -0.5).
alt_cfgpp_scale: 0

# CFG++ (see https://cfgpp-diffusion.github.io/)
cfgpp: false

### Reversible Settings ###

# Reversible ETA (used for reversible samplers).
reta: 1.0
# Scale of the reversible correction. Can also be set to a negative value.
reversible_scale: 1.0
# No effect unless both start and end are set. Will scale the reta value based on the
# percentage of sampling. In other words, reta*dyn_reta_start at the beginning,
# reta*dyn_reta_end at the end.
dyn_reta_start: null
dyn_reta_end: null

pre_filter: null

post_filter: null

### ODE Sampler Settings ###

# Solver type.
de_solver: dopri5 # Example - varies based on solver sampler.
# Relative tolerance (log 10)
de_rtol: -1.5
# Absolute tolerance (log 10)
de_atol: -3.5
# Max model calls allowed to compute the solution. If the limit is exceeded, it is an error.
de_max_nfe: 1000
# Min sigma to sample to. If the current step start <= min sigma, then the sampler will run
# a Euler step. If the current step end <= min sigma then the slover will sample to the min
# sigma and then to a Euler step from min sigma for the rest.
de_min_sigma: 0.0292
# Hack that seems to help results by stretching the down sigma a bit. Set to 0 to disable.
de_fixup_hack: 0.025

# Used to split the step into sections. Useful for fixed step methods.
# Applies to: solver_torchode, solver_diffrax
de_split: 1

# Initial step size (as a percentage).
# Applies to: solver_torchode, solver_diffrax
de_initial_step: 0.25

# Coefficients for the step size PID controller.
# See https://en.wikipedia.org/wiki/Proportional%E2%80%93integral%E2%80%93derivative_controller
# These values seem okay with dopri5.
# Applies to: solver_tode, solver_diffrax
de_ctl_pcoeff: 0.3
de_ctl_icoeff: 0.9
de_ctl_dcoeff: 0.2

# Controls whether to compile the solver. May or may not work,
# also may or may not be a speed increase as the compiled solver is
# not cached between substeps.
# Applies to: solver_torchode
tode_compile: false

### torchsde solver specific parameters.
tsde_noise_type: "scalar"
tsde_sde_type: "stratonovich"
tsde_levy_area_approx: "none"
tsde_noise_channels: 1
tsde_g_multiplier: 0.05
tsde_g_reverse_time: true
tsde_g_derp_mode: false
tsde_batch_channels: true

### diffrax solver specific parameters.

# Turns on adaptive stepping. When enabled, de_split is not used.
# When disabled, it may be desirable to set de_split.
diffrax_adaptive: false
# Hack to make some solver methods work. May not be safe.
diffrax_fake_pure_callback: true
# Some diffrax methods don't allow adaptive stepping, enabling this
# makes them usable although it's less efficient (3x cost, 2x accuracy).
diffrax_half_solver: false
diffrax_batch_channels: false
# Some solvers require specific types of Levy area approximation.
# See: https://docs.kidger.site/diffrax/api/brownian/#levy-areas
diffrax_levy_area_approx: "brownian_increment"
# Some solvers may require manually specifying the error order.
diffrax_error_order: null
# Enables SDE mode (and SDE-specific solvers). May not be worth using.
diffrax_sde_mode: false
# Noise multiplier when SDE mode is enabled.
diffrax_g_multiplier: 0.0
# Only applies when time scaling is enabled. Reverses time.
diffrax_g_reverse_time: false
# Scales the g multiplier based on the current time.
diffrax_g_time_scaling: false
# Experimental option to flip the sign on the g multiplier when time >= half the step.
# i.e. if you'd get 1,2,3,4 as g values for the step, with this it would be 1,2,-3,-4.
diffrax_g_split_time_mode: false


### Other Sampler Specific Parameters ###

# Used for some samplers that use history from previous steps.
# List of samplers and default value below:
#   dpmpp_2m: 1
#   dpmpp_2m_sde: 1
#   dpmpp_3m_sde: 2
#   reversible_heun_1s: 1
#   ipndm: 1 (max 3)
#   ipndm_v: 1 (max 3)
#   deis: 1 (max 3)
history_limit: 999 # Varies based on sampler.

# Used for some samplers with variable order. List of samplers and default value below:
#   heunpp2: 3
#   rk_dynamic: 4 - Can also be set to 0 to try to dynamically adjust the order based on calculated
#                   error from the last step (but that may not work well).
max_order: 999 # Varies based on sampler.

# Used for dpmpp_2m. One of midpoint, heun
solver_type: "midpoint"

# Coefficients mode for DEIS. One of tab or rhoab.
deis_mode: "tab"

# Used for samplers with cycle in the name. Controls how much noise is cycled per step.
cycle_pct: 0.25

# Used for ttm_jvp. Supposed works better when ETA > 0
alternate_phi_2_calc: true

# Parameters for dancing samplers:
# Number of steps to leap ahead.
leap: 2
# ETA for dance steps
deta: 1.0
# dyn_deta works the same as dyn_eta/reta. See above.
dyn_deta_start: null
dyn_deta_end: null
# One of lerp, lerp_alt, deta
dyn_deta_mode: "lerp"
</details>

OCS Param and OCS MultiParam

Allows specifying parameter inputs that can't be expressed with YAML, such as custom noise.

Node Parameters

Input Parameters

Text Parameters

Allows specifying extra parameters. You may use this block to rename a key, for example if your key type was custom_noise you could enter:

rename: test

in the OCS Param node and:

custom_noise: test

in the node that was connected to the params to have it use the custom noise specifically named test.

OCS MultiParam

MultiParam is the same as Param except it has multiple optional inputs like key_1, key_2, etc.

Text Parameters

Same as OCS Param (see above), however if set you should use an object with a key corresponding to the index of the param. For example, if you wanted to rename key_1 and key_2 you would do something like:

1:
    rename: test1
2:
    rename: test2

OCS SimpleRestartSchedule

Generates a restart schedule.

Node Parameters

Input Parameters

Text Parameters

JSON or YAML schedule in list form.

- [4, -3]
- [2, -1]
- 1

Each item should be one of:

The example above means:

  1. After 4 steps, jump back 3 steps.
  2. After 2 steps, jump back one step.
  3. Go to the second item (after 2 steps, jump back one step).

The node start_step parameter is effectively the same as [start_step, 0] as a schedule item.

OCSNoise to SONAR_CUSTOM_NOISE

Adapter that enables using OCS noise generators with nodes that accept SONAR_CUSTOM_NOISE.

Most built-in OCS nodes will accept either type currently.

OCSNoise PerlinSimple

Generates 2D or 3D Perlin noise with many tuneable parameters. Can be plugged in to samplers for ancestral or SDE sampling. For initial noise or img2img workflows, use the NoisyLatentLike node from ComfyUI-sonar (see Integration).

3D Perlin noise works by taking a slice in the depth dimension each time the noise sampler is called.

For more tuneable parameters, see the OCSNoise PerlinAdvanced node.

Note: The shape of the latent must be a multiple of lacunarity ** (octaves - 1) * res (** indicates raising something to a power). Most latent types will have one latent pixel equaling eight normal pixels - i.e. if your image is 512x512, the latent would be 64x64.

Node Parameters

OCSNoise PerlinAdvanced

Generates 2D or 3D Perlin noise with many tuneable parameters. Can be plugged in to samplers for ancestral or SDE sampling. For initial noise or img2img workflows, use the NoisyLatentLike node from ComfyUI-sonar (see Integration).

3D Perlin noise works by taking a slice in the depth dimension each time the noise sampler is called.

Note: The shape of the latent in the relevant dimension including padding must be a multiple of lacunarity ** (octaves - 1) * res. Most latent types will have one latent pixel equaling eight normal pixels - i.e. if your image is 512x512, the latent would be 64x64.

Node Parameters