Home

Awesome

Add Temporal Coherence to Single Image AI Upscaling Models in Vapoursynth

Also known as temporal consistency, line wiggle fix, stabilization, temporal denoising, or temporal fix.
This will not add extra work to the upscaling on the GPU and instead run in parallel on the CPU.
Intended for animation.

Comparisons (by hddvddegogo):
https://www.youtube.com/watch?v=BXc_Uddt2KA
https://www.youtube.com/watch?v=u6LHR9_m5rg

<p align="center"> <img src="README_example.gif"/> </p> <br />

Requirements

Setup

Put the vs_temporalfix.py file into your vapoursynth scripts folder.
Or install via pip: pip install git+https://github.com/pifroggi/vs_temporalfix.git

<br />

Usage

from vs_temporalfix import vs_temporalfix
clip = vs_temporalfix(clip, strength=400, tr=6, exclude="[10 20]", debug=False)

clip
Temporally unstable clip. Must be in YUV or GRAY format.
Should have no black borders. Full range (PC) input is recommended.

strength
Suppression strength of temporal inconsistencies. Higher means more aggressive. 400 works great in most cases.
The best way to finetune is to find a static scene and increase this till lines and details are stable.
Reduce if you get blending/ghosting on small movements, especially in dark or hazy scenes, or blocky artifacts.

tr
Temporal radius sets the number of frames to average over.
Higher means more stable, especially on slow pans and zooms, but makes it slower. 6 works great in most cases.
The best way to finetune is to find a slow pan or zoom and increase this till lines and details are stable.

exclude (optional)
Optionally exclude scenes with intended temporal inconsistencies (like TV noise), or in case this doesn't work.
Example setting 3 scenes: exclude="[10 20] [600 900] [2000 2500]"
First number in the brackets is the first frame of the scene, the second number is the last frame (inclusive).

debug (optional)
Shows protected areas that will be left untouched in pink. This includes areas with high motion, scene changes and excluded scenes. May help while tuning parameters to see if the area is even affected.

<br />

Tips & Troubleshooting

[!CAUTION]

[!TIP]

Benchmarks

HardwareResolutionTRAverage FPS
Ryzen 5900X1440x10806~7 fps
Ryzen 5900X2880x21606~4 fps
<br />

Alternative Usage Options

Several projects integrated this script to simplify usage without Vapoursynth knowledge.