Also known as atmospheric turbulance mitigation, warp stabilization, film shrink or VHS distortion fix, dewobble, dewiggle, detilt, rectification, heat haze removal. Can help with distortions from low bitrate compression or old codecs like MPEG2.
This does not do general video stabilization for shaky footage, only removes distortions within the frames. It is recommented to stabilize first if needed.
This is a partial implementation of the Turbulence Mitigation Transformer. (only distortion removal, no deblurring)
- pytorch
pip install numpypip install einops
Put the entire vs_undistort folder into your vapoursynth scripts folder.
Or install via pip: pip install git+https://github.com/pifroggi/vs_undistort.git
from vs_undistort import vs_undistort
clip = vs_undistort(clip, temp_window=10, tile_size=480, device="cuda")
clip
Distorted clip. Must be in RGBS format.
temp_window
Temporal window. Amount of frames to include in the calculation and size of chunks the clip will be processed in.
Larger means higher VRAM requirements, but better temporal averaging effect and slower distortions can be removed. If this is too small, some distortions may not get removed and seams from tile_size may become more obvious.
tile_size
Size of tiles to split the frames into. Must be a multiple of 16.
Larger means higher VRAM requirements, but better spatial averaging effect and larger/lower frequency distortions can be removed. If distortions are larger than tile_size, they can not be removed. If only some distortions are larger than tile_size, visible seams may appear.
device
Possible values are "cuda" to use with an Nvidia GPU, or "cpu". This will be extremely slow on CPU.
If you are getting "RuntimeError: CUDA error: invalid argument" you are likely running out of GPU memory. Try lowering tile_size or temp_window. Restarting vsedit can also clear memory, if it is in use.
If you have an undistorted reference clip, try this: https://github.com/pifroggi/vs_align
| Hardware | Resolution | Average FPS |
|---|---|---|
| RTX 4090 | 720x480 | ~14 fps |
| RTX 4090 | 1440x1080 | ~3.5 fps |
| RTX 4090 | 2880x2160 | ~1 fps |