Home

Awesome

SoftSplat-Full

Full model implementation of CVPR 2020 paper, "Softmax Splatting for Video Frame Interpolation"

<!-- The full model's weights pretrained on Vimeo90k data can be downloaded from this [link](https://drive.google.com/file/d/1wtUFS68D8hVKRg-LFr7jAibg8KgvyrMZ/view?usp=sharing). My reproduced model shows a ```PSNR of 35.59``` in Vimeo90k. -->

UPDATE NOTE: the pretrained weights below should be used with the version commited on June 25, 2022. The current version can be used as a reference, but may not lead to the exact performance reported, since some details may be missing. Will be updated soon.

Pre-trained weights for pre-defined Z using alpha.

Pre-trained weights for fine-tuned Z using a small neural network v.

Both weights approximately lead to a PSNR of 35.8 in Vimeo90k. (reported score from the paper is 36.10)

The model assumes to take a tensor in the shape of [B, C, T, H, W] where B is the batch size, C is the number of channels (3 RGB channels), T is the number of frames (2 frames: frame 0 & frame 1), H and W is the height and width of the frames.

from SoftSplatModel import SoftSplatBaseline

frame0frame1 = torch.randn([1, 3, 2, 448, 256]).cuda()  # batch size 1, 3 RGB channels, 2 frame input, H x W of 448 x 256
target_t = torch.tensor([0.5]).cuda()  # target t=0.5

model = SoftSplatBaseline().cuda()
model.load_state_dict(torch.load('./SoftSplat_predefinedZ.pth')['model'])

framet = model(frame0frame1, target_t)