Awesome

PWC Net TensorFlow

Tensorflow implementation of Pyramid, Warping and Cost Volume (PWC) Networks based on the paper presented at CVPR 2018. Currently, main.py simply downloads the FlyingChairs Dataset and starts training, following the outlined schedule. This code could easily be adapted to train on other datasets though.

Tested Environment

Ubuntu 16.04 Python3 Tensorflow 1.8 Cuda 9.0

Acknowledgements

This repo uses 3 custom written tensorfow ops in c++

The correlation op was taken from this tensorflow implementation of UnFlow by Simon Meister The ppm and flo decoding ops were taken from this collection of tf ops, from the Computer Vision Group, Albert-Ludwigs-Universität Freiburg

Usage

python3 main.py

A tensorboard session will automatically be started in a new tmux window (so that the visualisations are still available after the python session has ended). This tensorboard session will log the training/validation losses, as well as giffs of the flow as it trains.

Some general hyperparameters regarding the logging of data can be changed through task.py Other hyperparameters relating to the training schedule can be changed in the constructor of network.py The default training/validation split is to have 90% training, with 10% left for validation.

Example visualisations following training

From left to right, the images below indicate rgb image, ground truth flow, predicted flow, flow error

Examples from the training set:

Example Training Flow Result 1 Example Training Flow Result 2 Example Training Flow Result 3 Example Training Flow Result 4

Examples from the validation set:

Example Validation Flow Result 1

Example Training Loss

This is an example of the loss when training on the full flying chairs dataset (no validation was used on this occassion).

Example Loss