Awesome
netdef_models
Repository for different network models related to flow/disparity from the following papers:
NOTE: We only provide deployment code for these networks. We do not publish any training code and also do not offer support about questions for training networks.
-
Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow
(E. Ilg and T. Saikia and M. Keuper and T. Brox published at ECCV 2018) [paper] [video] -
Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow
(E. Ilg and Ö. Cicek and S. Galesso and A. Klein and O. Makansi and F. Hutter and T. Brox published at ECCV 2018) [paper] [video]
Setup
- Install tensorflow (1.4) (pip3 install tensorflow-gpu==1.4)
- Compile and install lmbspecialops. Please use the branch
eccv18
instead ofmaster
- Install netdef_slim
- Clone this repository
Running networks
- Change your directory to the network directory (Eg: FlowNet3)
- Run download_snapshots.sh. This takes a while to download all snapshots
- Now you should be ready to run the networks. Change your directory to a network type (Eg: css).
Use the following command to test the network on an image pair:
python3 controller.py eval image0_path image1_path out_dir
Output formats
The networks are executed using the controller.py scripts in the respective folders. Just running this controller will produce several output files in a folder (note that you can also obtain this output just as numpy arrays and write it to some custom files; see next section).
For optical flow we use the standard .flo
format.
The other modalities use a custom binary format called .float3
. To read .float3
files to numpy arrays, please use the
netdef_slim.utils.io module.
Example usage:
from netdef_slim.utils.io import read
occ_file = 'occ.float3'
occ_data = read(occ_file) # returns a numpy array
# to visualize
import matplotlib.pyplot as plt
plt.imshow(occ_data[:,:,0])
Controller eval
The eval method of the controller writes to the disk by default.
To avoid writing to disk, create a Controller object and use the eval
method available in the net_actions
member variable.
This can be useful if you want to process the output of our networks in memory and not incur additional disk I/O.
Example usage:
import netdef_slim as nd
nd.load_module('FlowNet3/css/controller.py')
c = Controller()
out = c.net_actions.eval(img0, img1)
# out is an OrderedDict with the following structure
#OrderedDict(['flow[0].fwd', np.array[...],
'occ[0].fwd', np.array[...],
'occ_soft[0].fwd', np.array[...],
'mb[0].fwd', np.array[...],
'mb_soft[0].fwd', np.array[...],
])
License
netdef_models is under the GNU General Public License v3.0