Home

Awesome

netdef_models

Repository for different network models related to flow/disparity from the following papers:

NOTE: We only provide deployment code for these networks. We do not publish any training code and also do not offer support about questions for training networks.

Setup

Running networks

Output formats

The networks are executed using the controller.py scripts in the respective folders. Just running this controller will produce several output files in a folder (note that you can also obtain this output just as numpy arrays and write it to some custom files; see next section).

For optical flow we use the standard .flo format. The other modalities use a custom binary format called .float3. To read .float3 files to numpy arrays, please use the netdef_slim.utils.io module.

Example usage:

from netdef_slim.utils.io import read 
occ_file = 'occ.float3'
occ_data = read(occ_file) # returns a numpy array

# to visualize
import matplotlib.pyplot as plt
plt.imshow(occ_data[:,:,0])

Controller eval

The eval method of the controller writes to the disk by default. To avoid writing to disk, create a Controller object and use the eval method available in the net_actions member variable. This can be useful if you want to process the output of our networks in memory and not incur additional disk I/O.

Example usage:

import netdef_slim as nd
nd.load_module('FlowNet3/css/controller.py')
c = Controller() 
out = c.net_actions.eval(img0, img1)
# out is an OrderedDict with the following structure
#OrderedDict(['flow[0].fwd',     np.array[...],
              'occ[0].fwd',      np.array[...],
              'occ_soft[0].fwd', np.array[...],
              'mb[0].fwd',       np.array[...],
              'mb_soft[0].fwd',  np.array[...],
              ])       

License

netdef_models is under the GNU General Public License v3.0