Home

Awesome

Fast Optical Flow using Dense Inverse Search (DIS)

Our code is released only for scientific or personal use. Please contact us for commercial use.

If used this work, please cite:

@inproceedings{kroegerECCV2016, Author = {Till Kroeger and Radu Timofte and Dengxin Dai and Luc Van Gool}, Title = {Fast Optical Flow using Dense Inverse Search}, Booktitle = {Proceedings of the European Conference on Computer Vision ({ECCV})}, Year = {2016}}

Is you use the variational refinement, please additionally cite:

@inproceedings{weinzaepfelICCV2013, TITLE = {{DeepFlow: Large displacement optical flow with deep matching}}, AUTHOR = {Weinzaepfel, Philippe and Revaud, J{\'e}r{\^o}me and Harchaoui, Zaid and Schmid, Cordelia}, BOOKTITLE = {{ICCV 2013 - IEEE International Conference on Computer Vision}}, YEAR = {2013}}

Compiling

The program was only tested under a 64-bit Linux distribution. SSE instructions from built-in X86 functions for GNU GCC were used.

The following will build four binaries: Two for optical flow (run_OF_*) and two for depth from stereo (run_DE_*). For each problem, a fast variant operating on intensity images (run_*_INT) and a slower variant operating on RGB images (run_*_RGB) is provided.

mkdir build
cd build
cmake ../
make -j

The code depends on Eigen3 and OpenCV. However, OpenCV is only used for image loading, scaling and gradient computation (run_dense.cpp). It can easily be replaced by other libraries.

Usage

The interface for all four binaries (run_*_*) is the same.

VARIANT 1 (Uses operating point 2 of the paper, automatically selects coarsest scale):

./run_*_* image1.png image2.png outputfile

VARIANT 2 (Manually select operating point X=1-4, automatically selects coarsest scale):

./run_*_* image1.png image2.png outputfile X

VARIANT 3 (Set all parameters explicitly):

./run_*_* image1.png image2.png outputfile p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18 p19 p20

Example for variant 3 using operating point 2 of the paper:

./run_OF_INT in1.png int2.png out.flo 5 3 12 12 0.05 0.95 0 8 0.40 0 1 0 1 10 10 5 1 3 1.6 2

Parameters:

1. Coarsest scale                               (here: 5)
2. Finest scale                                 (here: 3)
3/4. Min./Max. iterations                       (here: 12)
5./6./7. Early stopping parameters
8. Patch size                                   (here: 8)
9. Patch overlap                                (here: 0.4)
10.Use forward-backward consistency             (here: 0/no)
11.Mean-normalize patches                       (here: 1/yes)
12.Cost function                                (here: 0/L2)  Alternatives: 1/L1, 2/Huber, 10/NCC
13.Use TV refinement                            (here: 1/yes)
14./15./16. TV parameters alpha,gamma,delta     (here 10,10,5)
17. Number of TV outer iterations               (here: 1)
18. Number of TV solver iterations              (here: 3)
19. TV SOR value                                (here: 1.6)
20. Verbosity                                   (here: 2) Alternatives: 0/no output, 1/only flow runtime, 2/total runtime

The optical flow output is saves as .flo file. (http://sintel.is.tue.mpg.de/downloads)

The interface for depth from stereo is exactly the same. The output is saves as pfm file. (http://vision.middlebury.edu/stereo/code/)

NOTES:

  1. For better quality, increase the number iterations (param 3/4), use finer scales (param. 2), higher patch overlap (param. 9), more outer TV iterations (param. 17)
  2. L1/Huber cost functions (param. 12) provide better results, but require more iterations (param. 3/4)

Bugs and extensions

If you find bugs, etc., please feel free to contact me. Contact details are available on my webpage. http://www.vision.ee.ethz.ch/~kroegert/

History

July 2016 v1.0.0 - Initial Release August 2016 v1.0.1 - Minor Bugfix: Error in L1 and Huber error norm computation.

LICENCE CONDITIONS

GPLv3: http://gplv3.fsf.org/

All programs in this collection are free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.