Awesome

Fast Moving Object (FMO) Deblurring Benchmark

Simple Python library to evaluate your FMO deblurring methods.

Datasets

All three datasets (TbD, TbD-3D, Falling Objects) can be downloaded by running (after modifying the data storage folder path):

bash download_datasets.sh

Usage

Implement a function that takes as input image I [w, h, 3], background B [w, h, 3], bounding box of approximation FMO location, the required number of generated sub-frames n (temporal super-resolution), and an approximate object size. Your method should output temporal super-resolution mini-video of size [w, h, 3, n]. Optionally, if you want to evaluate trajectory accuracy, output the sub-frame object trajectory of size [2, n] or None.

An example of a dummy algorithm that always outputs the input image and does not evaluate the trajectory accuracy:

def my_deblur(I,B,bbox,nsplits,radius):
        return np.repeat(I[:,:,:,None], nsplits, 3), None

Baselines

We provide several baseline and state-of-the-art methods.

Dummy baselines

Two baselines, one that always outputs the input image, and another that output the background image. Example is shown in example_dummy.py.

Deblatting

To evaluate this method, please check out the deblatting sub-module. We provide three versions of deblatting: classical deblatting with single appearance (TbD), deblatting with chanring appearance (TbD-3D), and deblatting with trajectory oracle (TbD-O). Examples are shown in example_deblatting.py.

DeFMO - current state-of-the-art

The easiest way to evaluate DeFMO is using Kornia (kornia.feature.DeFMO). Example is shown in example_defmo.py.

To evaluate this method using the original source coude, please download DeFMO. Example is shown in example_defmo_source.py.

Scores

TbD-3D-Oracle has access to the ground-truth trajectory. Therefore, it's not a competitive baseline and is provided just for the reference.

Falling Objects dataset

Arbitrary shaped and textured objects.

TbD-3D dataset

Mostly spherical but significantly textured objects moving in 3D.

TbD dataset

Mostly spherical and uniformly colored objects moving in a plane parallel to the camera plane.

Reference

If you use this repository, please cite the following publication:

@inproceedings{defmo,
  author = {Denys Rozumnyi and Martin R. Oswald and Vittorio Ferrari and Jiri Matas and Marc Pollefeys},
  title = {DeFMO: Deblurring and Shape Recovery of Fast Moving Objects},
  booktitle = {CVPR},
  address = {Nashville, Tennessee, USA},
  month = jun,
  year = {2021}
}

The baseline TbD method:

@inproceedings{Kotera-et-al-ICCVW-2019,
  author = {Jan Kotera and Denys Rozumnyi and Filip Sroubek and Jiri Matas},
  title = {Intra-frame Object Tracking by Deblatting},
  booktitle = {Internatioal Conference on Computer Vision Workshop (ICCVW), 
  Visual Object Tracking Challenge Workshop, 2019},
  address = {Seoul, South Korea},
  month = oct,
  year = {2019}
}

The baseline TbD-3D or TbD-O method:

@inproceedings{Rozumnyi-et-al-CVPR-2020,
  author = {Denys Rozumnyi and Jan Kotera and Filip Sroubek and Jiri Matas},
  title = {Sub-frame Appearance and 6D Pose Estimation of Fast Moving Objects},
  booktitle = {CVPR},
  address = {Seattle, Washington, USA},
  month = jun,
  year = {2020}
}

Some ideas are taken from:

@inproceedings{Rozumnyi-et-al-CVPR-2017,
  author = {Denys Rozumnyi and Jan Kotera and Filip Sroubek and Lukas Novotny and Jiri Matas},
  title = {The World of Fast Moving Objects},
  booktitle = {CVPR},
  address = {Honolulu, Hawaii, USA},
  month = jul,
  year = {2017}
}