Awesome

DeepGyro

Keras implementation of the deblurring method: Gyroscope-Aided Motion Deblurring with Deep Networks [arXiv]

Testing

Download weights for DeepGyro from here and put them to the checkpoints folder. The weights for DeepBlind are also provided here.

To deblur images in the folder input/paper, execute: python deepgyro.py -i input/paper

To deblur images with DeepBlind, execute: python deepblind.py -i input/paper

Deblurring your own images

Before you can deblur your own images with DeepGyro, you need to compute gyro-based blur fields. You can use the code provided in the preprocess folder. Note that you will need gyroscope measurements, image timestamps, exposure times and some calibration information (see the next section). Once you have done this, put your data to the input/mydata folder and execute:

python deepgyro.py -i input/mydata

If you want to deblur images with DeepBlind, the blur fields are not needed. You can simply put your blurry images to input/mydata/blurred folder and execute:

python deepblind.py -i input/mydata

Make sure the folder structure is the same as in input/paper.

Generation of blur fields

You can generate your own blur fields by using the code in the preprocess folder. The raw input data is expected to be in the same format as the sample data in the preprocess/myrawdata folder. Generate blur fields using the command:

python generate.py -i myrawdata -o mydata

The output folder mydata can be moved to input folder to be processed by DeepGyro. Before you run generate.py on your data make sure to provide all necessary information as described in the following:

Blurry images myrawdata/images/xxx.jpg There can be more than one image. For example, you could capture a burst of images and deblur them all.

Image timestamps and exposure times myrawdata/images/images.txt The text file images.txt should have the same number of lines as there are input images. Each line contains the timestamp and exposure time (in nanoseconds). Example line: 2705480529000 30000000.

Inertial measurements myrawdata/imu/imu.txt The text file imu.txt contains inertial measurements including timestamps (in nanoseconds). Only gyroscope readings are required. The lines that start with “4” represent gyroscope. Example line: 4 358285205625 -1.5909724 -0.0620517 0.058423

Calibration information calibration.py Update the calibration file calibration.py so that it corresponds to your setup. The file should include intrinsic camera parameters, camera readout time (rolling shutter skew), IMU-camera temporal offset and the rotation matrix that aligns the IMU frame with the camera frame.

Citation

If you find our code helpful in your research or work, please cite our paper.

@InProceedings{Mustaniemi_2019_WACV,
  author = {Mustaniemi, Janne and Kannala, Juho and Särkkä, Simo and Matas, Jiri and Heikkilä, Janne},
  title = {Gyroscope-Aided Motion Deblurring with Deep Networks},
  booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
  month = {January},
  year = {2019}
}