Awesome
EvUnroll: Neuromorphic Events based Rolling Shutter Image Correction (CVPR22)
This repo contains the code and dataset for EvUnroll: Neuromorphic events based rolling shutter image correction by Xinyu Zhou, Peiqi Duan, Yi Ma, and Boxin Shi.
Examples of RS correction results on Gev-RS dataset
Data
- Gev-RS dataset: We capture GS frames from a high-speed camera, and simulate RS frames and corresponding event stream. You can download the dataset from Gev-RS.
- Real data examples: We collect real-data examples with an RS-event hybrid camera for testing, you can download it from RealData
Gev-RS dataset follows the below directory format:
├── path_to_your_dataset_folder/
├── Gev-RS/
├── train/
├── seq1/
├── rs_blur/
├── 00000.png
......
├── rs_sharp/
├── 00000.png
......
├── gt/
├── 00000.png
......
├── seq2/
......
├── test/
......
├── all_sequence/
├── train/
├── seq1.avi
......
├── test/
......
├── Gev-RS-DVS/
├── train/
├── seq1/
├── seq1.h5
├── seq1_events_viz.avi
......
├── test/
......
Usage
Dependency
pip install -r requirements.txt
Test
- Put the pretrained model in trained_model/* .
- Change the path to the dataset in util/config.py.
python test.py
Train
The training procedure consists of three steps:
- Train the synthesis module and flow module seperately.
- Freeze the weights of above two modules, and train the fusion module.
- Unfreeze weights of all three modules and finetune the entire network.
To reprocude the training procedure, you might need to slightly modify the training code in different training steps, and
python train.py
License
The datasets can be freely used for research and education only. Any commercial use is strictly prohibited.
Bibtex
@InProceedings{Zhou_2022_CVPR,
author = {Zhou, Xinyu and Duan, Peiqi and Ma, Yi and Shi, Boxin},
title = {EvUnroll: Neuromorphic Events Based Rolling Shutter Image Correction},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2022},
pages = {17775-17784}
}
Contact
If you have any questions, please send an email to zhouxiny@pku.edu.cn