Awesome
EventStream-SR
This repository is for the ICCV 2021 paper: Event Stream Super-Resolution via Spatiotemporal Constraint Learning.
This is our raw, unintegrated code.
Requirements
-
Python 3 with the following packages installed:
- torch==1.3.1
- torchvision==0.4.2
- tqdm==4.61.1
- numpy==1.19.2
- imageio==2.9.0
- Pillow==8.2.0
- tensorboardX==2.2
- pyyaml==5.4.1
-
slayerPytorch
- See https://github.com/bamsumit/slayerPytorch to install the slayerPytorch for the SNN simulation.
-
cuda
- A CUDA enabled GPU is required for training any model. We test our code with CUDA 10.0 V10.0.130 and cudnn 7.6.5.
Data preparing
- Our datasets are available at https://github.com/lisiqi19971013/event-based-datasets.
- Download the datasets and unzip them to the folder (./dataset/)
- Change the corresponding data path in each .py file
Train and test
Training
This is the raw and unintegrated code. If you want to train the model on a dataset, such as N-MNIST, run this code
>>> cd nMnist
>>> python trainNmnist.py --bs 64 --savepath './ckpt/' --epoch 30 --showFreq 50 --lr 0.1 --cuda '1' --j 4
or just change the file train.sh and run
>>> sh train.sh
Testing
After training, run the following code to generating results. NOTICE: the default output path is (./dataset/N-MNIST/ResConv/HRPre).
>>> python testNmnist.py
Calculate metrics
After generating results, run calRMSE.py to calculate the metrics. NOTICE: the output path should be changed.
>>> python calRMSE.py
Downstream application
-
Classification
The classification experiment is done with the N-MNIST, Cifar10-DVS and ASL-DVS datasets. After generating the super-resolution results, the classification can be done by the following code (for the N-MNIST dataset for example).
>>> cd nMnist >>> python trainNmnistClassification.py
In the classification experiment, the training sets are the ground truth event streams from the training sets of the super-resolution task. The test sets are the test set of the super-resolution task, containing the LR event streams, HR ground truth event streams, output event streams and others. The test mode and other parameters is selected in the trainNmnistClassification.py file.
-
Image reconstruction
The image reconstruction task is done following the E2VID, the code and the pretrain model is proposed in https://github.com/uzh-rpg/rpg_e2vid.