Awesome
Overview
This repo provides the code of Neuromorphic Imaging with Joint Image Deblurring and Event Denoising.
@article{zhang2024tip,
title = {Neuromorphic Imaging with Joint Image Deblurring and Event Denoising},
author = {Pei Zhang and Haosen Liu and Zhou Ge and Chutian Wang and Edmund Y. Lam},
journal = {IEEE Transactions on Image Processing},
volume = {33},
pages = {2318--2333},
month = {March},
year = {2024},
doi = {10.1109/TIP.2024.3374074},
}
Implementation
Before start, please glance at the (good and failed) sample files we upload in the folders data
and results
.
Preparation
- Put your image file and its corresponding event data in the folder you specify by
configs.data
indemo.m
(default:data
). - The event data must be in
.mat
and witht, x, y, p
entries.
Run
Run demo.m
with the following configurations:
configs.dir: your dir name
configs.data: folder that stores input data
configs.results: folder that stores output results
configs.blur: input image file
configs.evs: input event file
configs.dvs_resolution: DVS spatial resolution
configs.alpha: weight of the event regularizer
configs.beta: weight of the l_0 regularizer
configs.sigma: weight of the Gaussian regularizer
configs.weight: weight of gradient supervision
configs.N: find neighbors (1) or not (0)
configs.dx: spatial threshold to specify a square boundary of the neighbors
configs.dt: temporal threshold to specify a boundary of the neighbors
configs.case: specify a use case (-1, 1, 2)
For convenience, we split our algorithm into 3 functions, which are controlled by configs.case
:
configs.case = -1
for experiencing joint image deblurring and event denoising.configs.case = 1
for image deblurring only (if you have a blurry image and clean events). The following configurations are disabled (any value):configs.weight
,configs.N
,configs.dx
,configs.dt
.configs.case = 2
for event denoising only (if you have a sharp image and noisy events). The following configurations are disabled (any value):configs.alpha
,configs.beta
,configs.sigma
.
Results
Once done, up to 4 files are generated in the folder you specify by configs.results
in demo.m
(default: results
):
xxx_configs.mat
: record the configurations usedxxx_sharp.png
: restored sharp image (only forconfigs.case = -1
andconfigs.case = 1
)xxx_kernel.png
: estimated blur kernel (only forconfigs.case = -1
andconfigs.case = 1
)xxx_signals.mat
: denoised events (only forconfigs.case = -1
andconfigs.case = 2
)
Dataset
Our real dataset has multiple pairs of blurry images and noisy event streams recorded by a DAVIS346 camera on a rich range of scenarios. Download it from here with password zhang2024joint
.