Awesome
Ev-LaFOR (ICCV 2023 Oral)
This repository contains the official PyTorch implementation of the paper "Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events" paper (ICCV 2023, Oral). [Paper]
Qualitative Results on N-Caltech101 and N-ImageNet100 datasets
<img src="imgs/qual_result.png" width="100%" height="100%">Quantitative results on N-Caltech101 and N-ImageNet100 datasets
<img src="imgs/quan_result.png" width="100%" height="100%">Requirements
Dataset
Download N-Caltech101 datasets. Download N-ImageNet datasets.
For convenience, you can also use data split that we have used: Download N-Caltech101 & Caltech101 datasets. Download N-ImageNet100 & ImageNet100 datasets.
š Data structure
Our folder structure is as follows:
caltech-101 (For Image)
āāā caltech-101
āāā 101_ObjectCategories
āāā accordion
ā āāā image_0001.jpg
ā āāā ...
āāā airplanes
ā āāā image_0001.jpg
ā āāā ...
ā
āāā ...
Caltech101 (For Event)
āāā accordion
ā āāā image_0001.bin
ā āāā ...
āāā airplanes
ā āāā image_0001.bin
ā āāā ...
āāā ...
ImageNet (For Image)
āāā extracted_100_train
ā āāā n01443537
ā ā āāā n01443537_2.JPEG
ā ā āāā ...
ā āāā ...
āāā extracted_100_val
āāā ILSVRC2012_val_00000007.JPEG
āāā ILSVRC2012_val_00000017.JPEG
āāā ...
N_ImageNet (For Event)
āāā extracted_100_train
ā āāā n01443537
ā ā āāā n01443537_2.npz
ā ā āāā ...
ā āāā ...
āāā extracted_100_val
āāā n01443537
ā āāā ILSVRC2012_val_00000236.npz
ā āāā ...
āāā n01616318
ā āāā ILSVRC2012_val_00000018.npz
ā āāā ...
ā
āāā ...
Data Path Change
datasets/caltech_event_ours_unpair_noise.py -L136: data_dir = "your caltech-101 path", event_dir = "your N-Caltech 101 path"
datasets/N_imagenet100_noise.py -L115: data_dir = "your ImageNet path", event_dir = "your N-ImageNet path"
Training & Test Code
Train & Test on N-Caltech 101 Dataset
$ python pretraining_event_with_prototype_caltech.py -en $experiment_name$ -d caltech_ours --ssl_spatial --inverse --n_mask 6
Train & Test on N-ImageNet 100 Dataset
$ python pretraining_event_with_prototype_imagenet.py -en $experiment_name$ -d imagenet100 --ssl_spatial --inverse --n_mask 6
You can also use the multi prototype by adding the --multi_proto
Reference
Hoonhee Cho*, Hyeonseong Kim*, Yujeong Chae, and Kuk-Jin Yoon "Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events", In ICCV, 2023.
@inproceedings{cho2023label,
title={Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events},
author={Cho, Hoonhee and Kim, Hyeonseong and Chae, Yujeong and Yoon, Kuk-Jin},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={19866--19877},
year={2023}
}
Contact
If you have any question, please send an email to hoonhee cho (gnsgnsgml@kaist.ac.kr)
License
The project codes and datasets can be used for research and education only.