Home

Awesome

SFOD: Spiking Fusion Object Detector

Optional image alt text

This is the official implementation of the 'SFOD: Spiking Fusion Object Detector' .

Requirements

<p align="center">
RepositoryVersion
CUDA11.7
cuDNNV8.0.0
Python3.9.0
Pytorch1.12.1
Torchvision0.13.1
Torchmetrics0.11.4
Pytorch-lightning2.0.1
SpikingJelly0.0.12
</p>

Pretrained Checkpoints

We will provide the trained models in the pretrained folder, which will include pretrained backbone networks and pretrained SFOD.

Required Data

To evaluate or train SFOD you will need to download the datasets:

Dataset NameLink
NCARS DatasetDownload N-CARS Car Classification Dataset | PROPHESEE
GEN1 DatasetDownload Gen1 Automotive Detection Dataset | PROPHESEE

Training

Training for Backbone

python classification.py -devices auto -num_workers 8 -test -save_ckpt -model densenet-121_16 -loss_fun mse -encoding fre -early_stopping

python classification.py -devices auto -num_workers 8 -test -save_ckpt -model densenet-121_24 -loss_fun mse -encoding fre -early_stopping

Training for SFOD

python object_detection.py -devices auto -num_workers 4 -test -save_ckpt -backbone densenet-121_24 -pretrained_backbone ./pretrained/DenseNet121-24.ckpt -b 16 -fusion -fusion_layers 3 -mode res

Evaluation

When you perform evaluation, the corresponding pretrained model data needs to be replaced in the appropriate root folder.

Evaluation for Backbone

python classification.py -devices auto -num_workers 8 -test -no_train -model densenet-121_16 -loss_fun mse -encoding fre -pretrained DenseNet121-16.ckpt

python classification.py -devices auto -num_workers 8 -test -no_train -model densenet-121_24 -loss_fun mse -encoding fre -pretrained DenseNet121-24.ckpt

Evaluation for SFOD

python object_detection.py -num_workers 4 -test -no_train -pretrained SFOD.ckpt -backbone densenet-121_24 -fusion -fusion_layers 3 -mode res

Code Acknowledgments

This code is based on object-detection-with-spiking-neural-networks . Thanks to the contributors of object-detection-with-spiking-neural-networks .

@InProceedings{Cordone_2022_IJCNN,
    author    = {Cordone, Loic and Miramond, Benoît and Thierion, Phillipe},
    title     = {Object Detection with Spiking Neural Networks on Automotive Event Data},
    booktitle = {Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN)},
    month     = {July},
    year      = {2022},
    pages     = {}
}