Home

Awesome

Event-Based Angular Velocity Regression with Spiking Networks

<p align="center"> <a href="https://youtu.be/cffwH41ReF4"> <img src="http://rpg.ifi.uzh.ch/img/papers/ICRA20_Gehrig_vid_thumbnail_play.png" alt="SNN Regression" width="600"/> </a> </p>

This is the code for the paper Event-Based Angular Velocity Regression with Spiking Networks by Mathias Gehrig, Sumit Bam Shrestha, Daniel Mouritzen and Davide Scaramuzza.

You can find a pdf of the paper here. If you use any of this code, please cite the following publication:

@Article{Gehrig20icra,
  author        = {Mathias Gehrig and Sumit Bam Shrestha and Daniel Mouritzen and Davide Scaramuzza},
  title         = {Event-Based Angular Velocity Regression with Spiking Networks},
  journal       = {{IEEE} International Conference on Robotics and Automation (ICRA)},
  url           = {http://rpg.ifi.uzh.ch/docs/ICRA20_Gehrig.pdf},
  year          = 2020
}

Setup

Tested with:

using:

Data and Log Directory

git clone git@github.com:uzh-rpg/snn_angular_velocity.git
cd snn_angular_velocity

Either set up data and logs directory within the cloned repository with:

export data_dir=$(pwd)/data
export log_dir=$(pwd)/logs
mkdir -p $data_dir $log_dir/train $log_dir/test

or set up an external data and logs directory with symbolic links:

export data_dir="YOUR_DATA_PATH"
export log_dir="YOUR_LOG_PATH"

mkdir -p $data_dir $log_dir/train $log_dir/test
ln -s $data_dir data
ln -s $log_dir logs

or pass the data and logging directories with arguments. For more info:

python test.py -h

Installation with Anaconda

Adapt the CUDA toolkit version according to your setup. We also recommend to build PyTorch from source to avoid conflicts while compiling SLAYER. However the following option might work for you as well:

Create conda environment from scratch with precompiled PyTorch

cuda_version=10.0

conda create -y -n snn python=3.7
conda activate snn
conda install -y pytorch torchvision cudatoolkit=$cuda_version -c pytorch
conda install -y strictyaml tqdm -c conda-forge
conda install -y h5py pyyaml -c anaconda

# Setup for SLAYER
cd slayerpytorch
python setup.py install
cd ..

Download Dataset and Model

Download pretrained model.

wget "http://rpg.ifi.uzh.ch/data/snn_angular_velocity/models/pretrained.pt" -O pretrained/cnn5-avgp-fc1.pt

Download and extract test dataset. This requires zstd which you can get with sudo apt install zstd on Ubuntu.

wget "http://rpg.ifi.uzh.ch/data/snn_angular_velocity/dataset/test.tar.zst" -O $data_dir/test.tar.zst
cd $data_dir
zstd -vd test.tar.zst
tar -xvf test.tar
rm test.*

Similarly, you can get the training dataset

wget "http://rpg.ifi.uzh.ch/data/snn_angular_velocity/dataset/train.tar.zst" -O $data_dir/train.tar.zst

validation dataset

wget "http://rpg.ifi.uzh.ch/data/snn_angular_velocity/dataset/val.tar.zst" -O $data_dir/val.tar.zst

and the original panorama images to recreate the dataset from scratch

wget "http://rpg.ifi.uzh.ch/data/snn_angular_velocity/dataset/imgs.tar" -O $data_dir/imgs.tar

Test

To reproduce the numbers in the paper run:

python test.py

Config file

This uses by default the configuration file in test_config.yaml. Modify the test config if you want to change one of the following parameters:

Writing predictions to disk

If you would like to write the predictions of the network to disk:

python test.py --write

This will generate the following three files in $log_dir/test/*/out/: