Awesome
Event-based Background-Oriented Schlieren
This is the official repository for Event-based Background-Oriented Schlieren, IEEE T-PAMI 2023 by
Shintaro Shiba, Friedhelm Hamann, Yoshimitsu Aoki and Guillermo Callego.
Paper | Video | Dataset | News
</h2>If you use this work in your research, please cite it (see also here):
@Article{Shiba23pami,
author = {Shintaro Shiba and Friedhelm Hamann and Yoshimitsu Aoki and Guillermo Gallego},
title = {Event-based Background-Oriented Schlieren},
journal = {IEEE Trans. Pattern Anal. Mach. Intell. (T-PAMI)},
year = {2024},
volume = {46},
number = {4},
pages = {2011-2026},
doi = {10.1109/TPAMI.2023.3328188}
}
Setup
Requirements
-
python: 3.8.x, 3.9.x, 3.10.x
-
ffmpeg ... This is necessary for
ffmpeg-python
.
Tested environments
- Mac OS Monterey (both M1 and non-M1)
- Ubuntu (CUDA 11.1, 11.3, 11.8)
- PyTorch 1.9-1.12.1, or PyTorch 2.0.
Installation
Quick setup
Minimal quick setup is to use venv.
python3 -m venv ~/work/venv
source ~/work/venv/bin/activate # activate env
python3 -V # make sure the version, 3.8 or higher
pip3 install scipy==1.9.1 optuna==2.10.1 opencv-python matplotlib plotly \
ffmpeg-python h5py hdf5plugin PyYAML Pillow sklearn scikit-image argparse openpiv \
torch torchvision black isort kaleido
Full setup
You can use poetry.
poetry install
This install dev dependencies (format, test) too.
Download dataset
Please download the dataset and put into your local folder. You can download the files from the official source or Google Drive The structure of the folder is as follows:
(root)/datasets/
CCS/ # fixed keyword: dataset recorded with the co-capture system (CCS).
(sequence_name)/
basler_0/
frames.mp4 # frame data
config.yaml
prophesee_0/
cd_events.raw # event data in .raw format (Prophesee)
events.hdf5 # event data converted in HDF5 format
trigger_events.txt
homography.txt # calibration file
EBOS dataset recorded with Co-Capture System (CCS)
The Prophesee camera provides .raw
format file, which requires to install OpenEB
.
We converted this file into hdf5 file formats for better compatibility among languages and OS.
So, there is no need to install OpenEB.
Execution
Run
An example is:
python3 bos_event.py --config_file ./configs/hot_plate1.yaml --eval
Config
The configuration for each experiment is provided to the script through yaml files.
Please see the readme.
Development
Note you use poetry for development.
Format
You can quickly run format of the codes:
make fmt
<!--
# Citation
If you use this work in your research, please cite it as follows:
```bibtex
@Article{Shiba23pami,
author = {Shintaro Shiba and and Friedhelm Hamann and Yoshimitsu Aoki and Guillermo Gallego},
title = {Event-based Background-Oriented Schlieren},
booktitle = {IEEE T-PAMI},
pages = {},
doi = {10.1109/TPAMI.2023.3328188},
year = 2023
}
```
-->
Code Authors
LICENSE
Please check License.
Acknowledgement
TBD