Awesome
HPLFlowNet
This is the code for HPLFlowNet, a hierarchical permutohedral lattice flownet for scene flow estimation on large-scale point clouds. The code is developed and maintained by Xiuye Gu.
Prerequisites
Our model is trained and tested under:
-
Python 3.5.2 (testing under Python 3.6.5 also works)
-
NVIDIA GPU + CUDA CuDNN
-
PyTorch (torch == 0.4.0) (Should also support PyTorch 1.0 now)
-
Numba (numba == 0.38.1)
-
You may need to install cffi.
-
Mayavi for visualization.
-
Installation on Ubuntu:
pip3 install https://download.pytorch.org/whl/cu90/torch-0.4.0-cp35-cp35m-linux_x86_64.whl
pip3 install numba
pip3 install cffi
sudo apt-get -y install python-vtk
sudo pip3 install mayavi
sudo apt-get install python3-pyqt5
sudo pip3 install PyQt5
Data preprocess
- FlyingThings3D:
Download and unzip the "Disparity", "Disparity Occlusions", "Disparity change", "Optical flow", "Flow Occlusions" for DispNet/FlowNet2.0 dataset subsets from the FlyingThings3D website (we used the paths from this file, now they added torrent downloads)
. They will be upzipped into the same directory,
RAW_DATA_PATH
. Then run the following script for 3D reconstruction:
python3 data_preprocess/process_flyingthings3d_subset.py --raw_data_path RAW_DATA_PATH --save_path SAVE_PATH/FlyingThings3D_subset_processed_35m --only_save_near_pts
- KITTI Scene Flow 2015
Download and unzip KITTI Scene Flow Evaluation 2015 to directory
RAW_DATA_PATH
. Run the following script for 3D reconstruction:
python3 data_preprocess/process_kitti.py RAW_DATA_PATH SAVE_PATH/KITTI_processed_occ_final
Get started
Setup:
cd models; python3 build_khash_cffi.py; cd ..
Trained models
Out trained model can be downloaded in the trained_models
folder.
Inference
Set data_root
in the configuration file to SAVE_PATH
in the data preprocess section. Set resume
to be the path of your trained model or our trained model in trained_models
. Then run
python3 main.py configs/test_xxx.yaml
Current implementation only supports batch_size=1
.
Train
Set data_root
in the configuration file to SAVE_PATH
in the data preprocess section. Then run
python3 main.py configs/train_xxx.yaml
Visualization
If you set TOTAL_NUM_SAMPLES
in evaluation_bnn.py
to be larger than 0. Sampled results will be saved in a subdir of your checkpoint directory, VISU_DIR
.
Run
python3 visualization.py VISU_DIR
Citation
If you use this code for your research, please cite our paper.
@inproceedings{HPLFlowNet,
title={HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for
Scene Flow Estimation on Large-scale Point Clouds},
author={Gu, Xiuye and Wang, Yijie and Wu, Chongruo and Lee, Yong Jae and Wang, Panqu},
booktitle={Computer Vision and Pattern Recognition (CVPR), 2019 IEEE International Conference on},
year={2019}
}
Acknowledgments
Our permutohedral lattice implementation is based on Fast High-Dimensional Filtering Using the Permutohedral Lattice. The BilateralNN implementation is also closely related. Our hash table implementation is from khash-based hashmap in Numba.