Awesome
Multi-Scale Bidirectional Recurrent Network with Hybrid Correlation for Point Cloud Based Scene Flow Estimation
Wencan Cheng and Jong Hwan Ko
IEEE International Conference on Computer Vision (ICCV), 2023
Prerequisities
Our model is trained and tested under:
- Python 3.6.9
- NVIDIA GPU + CUDA CuDNN
- PyTorch (torch == 1.6.0)
- scipy
- tqdm
- sklearn
- numba
- cffi
- pypng
- pptk
- thop
Please follow this repo or the instructions below for compiling the furthest point sampling, grouping and gathering operation for PyTorch.
cd pointnet2
python setup.py install
cd ../
Data preprocess
We adopt the equivalent preprocessing steps in HPLFlowNet and PointPWCNet.
We copy the preprocessing instructions here for your convinience.
- FlyingThings3D:
Download and unzip the "Disparity", "Disparity Occlusions", "Disparity change", "Optical flow", "Flow Occlusions" for DispNet/FlowNet2.0 dataset subsets from the FlyingThings3D website (we used the paths from this file, now they added torrent downloads)
. They will be upzipped into the same directory,
RAW_DATA_PATH
. Then run the following script for 3D reconstruction:
python3 data_preprocess/process_flyingthings3d_subset.py --raw_data_path RAW_DATA_PATH --save_path SAVE_PATH/FlyingThings3D_subset_processed_35m --only_save_near_pts
- KITTI Scene Flow 2015
Download and unzip KITTI Scene Flow Evaluation 2015 to directory
RAW_DATA_PATH
. Run the following script for 3D reconstruction:
python3 data_preprocess/process_kitti.py RAW_DATA_PATH SAVE_PATH/KITTI_processed_occ_final
Evaluation
Set data_root
in the configuration file to SAVE_PATH
in the data preprocess section before evaluation.
We provide pretrained model in pretrain_weights
.
Please run the following instrcutions for evaluating.
python3 evaluate.py config_evaluate.yaml
Train
If you need a newly trained model, please set data_root
in the configuration file to SAVE_PATH
in the data preprocess section before evaluation at the first. Then excute following instructions.
python3 train_msbrn.py config_train.yaml
Acknowledgement
We thank repo for the corase-to-fine framework.