Awesome
PointINet: Point Cloud Frame Interpolation Network
Introduction
The repository contains the source code and pre-trained models of our paper (published on AAAI 2021): PointINet: Point Cloud Frame Interpolation Network
.
Environment
Our code is developed and tested on the following environment:
- Python 3.6
- PyTorch 1.4.0
- Cuda 10.1
- Numpy 1.19
We utilized several open source library to implement the code:
- kaolin
- pytorch3d
- PyTorchEMD (only for evaluation)
- Mayavi (only for visualization of demo)
- wandb (to record training process)
Usage
Dataset
We utilize two large scale outdoor LiDAR dataset:
To facilitate the implementation, we split the LiDAR point clouds in nuScenes dataset by scenes and the results are saved in data/scene-split
. Besides, all of the LiDAR files in nuScenes dataset are stored in one single folder (include sweeps and samples).
For the pre-training of FlowNet3D, please refer to FlowNet3D to download the pre-processed dataset (Flythings3D and Kitti scene flow dataset).
Demo
We provide a demo to visualize the result, please run
python demo.py --is_save IS_SAVE --visualize VISUALIZE
Training
Training of FlowNet3D
To train FlowNet3D, firstly train on Flythings3D dataset
python train_sceneflow.py --batch_size BATCH_SIZE --gpu GPU --dataset Flythings3D --root DATAROOT --save_dir CHECKPOINTS_SAVE_DIR --train_type init
Then train it on Kitti scene flow dataset
python train_sceneflow.py --batch_size BATCH_SIZE --gpu GPU --dataset Kitti --root DATAROOT --pretrain_model PRETRAIN_MODEL --save_dir CHECKPOINTS_SAVE_DIR --train_type init
After that train it on Kitti odometry dataset based on the model pretrained on Kitti scene flow dataset.
python train_sceneflow.py --batch_size BATCH_SIZE --gpu GPU --dataset Kitti --root DATAROOT --pretrain_model PRETRAIN_MODEL --save_dir CHECKPOINTS_SAVE_DIR --train_type refine
Also train it on nuScenes dataset based on the model pretrained on Kitti scene flow dataset.
python train_sceneflow.py --batch_size BATCH_SIZE --gpu GPU --dataset nuscenes --root DATAROOT --pretrain_model PRETRAIN_MODEL --save_dir CHECKPOINTS_SAVE_DIR --train_type refine
Training of PointINet
We only train the PointINet on Kitti odometry dataset, run
python train_interp.py --batch_size BATCH_SIZE --gpu GPU --dataset kitti --root DATAROOT --pretrain_model FLOWNET3D_PRETRAIN_MODEL --freeze 1
Testing
To test on Kitti odometry dataset, run
python test.py --gpu GPU --dataset kitti --root DATAROOT --pretrain_model POINTINET_PRETRAIN_MODEL --pretrain_flow_model FLOWNET3D_PRETRAIN_MODEL
To test on nuScenes dataset, run
python test.py --gpu GPU --dataset nuscenes --root DATAROOT --pretrain_model POINTINET_PRETRAIN_MODEL --pretrain_flow_model FLOWNET3D_PRETRAIN_MODEL --scenelist TEST_SCENE_LIST
Citation
@InProceedings{Lu2020_PointINet,
author = {Lu, Fan and Chen, Guang and Qu, Sanqing and Li, Zhijun and Liu, Yinlong and Knoll, Alois},
title = {PointINet: Point Cloud Frame Interpolation Network},
booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2021}
}