Home

Awesome

DGCNN.pytorch

[中文版]

This repo is a PyTorch implementation for Dynamic Graph CNN for Learning on Point Clouds (DGCNN)(https://arxiv.org/pdf/1801.07829). Our code skeleton is borrowed from WangYueFt/dgcnn.

Updates:

The network structure (Fig. 3) for classification in DGCNN paper is not consistent with the corresponding description in section 4.1 of the paper. The author of DGCNN adopts the setting of classification network in section 4.1, not Fig. 3. We fixed this mistake in Fig. 3 using PS and present the revised figure below.

 

<p float="left"> <img src="image/DGCNN.jpg"/> </p>

 

Tip: The result of point cloud experiment usually faces greater randomness than 2D image. We suggest you run your experiment more than one time and select the best result.

 

Requirements

 

Contents

Note: All following commands default use all GPU cards. To specify the cards to use, add CUDA_VISIBLE_DEVICES=0,1,2,3 before each command, where the user uses 4 GPU cards with card index 0,1,2,3. You can change the card number and indexes depending on your own needs.

 

Point Cloud Classification

Run the training script:

python main_cls.py --exp_name=cls_1024 --num_points=1024 --k=20 
python main_cls.py --exp_name=cls_2048 --num_points=2048 --k=40 

Run the evaluation script after training finished:

python main_cls.py --exp_name=cls_1024_eval --num_points=1024 --k=20 --eval=True --model_path=outputs/cls_1024/models/model.t7
python main_cls.py --exp_name=cls_2048_eval --num_points=2048 --k=40 --eval=True --model_path=outputs/cls_2048/models/model.t7

Run the evaluation script with pretrained models:

python main_cls.py --exp_name=cls_1024_eval --num_points=1024 --k=20 --eval=True --model_path=pretrained/model.cls.1024.t7
python main_cls.py --exp_name=cls_2048_eval --num_points=2048 --k=40 --eval=True --model_path=pretrained/model.cls.2048.t7

Performance:

ModelNet40 dataset

Mean Class AccOverall Acc
Paper (1024 points)90.292.9
This repo (1024 points)90.993.3
Paper (2048 points)90.793.5
This repo (2048 points)91.293.6

 

Point Cloud Part Segmentation

Note: The training modes 'full dataset' and 'with class choice' are different.

Run the training script:

python main_partseg.py --exp_name=partseg 
python main_partseg.py --exp_name=partseg_airplane --class_choice=airplane

Run the evaluation script after training finished:

python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=outputs/partseg/models/model.t7
python main_partseg.py --exp_name=partseg_airplane_eval --class_choice=airplane --eval=True --model_path=outputs/partseg_airplane/models/model.t7

Run the evaluation script with pretrained models:

python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=pretrained/model.partseg.t7
python main_partseg.py --exp_name=partseg_airplane_eval --class_choice=airplane --eval=True --model_path=pretrained/model.partseg.airplane.t7

Performance:

ShapeNet part dataset

Mean IoUAirplaneBagCapCarChairEarphoneGuitarKnifeLampLaptopMotorMugPistolRocketSkateboardTable
Shapes269076558983758697873921547451202184283661525271
Paper85.284.083.486.777.890.674.791.287.582.895.766.394.981.163.574.582.6
This repo85.284.580.384.779.891.176.892.087.383.895.769.694.383.751.576.182.8

Visualization:

Usage:

Use --visu to control visualization file.

Use --visu_format to control visualization file format.

Both .txt and .ply file can be loaded into MeshLab for visualization. For the usage of MeshLab on .txt file, see issue #8 for details. The .ply file can be directly loaded into MeshLab by dragging.

The visualization file name follows the format shapename_pred_miou.FILE_TYPE for prediction output or shapename_gt.FILE_TYPE for ground-truth, where miou shows the mIoU prediction for this shape.

Full dataset:

# Use trained model
python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=outputs/partseg/models/model.t7 --visu=airplane_0 --visu_format=ply

# Use pretrained model
python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=pretrained/model.partseg.t7 --visu=airplane_0 --visu_format=ply
# Use trained model
python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=outputs/partseg/models/model.t7 --visu=airplane --visu_format=ply

# Use pretrained model
python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=pretrained/model.partseg.t7 --visu=airplane --visu_format=ply
# Use trained model
python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=outputs/partseg/models/model.t7 --visu=all --visu_format=ply

# Use pretrained model
python main_partseg.py --exp_name=partseg_eval --eval=True --model_path=pretrained/model.partseg.t7 --visu=all --visu_format=ply

With class choice, for example airplane:

# Use trained model
python main_partseg.py --exp_name=partseg_airplane_eval --class_choice=airplane --eval=True --model_path=outputs/partseg_airplane/models/model.t7 --visu=airplane_0 --visu_format=ply

# Use pretrained model
python main_partseg.py --exp_name=partseg_airplane_eval --class_choice=airplane --eval=True --model_path=pretrained/model.partseg.airplane.t7 --visu=airplane_0 --visu_format=ply
# Use trained model
python main_partseg.py --exp_name=partseg_airplane_eval --class_choice=airplane --eval=True --model_path=outputs/partseg_airplane/models/model.t7 --visu=airplane --visu_format=ply

# Use pretrained model
python main_partseg.py --exp_name=partseg_airplane_eval --class_choice=airplane --eval=True --model_path=pretrained/model.partseg.airplane.t7 --visu=airplane --visu_format=ply

Results:

The visualization result of the airplane 0:

<p float="left"> <img src="image/partseg_visu.png"/> </p>

Color map:

<p float="left"> <img src="image/partseg_colors.png"/> </p>

 

Point Cloud Semantic Segmentation on the S3DIS Dataset

The network structure for this task is slightly different with part segmentation, without spatial transform and categorical vector. The MLP in the end is changed into (512, 256, 13) and only one dropout is used after 256.

You have to download Stanford3dDataset_v1.2_Aligned_Version.zip manually from https://goo.gl/forms/4SoGp4KtH1jfRqEj2 and place it under data/

Run the training script:

This task uses 6-fold training, such that 6 models are trained leaving 1 of 6 areas as the testing area for each model.

python main_semseg_s3dis.py --exp_name=semseg_s3dis_6 --test_area=6 

Run the evaluation script after training finished:

python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval_6 --test_area=6 --eval=True --model_root=outputs/semseg_s3dis/models/
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=outputs/semseg_s3dis/models/

Run the evaluation script with pretrained models:

python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval_6 --test_area=6 --eval=True --model_root=pretrained/semseg_s3dis/
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=pretrained/semseg_s3dis/

Performance:

Stanford Large-Scale 3D Indoor Spaces Dataset (S3DIS) dataset

Mean IoUOverall Acc
Paper56.184.1
This repo59.285.0

Visualization:

Usage:

Use --visu to control visualization file.

Use --visu_format to control visualization file format.

Both .txt and .ply file can be loaded into MeshLab for visualization. For the usage of MeshLab on .txt file, see issue #8 for details. The .ply file can be directly loaded into MeshLab by dragging.

The visualization file name follows the format roomname_pred_<miou>.FILE_TYPE for prediction output or roomname_gt.FILE_TYPE for ground-truth, where <miou> shows the mIoU prediction for this room.

Note: In semantic segmentation, you need to first run a training or evaluation command without visualization in the above sections to preprocess dataset. With the dataset well prepared, you can run a command with visualization in the following sections.

Evaluate in area 6 after the model is trained in area 1-5:

# Use trained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval_6 --test_area=6 --eval=True --model_root=outputs/semseg_s3dis/models/ --visu=area_6_office_1 --visu_format=ply

# Use pretrained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval_6 --test_area=6 --eval=True --model_root=pretrained/semseg_s3dis/ --visu=area_6_office_1 --visu_format=ply
# Use trained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval_6 --test_area=6 --eval=True --model_root=outputs/semseg_s3dis/models/ --visu=area_6 --visu_format=ply

# Use pretrained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval_6 --test_area=6 --eval=True --model_root=pretrained/semseg_s3dis/ --visu=area_6 --visu_format=ply

Evaluate in all areas after 6 models are trained:

# Use trained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=outputs/semseg_s3dis/models/ --visu=area_6_office_1 --visu_format=ply

# Use pretrained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=pretrained/semseg_s3dis/ --visu=area_6_office_1 --visu_format=ply
# Use trained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=outputs/semseg_s3dis/models/ --visu=area_6 --visu_format=ply

# Use pretrained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=pretrained/semseg_s3dis/ --visu=area_6 --visu_format=ply
# Use trained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=outputs/semseg_s3dis/models/ --visu=all --visu_format=ply

# Use pretrained model
python main_semseg_s3dis.py --exp_name=semseg_s3dis_eval --test_area=all --eval=True --model_root=pretrained/semseg_s3dis/ --visu=all --visu_format=ply

Results:

The visualization result of the office room 1 in area 6:

<p float="left"> <img src="image/semseg_s3dis_visu.png"/> </p>

Color map:

<p float="left"> <img src="image/semseg_s3dis_colors.png" width="800"/> </p>

 

Point Cloud Semantic Segmentation on the ScanNet Dataset

The DGCNN authors do not test on the ScanNet dataset. We try our best to implement the DGCNN model on the dataset.

Prepare dataset:

You need to change the directory to the prepare_data/ folder.

cd prepare_data/

Please download original dataset from website. You need to place the dataset under data/ScanNet/. The path data/ScanNet includes data/ScanNet/scans/ and data/ScanNet/scans_test/ folder.

To prepare the Scannet dataset for training and evaluation, run

python scannetv2_seg_dataset_rgb21c_pointid.py

This will generate four pickle files: scannet_train_rgb21c_pointid.pickle, scannet_val_rgb21c_pointid.pickle, scannet_val_rgb21c_pointid_keep_unanno.pickle, and scannet_test_rgb21c_pointid_keep_unanno.pickle.

Return to the root directory:

cd ..

Run the training script:

python main_semseg_scannet.py --exp_name=semseg_scannet

To train with both the training split and the validation split, use --train_val=True.

You can use TensorBoard to view the training log under outputs/semseg_scannet/logs/.

Run the evaluation script after training finished:

python main_semseg_scannet.py --eval=True --model_path=outputs/semseg_scannet/models/model_200.pth --exp_name=semseg_scannet_val --split=val
python main_semseg_scannet.py --eval=True --model_path=outputs/semseg_scannet/models/model_200.pth --exp_name=semseg_scannet_test --split=test

Run the evaluation script with pretrained models:

python main_semseg_scannet.py --eval=True --model_path=pretrained/semseg_scannet/models/model_200.pth --exp_name=semseg_scannet_val --split=val
python main_semseg_scannet.py --eval=True --model_path=pretrained/semseg_scannet/models/model_200.pth --exp_name=semseg_scannet_test --split=test

Since there are no ground-truth labels on the testing set, this script will directly save prediction result. You need to upload your prediction results to the website for evaluation.

Performance:

The validation set of the ScanNet Dataset

Mean IoUwallfloorcabinetbedchairsofatabledoorwindowbookshelfpicturecounterdeskcurtainrefrigeratorshower curtaintoiletsinkbathtubotherfurniture
This repo49.673.293.644.964.770.050.555.735.747.769.114.641.845.333.829.235.755.940.256.532.9

The testing set of the ScanNet Dataset

Mean IoUwallfloorcabinetbedchairsofatabledoorwindowbookshelfpicturecounterdeskcurtainrefrigeratorshower curtaintoiletsinkbathtubother furniture
This repo44.672.393.736.662.365.157.744.533.039.446.312.631.034.938.928.522.462.535.047.427.1

These is no official results of DGCNN on the ScanNet dataset. You can find our results on the website as DGCNN_reproduce.

Visualization:

Usage:

Use --visu to control visualization file.

Use --visu_format to control visualization file format.

Both .txt and .ply file can be loaded into MeshLab for visualization. For the usage of MeshLab on .txt file, see issue #8 for details. The .ply file can be directly loaded into MeshLab by dragging.

For example, if you want to visualize scene0568_00 on the pretrained model, run:

python main_semseg_scannet.py --eval=True --model_path=pretrained/semseg_scannet/models/model_200.pth --exp_name=semseg_scannet_val_visu --visu=scene0568_00

Results:

The visualization result of scene0568_00:

<p float="left"> <img src="image/semseg_scannet_visu.png"/> </p>

Color map:

<p float="left"> <img src="image/semseg_scannet_colors.png"/> </p>