Home

Awesome

ModelNet-C (now part of PointCloud-C)

:warning: This repo has been integrated into our new benchmark PointCloud-C! Please checkout PointCloud-C for any future updates!

Code for the paper "Benchmarking and Analyzing Point Cloud Classification under Corruptions". For the latest updates, see: sites.google.com/view/modelnetc/home

Benchmarking and Analyzing Point Cloud Classification under Corruptions
Jiawei Ren, Liang Pan, Ziwei Liu

arXiv 2022

<img src="figs/corruptions.png" alt="corruptions" width="400"/>

ModelNet-C [Download Link]

Get Started

Step 0. Clone the Repo

git clone https://github.com/jiawei-ren/ModelNet-C.git
cd ModelNet-C

Step 1. Set Up the Environment

Set up the environment by:

conda create --name modelnetc python=3.7.5
conda activate modelnetc
pip install -r requirements.txt
cd SimpleView/pointnet2_pyt && pip install -e . && cd -
pip install -e modelnetc_utils

Step 2. Prepare Data

Download ModelNet-C by:

cd data
gdown https://drive.google.com/uc?id=1KE6MmXMtfu_mgxg4qLPdEwVD5As8B0rm
unzip modelnet_c.zip && cd ..

Alternatively, you may download ModelNet40-C manually and extract it under data.

Step 3. Download Pretrained Models

Download pretrained models by

gdown https://drive.google.com/uc?id=11RONLZGg0ezxC16n57PiEZouqC5L0b_h
unzip pretrained_models.zip

Alternatively, you may download pretrained models manually and extract it under root directory.

Benchmark on ModelNet-C

Evaluation Commands

Evaluation commands are provided in EVALUATE.md.

Customize ModelNet-C Evaluation for Your Codebase

We have provided evaluation utilities to help you evaluate on ModelNet-C using your own codebase. Please follow CUSTOMIZE.md.

Benchmark Results

MethodReferenceStandalonemCEClean OA
DGCNNWang et al.Yes1.0000.926
PointNetQi et al.Yes1.4220.907
PointNet++Qi et al.Yes1.0720.930
RSCNNLiu et al.Yes1.1300.923
SimpleViewGoyal et al.Yes1.0470.939
GDANetXu et al.Yes0.8920.934
CurveNetXiang et al.Yes0.9270.938
PAConvXu et al.Yes1.1040.936
PCTGuo et al.Yes0.9250.930
RPCRen et al.Yes0.8630.930
DGCNN+PointWOLFKim et al.No0.8140.926
DGCNN+RSMixLee et al.No0.7450.930
DGCNN+WOLFMixRen et al.No0.5900.932
GDANet+WOLFMixRen et al.No0.5710.934

*Standalone indicates if the method is a standalone architecture or a combination with augmentation or pretrain.

Todos

Cite ModelNet-C

@article{
    ren2022modelnetc,
    title={Benchmarking and Analyzing Point Cloud Classification under Corruptions},
    author={Jiawei Ren and Liang Pan and Ziwei Liu},
    journal={arXiv:2202.03377},
    year={2022},
}

Acknowledgement

This codebase heavily borrows codes from the following repositories: