Awesome

Part2Object

By Cheng Shi, Yulin Zhang, Bin Yang, Jiajin Tang, Yuexin Ma and Sibei Yang

The official PyTorch implementation of the "Part2Object: Hierarchical Unsupervised 3D Instance Segmentation".

README structure

• Installation
• Self-Training and Data-efficient
• Main Result and Available Resources

Roadmap

• Installation
• Data download and Preprocessing
• Pseudo Mask Generation
• Upload Pseudo Mask Result
• Self-Training
• Upload Pretrained Models

Installation

We follow Mask3D to install our environment.

Dependencies

The main dependencies of the project are the following:

python: 3.10.9
cuda: 11.3

You can set up a conda environment as follows

# Some users experienced issues on Ubuntu with an AMD CPU
# Install libopenblas-dev (issue #115, thanks WindWing)
# sudo apt-get install libopenblas-dev

export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6"

conda env create -f environment.yml

conda activate part2object

pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
pip3 install torch-scatter -f https://data.pyg.org/whl/torch-1.12.1+cu113.html
pip3 install 'git+https://github.com/facebookresearch/detectron2.git@710e7795d0eeadf9def0e7ef957eea13532e34cf' --no-deps

mkdir third_party
cd third_party

git clone --recursive "https://github.com/NVIDIA/MinkowskiEngine"
cd MinkowskiEngine
git checkout 02fc608bea4c0549b0a7b00ca1bf15dee4a0b228
python setup.py install --force_cuda --blas=openblas

cd ..
git clone https://github.com/ScanNet/ScanNet.git
cd ScanNet/Segmentator
git checkout 3e5726500896748521a6ceb81271b0f5b2c0e7d2
make

cd ../../pointnet2
python setup.py install

cd ../../
pip3 install pytorch-lightning==1.7.2

Self-Training and Data-efficient

You can download our generated pseudo-labels here or generate by yourself with our code.

Train & Evaluation

To train or test the results of Part2Object, modify the file paths appropriately and run the following scripts.

sh scripts/scannet/scannet_val.sh

Train data efficient model

After getting the base model trained with pseudo-labeling, you can train the data efficient model by modifying the following script appropriately.

sh scripts/scannet/scannet_df.sh

Main Result and Available Resources

Pseudo Label

Model

Acknowledgement

We thank Mask3D for their valuable code bases.

If you find Part2Object useful in your research, please consider citing:

@article{shi2024part2object,
  title={Part2Object: Hierarchical Unsupervised 3D Instance Segmentation},
  author={Shi, Cheng and Zhang, Yulin and Yang, Bin and Tang, Jiajin and Ma, Yuexin and Yang, Sibei},
  journal={arXiv preprint arXiv:2407.10084},
  year={2024}
}