Awesome

MeshMAE: Masked Autoencoders for 3D Mesh Data Analysis, ECCV 2022

This is the PyTorch implementation of our MeshMAE.

Requirements

python 3.9+
CUDA 11.1+
torch 1.11+

To install python requirements:

pip install -r requirements.txt

To install PyGem, please refer to Pygem.

Fetch Data

Datasets

Here, we provide the download links of the datasets for pre-train, classification and segmentation.

ModelNet40 here
Humanbody here
COSEG-aliens here
ShapeNet here（we also provide the processed ShapeNet dataset as here）

Please create a new folder 'datasets' in the main root, and put the downloaded datasets in this folder. And '--dataroot' in the 'xxx.sh' refers to the root of datasets.

For example, the root of ModelNet40 should be:

--dataroot ./dataset/Manifold40-MAPS-96-3/

To process the raw data, please use data_preprocess/manifold.py, which can transform non-manifold mesh data into manifold and simplify it to 500 faces.

To remesh the meshes, you can refer to the datagen_maps.py of SubdivNet, which can generate hierarchical structures.

Models

Here, we provide the download links of the pre-trained models.

Task	Dataset	Baseline	Finetune
Pre-training	ShapeNet	here	---
Classification	ModelNet40	here	here
Segmentation	HumanBody	here	here
Segmentation	Coseg-alien	here	here

Pretrain

To pretrain on the ShapeNet dataset, run this command:

sh scripts/pretrain/train_pretrain.sh

Downstream Tasks

Classification

To train the classification model from scratch, run this command:

sh scripts/classification/train.sh

To finetune the classification model, run this command:

sh scripts/classification/train_finetune.sh

Segmentation

To train the segmentation model from scratch, run this command:

sh scripts/segmentation/<...>/train.sh

To finetune the segmentation model, run this command:

sh scripts/segmentation/<...>/train_finetune.sh

Finetune note

To finetune the model, please create a folder 'checkpoints' in the main root, and put the the pre-trained model in it. And '--checkpoint' in the 'train_finetune.sh' refers to the root the pre-trained model.

For example, the root of pre-trained model should be:

--checkpoint "./checkpoints/shapenet_pretrain.pkl"

Reference

@inproceedings{meshmae2022,
  title={MeshMAE: Masked Autoencoders for 3D Mesh Data Analysis},
  author={Liang, Yaqian and Zhao, Shanshan and Yu, Baosheng and Zhang, Jing and He, Fazhi},
  booktitle={European Conference on Computer Vision},
  year={2022},
}