Awesome

V-PRISM: Probabilistic Mapping of Unknown Tabletop Scenes

Overview

This is the code for the paper "V-PRISM: Probabilistic Mapping of Unknown Tabletop Scenes". Our method builds a 3D probabilistic map of a tabletop scene by using a feature transform and performing an EM algorithm. We employ a novel negative sampling technique before our feature transform to fully encode information about the scene. Our method not only reconstructs the scene accurately, but encodes valuable information about uncertainty due to occlusion. In our paper, we perform both qualitative and quantitative experiments to verify these claims. See below for a figure outlining our method.

Usage

Getting Started:

1. clone the repo

   git clone https://github.com/Herb-Wright/v-prism.git v_prism
   cd v_prism
   

2. create the conda environment (if you don't have cuda, use env_cpu.yml and v_prism_cpu instead):

   conda env create -f environment.yml
   conda activate v_prism
   

3. install the project

   pip install -e .
   

Using VPRISM:

Here is an example:

from v_prism import full_VPRISM_method

map = full_VPRISM_method(X, y, num_classes, cam_pos)

See ./notebooks/example_demo.ipynb for a more detailed example.

Running Experiments:

1. download the items you need to run the particular experiment:
   • ShapeNet Scenes - download
   • YCB Scenes - download
   • Objaverse Scenes - download
   • ShapeNetCore.v2 Dataset - website
   • YCB Dataset - website
   • Objaverse .obj files (Not Available)
   • PointSDF weights - download
2. run the script python ./scripts/eval_sim_data.py with desired arguments

Note. You might not be able to run objaverse experiments because I had to do some extra steps to convert certain meshes to obj files.

Resources

Please consider citing our work:

@article{wright2024vprism,
  title={V-PRISM: Probabilistic Mapping of Unknown Tabletop Scenes},
  author={Wright, Herbert and Zhi, Weiming and Johnson-Roberson, Matthew and Hermans, Tucker},
  journal={arXiv preprint arXiv:2403.08106},
  year={2024}
}

Links:

• Arxiv
• Website