Awesome
Diff-DOPE
A 6-DoF pose refiner that takes as input an image, a 3d model, and an initial object pose. The method then outputs the pose of the object using a differentiable renderer that minimizes the object reprojection error (rgb, depth, edges, mask, etc.). If you want to generate the figure above check out this link.
Installation
conda create -n diffdope python=3.9
conda activate diffdope
pip install -e .
Run simple scene
From the root folder of the this repository, call:
python examples/simple_scene.py
The first run will compile several CUDA kernels, which will take a brief amount of time. Subsequent runs will be faster. After the script finishes, you should see the resulting object pose displayed as a matrix, as well as a filename for a video animation of the optimization process.
Run on BOP
You can check examples/run_bop_scene.py
to see how to load poses from a dataset and run optization. This uses the pertubed poses from the diff-dope paper. The paths in the file are absolute so you are going to need to change them.
Features and TODO
- Add a point cloud visualizer to check the output pose, use open 3d
- Add an example that uses a 3rd party neural network to add as a loss, canny detection, latent space
- The
find_crop
behaviour is not amazing, need to rework.
Development notes
Pre-commit hooks are available to ensure code quality. To install them, run:
pip install pre-commit
pre-commit install
To manually run the hooks without making a commit, call:
pre-commit run --all-files