Awesome
Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer
PyTorch implementation of CtRNet https://arxiv.org/abs/2302.14332
Dependencies
Recommend set up the environment using Anaconda. Code is developed and tested on Ubuntu 20.04.
- Python(3.8)
- Numpy(1.22.4)
- PyTorch(1.10.0)
- torchvision(0.11.1)
- pytorch3d(0.6.2)
- Kornia(0.6.3)
- Transforms3d(0.3.1)
More details see environment.yml
.
Usage
- See
inference_single_frame.ipynb
for example single frame inference. - We provide ROS node for CtRNet, which subscribes image and joint state topics and publishes robot pose.
python ros_node/panda_pose.py
Dataset
Weights
Weights for Panda and Baxter can be found here.
Videos
Using CtRNet for visual servoing experiment with moving camera.