Home

Awesome

PHYSIM_6DPose

This tool performs 6DoF Pose estimation for shelf and table-top environments using multi-view RGB-D images. You get the option to use Faster-RCNN or FCN for object segmentation. It also gives option to use PCA and Super4PCS for computing pose estimates. Finally as a post processing one could chose from it performs ICP and physical reasoning (optional).

Installation

  1. install Matlab Robotics toolbox

  2. execute in matlab path-to-repo/ros-packages/src/pose_estimation/src/make.m

  3. setup caffe for Faster-RCNN

  4. setup caffe for FCN

  5. install Blender

  6. realsense camera setup

  7. run cd path-to-repo/ros-packages/src

  8. run catkin_init_workspace

  9. run cd ../

  10. run catkin_make

  11. run cd src/super4pcs

  12. run mkdir build && cd build

  13. run cmake -DCMAKE_BUILD_TYPE=Release -DANN_DIR=$PWD/../ann_1.1.2/ ..

  14. run make

in case dependecies are not installed refer to Super4PCS installation

  1. run cd path-to-repo/ros-packages/src/detection_package/lib

  2. run make

  3. Add the following to ~/.bashrc :- export PHYSIM_6DPose_PATH=path to PHYSIM_6DPose repository export BLENDER_PATH=path to blender

Run Pose Estimation on a demo scene

  1. download rcnn model from this webpage and store it in $PHYSIM_6DPose_PATH/ros-packages/src/detection_package/data/faster_rcnn_models/

  2. run cd $PHYSIM_6DPose_PATH

  3. run ./runMaster.sh

  4. execute in matlab ros-packages/src/pose_estimation/src/poseServiceStart.m

  5. run rosservice call /pose_estimation "path-to-tmp-directory" "path-to-calibration-folder"

Run Pose Estimation on a real setup

  1. run robot.launch (specific to Rutgers) which publishes the realsense camera pose.

  2. run rosrun marvin_convnet save_images _write_directory:="path-to-some-tmp-directory" _camera_service_name:="/realsense_camera"

  3. run rosservice call /save_images ["expo_dry_erase_board_eraser","other-object-names"] binId frameId (for table top you can use 13 as the bin id and for shelf from 1-12)

  4. run rosservice call /pose_estimation "path-to-tmp-directory" "path-to-calibration-folder"

References

  1. Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge : Andy Zeng, Kuan-Ting Yu, Shuran Song, Daniel Suo, Ed Walker Jr., Alberto Rodriguez and Jianxiong Xiao

  2. Super4PCS: Fast Global Pointcloud Registration via Smart Indexing : Mellado, Nicolas and Aiger, Dror and Mitra, Niloy J.

  3. Fast Global registration: Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun