Awesome
CloudAAE
This is an tensorflow implementation of "CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds"
Files
- log: directory to store log files during training.
- losses: loss functions for training.
- models: a python file defining model structure.
- object_model_tfrecord: full object models for data synthesizing and visualization purpose.
- tf_ops: tensorflow implementation of sampling operations (credit: Haoqiang Fan, Charles R. Qi).
- trained_network: a trained network.
- utils: utility files for defining model structure.
- ycb_video_data_tfRecords: synthetic training data and real test data for the YCB video dataset.
- evaluate_cloudAAE_ycbv.py: script for testing object 6d pose estimation with a trained network on test set in YCB video dataset.
- train_cloudAAE_ycbv.py: script for training a network on synthetic data for YCB objects.
Requirements
- Tensorflow-GPU (tested with 1.12.0)
- transforms3d
- open3d for visualization
Test a trained network
- Testing data in tfrecord format is available
- Download zip file
- Unzip and place all files in ycb_video_data_tfRecords/test_real/
- After activate tensorflow
python evaluate_cloudAAE_ycbv.py --trained_model trained_network/20200908-204328/model.ckpt --batch_size 1 --target_cls 0
- --trained_model: directory to trained model (*.ckpt).
- --batch_size: 1.
- --target_class: target class for pose estimation.
- Translation prediction is in unit meter.
- Rotation prediction is in axis-angle format.
- Result
- If you turn on visualization with b_visual=True, you will see the following displays which are partially observed point cloud segments (red) overlaid with object model (green) with pose estimates. The reconstructed point cloud is also presented (blue).
- The coordinate is the object coordinate, object segment is viewed in the camera coordinate
Train a network
- Training data is created synthetically using 3D object model and 6D poses.
- The 6D pose and class id of target object are in ycb_video_data_tfRecords/train_syn/
- The data synthesis pipeline takes the target 3D object model and creates a segment of the object in the desired 6D pose. Below is two examples of synthetic segment (red), two real segments (red) are also shown for comparison.
- Run script
python train_cloudAAE_ycbv.py
- Log files and trained model is store in log
Citation
If you use this code in an academic context, please consider cite the paper:
BiBTeX:
@inproceedings{gao2020cloudpose,
title={CloudAAE: Learning 6D Object Pose Regression with On-line Data
Synthesis on Point Clouds},
author={G. Gao, M. Lauri, X. Hu, J. Zhang and S. Frintrop},
booktitle={ICRA},
year={2021}
}
Link to Paper
TBA
Acknowledgement
- The building block for this system is PointNet and Dynamic Graph.