Awesome

A Toolbox for HANDS'19 Challenge

An analysis of the participated approaches and the results are summarised in:

Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction

Video presentations to be presented at ECCV'20 can be found here:

If you find our work useful in your research, please consider citing our paper:

@InProceedings{armaganeccv2020,
 title   = {Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction},
 author  = {Anil Armagan and Guillermo Garcia-Hernando and Seungryul Baek and Shreyas Hampali and Mahdi Rad and Zhaohui Zhang and Shipeng Xie and MingXiu Chen and Boshen Zhang and Fu Xiong and Yang Xiao and Zhiguo Cao and Junsong Yuan and Pengfei Ren and Weiting Huang and Haifeng Sun and Marek Hr\'{u}z and Jakub Kanis and Zden\v{e}k Kr\v{n}oul and Qingfu Wan and Shile Li and Linlin Yang and Dongheui Lee and Angela Yao and Weiguo Zhou and Sijia Mei and Yunhui Liu and Adrian Spurr and Umar Iqbal and Pavlo Molchanov and Philippe Weinzaepfel and Romain Br\'{e}gier and Gr\'{e}gory Rogez and Vincent Lepetit and Tae-Kyun Kim},
 booktitle = {European Conference on Computer Vision ({ECCV})},
 year = {2020}
}

Getting Started

• Download the challenge dataset. HANDS'19 is hosted on CodaLab for you to signin and participate. Please fill the corresponding form on CodaLab to get access to the dataset. Please follow the challenge website for more details on the dataset.
• The toolbox should be under the same directory as HANDS'19 dataset.
• Download the MANO right hand model from MANO download page and place it under the toolbox directory.
• Dataset images with ground-truth annotations and renderings of the MANO model can visualized with visualize_task1_task2.py for Task 1 and Task 2, and with visualize_task3.py for Task 3.

usage:  cd your_directory/HANDS19-Challenge-Toolbox

	python3 visualize_task1_task2.py --task-id=1 --frame-id=0 --mano-model-path=./MANO_RIGHT.pkl
	OR
	python3 visualize_task3.py --frame-id=0 --mano-model-path=./MANO_RIGHT.pkl

Dependencies

This code is tested under Ubuntu16.04 with python3.5.

Python dependencies can be installed with pip3 install -r requirements.txt. Some dependencies are listed as: torch, moderngl, opendr, opencv-python, argparse, pickle, numpy and matplotlib. Please see requirements.txt for version specifications.

Licence

Acknowledgements

This code has been written and modified by Anil Armagan and Seungryul Baek using the sources below and Shreyas Hampali's sources on visualization of HO-3D dataset.

Thanks to the below sources we are able to provide this code:

• MANO model is licensed under: http://mano.is.tue.mpg.de/license
• The provided MANO class is adapted from PyTorch implementation of a great work by Kanazawa et al..
• Rendering codes for Task 3 MANO model is implemented with OpenDr.

@article{MANO:SIGGRAPHASIA:2017,
      title = {Embodied Hands: Modeling and Capturing Hands and Bodies Together},
      author = {Romero, Javier and Tzionas, Dimitrios and Black, Michael J.},
      journal = {ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)},
      volume = {36},
      number = {6},
      series = {245:1--245:17},
      month = nov,
      year = {2017},
      month_numeric = {11}
}
@article{hampali2019arxivv1,
	author = {{Hampali}, Shreyas and {Oberweger}, Markus and {Rad}, Mahdi and {Lepetit}, Vincent},
	title = "{HO-3D: A Multi-User, Multi-Object Dataset for Joint 3D Hand-Object Pose Estimation}",
	journal = {arXiv e-print arXiv:1907.01481v1},
	year = {2019}
}
@InProceedings{sbaekcvpr2019,
	title={Pushing the envelope for RGB-based dense 3D hand pose estimation via neural rendering},
	author={Seungryul Baek and Kwang In Kim and Tae-Kyun Kim},
	journal={Proceedings of the {IEEE} Conference on Computer Vision and Pattern Recognition ({CVPR})},
	year={2019}
}
@InProceedings{hmrKanazawa17,
  title={End-to-end Recovery of Human Shape and Pose},
  author = {Angjoo Kanazawa and Michael J. Black and David W. Jacobs and Jitendra Malik},
  booktitle={Computer Vision and Pattern Regognition (CVPR)},
  year={2018}
}