Home

Awesome

Hand Biomechanical Constraints Pytorch

Unofficial PyTorch reimplementation of Hand-Biomechanical-Constraints (ECCV2020).

This project reimplement following components :

  1. 3 kinds of biomechanical soft constraints
  2. integrate BMC into training procedure (PyTorch version)

Usage

git clone https://github.com/MengHao666/Hand-BMC-pytorch
cd Hand-BMC-pytorch
conda env create --file=environment.yml
conda activate bmc

Download data

Download 3D joint location data joints.zip Google Drive or Baidu Pan (2pip), and . These statistics are from following datasets:

Note the data from these datasets under their own licenses.

Calculate BMC

BMC

Run the code

python calculate_bmc.py

You will get

And if u want to check the coordinate system, run the code

cd utils
python calculate_joint_angles.py
One viewAnother view

Run the code

python calculate_convex_hull.py

You will get CONVEX_HULLS.npy, i.e. convex hulls to encircle the anatomically plausible joint angles.

And you will also see every convex hull like following figure:

BMC

Run the code

python plot.py

You will see all the convex hulls

BMC

Integrate BMC into training (PyTorch version)

Run the code

python weakloss.py

Experiment results

To check influence of BMC, instead of reimplementing the network of origin paper, I integrate BMC into my own project,

Train and evaluation curve

(AUC means 3D PCK, and ACC_HM means 2D PCK) teaser

3D PCK AUC Diffenence

DatasetDetNetDetNet+BMC
RHD0.93390.9364
STB0.87440.8778
DO0.93780.9475
EO0.92700.9182

Note

Limitation

Citation

This is the unofficial pytorch reimplementation of the paper "Weakly supervised 3d hand pose estimation via biomechanical constraints (ECCV 2020).

If you find the project helpful, please star this project and cite them:

@article{spurr2020weakly,
  title={Weakly supervised 3d hand pose estimation via biomechanical constraints},
  author={Spurr, Adrian and Iqbal, Umar and Molchanov, Pavlo and Hilliges, Otmar and Kautz, Jan},
  journal={arXiv preprint arXiv:2003.09282},
  volume={8},
  year={2020},
  publisher={Springer}
}