Home

Awesome

arXiv code visitors GitHub license

ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction [CVPR 2023]

This is the official repository of the ACR.

🔥(CVPR 2023) ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction

Zhengdi Yu, Shaoli Huang, Chen Fang, Toby P. Breckon, Jue Wang

Conference on Computer Vision and Pattern Recognition (CVPR), 2023

[Paper][Project Page][Video]

<p float="left"> <img src="docs/p1.GIF" width="49%" /> <img src="docs/P2.GIF" width="49%" /> </p>

News :triangular_flag_on_post:

Requirements

Conda environments

conda create -n ACR python==3.8.8  
conda activate ACR 
conda install -n ACR pytorch==1.10.0 torchvision==0.11.1 cudatoolkit=10.2 -c pytorch
pip install -r requirements.txt

For rendering and visualization on headless server, please consider install pytorch3d follow the official instruction and set renderer to pytorch3d in configs/demo.yml. Note that pyrender can only be used on desktop.

Pre-trained model and data

Demo

Note: use -t to smooth your results. We provide examples in demos/

# Run a real-time demo:
python -m acr.main --demo_mode webcam -t

# Run on a single image:
python -m acr.main --demo_mode image --inputs <PATH_TO_IMAGE>

# Run on a folder of images:
python -m acr.main --demo_mode folder -t --inputs <PATH_TO_FOLDER> 

# Run on a video:
python -m acr.main --demo_mode video -t --inputs <PATH_TO_VIDEO> 

Finally, the visualization will be saved in demos_outputs/. In video or folder mode, the results will also be saved as <FILENAME>_output.mp4.

More qualitative results

image wild

Applications

<p float="left"> <img src="docs/1.gif" width="33%" /> <img src="docs/2.gif" width="31%" /> <img src="docs/3.gif" width="33%" /> </p> <p float="left"> <img src="docs/4.gif" width="49.5%" /> <img src="docs/5.gif" width="49.5%" /> </p>

Citation

@inproceedings{yu2023acr,
  title = {ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction},
  author = {Yu, Zhengdi and Huang, Shaoli and Chen, Fang and Breckon, Toby P. and Wang, Jue},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2023}
  }

Acknowledgement

The pytorch implementation of MANO is based on manopth. We use some parts of the great code from ROMP. For MANO segmentation and rendering, we follow zc-alexfan. We thank all the authors for their impressive works!

Contact

For technical questions, please contact z.yu23@imperial.ac.uk or ZhengdiYu@hotmail.com

For commercial licensing, please contact shaolihuang@tencent.com