Home

Awesome

ARCTIC 🥶: A Dataset for Dexterous Bimanual Hand-Object Manipulation

👉I plan to enter the job market in Summer/Fall 2025. If you have an openning, feel free to email!👈

<p align="center"> <img src="docs/static/arctic-logo.svg" alt="Image" width="600" height="100" /> </p>

[ Project Page ] [ Paper ] [ Video ] [ Register ARCTIC Account ] [ ECCV'24 Competition ] [ Leaderboard ]

<p align="center"> <img src="docs/static/teaser.jpeg" alt="Image" width="100%"/> </p>

This is a repository for preprocessing, splitting, visualizing, and rendering (RGB, depth, segmentation masks) the ARCTIC dataset. Further, here, we provide code to reproduce our baseline models in our CVPR 2023 paper (Vancouver, British Columbia 🇨🇦) and developing custom models.

Our dataset contains heavily dexterous motion:

<p align="center"> <img src="./docs/static/dexterous.gif" alt="Image" width="100%"/> </p>

News

✨CVPR 2024 Highlight: HOLD is the first method that jointly reconstructs articulated hands and objects from monocular videos without assuming a pre-scanned object template and 3D hand-object training data. See our project page for details.

<p align="center"> <img src="./docs/static/hold/mug_ours.gif" alt="HOLD Reconstruction Example" width="300"/> <!-- Adjust width as needed --> </p> <p align="center"> <img src="./docs/static/hold/mug_ref.png" alt="Reference for HOLD Reconstruction" width="300"/> <!-- Adjust width as needed --> </p>

Invited talks/posters at CVPR2023:

Why use ARCTIC?

Summary on dataset:

Potential tasks with ARCTIC:

Check out our project page for more details.

Third-party ARCTIC resources

Projects that use ARCTIC

Reconstruction:

Generation:

Create a pull request for missing projects.

Features

<p align="center"> <img src="./docs/static/viewer_demo.gif" alt="Image" width="80%"/> </p>

Getting started

Get a copy of the code:

git clone https://github.com/zc-alexfan/arctic.git

License

See LICENSE.

Citation

@inproceedings{fan2023arctic,
  title = {{ARCTIC}: A Dataset for Dexterous Bimanual Hand-Object Manipulation},
  author = {Fan, Zicong and Taheri, Omid and Tzionas, Dimitrios and Kocabas, Muhammed and Kaufmann, Manuel and Black, Michael J. and Hilliges, Otmar},
  booktitle = {Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2023}
}

Our paper benefits a lot from aitviewer. If you find our viewer useful, to appreciate their hard work, consider citing:

@software{kaufmann_vechev_aitviewer_2022,
  author = {Kaufmann, Manuel and Vechev, Velko and Mylonopoulos, Dario},
  doi = {10.5281/zenodo.1234},
  month = {7},
  title = {{aitviewer}},
  url = {https://github.com/eth-ait/aitviewer},
  year = {2022}
}

Acknowledgments

Constructing the ARCTIC dataset is a huge effort. The authors deeply thank: Tsvetelina Alexiadis (TA) for trial coordination; Markus Höschle (MH), Senya Polikovsky, Matvey Safroshkin, Tobias Bauch (TB) for the capture setup; MH, TA and Galina Henz for data capture; Priyanka Patel for alignment; Giorgio Becherini and Nima Ghorbani for MoSh++; Leyre Sánchez Vinuela, Andres Camilo Mendoza Patino, Mustafa Alperen Ekinci for data cleaning; TB for Vicon support; MH and Jakob Reinhardt for object scanning; Taylor McConnell for Vicon support, and data cleaning coordination; Benjamin Pellkofer for IT/web support; Neelay Shah, Jean-Claude Passy, Valkyrie Felso for evaluation server. We also thank Adrian Spurr and Xu Chen for insightful discussion. OT and DT were supported by the German Federal Ministry of Education and Research (BMBF): Tübingen AI Center, FKZ: 01IS18039B".

Contact

For technical questions, please create an issue. For other questions, please contact arctic@tue.mpg.de.

For commercial licensing, please contact ps-licensing@tue.mpg.de.

Star History

Star History Chart