Awesome

BABEL: Bodies, Action and Behavior with English Labels [CVPR 2021]

Abhinanda R. Punnakkal*, Arjun Chandrasekaran*, Nikos Athanasiou, Alejandra Quiros-Ramirez, Michael J. Black. * denotes equal contribution

Project Website | Paper | Video | Poster

BABEL is a large dataset with language labels describing the actions being performed in mocap sequences. BABEL labels about 43 hours of mocap sequences from AMASS [1] with action labels. Sequences have action labels at two possible levels of abstraction:

• Sequence labels which describe the overall action in the sequence
• Frame labels which describe all actions in every frame of the sequence. Each frame label is precisely aligned with the duration of the corresponding action in the mocap sequence, and multiple actions can overlap.

To download the BABEL action labels, visit our 'Data' page. You can download the mocap sequences from AMASS.

Tutorials

We release some helper code in Jupyter notebooks to load the BABEL dataset, visualize mocap sequences and their action labels, search BABEL for sequences containing specific actions, etc.

See notebooks/ for more details.

Action Recognition

We provide features, training and inference code, and pre-trained checkpoints for 3D skeleton-based action recognition.

Please see action_recognition/ for more details.

Acknowledgements

We thank the Software Workshop at MPI for building the action recognition test set evaluation web server. The notebooks in this repo are inspired by the those provided by AMASS. The Action Recognition code is based on the 2s-AGCN [2] implementation.

References

[1] Mahmood, Naureen, et al. "AMASS: Archive of motion capture as surface shapes." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. <br> [2] Shi, Lei, et al. "Two-stream adaptive graph convolutional networks for skeleton-based action recognition." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.

License

Software Copyright License for non-commercial scientific research purposes. Please read carefully the terms and conditions and any accompanying documentation before you download and/or use the BABEL dataset, and software, (the "Model & Software"). By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use of this GitHub repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License.

Contact

The code in this repository is developed by Abhinanda Punnakkal and Arjun Chandrasekaran, and tested by Nikos Athanasiou.

If you have any questions you can contact us at babel@tue.mpg.de.