Awesome

Awesome-Skeleton-Based-Models <!-- omit in toc -->

We collect existing skeleton-based models (369+ Papers & Codes) published in prominent conferences (CVPR, ICCV, ECCV, AAAI, IJCAI, ACMMM, ICLR, ICML, NeurIPS, etc) and journals (TPAMI, IJCV, TIP, TMM, TNNLS, PMLR, etc).

TODO <!-- omit in toc -->

• Paper List
  • Skeleton-Based Action Recognition
  • 3D Pose Estimation
  • Skeleton-Based Person Re-Identification (New!)
  • Skeleton-Based Gesture Recognition
  • Skeleton-Based Gait Recognition
  • Others (Skeleton-Based Motion Prediction, Interaction Recognition, etc)

Contents <!-- omit in toc -->

• 0. Skeleton Tracking

  • Microsoft Kinect
  • Other Tracking Devices

• 1. Skeleton-Based Action Recognition

  • 70 Datasets
  • Survey Papers
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018
  • 2017
  • Before 2017
  • arXiv papers
  • Skeleton-based Action Recognition under Adversarial Attack
  • Leaderboards on NTU-RGB+D and NTU-RGB+D 120 Datasets

• 2. 3D Pose Estimation

  • 13 Datasets
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018
  • 2017
  • Before 2017
  • Real-Time 3D Pose Estimation
  • Leaderboards

• 3. Skeleton-Based Person Re-Identification
  • 6 Datasets
  • 2022
  • 2021
  • 2020
  • 2019
  • Before 2019
  • Leaderboards

Skeleton-Based Action Recognition (2013-2022)

70 Datasets

Overview of 70 available datasets for action recognition and their statistics, provided by the paper of (TPAMI 2022) [arxiv] (Human Action Recognition from Various Data Modalities: A Review).

S: Skeleton, D: Depth, IR: Infrared, PC: Point Cloud, ES: Event Stream, Au: Audio, Ac: Acceleration, Gyr: Gyroscope, EMG: Electromyography. Bold shows the most-frequently used datasets in the literature.

Survey Papers

• Human Action Recognition and Prediction: A Survey Recognition Algorithms (IJCV 2022) [paper]

• Human Action Recognition from Various Data Modalities: A Review (TPAMI 2022) [arxiv]

• A Comparative Review of Recent Kinect-based Action Recognition Algorithms (TIP 2019) [arxiv]

2022

• Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition (TPAMI 2022) [paper] [Github] (Not uploaded) [Gitee]

• [Sym-GNN] Symbiotic Graph Neural Networks for 3D Skeleton-Based Human Action Recognition and Motion Prediction (TPAMI 2022) [paper] [Github] (Not uploaded)

• [X-CAR] X-Invariant Contrastive Augmentation and Representation Learning for Semi-Supervised Skeleton-Based Action Recognition (TIP 2022) [paper]

• [FGCN] Feedback Graph Convolutional Network for Skeleton-Based Action Recognition (TIP 2022) [paper]

• [Multi-LiSAAL] Multi-Localized Sensitive Autoencoder-Attention-LSTM For Skeleton-based Action Recognition (TMM 2022) [paper]

• [LAGA-Net] LAGA-Net: Local-and-Global Attention Network for Skeleton Based Action Recognition (TMM 2022) [paper]

• [CIASA] Adversarial Attack on Skeleton-Based Human Action Recognition (TNNLS 2022) [paper]

• [MTT] MTT: Multi-Scale Temporal Transformer for Skeleton-Based Action Recognition (IEEE Signal Process. Lett. 2022) [paper]

• [Graph2Net] Graph2Net: Perceptually-Enriched Graph Learning for Skeleton-Based Action Recognition (IEEE Trans. Circuits Syst. Video Technol. 2022) [paper] [Github]

• A Cross View Learning Approach for Skeleton-Based Action Recognition (IEEE Trans. Circuits Syst. Video Technol. 2022) [paper]

• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition (ACM Trans. Multim. Comput. Commun. Appl. 2022) [paper] [Github]

• Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognition (ACM Trans. Multim. Comput. Commun. Appl. 2022) [paper]

• [SparseShift-GCN] SparseShift-GCN: High precision skeleton-based action recognition (Pattern Recognit. Lett. 2022) [paper]

• [FR-AGCN] Forward-reverse adaptive graph convolutional networks for skeleton-based action recognition (Neurocomputing 2022) [paper] [Github]

• [ED-GCN] Enhanced discriminative graph convolutional network with adaptive temporal modelling for skeleton-based action recognition (Comput. Vis. Image Underst. 2022) [paper]

2021

• Quo Vadis, Skeleton Action Recognition ? (IJCV 2021) [paper] [Github]

• [CTR-GCN] Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition (ICCV 2021) [paper] [Github]

• Spatio-Temporal Difference Descriptor for Skeleton-Based Action Recognition (AAAI 2021) [paper]

• [AdaSGN] AdaSGN: Adapting Joint Number and Model Size for Efficient Skeleton-Based Action Recognition (ICCV 2021) [paper] [Github]

• Self-supervised 3D Skeleton Action Representation Learning with Motion Consistency and Continuity (ICCV 2021) [paper]

• Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning (ICCV 2021) [paper]

• 3D Human Action Representation Learning via Cross-View Consistency Pursuit (CVPR 2021) [arxiv][Github]

• [MST-GCN] Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition (AAAI 2021) [paper] [Github]

• Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition (ACMMM 2021) [paper] [Github] (Not uploaded)

• Modeling the Uncertainty for Self-supervised 3D Skeleton Action Representation Learning (ACMMM 2021) [paper]

• Skeleton-Contrastive 3D Action Representation Learning (ACMMM 2021) [paper] [Github] (Not uploaded)

• [STST] STST: Spatial-Temporal Specialized Transformer for Skeleton-based Action Recognition (ACMMM 2021) [paper] [Github] (Not uploaded)

• [CP-STN] Spatial Temporal Enhanced Contrastive and Pretext Learning for Skeleton-based Action Representation (PMLR 2021) [paper]

• Tripool: Graph triplet pooling for 3D skeleton-based action recognition (Pattern Recognit. 2021) [paper]

• Structural Knowledge Distillation for Efficient Skeleton-Based Action Recognition (TIP 2021) [paper] [Github]

• [ShiftGCN++] Extremely Lightweight Skeleton-Based Action Recognition With ShiftGCN++ (TIP 2021) [paper] [Github]

• [Hyper-GNN] Hypergraph Neural Network for Skeleton-Based Action Recognition (TIP 2021) [paper]

• [GCN-HCRF] A Multi-Stream Graph Convolutional Networks-Hidden Conditional Random Field Model for Skeleton-Based Action Recognition (TMM 2021) [paper]

• Hierarchical Soft Quantization for Skeleton-Based Human Action Recognition (TMM 2021) [paper]

• Pose Refinement Graph Convolutional Network for Skeleton-Based Action Recognition ({IEEE} Robotics Autom. Lett. 2021) [paper] [Github]

• [FDGCN] Skeleton-Based Action Recognition With Focusing-Diffusion Graph Convolutional Networks (IEEE Signal Process. Lett. 2021) [paper]

• [ST-GDN] Spatial Temporal Graph Deconvolutional Network for Skeleton-Based Human Action Recognition (IEEE Trans. Circuits Syst. Video Technol. 2021) [paper]

• Fuzzy Integral-Based {CNN} Classifier Fusion for 3D Skeleton Action Recognition (IEEE Trans. Circuits Syst. Video Technol. 2021) [paper] [Github]

• [SEFN] Symmetrical Enhanced Fusion Network for Skeleton-Based Action Recognition (IEEE Trans. Circuits Syst. Video Technol. 2021) [paper]

• [RA-GCN] Richly Activated Graph Convolutional Network for Robust Skeleton-Based Action Recognition (IEEE Trans. Circuits Syst. Video Technol. 2021) [paper] [Github]

• Dual-Stream Structured Graph Convolution Network for Skeleton-Based Action Recognition (ACM Trans. Multim. Comput. Commun. Appl. 2021) [paper]

• Action recognition using kinematics posture feature on 3D skeleton joint locations (Pattern Recognit. Lett. 2021) [paper]

• Scene image and human skeleton-based dual-stream human action recognition (Pattern Recognit. Lett. 2021) [paper]

• Skeleton-based action recognition using sparse spatio-temporal GCN with edge effective resistance (Neurocomputing 2021) [paper]

• [VE-GCN] Integrating vertex and edge features with Graph Convolutional Networks for skeleton-based action recognition (Neurocomputing 2021) [paper]

• [AMV-GCNs] Adaptive multi-view graph convolutional networks for skeleton-based action recognition (Neurocomputing 2021) [paper]

• Rethinking the ST-GCNs for 3D skeleton-based human action recognition (Neurocomputing 2021) [paper] [Github]

• [AAM-GCN] Attention adjacency matrix based graph convolutional networks for skeleton-based action recognition (Neurocomputing 2021) [paper]

• [ST-TR] Skeleton-based action recognition via spatial and temporal transformer networks (Comput. Vis. Image Underst. 2021) [paper] [Github]

• [AS-CAL] Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition (Inf. Sci. 2021) [paper] [Github]

2020

• (Mainly from [Github], adding latest status)

• [MV-IGNET] Learning Multi-View Interactional Skeleton Graph for Action Recognition (TPAMI 2020) [paper][Github]

• [MS-AAGCN] Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks (TIP 2020) [paper]

• [P&C FW-AEC] PREDICT & CLUSTER: Unsupervised Skeleton Based Action Recognition (CVPR 2020) [paper]

• [CA-GC] Context Aware Graph Convolution for Skeleton-Based Action Recognition (CVPR 2020) [paper]

• [Shift-GCN] Skeleton-Based Action Recognition With Shift Graph Convolutional Network (CVPR 2020) [paper][Github]

• [DMGNN] Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction (CVPR 2020) [paper]

• [SGN] Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition (CVPR 2020) [arxiv][Github]

• [MS-G3D] Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition (CVPR 2020) [arxiv] [Github]

• Adversarial Self-Supervised Learning for Semi-Supervised 3D Action Recognition (ECCV 2020) [arxiv]

• Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement (ECCV 2020) [arxiv] [Github]

• [DecoupleGCN-DropGraph] Decoupling GCN with DropGraph Module for Skeleton-Based Action Recognition (ECCV 2020) [arxiv] [Github]

• Ms2l: Multi-task self-supervised learning for skeleton based action recognition (ACMMM 2020) [arxiv]

• [Dynamic GCN] Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition (ACM-MM 2020)[arxiv]

• [GCN-NAS] Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching (AAAI 2020) [arxiv] [Github]

• [PA-ResGCN] Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition (ACM-MM 2020) [arxiv] [Github]

• [Poincare-GCN] Mix Dimension in Poincaré Geometry for 3D Skeleton-based Action Recognition (ACM-MM 2020) [arxiv]

• [STIGCN] Spatio-Temporal Inception Graph Convolutional for Skeleton-Based Action Recognition (ACM-MM 2020) [arxiv]

• [JOLO-GCN] JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognition (WACV 2021) [arxiv]

• [ST-TR-AGCN] Spatial Temporal Transformer Network for Skeleton-based Action Recognition (Under submission at Computer Vision and Image Understanding (CVIU)) [arxiv] [Github]

• [PCRP] Prototypical Contrast and Reverse Prediction: Unsupervised Skeleton Based Action Recognition [arxiv] [Github]

2019

• NTU-RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding (TPAMI 2019) [arxiv] [Homepage] [Github]
• [VA-NN] View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition (TPAMI 2019) [arxiv] [Github]
• Bayesian Graph Convolutional LSTM for Skeleton Based Action Recognition (ICCV 2019) [arxiv]
• [2s-SDGCN] Spatial Residual Layer and Dense Connection Block Enhanced Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition (ICCV Workshop 2019) [paper]
• [DGNN] Skeleton-Based Action Recognition With Directed Graph Neural Networks (CVPR 2019) [paper] [unofficial PyTorch implementation]
• [2s-AGCN] Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition (CVPR 2019) [paper] [Github]
• [AS-GCN] Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition (CVPR 2019) [arxiv] [Github]
• [AGC-LSTM] An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition (CVPR 2019) [arxiv]
• Optimized Skeleton-based Action Recognition via Sparsified Graph Regression (ACMMM 2019) [paper]
• [Motif-STGCN] Graph CNNs with Motif and Variable Temporal Block for Skeleton-based Action Recognition (AAAI 2019) [arxiv] [Github]
• Richly Activated Graph Convolutional Network for Action Recognition with Incomplete Skeletons (ICIP 2019) [arxiv] [Github]
• [TSRJI] Skeleton Image Representation for 3D Action Recognition based on Tree Structure and Reference Joints (SIBGRAPI) [arxiv] [Github]
• [SkeleMotion] SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition (AVSS) [arxiv] [Github]

2018

• Beyond Joints: Learning Representations from Primitive Geometries for Skeleton-based Action Recognition and Detection (TIP 2018) [paper] [Github]
• [DPRL] Deep progressive reinforcement learning for skeleton-based action recognition (CVPR 2018) [paper]
• [SR-TSL] Skeleton based action recognition with spatial reasoning and temporal stack learning (ECCV 2018) [arxiv]
• [HCN] Co-occurrence feature learning from skeleton data for action recognition and detection with hierarchical aggregation (IJCAI 2018) [arxiv] [Reimplementation]
• [MAN] Memory attention networks for skeleton-based action recognition (IJCAI 2018) [arxiv] [Github]
• [ST-GCN] Spatial temporal graph convolutional networks for skeleton-based action recognition (AAAI 2018) [arxiv] [Github]
• Unsupervised representation learning with long-term dynamics for skeleton based action recognition (AAAI 2018) [arxiv] [Github]
• Spatio-temporal graph convolution for skeleton based action recognition (AAAI 2018) [arxiv]
• Part-based Graph Convolutional Network for Action Recognition (BMVC 2018) [arxiv] [Github]
• A Fine-to-Coarse Convolutional Neural Network for 3D Human Action Recognition (BMVC 2018) [arxiv]
• A Large-scale Varying-view RGB-D Action Dataset for Arbitrary-view Human Action Recognition (ACMMM 2018) [arxiv]
• Unsupervised feature learning of human actions as trajectories in pose embedding manifold (WACV 2018) [arxiv]

2017

• Jointly learning heterogeneous features for RGB-D activity recognition (TPAMI 2017) [paper]
• [Visualization CNN] Enhanced skeleton visualization for view invariant human action recognition (Pattern Recognition 2017) [paper]
• Global context-aware attention lstm networks for 3d action recognition (CVPR 2017) [paper]
• [Two-stream RNN] Modeling temporal dynamics and spatial configurations of actions using two-stream recurrent neural networks (CVPR 2017) [arxiv] [Github]
• [C-CNN + MTLN] A new representation of skeleton sequences for 3d action recognition (CVPR 2017) [arxiv]
• [Ensemble TS-LSTM] Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks (ICCV 2017) [paper] [Github]
• [VA-LSTM] View adaptive recurrent neural networks for high performance human action recognition from skeleton data (ICCV 2017) [arxiv]
• Learning action recognition model from depth and skeleton videos (ICCV 2017) [paper]
• [STA-LSTM] An end-to-end spatio-temporal attention model for human action recognition from skeleton data (AAAI 2017) [arxiv]
• Skeleton-based action recognition using LSTM and CNN (ICME Workshop 2017) [arxiv]
• Skeleton-based action recognition with convolutional neural networks (ICME Workshop 2017) [arxiv]
• PKU-MMD: A large scale benchmark for continuous multi-modal human action understanding (ACMMM Workshop 2017) [arxiv]
• [Temporal Conv] Interpretable 3d human action analysis with temporal convolutional networks (CVPR Workshop 2017) [arxiv]

Before 2017

• [Trust Gate ST-LSTM] Spatio-temporal lstm with trust gates for 3d human action recognition (ECCV 2016) [arxiv] [Github]
• [Part-aware LSTM] NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (CVPR 2016) [arxiv]
• Rolling rotations for recognizing human actions from 3d skeletal data (CVPR 2016) [paper]
• Co-occurrence feature learning for skeleton based action recognition using regularized deep lstm networks (AAAI 2016) [paper]
• Skeleton based action recognition with convolutional neural network (ACPR 2015) [paper]
• [H-RNN] Hierarchical recurrent neural network for skeleton based action recognition (CVPR 2015) [paper]
• Jointly learning heterogeneous features for rgb-d activity recognition (CVPR 2015) [paper]
• [LieGroup] Human action recognition by representing 3d skeletons as points in a lie group (CVPR 2014) [paper]
• Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations (IJCAI 2013) [paper]

arXiv papers

• Skeleton-DML: Deep Metric Learning for Skeleton-Based One-Shot Action Recognition [arxiv][Github] (Accepted at WACV 2022)
• Sparse Semi-Supervised Action Recognition with Active Learning [arxiv]
• STAR: Sparse Transformer-based Action Recognition [arxiv] [Github]
• [DenseIndRNN] Deep Independently Recurrent Neural Network (Preprint) [arxiv] [Github]
• Skeleton-Based Action Recognition with Synchronous Local and Non-local Spatio-temporal Learning and Frequency Attention [arxiv] (Accepted at ICME 2019)
• [DSTA-Net] Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognition [arxiv] (Accepted at ACCV 2020)
• SynSE: Syntactically Guided Generative Embeddings for Zero Shot Skeleton Action Recognition [arxiv] [Github] (Accepted at ICIP 2021)
• [PoseC3D] Revisiting Skeleton-based Action Recognition [arxiv][Github]
• Leveraging Third-Order Features in Skeleton-Based Action Recognition [arxiv][Github]

Skeleton-based Action Recognition under Adversarial Attack

• Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack (CVPR 2021) [arxiv]
• BASAR:Black-box Attack on Skeletal Action Recognition (CVPR 2021) [arxiv]

Leaderboards on NTU-RGB+D and NTU-RGB+D 120 Datasets

NTU-RGB+D

NTU-RGB+D 120

Others

• Kinetics-Skeleton Dataset
• SYSU 3D
• N-UCLA
• SBU
• CAD-120
• UT-Kinect
• MSR Action3D
• J-HMDB
• CAD-120
• Florence 3D
• UAV-Human

Acknowledge

If you have any problems, suggestions or improvements, please feel free to contact me (haocongrao@gmail.com). Welcome to refine current taxonomy, enrich collection of skeleton-based models, and discuss any constructive ideas.