Awesome
Awesome AI Sign Language Papers & Popularization of Professional Knowledge [UPDATING]
This repository is all you need to open the door about AI sign languages!
Keywords: Sign Language, Sign Language Translation (SLT), Sign Language Recognition (SLR), Sign Language Production (SLP), Sign Language Retrieval, Sign Language Linguistics
<details><summary><b>What's New</b></summary> `Update 2023-12-23:` More concise layout & updated contents & Add the introduction of sign language knowledge. </details> <details open><summary><b name="table-of-content">Table of contents</b></summary>-
AI Sign Language in Well-Known Institutions [Papers]
- XMU (厦门大学) SL ->
active
- USTC (中国科学技术大学) SL
- ZJU (浙江大学) SL ->
active
- THU (清华大学) SL
- Germany-UK (英德) SL
- XMU (厦门大学) SL ->
Introduction <a name="introduction"></a>
This repository is for those interested in the field of AI sign language (SL). The collected papers have been categorized according to different criteria (btime, type, institution, etc.) for ease of searching.
If useful, please Star
this repo, we are keeping it updated.
NOTE: There is overlap between the different categories. Please feel free to submit your Pull Requests for any updates.
- Collaborators & Contributors (welcome to join us!):
- Jiangbin Zheng, Westlake University & Zhejiang University, China
- Yidong Chen & Pei Yu & Rui Zhao, Xiamen University, China
- Haodong Zhang, Zhejiang University, China
- Junfu Pu, University Of Science and Technology of China
Popularization of Professional Sign Language Knowledge <a name="sl_knowledge"></a>
- Why AI Sign Language Research? <a name="why_sl"></a>
<details open><summary><b name="table-of-content">Unfolding Details</b></summary> <!-- ![](./src/fig1.jpg) --> <center> <img style="border-radius: 0.3125em; box-shadow: 0 2px 4px 0 rgba(34,36,38,.12),0 2px 10px 0 rgba(34,36,38,.08);width:60%" src="./src/fig1.jpg"> <br> <div style="color:orange; border-bottom: 1px solid #d9d9d9; display: inline-block; color: #999; padding: 2px;">Figure 1. Communication gap between the hearing and the deaf. </div> </center>Sign languages are the primary language of the deaf community. However, most hearing people find it difficult to understand sign languages. With the development of AI, researchers are trying to help people understand sign languages using AI techniques that are designed to convert sign languages into spoken languages in textual form.
</details>- Do you know the differences between SLT and SLR? <a name="slt_vs_slr"></a>
<details open><summary><b name="table-of-content">Unfolding Details</b></summary> <!-- ![](./src/fig2.jpg) --> <center> <img style="border-radius: 0.3125em; box-shadow: 0 2px 4px 0 rgba(34,36,38,.12),0 2px 10px 0 rgba(34,36,38,.08);" src="./src/fig2.jpg"> <br> <div style="color:orange; border-bottom: 1px solid #d9d9d9; display: inline-block; color: #999; padding: 2px;">Figure 2. SLR (Sign Language Recognition) vs. SLT (Sign Language Translation). </div> </center>At the very beggining, I wanna explain the difference between SLT and SLR, as shown in Fig. 2. I'm sure this is very important for most of you!
In early efforts, researchers explored this problem (sign languages -> text-form glosses) as a recognition problem (i.e., SLR), which converts sign languages to glosses
word by word according to the sign languages sequentially. Although glosses are in textual form, they do not provide meaningful interpretations of what a signer is saying because sign languages and glosses have their own specific linguistic rules, which are quite different from spoken languages.
As a result, researchers find it terrible to ignore the linguistic properties of sign language. Contrary to SLR, sign language translaton (SLT) systems aim to translate sign language videos into spoken sentences directly.
As far as AI technology is concerned, SLR belongs to the field of Computer Vision + Text Recognition, while SLT belongs to the field of Computer Vision + Text Translation.
</details>- More details you must know about SLR tasks? <a name="slr_details"></a>
<details open><summary><b name="table-of-content">Unfolding Details</b></summary>By default, we refer to the current Continuous SLR
as SLR because this type of SLR is the mainstream of the deep learning era.
In fact, it is important to note that SLR tasks are generally divided into three categories: finger-spelling recognition, isolated word recognition and continuous sign language recognition. In the early days (before about 2018), SLR works mainly focused on lexical-level tasks, such as finger-spelling recognition and isolated word recognition. Nowadays, the more practical continuous SLR has become mainstream sign language research
For more categorization and details, we recommend you read this survey/review paper for a detailed look at AI Sign history.
</details>SURVEY/REVIEW PAPERS <a name="survey"></a>
- 【IEEE Access 2019 (
Recommended
)】Technical approaches to Chinese sign language processing: A review. Suhail Muhammad Kamal; Yidong Chen; Shaozi Li; Xiaodong Shi; Jiangbin Zheng. [paper]Tags:
Survey
AI Sign Language in Timeline [Papers] <a name='sl_paper_timeline'></a>
2024 <a name='sl_paper_2024'></a>
- 【WACV 2024】Sign Language Production With Latent Motion Transformer.. Pan Xie, Taiying Peng, Yao Du, Qipeng Zhang. [paper] code
Tags:
SLP - 【ACL 2024】Sign Language Translation with Sentence Embedding Supervision. Yasser Hamidullah, Josef van Genabith, Cristina Espa?a-Bonet. [paper] [code]
Tags:
SLT - 【ACL 2024】Towards Privacy-Aware Sign Language Translation at Scale. Phillip Rust,?Bowen Shi,?Skyler Wang,?Necati Cihan Camg?z,?Jean Maillard. [paper] [code]
Tags:
SLT - 【ACL 2024】T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text. Aoxiong Yin,?Haoyuan Li,?Kai Shen,?Siliang Tang,?Yueting Zhuang. [paper] code
Tags:
SLP - 【ACL Findings 2024】iSign: A Benchmark for Indian Sign Language Processing.. Abhinav Joshi, Romit Mohanty, Mounika Kanakanti, Andesha Mangla, Sudeep Choudhary, Monali Barbate, Ashutosh Modi. [paper] [code]
Tags:
SLP - 【ACL Findings 2024】Unsupervised Sign Language Translation and Generation.. Zhengsheng Guo, Zhiwei He, Wenxiang Jiao, Xing Wang, Rui Wang, Kehai Chen, Zhaopeng Tu, Yong Xu, Min Zhang. [paper] code
Tags:
SLT - 【NAACL Findings 2024】Signer Diversity-driven Data Augmentation for Signer-Independent Sign Language Translation. Honghao Fu+, Liang Zhang+, Biao Fu, Jinsong Su, Xiaodong Shi, Yidong Chen.* [paper] code
Tags:
SLT - 【ECAI 2024】Improving Non-autoregressive Sign Language Translation with Random Ordering Progressive Prediction Pre-training. Pei Yu, Changhao Lai, Cong Hu, Shan Liu, Liang Zhang, Biao Fu, Yidong Chen.* [paper] code
Tags:
SLT - 【ECCV 2024】Uncertainty-aware sign language video retrieval with probability distribution modeling. Xuan Wu, Hongxiang Li, yuanjiang luo, Xuxin Cheng, Xianwei Zhuang, Meng Cao, Keren Fu. [paper] code
Tags:
Sign Language Retrieval - 【ECCV 2024】EvSign: Sign Language Recognition and Translation with Streaming Events. Pengyu Zhang, Hao Yin, Zeren Wang, Wenyue Chen, Sheng Ming Li, Dong Wang, Huchuan Lu, Xu Jia. [paper] [code]
Tags:
SLR&SLT - 【ECCV 2024】A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars. Ronglai Zuo, Fangyun Wei, Zenggui Chen, Brian Mak, Jiaolong Yang, Xin Tong. [paper] [code]
Tags:
SLP - 【ECCV 2024】SignGen: End-to-End Sign Language Video Generation with Latent Diffusion. Fan Qi, Yu Duan, Changsheng Xu, Huaiwen Zhang. [paper] code
Tags:
SLP - 【ECCV 2024】Visual Alignment Pre-training for Sign Language Translation. Peiqi Jiao,?Yuecong Min, Xilin Chen. [paper] code
Tags:
SLT - 【CVPR 2024】SignGraph: A Sign Sequence is Worth Graphs of Nodes. Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Hongkai Wen, Lei Xie, Sanglu Lu. [paper] [code]
Tags:
SLR - 【CVPR 2024】Neural Sign Actors: A Diffusion Model for 3D Sign Language Production from Text. Vasileios Baltatzis, Rolandos Alexandros Potamias, Evangelos Ververas, Guanxiong Sun, Jiankang Deng, Stefanos Zafeiriou. [paper] [code]
Tags:
SLP - 【CVPR 2024】LLMs are Good Sign Language Translators. Jia Gong, Lin Geng Foo, Yixuan He, Hossein Rahmani, Jun Liu. [paper] code
Tags:
SLT - 【COLING 2024】Adaptive Simultaneous Sign Language Translation with Confident Translation Length Estimation. Tong Sun+, Biao Fu+, Cong Hu, Liang Zhang, Ruiquan Zhang, Xiaodong Shi, Jinsong Su, Yidong Chen.* [paper] code
Tags:
SLT - 【COLING 2024】Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translation. Zhigang Chen, Benjia Zhou, Jun Li, Jun Wan, Zhen Lei, Ning Jiang, Quan Lu, Guoqing Zhao. [paper] code
Tags:
SLT - 【AAAI 2024】Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment. Rui Zhao, Liang Zhang, Biao Fu, Cong Hu, Jinsong Su, Yidong Chen. [paper] [code]
Tags:
SLT - 【AAAI 2024】G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Model. Pan Xie, Qipeng Zhang, Taiyi Peng, Hao Tang, Yao Du, Zexian Li. [paper] [code]
Tags:
SLP - 【arXiv 2024】American Sign Language Video to Text Translation. Parsheeta Roy,?Ji-Eun Han,?Srishti Chouhan,?Bhaavanaa Thumu. [paper] code
Tags:
SLT - 【ECCV 2024】SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark. Zhengdi Yu,?Shaoli Huang,?Yongkang Cheng,?Tolga Birdal. [paper] [code]
Tags:
Dataset - 【arXiv 2024】SignVTCL: Multi-Modal Continuous Sign Language Recognition Enhanced by Visual-Textual Contrastive Learning. Hao Chen , Jiaze Wang , Ziyu Guo , Jinpeng Li , Donghao Zhou, Bian Wu, Chenyong Guan, Guangyong Chen2 , Pheng-Ann Heng. [paper] [code]
Tags:
SLR - 【arXiv 2024】Towards Online Sign Language Recognition and Translation. Ronglai Zuo, Fangyun Wei, Brian Mak. [paper] [code]
Tags:
SLR&SLT - 【arXiv 2024】Unsupervised Sign Language Translation and Generation. Zhengsheng Guo,?Zhiwei He,?Wenxiang Jiao,?Xing Wang,?Rui Wang,?Kehai Chen,?Zhaopeng Tu,?Yong Xu,?Min Zhang. [paper] code
Tags:
SLT - 【ICASSP 2024】VK-G2T: Vision and Context Knowledge enhanced Gloss2Text. Liqiang Jing, Xuemeng Song, Xinxing Zu, Na Zheng, Zhongzhou Zhao, Liqiang Nie. [paper] code
Tags:
G2T - 【ICASSP 2024】An Explicit Multi-modal Fusion Method for Sign Language Translation. Cong Hu, Biao Fu, Pei Yu, Liang Zhang, Xiaodong Shi, Yidong Chen. [paper] code
Tags:
SLT - 【ICLR 2024】Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation. Ryan Wong, Necati Cihan Camgoz, Richard Bowden. [paper] code
Tags:
SLT - 【NeurIPS 2024】ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition. Desai Aashaka, Berger Lauren, Minakov Fyodor O, Milan Vanessa, Singh Chinmay, Pumphrey Kriston, Ladner Richard E, Iii Hal Daum , Lu Alex X, Caselli Naomi, Bragg Danielle. [paper] [code]
Tags:
Dataset - 【NeurIPS 2024】Auslan-Daily: Australian Sign Language Translation for Daily Communication and News. Shen Xin, Yuan Shaozu, Sheng Hongwei, Du Heming, Yu Xin. [paper] code
Tags:
Dataset - 【NeurIPS 2024】PopSign ASL v1.0: An Isolated American Sign Language Dataset Collected via Smartphones. Starner Thad, Forbes Sean, So Matthew, Martin David, Sridhar Rohit, Deshpande Gururaj, Sepah Sam, Shahryar Sahir, Bhardwaj Khushi, Kwok Tyler, Sehgal Daksh, Hassan Saad, Neubauer Bill, Vempala Sofia Anandi, Tan Alec, Heath Jocelyn, Kumar Unnathi Utpal, Mosur Priyanka Vijayaraghavan, Hall Tavenner M, Singh Rajandeep, Cui Christopher Zhang, Cameron Glenn, Dane Sohier, Tanzer Garrett. [paper] code
Tags:
Dataset - 【NeurIPS 2024】YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus. Uthus David, Tanzer Garrett, Georg Manfred. [paper] [code]
Tags:
Dataset
2023 <a name='sl_paper_2023'></a>
- 【CVPR 2023
(Highlight Paper)
】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code]Tags:
SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University - 【NeurIPS 2023】Auslan-Daily: Australian Sign Language Translation for Daily Communication and News. Xin Shen, Shaozu Yuan, Hongwei Sheng, Heming Du, Xin Yu. [paper] [code]
Tags:
Dataset - 【David Uthus, Garrett Tanzer, Manfred Georg】YouTube-ASL: A Large-Scale. * Open-Domain American Sign Language-English Parallel Corpus.* [[paper]](NeurIPS 2023) [code]
Tags:
https://arxiv.org/pdf/2306.15162.pdf; https://github.com/google-research/google-research/tree/master/youtube_asl - 【NeurIPS 2023】PopSign ASL v1.0: An Isolated American Sign Language Dataset Collected via Smartphones. Thad Starner, Sean Forbes, Matthew So, David Martin, Rohit Sridhar, Gururaj Deshpande, Sam Sepah, Sahir Shahryar, Khushi Bhardwaj, Tyler Kwok, Daksh Sehgal, Saad Hassan, Bill Neubauer, Sofia Anandi Vempala, Alec Tan, Jocelyn Heath, Unnathi Utpal Kumar, Priyanka Vijayaraghavan Mosur, Tavenner M. Hall, Rajandeep Singh et al.. [paper] code
Tags:
Dataset - 【CVPR 2023】Continuous Sign Language Recognition with Correlation Network. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. [paper] [code]
Tags:
SLR - 【CVPR 2023】Gloss Attention for Gloss-free Sign Language Translation. Aoxiong Yin, Tianyun Zhong, Li Tang, Weike Jin, Tao Jin, Zhou Zhao. [paper] [code]
Tags:
SLT - 【CVPR 2023】Distilling Cross-Temporal Contexts for Continuous Sign Language Recognition. Leming Guo, Wanli Xue, Qing Guo, Bo Liu, Kaihua Zhang, Tiantian Yuan, Shengyong Chen. [paper] code
Tags:
SLR - 【CVPR 2023】Natural Language-Assisted Sign Language Recognition. Ronglai Zuo, Fangyun Wei, Brian Mak. [paper] [code]
Tags:
SLR - 【CVPR 2023】CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning. Yiting Cheng, Fangyun Wei, Jianmin Bao, Dong Chen, Wenqiang Zhang. [paper] [code]
Tags:
Sign Language Retrieval - 【CVPR 2023】Ham2Pose: Animating Sign Language Notation into Pose Sequences. Rotem Shalev Arkushin, Amit Moryossef, Ohad Fried. [paper] [code]
Tags:
Others - 【CVPR 2023】Sign Language Video Retrieval with Free-Form Textual Queries. Amanda Duarte,Samuel Albanie,Xavier Giro-i-Nieto ,Gul Varol. [paper] [code]
Tags:
Sign Language Retrieval - 【ICCV 2023】Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining. Benjia Zhou, Zhigang Chen, Albert Clapes, Jun Wan, Yanyan Liang,Sergio Escalera, Zhen Lei, Du Zhang. [paper] [code]
Tags:
SLT - 【NAACL 2023】Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation. Yong Cao, Wei Li, Xianzhi Li, Min Chen, Guangyong Chen, Long Hu, Zhengdao Li, Hwang Kai. [paper] [code]
Tags:
SLT - 【ICLR 2023】SLTUNET: A Simple Unified Model for Sign Language Translation. Biao Zhang, Mathias M��ller, Rico Sennrich. [paper] [code]
Tags:
SLT - 【NeurIPS 2023】ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition. Aashaka Desai,Lauren Berger,Fyodor O. Minakov,Vanessa Milan,Chinmay Singh,Kriston Pumphrey,Richard E. Ladner,Hal Daum�� III,Alex X. Lu, Naomi Caselli, Danielle Bragg. [paper] code
Tags:
nan - 【IJCAI 2023】Efficient Sign Language Translation with a Curriculum-based NAR Decoder. Pei Yu1, Liang Zhang, Biao Fu1, Yidong Chen. [paper] code
Tags:
SLT - 【ICASSP 2023】A Token-level Contrastive Framework for Sign Language Translation. Biao Fu, Peigen Ye, Liang Zhang, Pei Yu, Cong Hu, Yidong Chen, Xiaodong Shi. [paper] code
Tags:
SLT - 【AAAI 2023】Self-Emphasizing Network for Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang liu, Wei Feng. [paper] code
Tags:
SLR - 【EACL 2023】Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation. Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu. [paper] code
Tags:
Sign Gloss Translation
2022 <a name='sl_paper_2022'></a>
- 【Arxiv 2022】Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation. [paper] code
Tags:
SLT - 【CVPR 2022】A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation. Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, Stephen Lin. [paper] code
Tags:
SLT - 【CVPR 2022】MLSLT: Towards Multilingual Sign Language Translation. Aoxiong Yin, Zhou Zhao, Weike Jin, Meng Zhang, Xingshan Zeng, Xiaofei He. [paper] code
Tags:
SLT - 【CVPR 2022】C2SLR: Consistency-Enhanced Continuous Sign Language Recognition. Ronglai Zuo, Brian Mak. [paper] code
Tags:
SLR - 【NeurIPS 2022】Two-Stream Network for Sign Language Recognition and Translation. Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, Brian Mak. [paper] code
Tags:
SLT & SLR - 【EMNLP 2022】Open-Domain Sign Language Translation Learned from Online Video. Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu. [paper] code
Tags:
SLT - 【NAACL 2022 Findings】Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation. Yong Cao, Wei Li, Xianzhi Li, Min Chen, Guangyong Chen, Long Hu, Zhengdao Li, Hwang Kai. [paper] code
Tags:
SLT - 【ACL 2022】WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language. Federico Tavella, Viktor Schlegel, Marta Romeo, Aphrodite Galata, Angelo Cangelosi. [paper] code
Tags:
Dataset - 【ACL 2022】Searching for fingerspelled content in American Sign Language. Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu. [paper] code
Tags:
Fingerspelled SL - 【ECCV 2022】Automatic dense annotation of large-vocabulary sign language videos. Liliane Momeni, Hannah Bull, K R Prajwal, Samuel Albanie, G��l Varol, Andrew Zisserman. [paper] code
Tags:
Annotation - 【ECCV 2022】Temporal Lift Pooling for Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. [paper] code
Tags:
SLR - 【ACMMM 2022】MC-SLT: Towards Low-Resource Signer-Adaptive Sign Language Translation. Tao Jin, Zhou Zhao, Meng Zhang, Xingshan Zeng. [paper] code
Tags:
SLT - 【WACV 2022】Sign Language Translation With Hierarchical Spatio-Temporal Graph Neural Network. Jichao Kan, Kun Hu, Markus Hagenbuchner, Ah Chung Tsoi, Mohammed Bennamoun, Zhiyong Wang. [paper] code
Tags:
SLT
2021 <a name='sl_paper_2021'></a>
- 【Neurocomputing 2021】Enhancing neural sign language translation by highlighting the facial expression information. [paper] code
Tags:
SLT - 【TMM 2021】Spatial-Temporal Multi-Cue Network for Sign Language Recognition and Translation. Hao Zhou, Wengang Zhou, Yun Zhou, Houqiang Li. [paper] code
Tags:
SLT & SLR - 【ICCV 2021】YouRefIt: Embodied Reference Understanding With Language and Gesture. Yixin Chen; Qing Li; Deqian Kong; Yik Lun Kei; Song-Chun Zhu; Tao Gao; Yixin Zhu; Siyuan Huang. [paper] code
Tags:
SLR - 【ICCV 2021】Speech Drives Templates: Co-Speech Gesture Synthesis With Learned Templates. Shenhan Qian; Zhi Tu; Yihao Zhi; Wen Liu; Shenghua Gao. [paper] code
Tags:
SLR - 【ICCV 2021】Audio2Gestures: Generating Diverse Gestures From Speech Audio With Conditional Variational Autoencoders. Jing Li; Di Kang; Wenjie Pei; Xuefei Zhe; Ying Zhang; Zhenyu He; Linchao Bao. [paper] code
Tags:
SLR - 【ICCV 2021】Aligning Subtitles in Sign Language Videos. Hannah Bull; Triantafyllos Afouras; G��l Varol; Samuel Albanie; Liliane Momeni; Andrew Zisserman. [paper] code
Tags:
SLR - 【ICCV 2021】Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives. Ben Saunders; Necati Cihan Camgoz; Richard Bowden. [paper] code
Tags:
SLR - 【ICCV 2021】SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition. Hezhen Hu; Weichao Zhao; Wengang Zhou; Yuechen Wang; Houqiang Li. [paper] code
Tags:
SLR - 【ICCV 2021】Visual Alignment Constraint for Continuous Sign Language Recognition. Yuecong Min, Aiming Hao, Xiujuan Chai, and Xilin Chen. [paper] [code]
Tags:
SLR - 【ICCV 2021】Self-Mutual Distillation Learning for Continuous Sign Language Recognition. Aiming Hao, Yuecong Min, and Xilin Chen. [paper] code
Tags:
SLR - 【CVPR 2021】Improving Sign Language Translation With Monolingual Data by Sign Back-Translation. Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, Houqiang Li. [paper] code
Tags:
SLR - 【CVPR 2021】How2Sign: A Large-Scale Multimodal Dataset for Continuous American Sign Language. Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, Xavier Giro-i-Nieto. [paper] code
Tags:
SLR - 【CVPR 2021】Fingerspelling Detection in American Sign Language. Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu. [paper] code
Tags:
SLR - 【CVPR 2021】Read and Attend: Temporal Localisation in Sign Language Videos. G��l Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, Andrew Zisserman. [paper] code
Tags:
SLR - 【CVPR 2021】iMiGUE: An Identity-Free Video Dataset for Micro-Gesture Understanding and Emotion Analysis. Xin Liu, Henglin Shi, Haoyu Chen, Zitong Yu, Xiaobai Li, Guoying Zhao. [paper] code
Tags:
SLR - 【CVPR 2021】Body2Hands: Learning To Infer 3D Hands From Conversational Gesture Body Dynamics. Evonne Ng, Shiry Ginosar, Trevor Darrell, Hanbyul Joo. [paper] code
Tags:
SLR - 【CVPR 2021】Model-Aware Gesture-to-Gesture Translation. Hezhen Hu, Weilun Wang, Wengang Zhou, Weichao Zhao, Houqiang Li. [paper] code
Tags:
SLR - 【AAAI 2021】Hand-Model-Aware Sign Language Recognition. Hezhen Hu, Wengang Zhou, Houqiang Li. [paper] code
Tags:
SLR - 【AAAI 2021】Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition. Benjia Zhou, Yunan Li, Jun Wan. [paper] code
Tags:
SLR - 【WACV 2021】Hand Pose Guided 3D Pooling for Word-level Sign Language Recognition. Al Amin Hosain; Panneer Selvam Santhalingam; Parth Pathak; Huzefa Rangwala; Jana Kosecka. [paper] code
Tags:
SLR - 【WACV 2021】Whose hand is this? Person Identification from Egocentric Hand Gestures. Satoshi Tsutsui; Yanwei Fu; David Crandall. [paper] code
Tags:
SLR
2020 <a name='sl_paper_2020'></a>
- 【ACMMM 2020】INCLUDE: A Large Scale Dataset for Indian Sign Language Recognition. Sridhar, Advaith, Rohith Gandhi Ganesan, Pratyush Kumar, and Mitesh Khapra. [paper] code
Tags:
SLR - 【ACMMM 2020】Boosting Continuous Sign Language Recognition via Cross Modality Augmentation. Pu, Junfu, Wengang Zhou, Hezhen Hu, and Houqiang Li. [paper] code
Tags:
SLR - 【ACMMM 2020】Recognizing Camera Wearer from Hand Gestures in Egocentric Videos. Thapar, Daksh, Aditya Nigam, and Chetan Arora. [paper] code
Tags:
SLR - 【NeurIPS 2020】TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation. Li, Dongxu, Chenchen Xu, Xin Yu, Kaihao Zhang, Benjamin Swift, Hanna Suominen, and Hongdong Li. [paper] code
Tags:
SLR - 【FG 2020】Feature Selection for Zero-Shot Gesture Recognition. Naveen Madapana, Juan Wachs. [paper] code
Tags:
SLR - 【FG 2020】Image-Based Pose Representation for Action Recognition and Hand Gesture Recognition. Zeyi Lin, Wei Zhang, Xiaoming Deng, Cuixia Ma, Hongan Wang. [paper] code
Tags:
SLR - 【FG 2020】Neural Sign Language Translation by Learning Tokenization. Orbay, Alptekin, and Lale Akarun. [paper] code
Tags:
SLT - 【FG 2020】Sign Language Recognition in Virtual Reality. Jacob Schioppo, Zachary Meyer, Diego Fabiano, Shaun Canavan. [paper] code
Tags:
SLR - 【FG 2020】SILFA: Sign Language Facial Action Database for the Development of Assistive Technologies for the Deaf. Emely Puj��lli da Silva, Paula Dornhofer Paro Costa, Kate Mamhy Oliveira Kumada, Jos�� Mario De Martino. [paper] code
Tags:
Dataset - 【FG 2020】FineHand: Learning Hand Shapes for American Sign Language Recognition. Al Amin Hosain, Panneer Selvam Santhalingam, Parth Pathak, Huzefa Rangwala, Jana Ko?eck��. [paper] code
Tags:
SLR - 【FG 2020】Introduction and Analysis of an Event-Based Sign Language Dataset. Ajay Vasudevan, Pablo Negri, Bernabe Linares-Barranco, Teresa Serrano-Gotarredona. [paper] code
Tags:
Dataset - 【FG 2020】Towards a Visual Sign Language Dataset for Home Care Services. D. Kosmopoulos, I. Oikonomidis, C. Constantinopoulos, N. Arvanitis, K. Antzakas, A. Bifis, G. Lydakis, A. Roussos, A. Argyros. [paper] code
Tags:
Dataset - 【ECCV 2020】BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues. Samuel Albanie, G��l Varol, Liliane Momeni, Triantafyllos Afouras, Joon Son Chung, Neil Fox, Andrew Zisserman. [paper] code
Tags:
SLR - 【ECCV 2020】Progressive Transformers for End-to-End Sign Language Production. Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. [paper] [code]
Tags:
SLP - 【ECCV 2020】Stochastic Fine-grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition. Niu Zhe, Brian Mak. [paper] code
Tags:
SLR - 【ECCV 2020】Fully Convolutional Networks for Continuous Sign Language Recognition. Ka Leong Cheng, Zhaoyang Yang, Qifeng Chen, and Yu-Wing Tai. [paper] code
Tags:
SLR - 【ECCV 2020】Collaborative Learning of Gesture Recognition and 3D Hand Pose Estimation with Multi-Order Feature Analysis. Yang, Siyuan, Jun Liu, Shijian Lu, Meng Hwa Er, and Alex C. Kot. [paper] code
Tags:
SLR - 【ECCV 2020】Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach. Chaitanya Ahuja, Dong Won Lee, Yukiko I. Nakano, and Louis-Philippe Morency. [paper] code
Tags:
SLR - 【ECCV 2020】Towards Efficient Coarse-to-Fine Networks for Action and Gesture Recognition. Quader, Niamul, Juwei Lu, Peng Dai, and Wei Li. [paper] code
Tags:
SLR - 【CVPR 2020】Decoupled Representation Learning for Skeleton-Based Gesture Recognition. Jianbo Liu, Yongcheng Liu, Ying Wang, V��ronique Prinet, Shiming Xiang, Chunhong Pan. [paper] code
Tags:
SLR - 【CVPR 2020】An Efficient PointLSTM for Point Clouds Based Gesture Recognition. Yuecong Min, Yanxiao Zhang, Xiujuan Chai, Xilin Chen. [paper] code
Tags:
SLR - 【CVPR 2020】Transferring Cross-Domain Knowledge for Video Sign Language Recognition. Dongxu Li, Xin Yu, Chenchen Xu, Lars Petersson, Hongdong Li. [paper] code
Tags:
SLR - 【WACV 2020】Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. Dongxu Li, Cristian Rodriguez, Xin Yu, Hongdong Li. [paper] code
Tags:
Dataset - 【WACV 2020】Neural Sign Language Synthesis: Words Are Our Glosses. Jan Zelinka, Jakub Kanis. [paper] code
Tags:
SLP - 【CIN 2020】Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. [paper] code
Tags:
SLT - 【arXiv 2020】Multi-channel Transformers for Multi-articulatory Sign Language Translation. [paper] code
Tags:
SLT - 【CVPR 2020】Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation. Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. [paper] [code]
Tags:
SLT - 【arXiv 2020】Better Sign Language Translation with STMC-Transformer. Kayo Yin, Jesse Read. [paper] [code]
Tags:
SLT - 【arXiv 2020】Neural Sign Language Translation by Learning Tokenization. Alptekin Orbay and Lale Akarun. [paper] code
Tags:
SLT - 【ECCV 2020】Stochastic Fine-grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition. [paper] [code]
Tags:
SLR - 【ECCV 2020】Fully Convolutional Networks for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【AAAI 2020】Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【ECCV 2020】BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues. [paper] code
Tags:
SLR - 【WACV 2020】Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. [paper] [code]
Tags:
SLR - 【IJCV 2020】Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks. Stephanie Stoll, Necati Cihan Camgoz, Simon Hadfield, Richard Bowden. [paper] code
Tags:
Text2Sign - 【ECCV 2020】Progressive Transformers for End-to-End Sign Language Production. [paper] code
Tags:
Text2Sign - 【TCSVT 2020】Semantic Boundary Detection with Reinforcement Learning for Continuous Sign Language Recognition. [paper] code
Tags:
SLR
2019 <a name='sl_paper_2019'></a>
- 【IEEE Access 2019 (
Recommended
)】Technical approaches to Chinese sign language processing: A review. Suhail Muhammad Kamal; Yidong Chen; Shaozi Li; Xiaodong Shi; Jiangbin Zheng. [paper] codeTags:
Survey - 【BigMM 2019】Deep Grammatical Multi-classifier for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【ICIP 2019】Continuous Sign Language Recognition via Reinforcement Learning. [paper] code
Tags:
SLR - 【ICME 2019】Dynamic Pseudo Label Decoding for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【IJCAI 2019】Dense Temporal Convolution Network for Sign Language Translation. [paper] code
Tags:
SLR - 【IEEE TRANSACTIONS ON MULTIMEDIA 2019】A Deep Neural Framework for Continuous Sign
Language Recognition by Iterative Training. [paper] code
Tags:
SLR - 【CVPR 2019】Iterative Alignment Network for Continuous Sign Language. [paper] code
Tags:
SLR - 【TPAMI 2019】Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos. [paper] [code]
Tags:
SLR - 【arXiv 2019】SF-Net: Structured Feature Network for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【IEEE ACCESS 2019】Dynamic Sign Language Recognition Based on Video Sequence With BLSTM-3D Residual Networks. [paper] code
Tags:
SLR - 【ICCV 2019】Temporal Accumulative Features for Sign Language Recognition. [paper] code
Tags:
SLR - 【ICCCM 2019】Thai Sign Language Recognition Using 3D Convolutional Neural Networks. [paper] code
Tags:
SLR - 【DSAA 2019】Sign Language Recognition Analysis using Multimodal Data. [paper] code
Tags:
SLR - 【ICANN 2019】Spatial-Temporal Graph Convolutional Networks for Sign Language Recognition. [paper] [code]
Tags:
SLR - 【ICCV 2019】Fingerspelling recognition in the wild with iterative visual attention. [paper] code
Tags:
SLR - 【SITIS 2019】Translation of sign language glosses to text using sequence-to-sequence attention models. [paper] code
Tags:
Gloss2Text
2018 <a name='sl_paper_2018'></a>
- 【TCSVT 2018】Attention based 3D-CNNs for Large-Vocabulary Sign Language Recognition. [paper] code
Tags:
SLR - 【AAAI 2018】Video-based sign language recognition without temporal segmentation. [paper] code
Tags:
SLR - 【AAAI 2018】Hierarchical LSTM for Sign Language Translation. [paper] code
Tags:
SLT - 【ACMMM 2018】Connectionist Temporal Fusion for Sign Language Translation. [paper] code
Tags:
SLT - 【arXiv 2018】Neural Sign Language Translation based on Human Keypoint Estimation. Sang-Ki Ko, Chang Jo Kim, Hyedong Jung, Choongsang Cho. [paper] code
Tags:
SLT - 【CVPR 2018】Neural Sign Language Translation. Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, Richard Bowden. [paper] [code]
Tags:
SLT - 【IJCAI 2018】Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition. [paper] [code]
Tags:
SLR - 【ETRI 2018】Human-like sign-language learning method using deep learning. [paper] code
Tags:
SLR - 【IJCV 2018】Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs. [paper] code
Tags:
SLR - 【NIPS 2018】Attention in Convolutional LSTM for Gesture Recognition. [paper] [code]
Tags:
SLR
2017 <a name='sl_paper_2017'></a>
- 【CVPR 2017】Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs. [paper] [code]
Tags:
SLR - 【CVPR 2017】Recurrent Convolutional Neural Networks for Continuous Sign Language Recognition by Staged Optimization. [paper] code
Tags:
SLR - 【TOMM 2017】Online Early-Late Fusion Based on Adaptive HMM for Sign Language Recognition*. [paper] code
Tags:
SLR - 【ICCV 2017】SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition. [paper] [code]
Tags:
SLR - 【ICCV 2017】Learning Spatiotemporal Features Using 3DCNN and Convolutional LSTM for Gesture Recognition. [paper] [code]
Tags:
SLR
Earlier <a name='sl_paper_earlier'></a>
- 【ICME 2016】Chinese sign language recognition with adaptive HMM. [paper] code
Tags:
SLR - 【PCM2016】Sign Language Recognition with Multi-modal Features. [paper] code
Tags:
SLR - 【ICME2015】Sign language recognition using 3D convolutional neural networks. [paper] code
Tags:
SLR - 【ICIP 2016】Sign language recognition based on adaptive HMMS with data augmentation. [paper] code
Tags:
SLR - 【Pattern Recognit.Lett.2016】Continuous sign language recognition using level building based on fast hidden Markov model. [paper] code
Tags:
SLR - 【TACCESS 2016】Sign Transition Modeling and a Scalable Solution to Continuous Sign Language Recognition for Real-World Applications. [paper] code
Tags:
SLR - 【ICPR 2016】Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition. [paper] code
Tags:
SLR - 【CVPR 2016】Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks. [paper] code
Tags:
SLR - 【CVPR 2016】Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled. [paper] code
Tags:
SLR - 【BMVC 2016】Deep Sign: Hybrid CNN-HMM for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【ICIP 2016】Sign Language Recognition With Long Short-Term Memory. [paper] code
Tags:
SLR - 【ECCV 2016】Iterative Reference Driven Metric Learning for Signer Independent Isolated Sign Language Recognition. [paper] code
Tags:
SLR - 【LREC 2016】Automatic Alignment of HamNoSys Subunits for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【CVPRW 2015】Hand Gesture Recognition with 3D Convolutional Neural Networks. [paper] code
Tags:
SLR - 【ICCVW 2015】Deep Learning of Mouth Shapes for Sign Language. [paper] code
Tags:
SLR - 【ICME 2015】Sign Language Recognition using 3D convolutional neural networks. [paper] code
Tags:
SLR - 【ACCV 2014】Curve Matching from the View of Manifold for Sign Language Recognition. [paper] code
Tags:
SLR - 【ICIMCS 2014】A Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【ICTA 2013】English-ASL Gloss Parallel Corpus 2012: ASLG-PC12, The Second Release. [paper] code
Tags:
Dataset - 【SLPAT 2013】Improving Continuous Sign Language Recognition: Speech Recognition Techniques and System Design. [paper] code
Tags:
SLR - 【IWSLT 2013】Using Viseme Recognition to Improve a Sign Language Translation System. [paper] code
Tags:
SLR - 【AFGR 2013】Sign Language Recognition and Translation with Kinect. [paper] code
Tags:
SLR - 【CVPR 2012】Sign Language Recognition using Sequential Pattern Trees. [paper] code
Tags:
SLR - 【JMLR 2012】Sign language recognition using sub-units. [paper] code
Tags:
SLR - 【CVPRW 2011】Advances in phonetics-based sub-unit modeling for transcription alignment and sign language recognition. [paper] code
Tags:
SLR - 【Eng.Appl.Artif.Intell.2011】American Sign Language word recognition with a sensory glove using artificial neural networks. [paper] code
Tags:
SLR - 【INTERSPEECH 2007】Speech Recognition Techniques for a Sign Language Recognition System. [paper] code
Tags:
SLR - 【TSMC 2007】Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models. [paper] code
Tags:
SLR - 【TPAMI 1998】Real-time American sign language recognition using desk and wearable computer based video. [paper] code
Tags:
SLR
AI Sign Language in Well-Known Institutions [Papers] <a name="sl_paper_institution"></a>
XMU for AI Sign Language <a name='sl_paper_xmu'></a>
- 【NAACL Findings 2024】Signer Diversity-driven Data Augmentation for Signer-Independent Sign Language Translation. Honghao Fu+, Liang Zhang+, Biao Fu, Jinsong Su, Xiaodong Shi, Yidong Chen.* [paper] code
Tags:
SLT - 【ECAI 2024】Improving Non-autoregressive Sign Language Translation with Random Ordering Progressive Prediction Pre-training. Pei Yu, Changhao Lai, Cong Hu, Shan Liu, Liang Zhang, Biao Fu, Yidong Chen.* [paper] code
Tags:
SLT - 【COLING 2024】Adaptive Simultaneous Sign Language Translation with Confident Translation Length Estimation. Tong Sun+, Biao Fu+, Cong Hu, Liang Zhang, Ruiquan Zhang, Xiaodong Shi, Jinsong Su, Yidong Chen.* [paper] code
Tags:
SLT - 【AAAI 2024】Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment. Rui Zhao, Liang Zhang, Biao Fu, Cong Hu, Jinsong Su, Yidong Chen. [paper] [code]
Tags:
SLT - 【ICASSP 2024】An Explicit Multi-modal Fusion Method for Sign Language Translation. Cong Hu, Biao Fu, Pei Yu, Liang Zhang, Xiaodong Shi, Yidong Chen. [paper] code
Tags:
SLT - 【CVPR 2023
(Highlight Paper)
】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code]Tags:
SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University - 【IJCAI 2023】Efficient Sign Language Translation with a Curriculum-based NAR Decoder. Pei Yu1, Liang Zhang, Biao Fu1, Yidong Chen. [paper] code
Tags:
SLT - 【ICASSP 2023】A Token-level Contrastive Framework for Sign Language Translation. Biao Fu, Peigen Ye, Liang Zhang, Pei Yu, Cong Hu, Yidong Chen, Xiaodong Shi. [paper] code
Tags:
SLT - 【Arxiv 2022】Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation. [paper] code
Tags:
SLT - 【Neurocomputing 2021】Enhancing neural sign language translation by highlighting the facial expression information. [paper] code
Tags:
SLT - 【CIN 2020】Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. [paper] code
Tags:
SLT - 【IEEE Access 2019 (
Recommended
)】Technical approaches to Chinese sign language processing: A review. Suhail Muhammad Kamal; Yidong Chen; Shaozi Li; Xiaodong Shi; Jiangbin Zheng. [paper] codeTags:
Survey
USTC for AI Sign Language <a name='sl_paper_ustc'></a>
- 【ACMMM 2020】Boosting Continuous Sign Language Recognition via Cross Modality Augmentation. Pu, Junfu, Wengang Zhou, Hezhen Hu, and Houqiang Li. [paper] code
Tags:
SLR - 【AAAI 2020】Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【TCSVT 2020】Semantic Boundary Detection with Reinforcement Learning for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【BigMM 2019】Deep Grammatical Multi-classifier for Continuous Sign Language Recognition. [paper] code
Tags:
SLR - 【IJCAI 2019】Dense Temporal Convolution Network for Sign Language Translation. [paper] code
Tags:
SLR - 【IEEE TRANSACTIONS ON MULTIMEDIA 2019】A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training. [paper] code
Tags:
SLR - 【CVPR 2019】Iterative Alignment Network for Continuous Sign Language. [paper] code
Tags:
SLR - 【AAAI 2018】Video-based sign language recognition without temporal segmentation. [paper] code
Tags:
SLR - 【AAAI 2018】Hierarchical LSTM for Sign Language Translation. [paper] code
Tags:
SLT - 【ACMMM 2018】Connectionist Temporal Fusion for Sign Language Translation. [paper] code
Tags:
SLT - 【IJCAI 2018】Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition. [paper] [code]
Tags:
SLR - 【ICME 2016】Chinese sign language recognition with adaptive HMM. [paper] code
Tags:
SLR - 【PCM2016】Sign Language Recognition with Multi-modal Features. [paper] code
Tags:
SLR - 【ICME2015】Sign language recognition using 3D convolutional neural networks. [paper] code
Tags:
SLR - 【ICIP 2016】Sign language recognition based on adaptive HMMS with data augmentation. [paper] code
Tags:
SLR - 【ICIP 2016】Sign Language Recognition With Long Short-Term Memory. [paper] code
Tags:
SLR
ZJU for AI Sign Language <a name='sl_paper_zju'></a>
- 【ACL 2024】T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text. Aoxiong Yin,?Haoyuan Li,?Kai Shen,?Siliang Tang,?Yueting Zhuang. [paper] code
Tags:
SLP - 【arXiv 2024】SignVTCL: Multi-Modal Continuous Sign Language Recognition Enhanced by Visual-Textual Contrastive Learning. Hao Chen , Jiaze Wang , Ziyu Guo , Jinpeng Li , Donghao Zhou, Bian Wu, Chenyong Guan, Guangyong Chen2 , Pheng-Ann Heng. [paper] [code]
Tags:
SLR - 【CVPR 2023
(Highlight Paper)
】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code]Tags:
SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University
THU for AI Sign Language <a name='sl_paper_thu'></a>
- 【CVPR 2023
(Highlight Paper)
】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code]Tags:
SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University - 【CVPR 2022】A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation. Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, Stephen Lin. [paper] code
Tags:
SLT
Germany-UK for AI Sign Language <a name='sl_paper_germany-uk'></a>
- 【arXiv 2020】Multi-channel Transformers for Multi-articulatory Sign Language Translation. [paper] code
Tags:
SLT - 【CVPR 2020】Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation. Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. [paper] [code]
Tags:
SLT - 【TPAMI 2019】Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos. [paper] [code]
Tags:
SLR - 【CVPR 2018】Neural Sign Language Translation. Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, Richard Bowden. [paper] [code]
Tags:
SLT - 【CVPR 2017】Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs. [paper] [code]
Tags:
SLR - 【ICCV 2017】SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition. [paper] [code]
Tags:
SLR
Datasets <a name="datasets"></a>
Dataset | Language | Classes | Samples | Data Type | Language Level |
---|---|---|---|---|---|
ASLG-PC12paper | American | - | 87,709 | GLOSS&Sentences | isolated |
CSL Dataset I | Chinese | 500 | 125,000 | Videos&Depth from Kinect | isolated |
CSL Dataset II | Chinese | 100 | 25,000 | Videos&Depth from Kinect | continuous |
RWTH-PHOENIX-Weather 2014 | German | 1,081 | 6,841 | Videos | continuous |
RWTH-PHOENIX-Weather 2014 T | German | 1,066 | 8,257 | Videos | continuous |
ASLLVD | American | 3,300 | 9,800 | Videos(multiple angles) | isolated |
ASLLVD-Skeleton | American | 3,300 | 9,800 | Skeleton | isolated |
SIGNUM | German | 450 | 33,210 | Videos | continuous |
DGS Kinect 40 | German | 40 | 3,000 | Videos(multiple angles) | isolated |
DEVISIGN-G | Chinese | 36 | 432 | Videos | isolated |
DEVISIGN-D | Chinese | 500 | 6,000 | Videos | isolated |
DEVISIGN-L | Chinese | 2000 | 24,000 | Videos | isolated |
LSA64 | Argentinian | 64 | 3,200 | Videos | isolated |
GSL isol. | Greek | 310 | 40,785 | Videos&Depth from RealSense | isolated |
GSL SD | Greek | 310 | 10,290 | Videos&Depth from RealSense | continuous |
GSL SI | Greek | 310 | 10,290 | Videos&Depth from RealSense | continuous |
IIITA -ROBITA | Indian | 23 | 605 | Videos | isolated |
PSL Kinect | Polish | 30 | 300 | Videos&Depth from Kinect | isolated |
PSL ToF | Polish | 84 | 1,680 | Videos&Depth from ToF camera | isolated |
BUHMAP-DB | Turkish | 8 | 440 | Videos | isolated |
LSE-Sign | Spanish | 2,400 | 2,400 | Videos | isolated |
Purdue RVL-SLLL | American | 39 | 546 | Videos | isolated |
RWTH-BOSTON-50 | American | 50 | 483 | Videos(multiple angles) | isolated |
RWTH-BOSTON-104 | American | 104 | 201 | Videos(multiple angles) | continuous |
RWTH-BOSTON-400 | American | 400 | 843 | Videos | continuous |
WLASL | American | 2,000 | 21,083 | Videos | isolated |
AI Sign Language Related Fields <a name="related_fields"></a>
The related fields, such as Large Pretrained Language Model
,Corss-modal/Multi-modality
, Action Recognition
, Machine Translation
, Speech Recognition
, Video Captioning
, can help you gain more inspiration and knowledge