Awesome
<div align="center"><h1> Motion Mamba: Efficient and Long Sequence Motion Generation<br> <sub><sup><a href="https://eccv2024.ecva.net/">ECCV 2024</a></sup></sub> </h1>Zeyu Zhang<sup>*</sup>, Akide Liu<sup>*</sup>, Ian Reid, Richard Hartley, Bohan Zhuang, Hao Tang<sup>✉</sup>
<sup>*</sup>Equal contribution <sup>✉</sup>Corresponding author: bjdxtanghao@gmail.com
Human motion generation stands as a significant pursuit in generative computer vision, while achieving long-sequence and efficient motion generation remains challenging. Recent advancements in state space models (SSMs), notably Mamba, have showcased considerable promise in long sequence modeling with an efficient hardware-aware design, which appears to be a promising direction to build motion generation model upon it. Nevertheless, adapting SSMs to motion generation faces hurdles since the lack of a specialized design architecture to model motion sequence. To address these challenges, we propose Motion Mamba, a simple and efficient approach that presents the pioneering motion generation model utilized SSMs. Specifically, we design a Hierarchical Temporal Mamba (HTM) block to process temporal data by ensembling varying numbers of isolated SSM modules across a symmetric U-Net architecture aimed at preserving motion consistency between frames. We also design a Bidirectional Spatial Mamba (BSM) block to bidirectionally process latent poses, to enhance accurate motion generation within a temporal frame. Our proposed method achieves up to 50% FID improvement and up to 4 times faster on the HumanML3D and KIT-ML datasets compared to the previous best diffusion-based method, which demonstrates strong capabilities of high-quality long sequence motion modeling and real-time human motion generation.
<div align="center"> <img src="static/images/main.svg" style="width: 100%;"> <img src="static/images/block.svg" style="width: 80%;"> </div>News
<b>(07/22/2024)</b> 🎉 Our paper was invited for a talk at <a href="https://www.mihoyo.com/"><b>miHoYo</b></a>. You can find our slides <a href="https://steve-zeyu-zhang.github.io/MotionMamba/static/pdfs/Motion_Mamba_Slides_miHoYo.pdf"><b>here</b></a>!
<b>(07/05/2024)</b> 🎉 Our paper has been highlighted twice by <a href="https://wx.zsxq.com/dweb2/index/topic_detail/5122458815888184"><b>CVer</b></a>!
<b>(07/02/2024)</b> 🎉 Our paper has been accepted to <a href="https://eccv2024.ecva.net/"><b>ECCV 2024</b></a>!
<b>(03/15/2024)</b> 🎉 Our paper has been highlighted by <a href="https://twitter.com/Marktechpost/status/1768770427680424176"><b>MarkTechPost</b></a>!
<b>(03/13/2024)</b> 🎉 Our paper has been featured in <a href="https://twitter.com/_akhaliq/status/1767750847239262532"><b>Daily Papers</b></a>!
<b>(03/13/2024)</b> 🎉 Our paper has been highlighted by <a href="https://wx.zsxq.com/dweb2/index/topic_detail/1522541851241522"><b>CVer</b></a>!
Citation
@article{zhang2024motion,
title={Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM},
author={Zhang, Zeyu and Liu, Akide and Reid, Ian and Hartley, Richard and Zhuang, Bohan and Tang, Hao},
journal={arXiv preprint arXiv:2403.07487},
year={2024}
}