Awesome
Awesome-LLM-3D
<a href="" target='_blank'><img src="https://visitor-badge.laobi.icu/badge?page_id=activevisionlab.llm3d&left_color=gray&right_color=blue"> </a> ![arXiv](https://img.shields.io/badge/arXiv-2405.10255-b31b1b.svg)
📢 Survey paper available on arXiv now: [Paper]
🏠 About
Here is a curated list of papers about 3D-Related Tasks empowered by Large Language Models (LLMs). It contains various tasks including 3D understanding, reasoning, generation, and embodied agents. Also, we include other Foundation Models (CLIP, SAM) for the whole picture of this area.
This is an active repository, you can watch for following the latest advances. If you find it useful, please kindly star this repo.
🔥 News
- [2023-12-16] Xianzheng Ma and Yash Bhalgat curated this list and published the first version;
- [2024-01-06] Runsen Xu added chronological information and Xianzheng Ma reorganized it in Z-A order for better following the latest advances.
Table of Content
3D Understanding via LLM
3D Understanding via other Foundation Models
3D Reasoning
Date | keywords | Institute (first) | Paper | Publication | Others |
---|---|---|---|---|---|
2023-5-20 | 3D-CLR | UCLA | 3D Concept Learning and Reasoning from Multi-View Images | CVPR '23 | github |
- | Transcribe3D | TTI, Chicago | Transcribe3D: Grounding LLMs Using Transcribed Information for 3D Referential Reasoning with Self-Corrected Finetuning | CoRL '23 | github |
3D Generation
Date | keywords | Institute | Paper | Publication | Others |
---|---|---|---|---|---|
2023-11-29 | ShapeGPT | Fudan University | ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Model | Arxiv | github |
2023-11-27 | MeshGPT | TUM | MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers | Arxiv | project |
2023-10-19 | 3D-GPT | ANU | 3D-GPT: Procedural 3D Modeling with Large Language Models | Arxiv | github |
2023-9-21 | LLMR | MIT | LLMR: Real-time Prompting of Interactive Worlds using Large Language Models | Arxiv | github |
2023-9-20 | DreamLLM | MEGVII | DreamLLM: Synergistic Multimodal Comprehension and Creation | Arxiv | github |
2023-4-1 | ChatAvatar | Deemos Tech | DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance | ACM TOG | website |
3D Embodied Agent
3D Benchmarks
Contributing
your contributions are always welcome!
I will keep some pull requests open if I'm not sure if they are awesome for 3D LLMs, you could vote for them by adding 👍 to them.
If you have any questions about this opinionated list, please get in touch at xianzheng@robots.ox.ac.uk or Wechat ID: mxz1997112.
Acknowledgement
This repo is inspired by Awesome-LLM