Home

Awesome

RenderOcc

Paper | Video | Technical Report (UniOcc)

demo (Visualization of RenderOcc's prediction, which is supervised only with 2D labels.)

INTRODUCTION

RenderOcc is a novel paradigm for training vision-centric 3D occupancy models only with 2D labels. Specifically, we extract a NeRF-style 3D volume representation from multi-view images, and employ volume rendering techniques to establish 2D renderings, thus enabling direct 3D supervision from 2D semantics and depth labels.

demo

Getting Started

Model Zoo

MethodBackbone2D-to-3DLr SchdGTmIoUConfigLogDownload
RenderOccSwin-BaseBEVStereo12ep2D24.46configlogmodel

Acknowledgement

Many thanks to these excellent open source projects:

Related Projects:

BibTeX

If this work is helpful for your research, please consider citing:

@article{pan2023renderocc,
  title={RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision},
  author={Pan, Mingjie and Liu, Jiaming and Zhang, Renrui and Huang, Peixiang and Li, Xiaoqi and Liu, Li and Zhang, Shanghang},
  journal={arXiv preprint arXiv:2309.09502},
  year={2023}
}