Awesome

FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin

Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center

News

• 2024.09.16 Technical Report: FlashOcc can be insert to Bevdet with 1.1ms consumption while facilitating each other.
• 2024.09.16 Selected as reference algorithm for occupancy on horizon J6E/M
• 2024.06.10 Release the code for Panoptic-FlashOCC
• 2024.04.17 Support for ray-iou metric
• 2024.03.22 Release the code for FlashOCCV2
• 2024.02.03 Release the training code for FlashOcc on UniOcc
• 2024.01.20 TensorRT Implement Writen In C++ With Cuda Acceleration
• 2023.12.23 Release the quick testing code via TensorRT in MMDeploy.
• 2023.11.28 Release the training code for FlashOCC.

This repository is an official implementation of FlashOCC

and Panoptic-FlashOCC

Main Results

1. FlashOCC

FPS are tested via TensorRT on 3090 with FP16 precision. Please refer to Tab.2 in paper for the detail model settings for M-number.

2. Panoptic-FlashOCC

In Panoptic-FlashOCC, we have made the following 3 adjustments to FlashOCC:

• Without using camera mask for training. This is because its use significantly improves the prediction performance in the visible region, but at the expense of prediction in the invisible region.
• Using category balancing.
• Using stronger loss settings.
• Introducing instance center for panoptic occupancy

More results for different configurations will be released soon.

• Please note that the FPS here is measured with A100 GPU (PyTorch fp32 backend).

Get Started

1. Environment Setup
2. Model Training
3. Quick Test Via TensorRT In MMDeploy

4. Visualization

• [flashocc] : A detail video can be found at baidu

• [panoptic-flashocc] : first row is our prediction and second row is gt.

5. TensorRT Implement Writen In C++ With Cuda Acceleration

Acknowledgement

Many thanks to the authors of BEVDet, FB-BEV, RenderOcc and SparseBEV

Bibtex

If this work is helpful for your research, please consider citing the following BibTeX entry.

@article{yu2024ultimatedo,
  title={UltimateDO: An Efficient Framework to Marry Occupancy Prediction with 3D Object Detection via Channel2height},
  author={Yu, Zichen and Shu, Changyong},
  journal={arXiv preprint arXiv:2409.11160},
  year={2024}
}

@article{yu2024panoptic,
  title={Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center},
  author={Yu, Zichen and Shu, Changyong and Sun, Qianpu and Linghu, Junjie and Wei, Xiaobao and Yu, Jiangyong and Liu, Zongdai and Yang, Dawei and Li, Hui and Chen, Yan},
  journal={arXiv preprint arXiv:2406.10527},
  year={2024}
}

@article{yu2023flashocc,
      title={FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin}, 
      author={Zichen Yu and Changyong Shu and Jiajun Deng and Kangjie Lu and Zongdai Liu and Jiangyong Yu and Dawei Yang and Hui Li and Yan Chen},
      year={2023},
      eprint={2311.12058},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}