Awesome

English | 简体中文

<h2 align="center">RT-DETR: DETRs Beat YOLOs on Real-time Object Detection</h2> <p align="center">  <a href="https://github.com/lyuwenyu/RT-DETR/blob/main/LICENSE"> <img alt="license" src="https://img.shields.io/github/license/lyuwenyu/RT-DETR"> </a> <a href="https://github.com/lyuwenyu/RT-DETR/pulls"> <img alt="prs" src="https://img.shields.io/github/issues-pr/lyuwenyu/RT-DETR"> </a> <a href="https://github.com/lyuwenyu/RT-DETR/issues"> <img alt="issues" src="https://img.shields.io/github/issues/lyuwenyu/RT-DETR?color=pink"> </a> <a href="https://github.com/lyuwenyu/RT-DETR"> <img alt="issues" src="https://img.shields.io/github/stars/lyuwenyu/RT-DETR"> </a> <a href="https://arxiv.org/abs/2304.08069"> <img alt="arXiv" src="https://img.shields.io/badge/arXiv-2304.08069-red"> </a> <a href="mailto: lyuwenyu@foxmail.com"> <img alt="emal" src="https://img.shields.io/badge/contact_me-email-yellow"> </a> </p>

This is the official implementation of papers

🚀 Updates

[2024.10.10] Add sliced inference support for small object detecion. #468
[2024.09.23] Add ✅Regnet and DLA34 for RTDETR.
[2024.08.27] Add hubconf.py file to support torch hub.
[2024.08.22] Improve the performance of ✅ RT-DETRv2-S to 48.1 mAP (<font color=green>+1.6</font> compared to RT-DETR-R18).
[2024.07.24] Release ✅ RT-DETRv2!
[2024.02.27] Our work has been accepted to CVPR 2024!
[2024.01.23] Fix difference on data augmentation with paper in rtdetr_pytorch #84.
[2023.11.07] Add pytorch ✅ rtdetr_r34vd for requests #107, #114.
[2023.11.05] Upgrade the logic of remap_mscoco_category to facilitate training of custom datasets, see detils in Train custom data part. #81.
[2023.10.23] Add discussion for deployments, supported onnxruntime, TensorRT, openVINO.
[2023.10.12] Add tuning code for pytorch version, now you can tuning rtdetr based on pretrained weights.
[2023.09.19] Upload ✅ pytorch weights convert from paddle version.
[2023.08.24] Release RT-DETR-R18 pretrained models on objects365. 49.2 mAP and 217 FPS.
[2023.08.22] Upload ✅ rtdetr_pytorch source code. Please enjoy it!
[2023.08.15] Release RT-DETR-R101 pretrained models on objects365. 56.2 mAP and 74 FPS.
[2023.07.30] Release RT-DETR-R50 pretrained models on objects365. 55.3 mAP and 108 FPS.
[2023.07.28] Fix some bugs, and add some comments. 1, 2.
[2023.07.13] Upload ✅ training logs on coco.
[2023.05.17] Release RT-DETR-R18, RT-DETR-R34, RT-DETR-R50-m（example for scaled).
[2023.04.17] Release RT-DETR-R50, RT-DETR-R101, RT-DETR-L, RT-DETR-X.

📍 Implementations

🔥 RT-DETRv2
- paddle: code&weight
- pytorch: code&weight
🔥 RT-DETR
- paddle: code&weight
- pytorch: code&weight

Model	Input shape	Dataset	$AP^{val}$	$AP^{val}_{50}$	Params(M)	FLOPs(G)	T4 TensorRT FP16(FPS)
RT-DETR-R18	640	COCO	46.5	63.8	20	60	217
RT-DETR-R34	640	COCO	48.9	66.8	31	92	161
RT-DETR-R50-m	640	COCO	51.3	69.6	36	100	145
RT-DETR-R50	640	COCO	53.1	71.3	42	136	108
RT-DETR-R101	640	COCO	54.3	72.7	76	259	74
RT-DETR-HGNetv2-L	640	COCO	53.0	71.6	32	110	114
RT-DETR-HGNetv2-X	640	COCO	54.8	73.1	67	234	74
RT-DETR-R18	640	COCO + Objects365	49.2	66.6	20	60	217
RT-DETR-R50	640	COCO + Objects365	55.3	73.4	42	136	108
RT-DETR-R101	640	COCO + Objects365	56.2	74.6	76	259	74
RT-DETRv2-S	640	COCO	48.1 <font color=green>(+1.6)</font>	65.1	20	60	217
RT-DETRv2-M<sup>*<sup>	640	COCO	49.9 <font color=green>(+1.0)</font>	67.5	31	92	161
RT-DETRv2-M	640	COCO	51.9 <font color=green>(+0.6)</font>	69.9	36	100	145
RT-DETRv2-L	640	COCO	53.4 <font color=green>(+0.3)</font>	71.6	42	136	108
RT-DETRv2-X	640	COCO	54.3	72.8 <font color=green>(+0.1)</font>	76	259	74

Notes:

COCO + Objects365 in the table means finetuned model on COCO using pretrained weights trained on Objects365.

🦄 Performance

🏕️ Complex Scenarios

🌋 Difficult Conditions

Citation

If you use RT-DETR or RTDETRv2 in your work, please use the following BibTeX entries:

@misc{lv2023detrs,
      title={DETRs Beat YOLOs on Real-time Object Detection},
      author={Yian Zhao and Wenyu Lv and Shangliang Xu and Jinman Wei and Guanzhong Wang and Qingqing Dang and Yi Liu and Jie Chen},
      year={2023},
      eprint={2304.08069},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

@misc{lv2024rtdetrv2improvedbaselinebagoffreebies,
      title={RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer}, 
      author={Wenyu Lv and Yian Zhao and Qinyao Chang and Kui Huang and Guanzhong Wang and Yi Liu},
      year={2024},
      eprint={2407.17140},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2407.17140}, 
}