Awesome

R-YOLOv4

Introduction

The objective of this project is to adapt YOLOv4 model to detecting oriented objects. As a result, modifying the original loss function of the model is required. I got a successful result by increasing the number of anchor boxes with different rotating angle and combining smooth-L1-IoU loss function proposed by R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object into the original loss for bounding boxes.

Features

Loss Function (only for x, y, w, h, theta)

*UPDATE*

The followings are updated to the code to improve the performance of the original setting.

• Circular Smooth Labeling
• KFIoU
• Yolov5
• Yolov7

Setup

1. Clone repository

   $ git clone https://github.com/kunnnnethan/R-YOLOv4.git
   $ cd R-YOLOv4/
   

2. Create Environment

• Conda

  1. Create virual environment

     $ conda create -n ryolo python=3.8
     $ conda activate ryolo
     

  2. Install PyTorch and torchvision following the official instructions, e.g.,

     If you are using CUDA 11.8 version
     $ conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
     

  3. Install required libraries

     $ pip install -r requirements.txt
     

  4. Install detectron2 for calculating SkewIoU on GPU following the official instructions, e.g.,

     python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
     

• Docker

  $ docker build -t ryolo docker/
  $ sudo docker run -it --rm --gpus all --ipc=host --ulimit memlock=-1 -v ${your_path}:/workspace ryolo
  

6. Download pretrained weights</br> weights

7. Make sure your files arrangment looks like the following</br> Note that each of your dataset folder in data should split into three files, namely train, test, and detect.

   R-YOLOv4/
   ├── train.py
   ├── test.py
   ├── detect.py
   ├── xml2txt.py
   ├── environment.xml
   ├── requirements.txt
   ├── model/
   ├── datasets/
   ├── lib/
   ├── outputs/
   ├── weights/
       ├── pretrained/ (for training)
       └── UCAS-AOD/ (for testing and detection)
   └── data/
       └── UCAS-AOD/
           ├── class.names
           ├── train/
               ├── ...png
               └── ...txt
           ├── test/
               ├── ...png
               └── ...txt
           └── detect/
               └── ...png
   

Train

I have implemented methods to load and train three different datasets. They are UCAS-AOD, DOTA, and custom dataset respectively. You can check out how I loaded those dataset into the model at /datasets. The angle of each bounding box is limited in (- pi/2, pi/2], and the height of each bounding box is always longer than it's width.

Check all the setting in .yaml files that you are going to use in the /data folder.

$ python train.py --model_name DOTA_yolov7_csl_800 --config data/hyp.yaml --img_size 800 --data data/DOTA.yaml --epochs 100 --mode csl --ver yolov7

You can run display_inputs.py to visualize whether your data is loaded successfully.

UCAS-AOD dataset

Please refer to this repository to rearrange files so that it can be loaded and trained by this model.</br>

DOTA dataset

Download the official dataset from here. The original files should be able to be loaded and trained by this model.</br>

Train with custom dataset

1. Use labelImg2 to help label your data. labelImg2 is capable of labeling rotated objects.
2. Move your data folder into the R-YOLOv4/data folder.
3. Run xml2txt.py
   1. generate txt files: python xml2txt.py --data_folder your-path --action gen_txt
   2. delete xml files: python xml2txt.py --data_folder your-path --action del_xml

A trash custom dataset that I made are provided for your convenience.

Test

$ python test.py --data data/DOTA.yaml --hyp data/hyp.yaml --weight_path weights/DOTA_yolov7_csl_800/best.pth --batch_size 8 --img_size 800 --mode csl --ver yolov7

detect

$ python detect.py --data data/UCAS_AOD.yaml --hyp data/hyp.yaml --weight_path weights/DOTA_yolov7_csl_800/best.pth --batch_size 8 --img_size 800 --conf_thres 0.3 --mode csl --ver yolov7

Tensorboard

If you would like to use tensorboard for tracking traing process.

• Open additional terminal in the same folder where you are running program.
• Run command $ tensorboard --logdir=weights --port=6006
• Go to http://localhost:6006/

Results

UCAS_AOD

DOTA

<img src="https://github.com/kunnnnethan/R-YOLOv4/blob/main/outputs/DOTA/P0006.png" alt="DOTA" height="430"/><img src="https://github.com/kunnnnethan/R-YOLOv4/blob/main/outputs/DOTA/P0031.png" alt="DOTA" height="430"/>

trash (custom dataset)

References

WongKinYiu/yolov7</br> ultralytics/yolov5</br> Tianxiaomo/pytorch-YOLOv4</br> yangxue0827/RotationDetection</br> eriklindernoren/PyTorch-YOLOv3</br>

YOLOv4: Optimal Speed and Accuracy of Object Detection

@article{yolov4,
  title={YOLOv4: Optimal Speed and Accuracy of Object Detection},
  author={Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao},
  journal = {arXiv},
  year={2020}
}

R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object

@article{r3det,
  title={R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object},
  author={Xue Yang, Junchi Yan, Ziming Feng, Tao He},
  journal = {arXiv},
  year={2019}
}