Awesome

The current codebase will be updated soon to reflect the results found in the following paper:

Deep object detection for waterbird monitoring using aerial imagery<br/> Krish Kabra^*,1, Alexander Xiong^*,1, Wenbin Li^*,1, Minxuan Luo¹, William Lu¹, Raul Garcia¹, Dhananjay Vijay¹, Jiahui Yu¹, Maojie Tang¹, Tianjiao Yu¹, Hank Arnold², Anna Vallery², Richard Gibbons³, Arko Barman¹ <br/> ^* equal contribution <br/> ¹Rice University, Houston, TX 77005, USA <br/> ²Houston Audubon Society, Houston, TX 77079, USA <br/> ³American Bird Conservancy, The Plains, VA 20198, USA <br/>

Stay tuned!

<br> <img src="https://raw.githubusercontent.com/facebookresearch/detectron2/main/.github/Detectron2-Logo-Horz.svg" width="200"> <br>

The following open source packages are used in this project:

• Numpy
• Pandas
• Matplotlib
• OpenCV
• Detectron2
• WAndB

code
.
├── configs
├────── (useful sweep config files for WAndB)
├── scripts
├────── data_exploration.py
├── utils
├────── config.py
├────── cropping.py
├────── dataloader.py
├────── evaluation.py
├────── plotting.py
├────── trainer.py
├── README.md
├── requirements.txt
├── data_exploration.py  
├── Audubon-Bird-Detection-Tutorial.ipynb
├── train_net.py
├── wandb_train_net.py

git clone https://github.com/RiceD2KLab/Audubon_F21.git

pip3 install torch==1.10.0+cu102 torchvision==0.11.1+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html

python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
# (add --user if you don't have permission)

# Or, to install it from a local clone:
git clone https://github.com/facebookresearch/detectron2.git
python -m pip install -e detectron2

# On macOS, you may need to prepend the above commands with a few environment variables:
CC=clang CXX=clang++ ARCHFLAGS="-arch x86_64" python -m pip install ...

pip install requirements.txt

<br>

See train_net.py, wandb_train_net.py, or Colab Notebook for usage of code.

A caveat of this approach is that unavoidably some birds will be cut into two parts and appear in two neighboring patches, as seen in Figure 2. In addition, as counting the number of birds is among the objectives, the same problem needs to be tackled in the detection phase as well. In this case, only the generated image with over 50% fraction of the cropped bird keeps the bounding box, while the remaining fraction of the bounding box in another image is discarded. This means that we are training the model to detect both complete birds and partial birds.

In the detection stage, we will also try to come up with a proper merging mechanism to merge partial detection in neighboring patches and count as one if repeated counting is a common pattern in detection.

Deep learning models are effective with about 1,000 images per class, but some bird species do not have abundant training samples in our dataset. Our team plans to make deep learning models more robust via data augmentation, which means training models with synthetically modified data:

These data augmentation steps help models adapt to different orientations, locations, and scales of the same object class, and will boost the performance of the models.

We utilized the <i>imgaug</i> library to generate modified images. We have tried several types of augmentations: flipping, blurring, adding Gaussian noise and changing color contrasts.

<b> For the time being, our model is only trained on original data. </b> We plan to retrain our model on the augmented dataset and compare performances. We are generating a larger training set using the augmentation methods mentioned above. Specifically, both the original images and the transformed images will be fed to the model in the training phase, but only original images will be used for evaluation and testing purposes.

We utilize a RetinaNet and Faster R-CNN module both with a ResNet-50-FPN backbone. We first train our model to perform the simple task of detecting birds with no distinction of species. We then train the model to identify bird species: namely, Brown Pelicans, Laughing Gulls, Mixed Terns, Great Blue Herons, and Great Egrets/White Morphs.

Due to the lack of annotated data available for other bird species, we re-label all other bird species under the "Other/Unknown" category.

<i><b>Note:</b> The model weights used to initialize both the bird-only and bird-species detector come from a pre-trained model on the MS COCO dataset. </i>

The high AP of 93.7% using an IoU threshold of 0.50 is very promising.

The mAP of 43.7% is comparableto the state-of-the-art results for challenging object detection tasks such as on the COCO dataset.

The higher AP for all bird species using an IoU threshold of 0.50 in comparison to the bird-only detector is excellent, except for the “Other/Unknown” categroy, where the model drastically fails to classify. Nevertheless, we can combine the results from a bird-only detector and bird-species detector to recover the poor performance of the "Other/Unknown" bird category.

<b>Krish Kabra</b> <br>       Email: krish.kabra@rice.edu <a></a> <br>       GitHub: <a href="https://github.com/krishk97">@krishk97</a> <br>

<b>Minxuan Luo</b> <br>       Email: ml122@rice.edu<a></a> <br>       GitHub: <a href="https://github.com/minxuanluo">@minxuanluo</a> <br>

<b>Alexander Xiong</b> <br>       Email: xionga27@rice.edu<a></a> <br>       GitHub: <a href="https://github.com/awx1">@awx1</a> <br>

<b>William Lu</b> wyl1@rice.edu<br>       Email: <a></a> <br>       GitHub: <a href="">@</a> <br>

<b>Anna Vallery</b> <br>       Email: avallery@houstonaudubon.org<a></a> <br>

<b>Richard Gibbons Lu</b> <br>       Email: rgibbons@houstonaudubon.org<a></a> <br>

<b>Hank Arnold</b> <br>       Email: hmarnold@msn.com<a></a> <br>