Home

Awesome

The Art of Camouflage: Few-shot Learning for Animal Detection and Segmentation

This repository is the official implementation of the paper entitled: The Art of Camouflage: Few-shot Learning for Animal Detection and Segmentation, IEEE Access, 2024.

Authors: Thanh-Danh Nguyen, Anh-Khoa Nguyen Vu, Nhat-Duy Nguyen, Vinh-Tiep Nguyen, Thanh Duc Ngo, Thanh-Toan Do, Minh-Triet Tran, Tam V. Nguyen*.

[Paper] [ArXiv] [Code] [Project Page]

Updates

[2024/7] We have released the checkpoints, visualization, and initial instructions for FS-CDIS⚡!

1. Environment Setup

Download and install Anaconda with the recommended version from Anaconda Homepage: Anaconda3-2019.03-Linux-x86_64.sh

git clone https://github.com/danhntd/FS-CDIS.git
cd FS-CDIS
curl -O https://repo.anaconda.com/archive/Anaconda3-2019.03-Linux-x86_64.sh
bash Anaconda3-2019.03-Linux-x86_64.sh

After completing the installation, please create and initiate the workspace with the specific versions below. The experiments were conducted on a Linux server with a single GeForce RTX 2080Ti GPU, CUDA 10.1/10.2, Torch 1.7.

conda create --name FSCDIS python=3
conda activate FSCDIS
conda install pytorch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 cudatoolkit=10.2 -c pytorch

This source code is based on Detectron2. Please refer to INSTALL.md for the pre-built or building Detectron2 from source.

After setting up the dependencies, use the command pip install -e . in this root to finish.

2. Data Preparation

Download the datasets

The proposed CAMO-FS is available at this link.

Register datasets

Detectron2 requires a step of data registration for those who want to use the external datasets (Detectron2 Docs), which is already prepared in this repository.

3. Training Pipeline

<!-- Our proposed FS-CDIS framework: <img align="center" src="/visualization/framework.png"> -->

Our detailed proposals of instance triplet loss and instance memory storage: <img align="center" src="/visualization/framework_fs-cdis-memo-redesign-ieee-access.png">

Configurations

All configs can be found in the ./configs/ directory.

<!-- Initial parameters: ``` ``` ### Training ``` ``` --> <!-- ### Pre-defined variables ``` export CUDA_VISIBLE_DEVICES=0 export NGPUS=1 cfg_MODEL=' MODEL.ROI_HEADS.NUM_CLASSES 16 SOLVER.MAX_ITER 2000 ' MODEL_NAME='novel1_1shot' OUTPUT_DIR=checkpoints/camo_mtfa_default/camo_model_${MODEL_NAME}_mask_rcnn_R_101_FPN_mtfa config=configs/CAMO-shot_mtfa_default/mask_rcnn_R_101_FPN_ft_fsdet_cos_${MODEL_NAME}.yaml WEIGHT=weights/mrcnn_r101_fpn_80cls.pkl ``` ### Testing ``` python tools/run_train.py --num-gpus ${NGPUS} \ --dist-url auto \ --resume \ --config-file ${config} \ --opts MODEL.WEIGHTS ${WEIGHT} OUTPUT_DIR ${OUTPUT_DIR} ${cfg_MODEL} SOLVER.STEPS "(40000, 54000)" ``` -->

The whole script commands can be found in ./scripts/*.

Released checkpoints and results:

We provide the checkpoints of our final model:

<!-- | Model R-101 | FS-CDIS-ITL | FS-CDIS-IMS | | ----------- |:---------------:|:----------------:| | 1-shot |[link](https://) | [link](https://) | | 2-shot |[link](https://) | [link](https://) | | 3-shot |[link](https://) | [link](https://) | | 5-shot |[link](https://) | [link](https://) | -->
Model R-101FS-CDIS-ITLmask APbox APFS-CDIS-IMSmask APbox AP
1-shotlink4.464.04link5.464.50
2-shotlink5.577.28link6.956.95
3-shotlink6.417.49link7.367.55
5-shotlink8.489.76link9.6110.36

4. Visualization

<p align="center"> <img width="800" src="/visualization/visualization.png"> </p>

Citation

Please use the following bibtex to cite this repository:

@article{nguyen2024art,
  title={The Art of Camouflage: Few-shot Learning for Animal Detection and Segmentation},
  author={Nguyen, Thanh-Danh and Vu, Anh-Khoa Nguyen and Nguyen, Nhat-Duy and Nguyen, Vinh-Tiep and Ngo, Thanh Duc and Do, Thanh-Toan and Tran, Minh-Triet and Nguyen, Tam V},
  journal={IEEE Access},
  volume={-},
  pages={-},
  year={2024},
  publisher={IEEE}
}

Acknowledgements

iMTFA Detectron2