Awesome
This repository is the official implementation of IJCV (accepted in 2024) "LSKNet: A Foundation Lightweight Backbone for Remote Sensing" at: IJCV or arxiv
Our conference version: ICCV 2023 "Large Selective Kernel Network for Remote Sensing Object Detection" at: ICCV Open Access
Abstract
Recent research on remote sensing object detection has largely focused on improving the representation of oriented bounding boxes but has overlooked the unique prior knowledge presented in remote sensing scenarios. Such prior knowledge can be useful because tiny remote sensing objects may be mistakenly detected without referencing a sufficiently long-range context, and the long-range context required by different types of objects can vary. In this paper, we take these priors into account and propose the Large Selective Kernel Network (LSKNet). LSKNet can dynamically adjust its large spatial receptive field to better model the ranging context of various objects in remote sensing scenarios. To the best of our knowledge, this is the first time that large and selective kernel mechanisms have been explored in the field of remote sensing object detection. Without bells and whistles, our lightweight LSKNet sets new state-of-the-art scores on standard remote sensing classification, object detection and semantic segmentation benchmarks. Based on a similar technique, we rank 2nd place in 2022 the Greater Bay Area International Algorithm Competition
Introduction
This repository is the official implementation of IJCV 2024 "LSKNet: A Foundation Lightweight Backbone for Remote Sensing" at: arxiv
The master branch is built on MMRotate which works with PyTorch 1.6+.
LSKNet backbone code is placed under mmrotate/models/backbones/, and the train/test configure files are placed under configs/lsknet/
Results and models
Imagenet 300-epoch pre-trained LSKNet-T backbone: Download
Imagenet 300-epoch pre-trained LSKNet-S backbone: Download
DOTA1.0
| Model | mAP | Angle | lr schd | Batch Size | Configs | Download | note |
|---|---|---|---|---|---|---|---|
| RTMDet-l (1024,1024,-) | 81.33 | - | 3x-ema | 8 | - | - | Prev. Best |
| LSKNet_T (1024,1024,200) + ORCNN | 81.37 | le90 | 1x | 2*8 | lsk_t_fpn_1x_dota_le90 | model | log | |
| LSKNet_S (1024,1024,200) + ORCNN | 81.64 | le90 | 1x | 1*8 | lsk_s_fpn_1x_dota_le90 | model | log | |
| LSKNet_S* (1024,1024,200) + ORCNN | 81.85 | le90 | 1x | 1*8 | lsk_s_ema_fpn_1x_dota_le90 | model | log | EMA Finetune |
| LSKNet_S (1024,1024,200) + Roi_Trans | 81.22 | le90 | 1x | 2*8 | lsk_s_roitrans_fpn_1x_dota | model | log | |
| LSKNet_S (1024,1024,200) + R3Det | 80.08 | oc | 1x | 2*8 | lsk_s_r3det_fpn_1x_dota | model | log | |
| LSKNet_S (1024,1024,200) + S2ANet | 81.32 | le135 | 1x | 2*8 | lsk_s_s2anet_fpn_1x_dota | model | log |
FAIR1M-1.0
| Model | mAP | Angle | lr schd | Batch Size | Configs | Download | note |
|---|---|---|---|---|---|---|---|
| O-RCNN (1024,1024,200) | 45.60 | le90 | 1x | 1*8 | oriented_rcnn_r50_fpn_1x_fair_le90 | - | Prev. Best |
| LSKNet_S (1024,1024,200) | 47.87 | le90 | 1x | 1*8 | lsk_s_fpn_1x_dota_le90 | model | log |
HRSC2016
| Model | mAP(07) | mAP(12) | Angle | lr schd | Batch Size | Configs | Download | note |
|---|---|---|---|---|---|---|---|---|
| RTMDet-l | 90.60 | 97.10 | le90 | 3x | - | - | - | Prev. Best |
| ReDet | 90.46 | 97.63 | le90 | 3x | 2*4 | redet_re50_refpn_3x_hrsc_le90 | - | Prev. Best |
| LSKNet_S | 90.65 | 98.46 | le90 | 3x | 1*8 | lsk_s_fpn_3x_hrsc_le90 | model | log |
Installation
MMRotate depends on PyTorch, MMCV and MMDetection. Below are quick steps for installation. Please refer to Install Guide for more detailed instruction.
conda create --name openmmlab python=3.8 -y
conda activate openmmlab
conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=10.2 -c pytorch
pip install -U openmim
mim install mmcv-full
mim install mmdet
git clone https://github.com/zcablii/Large-Selective-Kernel-Network.git
cd Large-Selective-Kernel-Network
pip install -v -e .
Get Started
Please see get_started.md for the basic usage of MMRotate. We provide colab tutorial, and other tutorials for:
LSKNet for Remote Sensing Segmentation
We further extend our work to segmentation tasks on the Potsdam, Vaihingen, LoveDA, and UAVid datasets. Please visit LSKNet + GeoSeg. To facilitate easy reproduction and swift initiation for beginners, we offer our prepared remote sensing segmentation datasets here.
-Vaihingen -Potsdam -LoveDA -uavid
Acknowledgement
MMRotate is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new methods.
Citation
If you use this toolbox or benchmark in your research, please cite this project.
@article{Li_2024_IJCV,
title={LSKNet: A Foundation Lightweight Backbone for Remote Sensing},
author={Li, Yuxuan and Li, Xiang and Dai, Yimain and Hou, Qibin and Liu, Li and Liu, Yongxiang and Cheng, Ming-Ming and Yang, Jian},
journal={International Journal of Computer Vision},
year={2024},
doi = {https://doi.org/10.1007/s11263-024-02247-9},
publisher={Springer}
}
@InProceedings{Li_2023_ICCV,
author = {Li, Yuxuan and Hou, Qibin and Zheng, Zhaohui and Cheng, Ming-Ming and Yang, Jian and Li, Xiang},
title = {Large Selective Kernel Network for Remote Sensing Object Detection},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {16794-16805}
}
License
Licensed under a Creative Commons Attribution-NonCommercial 4.0 International for Non-commercial use only. Any commercial use should get formal permission first.