Awesome
RMFormer
This Repo. is used for our ACM MM2023 paper:
Recurrent Multi-scale Transformer for High-Resolution Salient Object Detection, ACM MM2023 (https://arxiv.org/abs/2308.03826)
HRS10K Dataset
We contribute a new HRSOD dataset named HRS10K, which contains a total of 10,500 images, dividing 8,400 images for training and 2,100 images for testing.
Download Link:
Baidu [Extracting code: a750]
Usage
Conda Enviorment
- PyTorch 1.12.1
- python 3.8
- cudatoolkit 11.6
- OpenCV
- einops
- timm
- skimage
- tqdm
- tenosorboard
You can initialize a new conda enviroment as follow:
# Create New Enviorment
conda create -n RMFormer python=3.8
# CUDA & PyTorch
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6 -c pytorch -c conda-forge
# Others
conda install tqdm tensorboard tensorboardX
pip install opencv-python einops timm scikit-image
Directory
The directory should be like this:
-- loss (loss function)
-- model (model structure)
|-- RMFormer.py
-- save_models (pretrained model)
|-- Atemp (save training process models)
|-- pretrain (pretrain swin model)
| |-- swin_base_patch4_window12_384_22k.pth
-- train_data (train/test datasets, change in ./myconfig.py)
|-- HRS10K
| |-- image
| |-- mask
| |-- Results
|-- UHRSD
| |-- image
| |-- mask
| |-- Results
|-- HRSOD
| |-- image
...
Edge Map Generate
Modify the dir in ./gen_edgemap.py, then run:
python gen_edgemap.py
Train
Download Swin pretrain model, save them in ./save_models/pretrain
The link below is from https://github.com/microsoft/Swin-Transformer
Baidu [Extracting code:swin]
Change the dataset directory, training setting in ./myconfig.py
Make sure --itr_epoch in training_param is set correctly
then run:
bash train.sh
- After training, models will be saved in
./save_models/Atemp - Tensorboard log files are in
./runs1
Interference
We trained model in three different training setting: DH, UH and KUH
These trained models be download here:
Baidu [Extracting code:iavg] Google Drive
The saliency maps of our RMFormer can be download here:
Baidu [Extracting code:4h2g] Google Drive
Notice: We have reorganized the code and achieved enhanced performance by adjusting the parameter initialization before training. If you would like to access the improved models, please download them from the provided link below.
Trained models: Baidu [Extracting code:a2r6] Google Drive
Saliency maps: Baidu [Extracting code:akhg] Google Drive
Change the paths in ./test.py, then run:
python test.py
- After testing, saliency maps will be saved in the
'prediction_dir'
Evaluation
The code in ./eval.py is from:
https://github.com/xuebinqin/Binary-Segmentation-Evaluation-Tool
We also use the evalutation below to generate results in our paper:
https://github.com/Jun-Pu/Evaluation-on-salient-object-detection
Some modifications are made in our codes for generating mBA results.
You can refer to codes in ./evaluation_code.zip
If you have any problems. Please concat