Awesome

CAIR: Multi-Scale Color Attention Network for Instagram Filter Removal (ECCV 2022 Workshop)

Woon-Ha Yeo, Wang-Taek Oh, Kyung-Su Kang, Young-Il Kim, Han-Cheol Ryu

Challenge

• AIM 2022 Instagram Filter Removal Challenge

Installation & Setting

This implementation is based on NAFNet.

python 3.9.5
pytorch 1.11.0
cuda 11.3

pip install -r requirements.txt
python setup.py develop --no_cuda_ext

CAIR

Data Prepration

You can download IFFI(Instagram Filter Fashion Image) dataset on challenge website

You can train on IFFI dataset by following these steps:

  datasets
  └──IFFI
     └──IFFI-dataset-train
     |  └──0
     |  └──1
     |  └──2
     |  └──...
     └──IFFI-dataset-lr-train
     |  └──0
     |  └──1
     |  └──2
     |  └──...
     └──IFFI-dataset-lr-challenge-test-wo-gt
        └──0
        └──1
        └──2
        └──...

Your Instagram Filter Removal Challenge dataset to lmdb format. If you want to use ensemble learning, set --ensemble true.

python scripts/data/dataset_to_lmdb.py --basedir ./datasets/IFFI --ensemble false
python scripts/data/dataset_to_lmdb.py --basedir ./datasets/IFFI --ensemble true

Result data into submission format

python scripts/data/test_dataset.py --resultdir ./results/model_name

Train/Test

If you want to train/test other model, replace option file with others in options/ folder.

• Train

python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/train.py -opt options/train/CAIR/CAIR_M-width32.yml --launcher pytorch

• Test

# General test
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt options/test/CAIR/CAIR_M-width32.yml --launcher pytorch
#  TTA
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt options/test/CAIR-TTA/CAIR_M-width32.yml --launcher pytorch

Ensemble learning

• Data preparation

  For ensemble learning, train and test set should be inferenced with three models, and then three images from three models should be contatenated.

# CAIR_M-width32
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt ./options/test/Ensemble/CAIR_M-width32.yml --launcher pytorch
# CAIR_S-width32
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4319 basicsr/test.py -opt ./options/test/Ensemble/CAIR_S-width32.yml --launcher pytorch
python scripts/data/concat_ensemble_input.py

• Train ensemble network

python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/train.py -opt options/train/Ensemble/CAIR_Ensemble.yml --launcher pytorch

• Test ensemble network

python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt options/test/Ensemble/CAIR_Ensemble.yml --launcher pytorch

Results

Citation

@article{yeo2022cair,
  title={CAIR: Fast and Lightweight Multi-Scale Color Attention Network for Instagram Filter Removal},
  author={Yeo, Woon-Ha and Oh, Wang-Taek and Kang, Kyung-Su and Kim, Young-Il and Ryu, Han-Cheol},
  journal={arXiv preprint arXiv:2208.14039},
  year={2022}
}

Contacts

If you have any question, please contact mm074111@gmail.com

Acknowledgements

• https://github.com/XPixelGroup/BasicSR.git
• https://github.com/megvii-research/NAFNet.git
• https://github.com/swz30/CycleISP.git