Awesome
Dual Composition Network (DCNet)
This project hosts the code for our paper.
- Jongseok Kim, Youngjae Yu, Hoeseong Kim and Gunhee Kim. Dual Compositional Learning in Interactive Image Retrieval. In AAAI, 2021.
This project is an Winning Solution in FashionIQ 2020.
Reference
If you use this code as part of any published research, please refer following paper,
@inproceedings{kim:2021:AAAI,
title="{Dual Compositional Learning in Interactive Image Retrieval}",
author={Kim, Jongseok and Yu, Youngjae and Kim, Hoeseong and Kim, Gunhee},
booktitle={AAAI},
year=2021
}
Getting Started
Prerequisites
Language: python==3.7.7, pytorch==1.4.0
pip install -r requirement.txt
Datasets
-
Download Fashion-IQ dataset images from here. Save it under ./dataset/fashioniq/images/
(May usepreprocess/download_fashioniq.py
for downloading images.) -
Download Fashion-IQ dataset annotations from here. Save it under ./dataset/fashioniq/ (Files under folder
captions
andimage_splits
) -
Download Shoe dataset images from here. Save it under ./dataset/shoe/images/
-
Download Shoe dataset annotations from here. Save it under ./dataset/shoe/
(We only needrelative_captions_shoes.json
,eval_im_names.txt
, andtrain_im_names.txt
.)
Preprocessing
Below code resize images and create glove embedded caption files.
cd preprocess
python -m nltk.downloader 'punkt'
python -m spacy download en_vectors_web_lg
python process_cap.py
python gen_shoe_cap.py
python resize_img.py
How to Run the code
Evaluation
For evaluation, first download checkpoint and config.json file from Fashion IQ, Shoe under ./logdir/fashioniq_dcnet
, ./logdir/shoe_dcnet
respectively.
Then run below,
python test.py --resume logdir/fashioniq_dcnet/
python test.py --resume logdir/shoe_dcnet/
Training
For training run below,
python train.py --config configs/ce/fashioniq_dcnet.json --logdir fashioniq
python train.py --config configs/ce/shoe_dcnet.json --logdir shoe
You can change json file for other settings.
Experiments
For qualitative result and attention plot for text expert, refer
./experiments/result.ipynb
./experiments/textatt.ipynb
Deepfashion pretrained densenet
For Fashion IQ challenge performance, we provide deepfashion pretrained densenet as backbone (you may need larger VRAM)
- Download backbone checkpoint from here and save it under ./deepfashion/logdir/deepfashion_densenet
- Run following command to train
python train.py --config configs/ce/fashioniq_dcnet_deep.json --logdir fashioniq_deep