Home

Awesome

Dual Composition Network (DCNet)

This project hosts the code for our paper.

This project is an Winning Solution in FashionIQ 2020.

Reference

If you use this code as part of any published research, please refer following paper,

@inproceedings{kim:2021:AAAI,
    title="{Dual Compositional Learning in Interactive Image Retrieval}",
    author={Kim, Jongseok and Yu, Youngjae and Kim, Hoeseong and Kim, Gunhee},
    booktitle={AAAI},
    year=2021
}

Getting Started

Prerequisites

Language: python==3.7.7, pytorch==1.4.0

pip install -r requirement.txt

Datasets

Preprocessing

Below code resize images and create glove embedded caption files.

cd preprocess
python -m nltk.downloader 'punkt'
python -m spacy download en_vectors_web_lg
python process_cap.py
python gen_shoe_cap.py
python resize_img.py

How to Run the code

Evaluation

For evaluation, first download checkpoint and config.json file from Fashion IQ, Shoe under ./logdir/fashioniq_dcnet, ./logdir/shoe_dcnet respectively. Then run below,

python test.py --resume logdir/fashioniq_dcnet/
python test.py --resume logdir/shoe_dcnet/

Training

For training run below,

python train.py --config configs/ce/fashioniq_dcnet.json --logdir fashioniq
python train.py --config configs/ce/shoe_dcnet.json --logdir shoe

You can change json file for other settings.

Experiments

For qualitative result and attention plot for text expert, refer

./experiments/result.ipynb
./experiments/textatt.ipynb

Deepfashion pretrained densenet

For Fashion IQ challenge performance, we provide deepfashion pretrained densenet as backbone (you may need larger VRAM)

  1. Download backbone checkpoint from here and save it under ./deepfashion/logdir/deepfashion_densenet
  2. Run following command to train
python train.py --config configs/ce/fashioniq_dcnet_deep.json --logdir fashioniq_deep