Home

Awesome

Virtual Try-On with Garment-Pose Keypoints Guided Inpainting

This codes repository provides the pytorch implementation of the KGI virtual try-on method proposed in ICCV23 paper Virtual Try-On with Garment-Pose Keypoints Guided Inpainting.

Experimental Environment

Please follow the steps below to build the environment and install the required packages.

conda create -n kgi python=3.8 -y
conda activate kgi
bash install_pkgs.sh

Data Preparation

  1. The VITON-HD dataset could be downloaded from VITON-HD . Please place the dataset under the directory KGI/data/. The dataset contains the following content:

    ContentComment
    agnostic-masknot in use in KGI
    agnostic-v3.2not in use in KGI
    cloth
    cloth-masknot in use in KGI
    image
    image-denseposenot in use in KGI
    image-parse-agnostic-v3.2not in use in KGI
    image-parse-v3
    openpose_imgnot in use in KGI
    openpose_json
  2. In addition to above content, some other preprocessed conditions are in use in KGI. The content are generated with the data preprocessing codes [WIP]. The preprocessed data could be downloaded, respectively.

    ContentTrainTest
    image-landmark-jsonGoogle DriveGoogle Drive
    cloth-landmark-jsonGoogle DriveGoogle Drive
    labelGoogle DriveGoogle Drive
    parseGoogle DriveGoogle Drive
    parse_ag_fullGoogle DriveGoogle Drive
    ag_maskGoogle DriveGoogle Drive
    skin_maskGoogle DriveGoogle Drive
    • Data Preprocessing [WIP]
  3. Download the demo_paired_pairs.txt and demo_unpaired_pairs.txt under the directory KGI/data/zalando-hd-resized/ for in-training visualization.

  4. The structure of processed dataset should be as below:

    • KGI/data/zalando-hd-resized/
      • test/
        • ag_mask/
        • cloth/
        • cloth-landmark-json/
        • image/
        • image-landmark-json/
        • image-parse-v3/
        • openpose_json/
        • parse/
        • parse_ag_full/
        • skin_mask/
        • label.json
      • train/
        • ...
      • demo_paired_pairs.txt
      • demo_unpaired_pairs.txt
      • test_pairs.txt
      • train_pairs.txt

Model Training

The model training of the KGI method consists of three steps: Training of the Keypoints Generator, Training of the Parse Generator, Training of the Semantic Conditioned Inpainting Model.

Keypoints Generator

Parse Generator

Semantic Conditioned Inpainting Model

Demo with Pretrained Model

With the pretrained models, the final try-on results and the visualizations of the intermediate results could be generated with the following demo scripts:

python3 generate_demo.py

The final try-on results will be saved under KGI/example/generate_demo/final_results/ and the visualizations of the intermediate results will be saved under KGI/example/generate_demo/vis/. Below is an example of demo results. Demo Image 3

Acknowledgement and Citations