Awesome
GR-GAN
Pytorch implementation for reproducing GR-GAN results in the paper GRADUAL REFINEMENT TEXT-TO-IMAGE GENERATION
<img src="GAN_Model_V4.png" width="900px" height="550px"/>Usage
Get the code from github:git clone https://github.com/BoO-18/GR-GAN.git
Create a new conda env:conda create -n grgan python=3.7
and conda activate grgan
Install torch==1.7.1
: conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0
Folder CLIP is code from OPENAI with some changes to the output of the image encoder and text encoder. You should run:pip install CLIP
to install it.
Data
- Download our preprocessed metadata for coco and save them to
data/
- Download coco dataset and extract the images to
data/coco/
Training
-
Pre-train ITM models:
python pretrain_ITM.py --cfg cfg/ITM/coco.yml --gpu 0
-
Train GR-GAN models:
python main.py --cfg cfg/coco_GRAGN.yml --gpu 1
-
*.yml
files are example configuration files for training/evaluation our models.
Pretrained Model
- ITM for coco or Use Alternate address提取码(Extraction code):
jnzz
. Download and save it tomodels/
- GR-GAN for coco or Use Alternate address提取码(Extraction code):
lbre
. Download and save it tomodels/
Demo
- File
code/demo.ipynb
is a detailed usage example for GR-GAN.
Validation
- Generate images for GR-GAN models on MS-COCO:
python main.py --cfg cfg/eval_coco.yml --gpu 2
- We compute inception score for models trained on coco:
python CMD_score.py --path1 image_feature.npz --path2 sent_feature.npz --path3 (your image path) --gpu 3