Home

Awesome

Python >=3.8 PyTorch >=1.1

Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation

The official repository for Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation.

 The overall framework of ATVC.

Requirements

Installation

We provide an environment file; environment.yml containing the required dependencies. Clone the repo and run the following command in the root of this directory:

conda env create -f environment.yml

Dataset

Please refer to DOWNLOAD.md for dataset preparation.

Pretrained Models

Please refer to pretrained-models to download the released models.

Train

Training commands

bash dist_train_vae.sh ${DATA_NAME} ${NODES} ${GPUS}
bash dist_train_atvc.sh ${VAE_PATH} ${DATA_NAME} ${NODES} ${GPUS}

Arguments

Test

Testing commands

bash gen_vae.sh ${GPU} ${VAE_PATH} ${IMAGE_PATH}
bash gen_atvc.sh ${GPU} ${ATVC_PATH} ${TEXT_QUERY} ${IMAGE_PATH}

Arguments

License

ATVC is released under the Apache 2.0 license.

Citation

If you find this code useful for your research, please cite our paper

@article{zhang2023accountable,
  title={Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation},
  author={Zhang, Zhiwei and Liu, Yuliang},
  journal={arXiv preprint arXiv:2303.05983},
  year={2023}
}

Acknowledgement

Our code is learned from DALLE-pytorch and CLIP. We would like to thank all the people who help label text-image pairs and participate in human evaluation experiments. We hope our explorations and findings contribute valuable insights regarding the accountability of textual-visual generative models.

Contact

This project is developed by Zhiwei Zhang (@zzw-zwzhang) and Yuliang Liu (@Yuliang-Liu).