Home

Awesome

python version License: AGPL linting workflow

<div align="center"> <h2>ECCV 2024</h2> <h1>Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization</h1>

Ming-Yang Ho<sup>1</sup>,   Che-Ming Wu<sup>2</sup>,   Min-Sheng Wu<sup>3</sup>,   Yufeng Jane Tseng<sup>1</sup>

<sup>1</sup>National Taiwan University,   <sup>2</sup>Amazon Web Services,   <sup>3</sup>aetherAI<br>

[Paper] [Project Page] <br><br><image src="./images/teaser.jpg"/> <br><image src="./images/framework.jpg"/>

</div>

Get started with an example

We provide a simple example (one image from the Kyoto summer2autumn dataset) for you to translate an UHR image with our DN.

Download example data

$ ./download.sh
$ unzip simple_example.zip

Environment preparation

  1. Please check your GPU driver version and modify Dockerifle accordingly
  2. Then, execute 
    $ docker-compose up --build -d
  3. Get into the docker container 
    $ docker exec -it dn-env bash

Inference

  1. In the docker container, please execute 
    $ python3 transfer.py -c data/japan/config.yaml
  2. Then, you can see a translated image at experiments/japan_CUT/test/IMG_6610/combined_dn_10.png
  3. To see the image conveniently, you can leverage the provided visualization.ipynb. The setup of jupyter notebbok can be achived by
    • a. modify a port mapping setting in docker-compose.yml; e,g, - 19000:8888
    • b. install jupyter in the container
    • c. run your jupyter notebook by nohup jupyter notebook --ip=0.0.0.0 --port=8888 --allow-root &
    • d. open the jupter notebook service on your port (19000 here)

Datasets

real2paint Dataset

For the real domain, please download the UHDM dataset from its official website. For the painting domain, we have curated a dataset of high-resolution Vincent van Gogh paintings, which can be downloaded at link1 or link2. Please note that we do not own these images; users should ensure their use does not trigger legal issues.

Kyoto-summer2autumn Dataset

Please download it at link.

ANHIR Dataset

Please download it at link. Please note that we do not own these images; users should ensure their use does not trigger legal issues.

ACROBAT Dataset

Please download it at link. Please note that we do not own these images; users should ensure their use does not trigger legal issues.

Train your model

The training of I2I model is the same as KIN. DN is a plugin for any I2I model with InstanceNorm layers.

Citation

@misc{ho2024pixelmomentsultrahighresolutionunpaired,
      title={Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization}, 
      author={Ming-Yang Ho and Che-Ming Wu and Min-Sheng Wu and Yufeng Jane Tseng},
      year={2024},
      eprint={2407.04245},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2407.04245}, 
}