Awesome
ManiFest: Manifold Deformation for Few-shot Image Translation
ManiFest: Manifold Deformation for Few-shot Image Translation
Fabio Pizzati, Jean-François Lalonde, Raoul de Charette
ECCV 2022
Preview
Citation
To cite our paper, please use
@inproceedings{pizzati2021manifest,
title={{ManiFest: Manifold Deformation for Few-shot Image Translation}},
author={Pizzati, Fabio and Lalonde, Jean-François and de Charette, Raoul},
booktitle={ECCV},
year={2022}
}
Prerequisites
Please create an environment using the requirements.yml
file provided.
conda env create -f requirements.yml
Download the pretrained models and the pretrained VGG used for the style alignment loss by following the link:
https://www.rocq.inria.fr/rits_files/computer-vision/manifest/manifest_checkpoints.tar.gz
Move the VGG network weights in the res
folder and the checkpoints in the checkpoints
one.
Inference
We provide pretrained models for the day2night, day2twilight and clear2fog tasks as described in the paper.
To perform general
inference using the pretrained model, please run the following command:
python inference_general.py --input_dir <input_directory> --output_dir <output_directory> --checkpoint <checkpoint_path>
To perform exemplar
inference, please use
python inference_exemplar.py --input_dir <input_directory> --output_dir <output_directory> --checkpoint <checkpoint_path> --exemplar_image <path_to_exemplar_image>
Training
We provide training code for all three tasks.
Download the ACDC, VIPER and Dark Zurich datasets.
Then, run the scripts provided in the `datasets' directory to create symbolic links.
python create_dataset.py --root_acdc <root acdc> --root_viper <root viper> --root_dz <root_dark_zurich>
To start training, modify the data/anchor_dataset.py
file and choose among day2night
, day2twilight
or clear2fog
in the root
option. Finally, start the training with
python train.py --comment "review training" --model fsmunit --dataset anchor
If you don't have a WANDB api key, please run
WANDB_MODE=offline python train.py --comment "review training" --model fsmunit --dataset anchor
Code structure
When extending the code, please consider the following structure. The train.py
file intializes logging utilities and set up callbacks for model saving and debug. The main training logic
is in networks/fsmunit_model.py
. In networks/backbones/fsmunit.py
it's possible to find the architectural components.