Awesome
Contents
- PanoGAN
- Installation
- Dataset Preparation
- Generating Images Using Pretrained Model
- Train and Test New Models
- Evaluation
- Acknowledgments
- Related Projects
- Contributions
PanoGAN
Cross-View Panorama Image Synthesis
Songsong Wu*, Hao Tang*, Xiaoyuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Sebe, and Yan Yan.<br>
(* equal contributions) <br>
IEEE Transactions on Multimedia, 2022. <br>
Framework
<img src='./imgs/panogan_architecture.jpg' width=800> <br>Generator
<img src='./imgs/Generator_Feedback.jpg' width=800> <br>Discriminator
<img src='./imgs/Discrimination.jpg' width=800> <br>Comparison Results on the CVUSA Dataset
<img src='./imgs/cvusa_sota_comparison.jpg' width=800> <br>Comparison Results on the OP Dataset
<img src='./imgs/op_sota_comparison.jpg' width=800> <br>License
All rights reserved. Licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International)
The code is released for academic research use only. For commercial use, please contact sswuai@gmail.com.
Installation
Clone this repo.
git clone https://github.com/sswuai/PanoGAN.git
cd PanoGAN/
This code requires PyTorch 1.0.0 and python 3.6.9+. Please install dependencies by
pip install -r requirements.txt (for pip users)
or
./scripts/conda_deps.sh (for Conda users)
To reproduce the results reported in the paper, you need to run experiments with at least 1 NVIDIA 2080Ti GPU.
Dataset Preparation
Please download the dataset CVUSA from google drive: Training sample set for ablation study, Test sample set.
Please download the dataset OP from google drive: Training sample set, Test sample set.
Generating Images Using Pretrained Model
PanoGAN model tranined on CVUSA dataset with all training samples
- Download the model trained on CVUSA dataset from PanoGAN model on CVUSA.
- Unzip the file to checkpoints/
- Change several parameters in
test_panogan_cvusa.sh
. - Run
sh test_panogan_cvusa.sh
for testing.
PanoGAN model tranined on OP dataset
- Download the model trained on OP dataset from PanoGAN model on OP.
- Unzip the file to checkpoints/
- Change several parameters in
test_panogan_op.sh
. - Run
sh test_panogan_op.sh
for testing.
Train and Test New Models
CVUSA dataset
- Go to the scripts folder.
- Change several parameters in
train_panogan_cvusa.sh
(ortrain_panogan_cvusa_ablation_study.sh
). - Run
sh train_panogan_cvusa.sh
(orsh train_panogan_cvusa_ablation_study.sh
) for training. - Change several parameters in
test_panogan_cvusa.sh
. - Run
sh test_panogan_cvusa.sh
for testing.
OP dataset
- Go to the scripts folder.
- Change several parameters in
train_panogan_op.sh
. - Run
sh train_panogan_op.sh
for training. - Change several parameters in
test_panogan_op.sh
. - Run
sh test_panogan_op.sh
for testing.
Evaluation
We adopt Prediction Accuracy, Inception Score, KL Score, SSIM, PSNR, and SD for evaluation of all the involved methods. Please refer to Evaluation for more details.
Acknowledgments
This source code is inspired by both Pix2Pix and SelectionGAN.
Related Projects
CrossMLP | SelectionGAN | LGGAN | DAGAN | DPGAN | Guided-I2I-Translation-Papers
Citation
@article{wu2022cross,
title={Cross-View Panorama Image Synthesis},
author={Wu, Songsong and Tang, Hao and Jing, Xiao-Yuan and Zhao, Haifeng and Qian, Jianjun and Sebe, Nicu and Yan, Yan},
journal={IEEE Transactions on Multimedia (TMM)},
year={2022}
}
Contributions
If you have any questions/comments/bug reports, feel free to open a github issue or pull a request or e-mail to the author Songsong Wu (sswuai@gmail.com).
Collaborations
I'm always interested in meeting new people and hearing about potential collaborations. If you'd like to work together or get in contact with me, please email bjdxtanghao@gmail.com. Some of our projects are listed here.