Awesome
Temporal Generative Adversarial Nets
The new version of TGAN has been published and the code is available: TGANv2.
This repository contains a collection of scripts used in the experiments of Temporal Generative Adversarial Nets with Singular Value Clipping.
Disclaimer: PFN provides no warranty or support for this implementation. Use it at your own risk. See license for details.
Results
Requirements
These scripts require the following python libraries.
- Chainer 2.0.0+
- h5py
- numpy
- pandas
- PIL
- PyYAML
- matplotlib
Note that they also require ffmpeg to produce a video from a set of images.
Usage
Datasets
In order to run our scripts, you need to prepare MovingMNIST and UCF-101 datasets as follows.
MovingMNIST
- Download
mnist_test_seq.npy
from here. - Put it on
path-to-tgans/data/mnist_test_seq.npy
.
UCF-101
There are two ways to create an UCF-101 dataset for this script.
- Transforms all the videos in the UCF-101 dataset to the images.
- Resizes these images to the appropriate resolution, and concatenate
them into as single hdf5 format represented as (time, channel, rows, cols).
In this transformation we used
make_ucf101.py
in this repository. Note that this script also produces a config file that describes videos and these corresponding label information. - puts them on
path-to-tgans/data
.
Another way is to simply download these files; please download them from this url, and put them on the same directory.
Training
TGAN with WGAN and Singular Value Clipping
python train.py --config_path configs/moving_mnist/mnist_wgan_svd_zdim-100_no-beta-all_init-uniform-all.yml --gpu 0
python train.py --config_path configs/ucf101/ucf101_wgan_svd_zdim-100_no-beta.yml --gpu 0
TGAN (WGAN and weight clipping)
python train.py --config_path configs/moving_mnist/mnist_wgan_clip_zdim-100_no-beta-all_init-uniform-all.yml --gpu 0
python train.py --config_path configs/ucf101/ucf101_wgan_clip_zdim-100_no-beta.yml --gpu 0
TGAN (vanilla GAN)
python train.py --config_path configs/ucf101/ucf101_vanilla_zdim-100_no-beta.yml --gpu 0
Quantitative evaluation on UCF101 (2019/08/20)
We have uploaded mean2.npz
on GitHub because there are many inquiries about the mean file in the UCF101.
If you want to perform a quantitative evaluation, please download it from
this url.
Citation
Please cite the paper if you are interested in:
@inproceedings{TGAN2017,
author = {Saito, Masaki and Matsumoto, Eiichi and Saito, Shunta},
title = {Temporal Generative Adversarial Nets with Singular Value Clipping},
booktitle = {ICCV},
year = {2017},
}
License
MIT License. Please see the LICENSE file for details.