Home

Awesome

Noisy Natural Gradient as Variational Inference

PyTorch implementation of Noisy Natural Gradient as Variational Inference.

<img src="./assets/model.png" width="100%">

Requirements

Comments

Experimental comments

Usage

Currently only MNIST dataset are currently supported, and only fully connected layer is implemented.

Options

More details in option_parser.py

Train

$ python train.py --model=FFG --batch_size=100 --lr=1e-3 --dataset=MNIST
$ python train.py --model=MVG --batch_size=100 --lr=1e-2 --dataset=MNIST --n=60000

Visualize

Test

$ python test.py --epoch=20

Training Graphs

1. MNIST

<p> <img src="./assets/ffg_loss_plot.png" width="80%"> </p> <p> <img src="./assets/mvg_loss_plot.png" width="80%"> </p>

Implementation detail

TODOs

Code reference

Visualization code(visualizer.py, utils.py) references to pytorch-CycleGAN-and-pix2pix(https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) by Jun-Yan Zhu

Author

Tony Kim