Awesome

Note: We're moving to PyTorch Lightning! Read about the move here. From the end of February, torchbearer will no longer be actively maintained. We'll continue to fix bugs when they are found and ensure that torchbearer runs on new versions of pytorch. However, we won't plan or implement any new functionality (if there's something you'd like to see in a training library, consider creating an issue on PyTorch Lightning).

<a href="http://pytorchbearer.org">Website</a> • <a href="https://torchbearer.readthedocs.io/en/latest/">Docs</a> • <a href="#examples">Examples</a> • <a href="#install">Install</a> • <a href="#citing">Citing</a> • <a href="#related">Related</a>

A PyTorch model fitting library designed for use by researchers (or anyone really) working in deep learning or differentiable programming. Specifically, we aim to dramatically reduce the amount of boilerplate code you need to write without limiting the functionality and openness of PyTorch.

Examples

General

<table> <tr> <td rowspan="3" width="160"> <img src="http://www.pytorchbearer.org/assets/img/examples/quickstart.jpg" width="256"> </td> <td rowspan="3"> Quickstart: Get up and running with torchbearer, training a simple CNN on CIFAR-10. </td> <td align="center" width="80"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/quickstart.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/quickstart.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/quickstart.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/callbacks.jpg" width="256"> </td> <td rowspan="3"> Callbacks: A detailed exploration of callbacks in torchbearer, with some useful visualisations. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/callbacks.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/callbacks.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/callbacks.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/imaging.jpg" width="256"> </td> <td rowspan="3"> Imaging: A detailed exploration of the imaging sub-package in torchbearer, useful for showing visualisations during training. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/imaging.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/imaging.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/imaging.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3" colspan="2"> Serialization: This guide gives an introduction to serializing and restarting training in torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/serialization.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/serialization.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/serialization.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3" colspan="2"> History and Replay: This guide gives an introduction to the history returned by a trial and the ability to replay training. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/history.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/history.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/history.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3" colspan="2"> Custom Data Loaders: This guide gives an introduction on how to run custom data loaders in torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/custom_loaders.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/custom_loaders.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/custom_loaders.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3" colspan="2"> Data Parallel: This guide gives an introduction to using torchbearer with DataParrallel. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/data_parallel.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/data_parallel.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/data_parallel.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/livelossplot.jpg" width="256"> </td> <td rowspan="3"> LiveLossPlot: A demonstration of the LiveLossPlot callback included in torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/livelossplot.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/livelossplot.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/livelossplot.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/pycm.jpg" width="256"> </td> <td rowspan="3"> PyCM: A demonstration of the PyCM callback included in torchbearer for generating confusion matrices. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/pycm.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/pycm.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/pycm.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3" colspan="2"> NVIDIA Apex: A guide showing how to perform half and mixed precision training in torchbearer with NVIDIA Apex. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/apex_torchbearer.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/apex_torchbearer.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/apex_torchbearer.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> </table>

Deep Learning

<table> <tr> <td rowspan="3" width="160"> <img src="http://www.pytorchbearer.org/assets/img/examples/vae.jpg" width="256"> </td> <td rowspan="3"> Training a VAE: A demonstration of how to train (add do a simple visualisation of) a Variational Auto-Encoder (VAE) on MNIST with torchbearer. </td> <td align="center" width="80"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/vae.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/vae.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/vae.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/gan.jpg" width="256"> </td> <td rowspan="3"> Training a GAN: A demonstration of how to train (add do a simple visualisation of) a Generative Adversarial Network (GAN) on MNIST with torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/gan.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/gan.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/gan.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/adversarial.jpg" width="256"> </td> <td rowspan="3"> Generating Adversarial Examples: A demonstration of how to perform a simple adversarial attack with torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/adversarial.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/adversarial.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/adversarial.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/transfer.jpg" width="256"> </td> <td rowspan="3"> Transfer Learning with Torchbearer: A demonstration of how to perform transfer learning on STL10 with torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/transfer_learning.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/transfer_learning.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/transfer_learning.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/regulariser.jpg" width="256"> </td> <td rowspan="3"> Regularisers in Torchbearer: A demonstration of how to use all of the built-in regularisers in torchbearer (Mixup, CutOut, CutMix, Random Erase, Label Smoothing and Sample Pairing). </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/regularisers.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/regularisers.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/regularisers.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3" colspan="2"> Manifold Mixup: A demonstration of how to use the Manifold Mixup callback in Torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/manifold_mixup.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/manifold_mixup.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/manifold_mixup.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/cam.jpg" width="256"> </td> <td rowspan="3"> Class Appearance Model: A demonstration of the Class Appearance Model (CAM) callback in torchbearer. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/cam.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/cam.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/cam.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> </table>

Differentiable Programming

<table> <tr> <td rowspan="3" width="160"> <img src="http://www.pytorchbearer.org/assets/img/examples/optimisers.jpg" width="256"> </td> <td rowspan="3"> Optimising Functions: An example (and some fun visualisations) showing how torchbearer can be used for the purpose of optimising functions with respect to their parameters using gradient descent. </td> <td align="center" width="80"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/basic_opt.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/basic_opt.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/basic_opt.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/svm.jpg" width="256"> </td> <td rowspan="3"> Linear SVM: Train a linear support vector machine (SVM) using torchbearer, with an interactive visualisation! </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/svm_linear.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/svm_linear.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/svm_linear.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> <tr> <td rowspan="3"> <img src="http://www.pytorchbearer.org/assets/img/examples/amsgrad.jpg" width="256"> </td> <td rowspan="3"> Breaking Adam: The Adam optimiser doesn't always converge, in this example we reimplement some of the function optimisations from the AMSGrad paper showing this empirically. </td> <td align="center"> <a href="https://nbviewer.jupyter.org/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/amsgrad.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/nbviewer_logo.svg" height="34"> </a> </td> </tr> <tr> <td align="center"> <a href="https://github.com/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/amsgrad.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/github_logo.png" height="32"> </a> </td> </tr> <tr> <td align="center"> <a href="https://colab.research.google.com/github/pytorchbearer/torchbearer/blob/master/docs/_static/notebooks/amsgrad.ipynb"> <img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28"> </a> </td> </tr> </table>

Install

The easiest way to install torchbearer is with pip:

pip install torchbearer

Alternatively, build from source with:

pip install git+https://github.com/pytorchbearer/torchbearer

Citing Torchbearer

If you find that torchbearer is useful to your research then please consider citing our preprint: Torchbearer: A Model Fitting Library for PyTorch, with the following BibTeX entry:

@article{torchbearer2018,
  author = {Ethan Harris and Matthew Painter and Jonathon Hare},
  title = {Torchbearer: A Model Fitting Library for PyTorch},
  journal  = {arXiv preprint arXiv:1809.03363},
  year = {2018}
}

Torchbearer isn't the only library for training PyTorch models. Here are a few others that might better suit your needs (this is by no means a complete list, see the awesome pytorch list or the incredible pytorch for more):

skorch, model wrapper that enables use with scikit-learn - crossval etc. can be very useful
PyToune, simple Keras style API
ignite, advanced model training from the makers of PyTorch, can need a lot of code for advanced functions (e.g. Tensorboard)
TorchNetTwo (TNT), can be complex to use but well established, somewhat replaced by ignite
Inferno, training utilities and convenience classes for PyTorch
Pytorch Lightning, lightweight wrapper on top of PyTorch with advanced multi-gpu and cluster support
Pywick, high-level training framework, based on torchsample, support for various segmentation models