Awesome

README

Implementation of OpenAI paper "An Empirical Model of Large-Batch Training" for Fastai V2.

The code is based on the batch size finder implementation for Fastai V1 by DanyWind (repo V1 / blog / discussion).

This implementation differs on:

1. It implements exactly the original article and not an aproximation (by default).
2. Fixes a couple of bugs in noise and scale values. However, they didn't affect on Simple Noise Scale value.

However, you could use the DanyWind aproximation by settting simulate_multi_gpus to False. DanyWind aproximation is faster but numerically more inestable and finds a Simple Noise Scale smaller than the original Simple Noise Scale.

It's tested with fastai 2.1. It should work with fastai>=2.0

TODO:

• Port description improvements from fastai2 PR.