Home

Awesome

Grokking

An implementation of the OpenAI 'Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets' paper in PyTorch.

<img src="figures/Figure_1_left_accuracy.png" height="300">

Installation

Usage

The project uses Weights & Biases to keep track of experiments. Run wandb login to use the online dashboard, or wandb offline to store the data on your local machine.

References

Code:

Paper: