Awesome
Empirical Experiments in "Feature Learning in Infinite-width Neural Networks"
This repo contains code to replicate our experiments (Word2Vec, MAML) in our paper
Feature Learning in Infinite-Width Neural Networks<br> Greg Yang, Edward Hu
In short, the code here will allow you to train feature learning infinite-width neural networks on Word2Vec and on Omniglot (via MAML).
Our results on Word2Vec:
<img src="word2vec.png" alt="Word2Vec Results" width="325"/>Our Results on MAML:
<img src="MAML.png" alt="MAML Results" width="200"/>Please see the README in individual folders for more details.
This is the 4th paper in the Tensor Programs series ([0][1][2][3]). Also see here for code in previous papers for calculating the GP and NTK limits of wide neural networks.