Awesome
<br /> <p align="center"> <a href="https://github.com/Joxis/pytorch-hebbian"> <img src="docs/neural_net-local.png" alt="Logo" width="200"> </a> <h3 align="center">PyTorch Hebbian</h3> <p align="center"> A lightweight framework for Hebbian learning based on PyTorch Ignite. Presented at the <a href="https://beyondbackprop.github.io/">Beyond Backpropagation</a> NeurIPS 2020 worskhop. </p> </p> <!-- TABLE OF CONTENTS -->Table of Contents
About the project
A lightweight and flexible framework for Hebbian learning in PyTorch.
Getting started
To get a local copy up and running follow these simple steps.
Prerequisites
Depending on your system (OS/GPU/CUDA support) you may need to manually install a specific PyTorch version. Please see the PyTorch website for more information.
Installation
Clone the repository:
pip install git+https://github.com/Joxis/pytorch-hebbian.git
Usage
Similar to a classic PyTorch pipeline, a model, dataset and data loader are first initiated. Next, the framework requires you to specify a learning rule, optimizer and trainer. You can then start Hebbian learning.
Simple example
# Creating the model
model = models.create_fc1_model([28 ** 2, 400])
# Creating the dataset and data loaders
dataset = datasets.mnist.MNIST(root=config.DATASETS_DIR, download=True,
transform=transforms.ToTensor())
train_loader = DataLoader(dataset, batch_size=1024, shuffle=True)
# Creating the learning rule, optimizer and trainer
learning_rule = KrotovsRule()
optimizer = Local(named_params=model.named_parameters(), lr=0.01)
trainer = HebbianTrainer(model=model, learning_rule=learning_rule,
optimizer=optimizer)
# Running the trainer
trainer.run(train_loader=train_loader, epochs=10)
Visualizations
Various TensorBoard visualizations are supported. Below are some examples.
<img src="docs/visualizers.jpg" alt="Logo" width="600">See examples for more.
Contributing
Any contributions you make are greatly appreciated. For major changes, please open an issue first to discuss what you would like to change.
- Fork the project
- Create your feature branch (
git checkout -b feature/amazing-feature
) - Commit your changes (
git commit -m 'Add some amazing feature'
) - Push to the branch (
git push origin feature/amazing-feature
) - Open a pull request
License
Distributed under the MIT License. See LICENSE for more information.
Reference
Please consider citing this work if it helps your research.
@inproceedings{talloen2020pytorchhebbian,
author = {Jules Talloen and Joni Dambre and Alexander Vandesompele},
location = {Online},
title = {PyTorch-Hebbian: facilitating local learning in a deep learning framework},
year = {2020},
}
Contact
Jules Talloen - <firstname>@<lastname>.eu
Project Link: https://github.com/Joxis/pytorch-hebbian
<!-- ACKNOWLEDGEMENTS -->