Awesome

🌟 Welcome to EthoML/VAME (Variational Animal Motion Encoding), an open-source machine learning tool for behavioral segmentation and analyses.

VAME documentation.

We are a group of behavioral enthusiasts, comprising the original VAME developers Kevin Luxem and Pavol Bauer, behavioral neuroscientists Stephanie R. Miller and Jorge J. Palop, and computer scientists and statisticians Alex Pico, Reuben Thomas, and Katie Ly). Our aim is to provide scalable, unbiased and sensitive approaches for assessing mouse behavior using computer vision and machine learning approaches.

We are focused on the expanding the analytical capabilities of VAME segmentation by providing curated scripts for VAME implementation and tools for data processing, visualization, and statistical analyses.

Recent Improvements to VAME

Curated scripts for VAME implementation
Addition of a new cost function for community dendrogram generation
Addition of a new egocentric alignment method
Addition of hardware instructions for video capturing of mouse behavior.
Addition of mouse behavioral videos for practicing VAME and for benchmarking purposes
Refined output filename structure

[Insert open field videos]

Authors and Code Contributors

VAME was developed by Kevin Luxem and Pavol Bauer (Luxem et. al., 2022). The original VAME repository was deprecated, forked, and is now being maintained here at https://github.com/EthoML/VAME.

The development of VAME is heavily inspired by DeepLabCut. As such, the VAME project management codebase has been adapted from the DeepLabCut codebase. The DeepLabCut 2.0 toolbox is © A. & M.W. Mathis Labs deeplabcut.org, released under LGPL v3.0. The implementation of the VRAE model is partially adapted from the Timeseries clustering repository developed by Tejas Lodaya.

VAME in a Nutshell

VAME is a framework to cluster behavioral signals obtained from pose-estimation tools. It is a PyTorch-based deep learning framework which leverages the power of recurrent neural networks (RNN) to model sequential data. In order to learn the underlying complex data distribution, we use the RNN in a variational autoencoder setting to extract the latent state of the animal in every step of the input time series. The workflow of VAME consists of 5 steps and we explain them in detail here

Installation

To get started we recommend using Anaconda with Python 3.11 or higher. Here, you can create a virtual enviroment to store all the dependencies necessary for VAME. You can also use the VAME.yaml file supplied here, by simply opening the terminal, running git clone https://github.com/LINCellularNeuroscience/VAME.git, then typ cd VAME then run: conda env create -f VAME.yaml).

Go to the locally cloned VAME directory and run python setup.py install in order to install VAME in your active conda environment.
Install the current stable Pytorch release using the OS-dependent instructions from the Pytorch website. Currently, VAME is tested on PyTorch 2.2.2. (Note, if you use the conda file we supply, PyTorch is already installed and you don't need to do this step.)

Getting Started

First, you should make sure that you have a GPU powerful enough to train deep learning networks. In our original 2022 paper, we were using a single Nvidia GTX 1080 Ti GPU to train our network. A hardware guide can be found here. VAME can also be trained in Google Colab or on a HPC cluster. Once you have your computing setup ready, begin using VAME by following the workflow guide.

Once you have VAME installed, you can try VAME out on a set of mouse behavioral videos and .csv files publicly available in the examples folder.

References

Original VAME publication: Identifying Behavioral Structure from Deep Variational Embeddings of Animal Motion <br/> Kingma & Welling: Auto-Encoding Variational Bayes <br/> Pereira & Silveira: Learning Representations from Healthcare Time Series Data for Unsupervised Anomaly Detection

License: GPLv3

See the LICENSE file for the full statement.