Awesome
VampNet
This repository contains recipes for training generative music models on top of the Descript Audio Codec.
Listen to the audio examples here!
HuggingFace Space coming soon!
Setting up
install
git clone https://github.com/hugofloresgarcia/vampnet2.git
pip install -e ./vampnet2
A note on argbind
This repository relies on argbind to manage CLIs and config files.
Config files are stored in the conf/
folder.
Getting the Pretrained Models
Download the pretrained models from this link. Then, extract the models to the models/
folder.
Usage
First, you'll want to set up your environment
source ./env/env.sh
Launching the Gradio Interface
TODO: add instructions for the new model when we have it?
python demo.py --args.load conf/interface/spotdl.yml --Interface.device cuda
Training a model
Staging a Run
Staging a run makes a copy of all the git-tracked files in the codebase and saves them to a folder for reproducibility. You can then run the training script from the staged folder.
stage --name my_run --run_dir /path/to/staging/folder
now, run the training script.
python scripts/exp/train.py --args.load conf/vampnet.yml --save_path /path/to/checkpoint
see python scripts/exp/train.py -h
for more options.
Evaluating a model
To evaluate a model, you'll need to generate samples from the test set, then compute metrics on those samples.
Generating Samples
use the scripts/utils/vamp_folder.py
script to generate samples from a folder of audio files.
TODO: update interface ckpt
python scripts/utils/vamp_folder.py --args.load conf/interface/spotdl.yml --Interface.device cuda --exp_type sampling-steps
see python scripts/utils/vamp_folder.py -h
for more options.
Fine-tuning
TODO: will the new model need coarse and c2f? should update this script
To fine-tune a model, use the script in scripts/exp/fine_tune.py
to generate 3 configuration files: c2f.yml
, coarse.yml
, and interface.yml
.
The first two are used to fine-tune the coarse and fine models, respectively. The last one is used to fine-tune the interface.
python scripts/exp/fine_tune.py "/path/to/audio1.mp3 /path/to/audio2/ /path/to/audio3.wav" <fine_tune_name>
This will create a folder under conf/<fine_tune_name>/
with the 3 configuration files.
The save_paths will be set to runs/<fine_tune_name>/coarse
and runs/<fine_tune_name>/c2f
.
launch the coarse job:
python scripts/exp/train.py --args.load conf/<fine_tune_name>/coarse.yml
this will save the coarse model to runs/<fine_tune_name>/coarse/ckpt/best/
.
launch the c2f job:
python scripts/exp/train.py --args.load conf/<fine_tune_name>/c2f.yml
launch the interface:
python demo.py --args.load conf/generated/<fine_tune_name>/interface.yml
Launching the Gradio Interface
python demo.py --args.load conf/interface/spotdl.yml --Interface.device cuda