Awesome

ronn

Randomized Overdrive Neural Networks

What is ronn?

Throughout audio technology history, engineers, circuit designers, and guitarists have searched tirelessly for novel, extreme, and exciting effects as a result of clipping audio signals. Whether it be vacuum tubes (valves), diodes, transistors, op-amps, microchips, or broken speaker drivers doing the distorting, it seems that we have tried them all. But maybe there is at least one area left relatively under-explored, and thats the realm of neural networks.

Now neural networks, have been a round for a bit. They have actually ALREADY been used to model distortion and overdrive effects from guitar amplifier and pedals quite a bit (such as here, here, here, and here). So then you may be asking, "well how is this any different?" And the answer is, ronn doesn't model ANY pre-existing audio circuit, we don't even bother to train anything! Instead we treat the concept of the neural network as a system which can distort a signal, and then we give the user control over that system to explore new effects. Get your hands dirty building neural networks without even touching TensorFlow or PyTorch.

Setup

Download

We supply pre-built VST/AU plugins here.

Once downloaded, unzip and move to:

AU: Macintosh HD/Library/Audio/Plug-Ins/Components
VST3: Macintosh HD/Library/Audio/Plug-Ins/VST3

Currently, we only have macOS builds.

Build

You can also build from source. The following steps are for building on macOS, assuming you have XCode and the Command Line Tools, as well as CMake already installed. The steps should be similar for other platforms.

1. Grab the modules and then change to the root of the plugin project

git submodule init
git submodule update
cd plugin/

2. Download the .zip file containing the libtorch (PyTorch C++ API) source.

wget https://download.pytorch.org/libtorch/cpu/libtorch-macos-1.7.1.zip
unzip libtorch-macos-1.7.1.zip

4. Run the following cmake commands to build the plugin

cmake -Bbuild -GXcode "-DCMAKE_OSX_ARCHITECTURES==i386;x86_64"
cmake --build build --target ronn_AU ronn_VST3 --config Release

5. Move the plugins to the system directory (macOS)

cp -r build/ronn_artefacts/Release/AU/ronn.component "/Volumes/Macintosh HD/Library/Audio/Plug-Ins/Components"
cp -r build/ronn_artefacts/Release/VST3/ronn.vst3 "/Volumes/Macintosh HD/Library/Audio/Plug-Ins/VST3"

Details

The ronn plugin enables users to run their audio directly through randomly weighted temporal convolutional networks (TCNs). Interestingly, using networks that have not been trained can produce a wide range of compelling audio effects simply by adjusting the architectural elements. These effects range from subtle distortion and overdrive, to more extreme drone-like and glitch effects.

Features

• Construct different neural networks and listen to the results in real-time.
• Adjust architectural elements:
  • Depth of the network
  • Convolutional kernel size
  • Width of the convolutional layers
  • Dilation growth factor
  • Activation functions
  • Weight initialization scheme
• Global seed control enables presets and recallability.
• Link the input/output gain to control overall drive level.
• Use depthwise convolutions for less CPU impact.
• Inspect the receptive field of the network and number of parameters.

More to come in the future...

• a GUI that shows the construction of the network as you add layers, change activations, etc.
• a transfer function window that shows the shape of the current activation function being used.
• use FiLM as a further way to adjust the distortion character. For example, create a 2D plane the user will sample from and then use a set of linear layers to project this to gamma and beta coefficients for each layers (using linear adaptors), then the user can control the input to this MLP.
• What if you randomized the weights in each layer every time you accessed that layer?

Citation

@inproceedings{steinmetz2020overdrive,
          title={Randomized Overdrive Neural Networks},
          author={Steinmetz, Christian J. and Reiss, Joshua D.},
          booktitle={4th Workshop on Machine Learning for Creativity and Design at NeurIPS 2020},
          year={2020}}