Home

Awesome

doodad

A library for launching python programs on different machines. Currently supports running locally and over EC2 and SSH (via Docker) with minimal (if any) modification to your existing program.

EC2 code is based on rllab's code.

Setup

export PYTHONPATH=$PYTHONPATH:/path/to/this/repo
pip install -r requirements.txt
python scripts/ec2_setup.py

Example

See ec2_launch_test.py for an example on how to run scripts on EC2, over SSH, or locally.

Tutorial

See the wiki

Changelog

17 March 2020 (v0.2.4)

04 March 2020 (v0.2.3)

26 February 2020 (v0.2.2)

TODOs

doodad.easy_launch

The doodad.easy_launch module is intended to make it very easy to launch experiments. Usage:

from doodad.easy_launch.python_function import run_experiment


def function(doodad_config, variant):
    print("The learning rate is", variant['learning_rate'])
    print("You are", variant['parameter'])
    print("Save to", doodad_config.base_log_dir)
    # save outputs (e.g. logs, parameter snapshots, etc.) to
    # doodad_config.base_log_dir

if __name__ == "__main__":
    variant = dict(
        learning_rate=1e-4,
        parameter='awesome',
    )
    run_experiment(
        function,
        exp_name='ec2-test-doodad-easy-launch',
        mode='ec2',
        variant=variant,
    )

To use it you well need to create a private version of the configuration file:

cp doodad/easy_launch/config.py doodad/easy_launch/config_private.py

See the README for more details.