Home

Awesome

dynamic-yaml

Dynamic YAML is a couple of classes and functions that add extra functionality to YAML that turns it into a great configuration language for Python. If you prefer JSON, then see dynamic-json.

YAML already provides:

In addition, the PyYAML parser provides:

Finally, the classes introduced by Dynamic YAML enable:

Dynamic PyYAML requires PyYAML (https://bitbucket.org/xi/pyyaml).

Usage

The key feature that was introduced is the ability for a string scalar to reference other parts of the configuration tree. This is done using the Python string formatting syntax. The characters '{' and '}' enclose a reference to another entry in the configuration structure. The reference takes the form key1.key2 where key1 maps to another mapping object and can be found in the root mapping, and key2 can be found in key1's mapping object. Multiple levels of nesting can be used (eg. key1.key2.key3 etc...). If you need brace literals, they can be escaped by doubling them up, as described by the Python format string documentation.

An example yaml configuration:

project_name: hello-world
dirs:
    home: /home/user
    venv: "{dirs.home}/venvs/{project_name}"
    bin: "{dirs.venv}/bin"
    data: "{dirs.venv}/data"
    errors: "{dirs.data}/errors"
    sessions: "{dirs.data}/sessions"
    databases: "{dirs.data}/databases"
    output: "{dirs.data}/output-{parameters.parameter1}-{parameters.parameter2}"
exes:
    main: "{dirs.bin}/main"
    test: tests
parameters:
    parameter1: a
    parameter2: b

Reading in a yaml file:

import dynamic_yaml

with open('/path/to/file.yaml') as fileobj:
    cfg = dynamic_yaml.load(fileobj)
    assert cfg.dirs.venv == '/home/user/venvs/hello-world'
    assert cfg.dirs.output == '/home/user/venvs/hello-world/data/output-a-b'

As the variables are dynamically resolved, it is also possible to combine this with argparse:

import dynamic_yaml

from argparse import ArgumentParser

with open('/path/to/file.yaml') as fileobj:
    cfg = dynamic_yaml.load(fileobj)
    parser = ArgumentParser()
    parser.add_argument('--parameter1')
    parser.add_argument('--parameter2')
    parser.parse_args('--parameter1 c --parameter2 d'.split(), namespace=cfg.parameters)
    assert cfg.dirs.output == '/home/user/venvs/hello-world/data/output-c-d'

Writing yaml will resolve all references:

import dynamic_yaml
import yaml

with open('/path/to/file.yaml') as fileobj:
    cfg = dynamic_yaml.load(fileobj)
    assert yaml.safe_load(dynamic_yaml.dump(cfg)) == yaml.safe_load('''
project_name: hello-world
dirs:
    home: /home/user
    venv: /home/user/venvs/hello-world
    bin: /home/user/venvs/hello-world/bin
    data: /home/user/venvs/hello-world/data
    errors: /home/user/venvs/hello-world/data/errors
    sessions: /home/user/venvs/hello-world/data/sessions
    databases: /home/user/venvs/hello-world/data/databases
    output: /home/user/venvs/hello-world/data/output-a-b}
exes:
    main: /home/user/venvs/hello-world/bin/main
    test: tests
parameters:
  - 0.5
  - 0.1
''')

Installation

To install, simply run:

pip install dynamic-yaml

Restrictions

Due to the short amount of time I was willing to spend on working upon this, there are a few restrictions that I could not overcome.