Home

Awesome

afroisalreadyinu

PyPI version

<img src="https://github.com/afroisalreadyinu/miniboss/raw/main/logo.png" width="200">

miniboss

miniboss is a Python application for locally running a collection of interdependent docker services, individually rebuilding and restarting them, and managing application state with lifecycle hooks. Services definitions can be written in Python, allowing the use of programming logic instead of markup.

Why not docker-compose?

First and foremost, good old Python instead of YAML. docker-compose is in the school of yaml-as-service-description, which means that going beyond a static description of a service set necessitates templates, or some kind of scripting. One could just as well use a full-blown programming language, while trying to keep simple things simple. Another thing sorely missing in docker-compose is lifecycle hooks, i.e. a mechanism whereby scripts can be executed when the state of a container changes. Lifecycle hooks have been requested multiple times, but were not deemed to be in the domain of docker-compose.

Installation

miniboss is on PyPi; you can install it with the following:

pip install miniboss

Usage

Here is a very simple service specification:

#! /usr/bin/env python3
import miniboss

miniboss.group_name('readme-demo')

class Database(miniboss.Service):
    name = "appdb"
    image = "postgres:10.6"
    env = {"POSTGRES_PASSWORD": "dbpwd",
           "POSTGRES_USER": "dbuser",
           "POSTGRES_DB": "appdb" }
    ports = {5432: 5433}

class Application(miniboss.Service):
    name = "python-todo"
    image = "afroisalreadyin/python-todo:0.0.1"
    env = {"DB_URI": "postgresql://dbuser:dbpwd@appdb:5432/appdb"}
    dependencies = ["appdb"]
    ports = {8080: 8080}
    stop_signal = "SIGINT"

if __name__ == "__main__":
    miniboss.cli()

The first use of miniboss is in the call to miniboss.group_name, which specifies a name for this group of services. If you don't set it, sluggified form of the directory name will be used. Group name is used to identify the services and the network defined in a miniboss file. Setting it manually to a non-default value will allow miniboss to manage multiple collections in the same directory.

A service is defined by subclassing miniboss.Service and overriding, in the minimal case, the fields image and name. The env field specifies the environment variables. As in the case of the appdb service, you can use ordinary variables anywhere Python accepts them. The other available fields are explained in the section Service definition fields. In the above example, we are creating two services: The application service python-todo (a simple Flask todo application defined in the sample-apps directory) depends on appdb (a Postgresql container), specified through the dependencies field. As in docker-compose, this means that python-todo will get started after appdb reaches running status.

The miniboss.cli function is the main entry point; you need to call it in the main section of your script. Let's run the script above without arguments, which leads to the following output:

Usage: miniboss-main.py [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  start
  stop

We can start our small collection of services by running ./miniboss-main.py start. After spitting out some logging text, you will see that starting the containers failed, with the python-todo service throwing an error that it cannot reach the database. The reason for this error is that the Postgresql process has started, but is still initializing, and does not accept connections yet. The standard way of dealing with this issue is to include backoff code in your application that checks on the database port regularly, until the connection is accepted. miniboss offers an alternative with lifecycle events. For the time being, you can simply rerun ./miniboss-main.py start, which will restart only the python-todo service, as the other one is already running. You should be able to navigate to http://localhost:8080 and view the todo app page.

You can also exclude services from the list of services to be started with the --exclude argument; ./miniboss-main.py start --exclude python-todo will start only appdb. If you exclude a service that is depended on by another, you will get an error. If a service fails to start (i.e. container cannot be started or the lifecycle events fail), it and all the other services that depend on it are registered as failed.

Stopping services

Once you are done working with a collection, you can stop the running services with miniboss-main.py stop. This will stop the services in the reverse order of dependency, i.e. first python-todo and then appdb. Exclusion is possible also when stopping services with the same --exclude argument. Running ./miniboss-main.py stop --exclude appdb will stop only the python-todo service. If you exclude a service whose dependency will be stopped, you will get an error. If, in addition to stopping the service containers, you want to remove them, include the option --remove. If you don't remove the containers, miniboss will restart the existing containers (modulo changes in service definition) instead of creating new ones the next time it's called with start. This behavior can be modified with the always_start_new field; see the details in Service definition fields.

Reloading a service

miniboss also allows you to reload a specific service by building a new container image from a directory. You need to provide the path to the directory in which the Dockerfile and build context of a service resides in order to use this feature. You can also provide an alternative Dockerfile name. Here is an example:

class Application(miniboss.Service):
    name = "python-todo"
    image = "afroisalreadyin/python-todo:0.0.1"
    env = {"DB_URI": "postgresql://dbuser:dbpwd@appdb:5432/appdb"}
    dependencies = ["appdb"]
    ports = {8080: 8080}
    build_from = "python-todo/"
    dockerfile = "Dockerfile"

The build_from option has to be a path relative to the main miniboss file. With such a service configuration, you can run ./miniboss-main.py reload python-todo, which will cause miniboss to build the container image, stop the running service container, and restart the new image. Since the context generated at start is saved in a file, any context values used in the service definition are available to the new container.

Lifecycle events

One of the differentiating feature of miniboss is lifecycle events, which are hooks that can be customized to execute code at certain points in a service's or the whole collection's lifecycle.

Per-service events

For per-service events, miniboss.Service has three methods that can be overridden in order to correctly change states and execute actions on the container:

These methods are noop by default. A service is not registered as properly started before lifecycle methods are executed successfully; only then are the dependent services started.

The ping method is particularly useful if you want to avoid the situation described above, where a container starts, but the main process has not completed initializing before any dependent services start. Here is an example for how one would ping the appdb service to make sure the PostgreSQL database is accepting connections:

import psycopg2

class Database(miniboss.Service):
    # fields same as above

    def ping(self):
        try:
            connection = psycopg2.connect("postgresql://dbuser:dbpwd@localhost:5433/appdb")
            cur = connection.cursor()
            cur.execute('SELECT 1')
        except psycopg2.OperationalError:
            return False
        else:
            return True

One thing to pay attention to is that, in the call to psycopg2.connect, we are using localhost:5433 as host and port, whereas the python-todo environment variable DBURI has appdb:5433 instead. This is because the ping method is executed on the host computer. The next section explains the details.

Collection events

It is possible to hook into collection change commands using the following hooks. You can call them on the base miniboss module and set a hook by passing it in as the sole argument, e.g. as follows:

import miniboss

def print_services(service_list):
    print("Started ", ' '.join(service_list))

miniboss.on_start_services(print_services)

Ports and hosts

miniboss starts services on an isolated bridge network, mapping no ports by default. The name of this service can be specified with the --network-name argument when starting a group. If it's not specified, the name will be generated from the group name by prefixing it with miniboss-. On the collection network, services can be contacted under the service name as hostname, on the ports they are listening on. The appdb Postgresql service above, for example, can be contacted on the port 5432, the default port on which Postgresql listens. This is the reason the host part of the DB_URI environment variable on the python-todo service is appdb:5432. If you want to reach appdb on the port 5433 from the host system, which would be necessary to implement the ping method as above, you need to make this mapping explicit with the ports field of the service definition. This field accepts a dictionary of integer keys and values. The key is the service container port, and the value is the host port. In the case of appdb, the Postgresql port of the container is mapped to port 5433 on the local machine, in order not to collide with any local Postgresql instances. With this configuration, the appdb database can be accessed at localhost:5433.

The global context

The object miniboss.Context, derived from the standard dict class, can be used to store values that are accessible to other service definitions, especially in the env field. For example, if you create a user in the post_start method of a service, and would like to make the ID of this user available to a dependent service, you can set it on the context with Context['user_id'] = user.id. In the definition of the second service, you can refer to this value in a field with the standard Python keyword formatting syntax, as in the following:

class DependantService(miniboss.Service):
    # other fields
	env = {'USER_ID': '{user_id}'}

You can of course also programmatically access it as Context['user_id'] once a value has been set.

When a service collection is started, the generated context is saved in the file .miniboss-context, in order to be used when the same containers are restarted or a specific service is reloaded.

Service definition fields

Release notes

0.3.0

0.4.0

0.4.1

0.4.2

0.4.3

0.4.3

Todos