Home

Awesome

License FOSSA Status Go Version Release Mentioned in Awesome Go Downloads

CI Go Report Card Maintainability Test Coverage Issues

decompose

Reverse-engineering tool for docker environments.

Takes all network connections from your docker containers and exports them as:

rationale

I was in need for a tool to visualize and inspect big (more than 470 containers) dockerized legacy system without any schemes and having a bare minimum of documentation

analogs

Closest analogs, i can find, that not suit my needs very well:

features

known limitations

installation

usage

decompose [flags]


-cluster string
    json file with clusterization rules, or auto:<similarity> for auto-clustering, similarity is float in (0.0, 1.0] range
-compress
    compress graph
-deep
    process-based introspection
-follow string
    follow only this container by name(s), comma-separated or from @file
-format string
    output format: csv, dot, json, puml, sdsl, stat, tree, yaml (default "json")
-help
    show this help
-load value
    load json stream, can be used multiple times
-local
    skip external hosts
-meta string
    json file with metadata for enrichment
-no-loops
    remove connection loops (node to itself) from output
-no-orphans
    remove orphaned (not connected) nodes from output
-out string
    output: filename or "-" for stdout (default "-")
-proto string
    protocol to scan: tcp,udp,unix or all (default "all")
-silent
    suppress progress messages in stderr
-skip-env string
    environment variables name(s) to skip from output, case-independent, comma-separated
-version
    show version

environment variables:

json stream format

type Item struct {
    Name       string              `json:"name"` // container name
    IsExternal bool                `json:"is_external"` // this host is external
    Image      *string             `json:"image,omitempty"` // docker image (if any)
    Container  struct{
        Cmd    []string          `json:"cmd"`
        Env    []string          `json:"env"`
        Labels map[string]string `json:"labels"`
    } `json:"container"` // container info
    Listen     map[string][]{
        Kind   string            `json:"kind"`  // tcp / udp / unix
        Value  string            `json:"value"`
        Local  bool              `json:"local"` // bound to loopback
    } `json:"listen"` // ports with process names
    Networks   []string            `json:"networks"` // network names
    Tags       []string            `json:"tags"` // tags, if meta presents
    Volumes    []*struct{
        Type string `json:"type"`
        Src  string `json:"src"`
        Dst  string `json:"dst"`
    } `json:"volumes"`           // volumes info, only when '-full'
    Connected  map[string][]string `json:"connected"` // name -> ports slice
}

Single node example with full info and metadata filled:

{
    "name": "foo-1",
    "is_external": false,
    "image": "repo/foo:latest",
    "container": {
        "cmd": [
            "foo",
            "-foo-arg"
        ],
        "env": [
            "FOO=1"
        ],
        "labels": {}
    },
    "listen": {"foo": [
        {"kind": "tcp", "value": "80"}
    ]},
    "networks": ["test-net"],
    "tags": ["some"],
    "volumes": [
        {
            "type": "volume",
            "src": "/var/lib/docker/volumes/foo_1/_data",
            "dst": "/data"
        },
        {
            "type": "bind",
            "src": "/path/to/foo.conf",
            "dst": "/etc/foo.conf"
        }
    ],
    "connected": {
        "bar-1": [
            {"src": "foo", "dst": "[remote]", "port": {"kind": "tcp", "value": "443"}}
        ]
    }
}

See stream.json for simple stream example.

metadata format

To enrich output with detailed descriptions, you can provide additional json file, with metadata i.e.:

{
    "foo": {
        "info": "info for foo",
        "docs": "https://acme.corp/docs/foo",
        "repo": "https://git.acme.corp/foo",
        "tags": ["some"]
    },
    "bar": {
        "info": "info for bar",
        "tags": ["other", "not-foo"]
    }
}

Using this file decompose can enrich output with info and additional tags, for every container that match by name with one of provided keys, like foo-1 or bar1 for this example.

See csv2meta.py for example how to create such json fom csv, and meta.json for metadata sample.

clusterization

with rules

You can join your services into clusters by flexible rules, in dot, structurizr and stat output formats. Example json (order matters):

[
    {
        "name": "cluster-name",
        "weight": 1,
        "if": "<expression>"
    },
    ...
]

Weight can be omitted, if not specified it equals 1.

Where <expression> is expr dsl, having env object node with follownig fields:

type Node struct {
    Listen     PortMatcher  // port matcher with two methods: `HasAny(...string) bool` and `Has(...string) bool`
    Name       string       // container name
    Image      string       // container image
    Cmd        string       // container cmd
    Args       []string     // container args
    Tags       []string     // tags, if meta present
    IsExternal bool         // external flag
}

See: cluster.json for detailed example.

automatic

Decompose provides automatic clusterization option, use -cluster auto:<similarity> to try it out, similarity is a float in (0.0, 1.0] range, representing how much similar ports nodes must have to be placed in same cluster (1.0 - must have all ports equal).

examples

Save full json stream:

sudo decompose > nodes-1.json

Get dot file:

decompose -format dot > connections.dot

Get tcp and udp connections as dot:

decompose -proto tcp,udp -format dot > tcp.dot

Merge graphs from json streams, filter by protocol, skip remote hosts and save as dot:

decompose -local -proto tcp -load "nodes-*.json" -format dot > graph-merged.dot

Load json stream, enrich and save as structurizr dsl:

decompose -load nodes-1.json -meta metadata.json -format sdsl > workspace.dsl

Save auto-clustered graph, with similarity factor 0.6 as structurizr dsl:

decompose -cluster auto:0.6 -format sdsl > workspace.dsl

example result

Scheme taken from redis-cluster:

svg it may be too heavy to display it with browser, use save image as and open it locally

Steps to reproduce:

git clone https://github.com/s0rg/redis-cluster-compose.git
cd redis-cluster-compose
docker compose up -d

then:

decompose -format dot | dot -Tsvg > redis-cluster.svg

license

FOSSA Status