Awesome

pipen-board

Visualize configuration and running of pipen pipelines on the web.

Installation

pip install pipen-board

Usage

$ pipen board --help
Usage: pipen board [options] <pipeline> -- [pipeline options]

Configure and run pipen pipelines from the web

Required Arguments:
  pipeline              The pipeline and the CLI arguments to run the pipeline. For the
                        pipeline either `/path/to/pipeline.py:<pipeline>` or
                        `<module.submodule>:<pipeline>` `<pipeline>` must be an instance of
                        `Pipen` and running the pipeline should be called under `__name__ ==
                        '__main__'.

Optional Arguments:
  -h, --help            show help message and exit
  -p PORT, --port PORT  Port to serve the UI wizard [default: 18521]
  -a FILE, --additional FILE
                        Additional arguments for the pipeline, in YAML, INI, JSON or TOML
                        format. Can have sections `ADDITIONAL_OPTIONS` and `RUNNING_OPTIONS`.
                        It can also have other sections and items to override the
                        configurations generated from the pipeline. If the pipeline is
                        provided as a python script, such as
                        `/path/to/pipeline.py:<pipeline>`, and `<pipeline>` runs under
                        `__name__ == '__main__'`, the additional file can also be specified
                        as `auto` to generate a `RUNNING OPTIONS/Local` section to run the
                        pipeline locally.
  --loglevel {auto,debug,info,warning,error,critical}
                        The logging level. If `auto`, it will be set to `debug` if `--dev` is
                        set, otherwise `info`. [default: auto]
  --dev                 Run the pipeline in development/debug mode. This will reload the
                        server when changes are made to this package and reload the pipeline
                        when page reloads for new configurations. Page cache is also disabled
                        in this mode.
  -w WORKDIR, --workdir WORKDIR
                        The working directory of the pipeline. [default: .pipen]
  -s SCHEMA_DIR, --schema-dir SCHEMA_DIR
                        The directory to store the configuration schemas. [default: ~/.pipen-
                        board]

Describing arguments in docstring

Docstring schema

class ProcessOrProcessGroup:
    """Short summary

    Long description
    Long description

    Args:
        arg1 (<metadata>): description
            - subarg1 (<metadata>): description
            - subarg2 (<metadata>): description
        arg2 (<metadata>): description

    <Other Sections>:
        <content>
    """

The metadata can have multiple attributes, separated by semicolon (;). For example:

arg1 (action=ns;required): description

Marks

You can mark a process using pipen.utils.mark(<mark>=<value>) as a decorator to decorate a process. For example:

from pipen import Proc
from pipen.utils import mark

@mark(board_config_no_input=True)
class MyProc(Proc):
    pass

Available marks:

• board_config_no_input: Whether to show the input section for the process in configuation page. Only affects the start processes. Default to False.
• board_config_hidden: Whether to hide the process options in the configuration page. Note that the process is still visible in the process list. Default to False.

Metadata for arguments

• argx*: An argument parser for Python, compatible with argparse.
• auto*: Automatically infer the type from a string value.
  • Any of True, TRUE, true, False, FALSE, false will be inferred as a bool value.
  • Any of None, NONE, none, null, NULL will be inferred as None.
  • Any integers will be inferred as int.
  • Any floats will be inferred as float.
  • Try to parse the value as JSON. If succeed, the value will be inferred as json.
  • Otherwise, the value will be inferred as str.

Types of options in the UI

The type of an option in the UI is determined by the btype, action or type metadata. If neither is specified, a PlainText will be used.

• BoolOption: Shown as a switch
• TextOption: Shown as a textarea (allow multiple lines)
• ChoiceOption: Shown as a dropdown list (subarg1 and subarg2 in the example above are used as the choices)
• MChoiceOption: Shown as a multiple choice list (subarg1 and subarg2 in the example above are used as the choices)
• JsonOption: Shown as a textarea, but the value will be validated and parsed as JSON
• ArrayOption: Shown as a tag input. Items can be added or removed.
• AutoOption: Shown as a 1-row textarea, and the value will be parsed automatically
• PlainText: Shown as a plain text. No validation or parsing will be performed.
• MoreLikeOption: Show as a box with buttons to add or remove sub-options. It's usally used together with ns type. If there is a sub-option under the option in the docstring wrapped by <...>, it indicates that we may have more sub-options.