Awesome
<h1 align="center"> <img src="docs/logo.svg" alt="tork" width="300px"> <br> </h1> <p align="center"> <a href="https://opensource.org/licenses/MIT"> <img src="https://img.shields.io/badge/license-MIT-_red.svg"> </a> <a href="https://goreportcard.com/report/github.com/runabol/tork"> <img src="https://goreportcard.com/badge/github.com/runabol/tork"> </a> <a href="https://github.com/runabol/tork/releases"> <img src="https://img.shields.io/github/release/runabol/tork"> </a> <img src="https://github.com/runabol/tork/workflows/ci/badge.svg"> </p> <p align="center"> <a href="#features">Features</a> • <a href="https://www.tork.run/installation">Installation</a> • <a href="https://www.tork.run">Documentation</a> • <a href="#quick-start">Quick Start</a> • <a href="https://www.tork.run/rest">REST API</a> • <a href="https://www.tork.run/web-ui">Web UI</a> </p>Tork is a highly-scalable, general-purpose workflow engine.
Features:
<h1 align="center"> <img src="docs/cli_v3.jpg" alt="tork" width="700px"> <br> </h1>- REST API
- Highly extensible
- Horizontally scalable
- Task isolation - tasks are executed within a container to provide isolation, idempotency, and in order to enforce resource limits
- Automatic recovery of tasks in the event of a worker crash
- Supports both stand-alone and distributed setup
- Retry failed tasks
- Middleware
- Pre/Post tasks
- No single point of failure
- Task timeout
- Full-text search
- Expression Language
- Conditional Tasks
- Parallel Tasks
- For-Each Task
- Subjob Task
- Task Priority
- Sandbox Mode
- Secrets
- Web UI
Documentation
See tork.run for the full documentation.
Quick Start
-
Ensure you have Docker with API Version >= 1.42 (use
docker version | grep API
to check). -
Download the binary for your system from the releases page.
Hello World
Start in standalone
mode:
./tork run standalone
Submit a job in another terminal:
# hello.yaml
---
name: hello job
tasks:
- name: say hello
image: ubuntu:mantic #docker image
run: |
echo -n hello world
- name: say goodbye
image: ubuntu:mantic
run: |
echo -n bye world
JOB_ID=$(curl \
-s \
-X POST \
--data-binary @hello.yaml \
-H "Content-type: text/yaml" \
http://localhost:8000/jobs | jq -r .id)
Query for the status of the job:
curl -s http://localhost:8000/jobs/$JOB_ID | jq .
{
"id": "ed0dba93d262492b8cf26e6c1c4f1c98",
"state": "COMPLETED",
...
"execution": [
{
...
"state": "COMPLETED",
}
],
}
A slightly more interesting example
The following job:
- Downloads a remote video file using a
pre
task to a shared/tmp
volume. - Converts the first 5 seconds of the downloaded video using
ffmpeg
. - Uploads the converted video to a destination using a
post
task.
# convert.yaml
---
name: convert a video
inputs:
source: https://upload.wikimedia.org/wikipedia/commons/1/18/Big_Buck_Bunny_Trailer_1080p.ogv
tasks:
- name: convert the first 5 seconds of a video
image: jrottenberg/ffmpeg:3.4-alpine
run: |
ffmpeg -i /tmp/input.ogv -t 5 /tmp/output.mp4
mounts:
- type: volume
target: /tmp
pre:
- name: download the remote file
image: alpine:3.18.3
env:
SOURCE_URL: "{{ inputs.source }}"
run: |
wget \
$SOURCE_URL \
-O /tmp/input.ogv
post:
- name: upload the converted file
image: alpine:3.18.3
run: |
wget \
--post-file=/tmp/output.mp4 \
https://devnull-as-a-service.com/dev/null
Submit the job in another terminal:
JOB_ID=$(curl \
-s \
-X POST \
--data-binary @convert.yaml \
-H "Content-type: text/yaml" \
http://localhost:8000/jobs | jq -r .id)
More examples
Check out the examples folder.
REST API
See the REST API documentation.
Swagger Docs
Make sure you have CORS configured in your config file:
[middleware.web.cors]
enabled = true
Start Tork in standalone
or coordinator
mode.
go run cmd/main.go run standalone
Serve the Swagger Docs
docker compose up -d swagger
Visit http://localhost:9000
Web UI
Tork Web is a web based tool for interacting with Tork.
License
Copyright (c) 2023-present Arik Cohen. Tork is free and open-source software licensed under the MIT License.