Home

Awesome

Dask docker images

Docker build

ImageDescriptionVersions
ghcr.io/dask/daskBase image to use for Dask scheduler and workers <br /> ![][daskdev-dask-py39-release] <br />
ghcr.io/dask/dask-notebookJupyter Notebook image to use as helper entrypoint <br /> ![][daskdev-dask-notebook-py39-release] <br />

Example

An example docker-compose.yml file is included for starting a small cluster.

docker-compose up

Open the notebook using the URL that is printed by the output so it has the token.

On a new notebook run:

from dask.distributed import Client
client = Client()  # The address is automatically set by the DASK_SCHEDULER_ADDRESS environment variable
client.ncores()

It should output something like this:

{'tcp://172.23.0.4:41269': 4}

Environment Variables

The following environment variables are supported for both the base and notebook images:

docker run -e EXTRA_CONDA_PACKAGES="-c dask/label/dev dask" daskdev/dask:latest

The notebook image supports the following additional environment variables:

Building images

Docker compose provides an easy way to building all the images with the right context

cd build

# Use legacy builder as buildkit still doesn't support subdirectories when building from git repos
export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

docker-compose build

# Just build one image e.g. notebook
docker-compose build notebook

Cross building

The images can be cross-built using docker buildx bake. However buildx bake does not listen to depends_on (since in theory that is only a runtime not a build time constraint https://github.com/docker/buildx/issues/447). To work around this we first need to build the "docker-stacks-foundation" image.

cd build

# If you have permission to push to daskdev/
docker buildx bake --progress=plain --set *.platform=linux/arm64,linux/amd64 --push docker-stacks-foundation
docker buildx bake --progress=plain --set *.platform=linux/arm64,linux/amd64 --push

# If you don'tset DOCKERUSER to your dockerhub username.
export DOCKERUSER=holdenk
docker buildx bake --progress=plain --set *.platform=linux/arm64,linux/amd64 --set docker-stacks-foundation.tags.image=${DOCKERUSER}/docker-stacks-foundation:lab-py38 --push docker-stacks-foundation
docker buildx bake --progress=plain --set *.platform=linux/arm64,linux/amd64 --set scheduler.tags=${DOCKERUSER}/dask --set worker.tags=${DOCKERUSER}/dask --set notebook.tags=${DOCKERUSER}/dask-notebook --set docker-stacks-foundation.tags=${DOCKERUSER}/docker-stacks-foundation:lab-py38 --set notebook.args.base=${DOCKERUSER} --push

Releasing

Building and releasing new image versions is done automatically.