Awesome
BuildKit CLI for kubectl
BuildKit CLI for kubectl is a tool for building OCI and Docker images with your kubernetes cluster.
It replaces the docker build
command to let you quickly and easily build your single and
multi-architecture container images.
Features
Drop in replacement for docker build
The BuildKit CLI for kubectl replaces the docker build
command with kubectl build
to build
images in your kubernetes cluster, instead of on a single node. Your Dockerfile will be parsed
the same way as with the existing docker build
command, and build flags should feel almost
the same.
Uses containerd or docker runtime environments
Regardless of whether your Kubernetes cluster is using pure containerd or docker, the builder will be able to build OCI compatible images. These images can be used inside of your cluster, or pushed to an image registry for distribution.
Works in numerous kubernetes environments
The BuildKit builder should work in most Kubernetes environments. We tested it with:
- vSphere Tanzu
- Amazon EKS
- Minikube
- VMware Fusion
- Docker Desktop
- k0s
- k3d (limited support)
You should use Kubernetes version 1.14 or greater.
Supports building multi-architecture images
If you need to support building multi-architecture images, the builder can create those by cross-compiling your binaries. Multi-architecture images allow you to create images with the same tag so that when they are pulled, the correct image for the architecture will be used. This includes architectures such as:
- linux/amd64 (Intel based 64 bit images)
- linux/386 (Intel based 32 bit images)
- linux/arm/v7 (ARM based 32 bit images - Raspberry Pi)
- linux/arm64 (ARM based 64 bit images)
- windows/amd64 (Intel based 64 bit Windows images)
Future versions will include native builds in mixed clusters, as well as being able to emulate architectures when it's difficult to cross-compile. Check out the multi-architecture documentation for more information.
Getting started
Installing on MacOS and Linux with Homebrew
We include a Brew Tap in this repo. First add the Tap with
brew tap vmware-tanzu/buildkit-cli-for-kubectl https://github.com/vmware-tanzu/buildkit-cli-for-kubectl
Then you can install with
brew install kubectl-buildkit
Keep up-to-date with brew upgrade kubectl-buildkit
(or brew upgrade
to upgrade everything)
Installing on Windows
We produce Choco compatible packages in our https://github.com/vmware-tanzu/buildkit-cli-for-kubectl/releases
Just download the latest nupkg
artifact and in a PowerShell Admin terminal, run
choco install .\kubectl-buildkit*.nupkg
Note: if you haven't already done so, you will need to run choco install kubernetes-cli
once to get kubectl
installed.
Installing on Linux
We also produce .deb
and .rpm
packages which should work for most distros in our https://github.com/vmware-tanzu/buildkit-cli-for-kubectl/releases
Just download the latest artifact and use yum install ./kubectl-buildkit*.rpm
or apt install ./kubectl-buildkit*.deb
Installing the tarball
Head over to https://github.com/vmware-tanzu/buildkit-cli-for-kubectl/releases and download the tgz
asset for your platform.
Once downloaded, extract it into your path. For example, on MacOS
cat darwin-*.tgz | tar -C /usr/local/bin -xvf -
Test your install with
kubectl build --help
Building from source
Check out our contributing guide for instructions on setting up your environment and build instructions
Changing contexts
If you're using more than one kubernetes environment, switch to the context you wish to use with the buildkit builder.
kubectl config get-contexts
kubectl config use-context <context name>
Creating a Kubernetes Registry Secret and Pushing
If you're going to push a newly created image to a container registry, you will need to store your container registry credentials. This can be done either by passing a secret through a multi-stage Dockerfile, or with a registry secret inside of Kubernetes. Although either method can store your plaintext password, you should consider creating an API key in your registry to be more secure.
To create a registry secret inside of Kubernetes, use the command:
read -s REG_SECRET
kubectl create secret docker-registry mysecret --docker-server='<registry hostname>' --docker-username=<user name> --docker-password=$REG_SECRET
kubectl build --push --registry-secret mysecret -t <registry hostname>/<namespace>/<repo>:<tag> -f Dockerfile ./
Hint: if you're trying to authenticate to Docker Hub, use https://index.docker.io/v1/
as the --docker-server
Registry-based Caching
BuildKit is smart about caching prior build results for efficient incremental builds. This works great for a single-node scenario, but if you want to build on a multi-node cluster, you can take advantage of BuildKit's ability to use a registry for cache persistence. This can have a significant improvement on incremental build times regardless of which node in the cluster your build lands on. For best performance, this registry should be "local" to the cluster. The following examples demonstrate this pattern:
- ./examples/local-registry.yaml A kubernetes Deployment+Service to run a local registry (unauthenticated)
- ./examples/local-registry-buildkitd.toml A BuildKit TOML configuration example for the above Registry that configures it for "insecure" access
To setup from the root of this tree:
kubectl apply -f ./examples/local-registry.yaml
kubectl buildkit create --config ./examples/local-registry-buildkitd.toml
You can then build using the registry cache with the command:
kubectl build -t myimage --cache-to=type=registry,ref=registry:5000/cache --cache-from=type=registry,ref=registry:5000/cache .
Custom Certs for Registries
If you happen to run a container image registry with non-standard certs (self signed, or signed by a private CA) you can configure buildkitd to trust those certificates.
First load your certs into a ConfigMap:
kubectl create configmap custom-root-certs --from-file=root_ca1.pem --from-file=root_ca2.pem
Then create a buildkitd.toml
file for your registries where the certs above will be mounted into /etc/config
debug = false
[worker.containerd]
namespace = "k8s.io"
[registry."myregistry1.acme.local"]
ca=["/etc/config/root_ca1.pem"]
[registry."myregistry2.acme.local:5001"]
ca=["/etc/config/root_ca2.pem"]
You can then create a customized builder with
kubectl buildkit create --custom-config=custom-root-certs --config ./buildkitd.toml
Get in Touch
If you encounter issues or have questions/comments, you can find us on the CNCF Slack at #buildkit-cli-for-kubectl
Contributing
Check out our contributing for guidance on how to help contribute to the project.
This project wouldn't have been possble without the awesome upstream work from BuildKit which serves as the back-end builder, and buildx which this CLI was derived from.