Home

Awesome

Byzer-build

Project byzer-build comes with tools to build

Byzer Sandbox Docker Image

With Byzer Sandbox docker image, users are able to take a quick glance into Byzer stack. Based on spark 2.4.3:

docker run -d --name sandbox -p 9002:9002 -p 9003:9003 -e MYSQL_ROOT_PASSWORD=root byzer/byzer-sandbox:2.4.3-latest

Based on spark 3.1.1:

docker run -d --name sandbox -p 9002:9002 -p 9003:9003 -e MYSQL_ROOT_PASSWORD=root byzer/byzer-sandbox:3.1.1-latest

Based on spark 3.3.0:

docker run -d --name sandbox -p 9002:9002 -p 9003:9003 -e MYSQL_ROOT_PASSWORD=root byzer/byzer-sandbox:3.3.0-latest

Building Sandbox

Click for details

Byzer-lang Container

docker run -d --name byzer-lang -e DRIVER_MEMORY=8g -e MASTER=local[*] -p9003:9003 byzer/byzer-lang:3.3.0-latest This command launches byzre-lang container with

Byzer-lang K8S image

The image is built on Ubuntu 20.04, and packaged with

An example to build the image

export BYZER_LANG_VERSION=2.4.0-SNAPSHOT
export SPARK_VERSION=3.3.0
## Run the script on dev directory, or it fails
cd dev
./bin/build-byzer-lang-k8s-azure-image.sh

Directory structure

/work
├── jdk-14                        
├── logs
├── spark-3.1.1-bin-hadoop3.2
└── user

/home/deploy
├── byzer-lang
├── entrypoint.sh
├── logs
└── spark-warehouse

/opt/conda
├── LICENSE.txt
├── bin
├── compiler_compat
├── conda-meta
├── condabin
├── envs
├── etc
├── include
├── lib
├── [Apr 23 09:29]  pkgs
├── share
├── shell
├── ssl
├── x86_64-conda-linux-gnu
└── x86_64-conda_cos6-linux-gnu

Byzer all-in-one

To install and run the all-in-one, please visit all-in-one doc . Tar ball naming convention is byzer-lang-all-in-one-<os>-amd64-<byzer_spark_version>-<byzer_lang_version>.tar.gz.

ParameterExplanation
byzer_lang_versionThe Byzer Lang version
byzer_spark_version3.3 for Spark 3.3.0, 3.0 for Spark 3.1.1 2.4 for Spark 2.4.3
oslinux darwin(mac) win

To build the Byzer all-in-one

export JUICEFS_VERSION=1.0.0
export SPARK_VERSION=3.3.0
export BYZER_LANG_VERSION=2.4.0-SNAPSHOT
export OS=linux
# darwin for MacOS, win for Windows
./dev/bin/app/build-byzer-cli-release.sh

Multi-container deployment

We provide scripts to start multiple microservices at once to facilitate us to build and start docker containers flexibly. Each service is deployed in an independent Ubuntu environment, which can isolate resources and environments.