Awesome
Mason
❌ UNMAINTAINED: this project is no longer maintained and no additional development is planned. Nor is any kind of deprecation planned. Rather this code is now frozen. If you are depending on this module in any of your applications, it is recommended you remove the dependency and/or find a replacement solution
Mason is a cross-platform, command-line package manager for C/C++ applications.
Mason is like:
- npm because it installs packages in the current working directory (
./mason_packages
) rather than globally (and therefore does not require privileges for, or conflict with, system-wide packages) - homebrew because it requires no use of
sudo
to install packages - apt-get or yum because it works on Linux
Mason is unlike:
-
all of the above...
Mason is a collection of bash scripts and does not depend on any specific runtime language, such as python, node.js, or ruby. It can build and publish a single set of binaries (>= OS X 10.8 and >= Ubuntu Precise), publish header-only files, and install packages. Mason has integrations with Travis CI and Amazon S3 for automated build and deployment.
Mason strongly prefers static libraries over shared libraries and has zero understanding of dependency trees: it leaves complete control to the developer for how packages relate.
Mason works on both OS X and Linux.
Table of Contents
- Installation
- Configuration
- Usage
- Creating a package
- Releasing a package
- Using a package
- Mason internals
- Troubleshooting
Installation
There are three recommended ways to install mason, via:
Curl
To install mason locally:
mkdir ./mason
curl -sSfL https://github.com/mapbox/mason/archive/v0.23.0.tar.gz | tar -z --extract --strip-components=1 --exclude="*md" --exclude="test*" --directory=./mason
Then you can use the mason
command like: ./mason/mason install <package> <version>
To install mason globally (to /tmp):
curl -sSfL https://github.com/mapbox/mason/archive/v0.23.0.tar.gz | tar -z --extract --strip-components=1 --exclude="*md" --exclude="test*" --directory=/tmp
Then you can use the mason
command like: /tmp/mason install <package> <version>
Submodule
Mason can also be added as a submodule to your repository. This is helpful for other contributors to get set up quickly.
Optionally a convention when using submodules, is to place the submodule at a path starting with .
to make the directory hidden to most file browsers. If you want your mason folder hidden then make sure to include the final part of the following command .mason/
so your submodule path has the leading .
instead of just being mason/
.
git submodule add git@github.com:mapbox/mason.git .mason/
This will append a few lines to your .gitmodules
file. Make sure to change the url
parameter to https
instead of git@github
ssh protocol.
[submodule ".mason"]
path = .mason
url = https://github.com/mapbox/mason.git
Update your Makefile
to point to the mason scripts and provide an installation script for the necessary dependencies. The following installs two Mason packages with the make mason_packages
command.
MASON ?= .mason/mason
$(MASON):
git submodule update --init
mason_packages: $(MASON)
$(MASON) install geometry 0.7.0
$(MASON) install variant 1.1.0
mason.cmake
Copy the https://raw.githubusercontent.com/mapbox/mason/master/mason.cmake into your cmake project. A common convention is to place it at <your project>/cmake/mason
mkdir cmake
wget -O cmake/mason.cmake https://raw.githubusercontent.com/mapbox/mason/master/mason.cmake
Then in your CmakeLists.txt
install packages like:
mason_use(<package name> VERSION <package version> HEADER_ONLY)
Note: Leave out HEADER_ONLY
if the package is a precompiled library. You can see if a package is HEADER_ONLY
by looking inside the script.sh
for MASON_HEADER_ONLY=true
like https://github.com/mapbox/mason/blob/68871660b74023234fa96d482898c820a55bd4bf/scripts/geometry/0.9.0/script.sh#L5
Configuration
By default Mason publishes packages to a Mapbox-managed S3 bucket. If you want to publish to a different bucket we recommend taking the following steps:
- Fork Mason and rename it to
mason-{your_org}
- Set
MASON_BUCKET
to your own S3 bucket - Publish packages to the new location
Usage
Most commands are structured like this:
mason <command> <library> <version>
The command
can be one of the following
- install - installs the specified library/version
- remove - removes the specified library/version
- build - forces a build from source (= skip pre-built binary detection)
- publish - uploads packages to the S3 bucket
- prefix - prints the absolute path to the library installation directory
- version - prints the actual version of the library (only useful when version is
system
) - cflags - prints C/C++ compiler flags
- ldflags - prints linker flags
- link - creates symlinks for packages in
mason_packages/.link
- trigger - trigger a build and publish operation on Travis CI
install
$ mason install libuv 0.11.29
* Downloading binary package osx-10.10/libuv/0.11.29.tar.gz...
######################################################################## 100.0%
* Installed binary package at /Users/user/mason_packages/osx-10.10/libuv/0.11.29
Installs libuv into the current folder in the mason_packages
directory. Libraries are versioned by platform and version number, so you can install several different versions of the same library along each other. Similarly, you can also install libraries for different platforms alongside each other, for example library binaries for OS X and iOS.
The install
command first checks if the specified library/version is already present for this platform, and if so, exits. This means you can run it multiple times (e.g. as part of a configuration script) without doing unnecessary work.
Next, Mason checks whether there are pre-built binaries available in the S3 bucket for the current platform. If that is the case, they are downloaded and unzipped and the installation is complete.
If no pre-built binaries are available, Mason is going to build the library according to the script in the mason_packages/.build
folder, and install into the platform- and library-specific directory.
remove
$ mason remove libuv 0.11.29
* Removing existing package...
/Users/user/mason_packages/osx-10.10/libuv/0.11.29/include/uv-darwin.h
[...]
/Users/user/mason_packages/osx-10.10/libuv/0.11.29
Removes the specified library/version from the package directory.
build
This command works like the install
command, except that it doesn't check for existing library installations, and that it doesn't check for pre-built binaries, i.e. it first removes the current installation and always builds the library from source. This is useful when you are working on a build script and want to fresh builds.
publish
Creates a gzipped tarball of the specified platform/library/version and uploads it to the mason-binaries
S3 bucket. If you want to use this feature, you need write access to the bucket and need to specify the environment variables AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
.
prefix
~ $ mason prefix libuv 0.11.29
/Users/user/mason_packages/osx-10.10/libuv/0.11.29
This prints the absolute path to the installation directory of the the library/version. Likely, this folder has the typical include
and lib
folders.
version
~ $ mason version zlib system
1.2.11
This prints the version of the library, which is only useful when version is system
. See System packages for more details.
cflags
~ $ mason cflags libuv 0.11.29
-I/Users/user/mason_packages/osx-10.10/libuv/0.11.29/include
Prints the C/C++ compiler flags that are required to compile source code with this library. Likely, this is just the include path, but may also contain other flags.
ldflags
~ $ mason ldflags libuv 0.11.29
-L/Users/user/mason_packages/osx-10.10/libuv/0.11.29/lib -luv -lpthread -ldl
Prints the linker flags that are required to link against this library.
link
~ $ mason link libuv 0.11.29
This command only works if the package has already been installed. When run it symlinks the versioned lib
, include
, share
, and bin
folders of the package into a shared structure that is unversioned. For example if mason prefix libuv 0.11.29
was ./mason_packages/osx-10.10/libuv/0.11.29
then the library would become available at ./mason_packages/.link/lib/libuv.a
trigger
In order to ensure that all pre-built binaries are consistent and reproducible, we perform the final build and publish operation on Travis CI.
First set the MASON_TRAVIS_TOKEN
environment variable. You can do this either by installing the travis
gem and running travis token
or by using curl
to hit the Travis api directly. See details on this below. WARNING: be careful to keep this token safe. Cycling it requires emailing support@travis-ci.com. Giving someone an access token is like giving them full access to your Travis account.
Once you are set up with your MASON_TRAVIS_TOKEN
then use the trigger
command to kick off a build:
./mason trigger <package name> <package version>
Run this command from the root of a local mason repository checkout. It makes a request to the Travis API to build and publish the specified version of the package, using the Travis configuration in ./scripts/${MASON_NAME}/${MASON_VERSION}/.travis.yml
.
- Using curl and travis api to generate MASON_TRAVIS_TOKEN
First generate a github personal access token that has repo
scope by going to https://github.com/settings/tokens. More details at https://help.github.com/articles/creating-an-access-token-for-command-line-use/.
Then set that in your environment and run:
GITHUB_TOKEN=<github token>
curl -s -i https://api.travis-ci.com/auth/github \
-H "User-Agent: Travis/1.0" \
-H "Content-Type: application/json" \
-H "Accept: application/vnd.travis-ci.2+json" \
-H "Host: api.travis-ci.com" \
-d "{\"github_token\": \"${GITHUB_TOKEN}\"}"
- Use the travis command
For details see https://docs.travis-ci.com/user/triggering-builds and https://github.com/travis-ci/travis.rb#readme
Creating a package
Prerequisites
Before getting started you should be able to answer the following questions.
What are you packaging?
There are different steps that you will need to follow depending on the type of library you are packaging. For example, there are fewer steps you need to take if you are creating a package of header-only code. Packaging compiled code has more steps because you'll need to tell Mason how to build your binaries. Another type of package is a System package which has a unique process as well.
Are there previous versions already published?
Check the list of packages to see if a previous version of your package exists. It's helpful to copy scripts from a previous version rather than creating code from scratch.
Where can Mason download your code?
Every package needs to tell Mason where to download the code that it will build and publish, for example:
https://github.com/mapbox/geometry.hpp/archive/v0.9.2.tar.gz
for a Github release: geometry 0.9.2https://github.com/mapbox/geometry.hpp/archive/b0e41cc5635ff8d50e7e1edb73cadf1d2a7ddc83.zip
for pre-release code hosted on Github: geometry b0e41cc
Note: Your code doesn't need to be hosted on Github in order for Mason to work. Your code can be hosted anywhere. Another common location is SourceForge.
Getting started
These are just basic steps to help get you started. Depending on the complexity of building your code, you might have to review the Mason scripts section to get a better idea of how to further configure Mason to be able to create your package.
-
Create a new directory for your package.
From inside your
mason
checkout, create a directory named${package}/${version}
. For example, if you have a library namedyour-lib
that is version0.1.0
you would:mkdir -p scripts/your-lib/0.1.0
-
Add scripts for building and publishing your package.
Each package must have the following two files:
script.sh
and.travis.yml
. Copy these two files from a previous version of your package.If no previous version of your package exists, it is recommended to copy a simple package that has mostly boiler plate code:
For example, if you're creating the first package of a library named
your-lib
that is version0.1.0
you would copy scripts from thegeometry
package:cp -r scripts/geometry/0.9.1 scripts/your-lib/0.1.0
-
Edit Mason variables in
script.sh
.You must set the follow Mason variables:
MASON_NAME
: set to the name of your package, e.g.your-lib
MASON_VERSION
: set to the package version, e.g.0.1.0
MASON_BUILD_PATH
: set to the location Mason will use to store header files or binaries before it packages and publishes them
You may also need to set the follow Mason variables:
- Other Mason variables
-
Override Mason functions in
script.sh
.You must override the follow Mason functions:
mason_load_source
: you must callmason_download
and update its parameters:- url (first parameter): set to the location of your source code archive, e.g.
https://github.com/mapbox/your-lib/archive/v${MASON_VERSION}.tar.gz
- checksum (second parameter): set to the checksum you get back after running a checksum function on the source code you want to download. The easiest way to get this checksum is to run
./mason build your-lib 0.1.0
(after setting themason_download
url) which will fail with an error message that will contain the correct checksum
- url (first parameter): set to the location of your source code archive, e.g.
mason_compile
- for header-only see geometry 0.9.2 for an example
- for code that needs to be compiled see zlib 1.2.8 for an example
You may also need to override the follow Mason functions:
- Other Mason functions
System packages
Some packages ship with operating systems or can be easily installed with operating-specific package managers. For example, libpng
is available on most systems and the version you're using doesn't really matter since it is mature and hasn't added any significant new APIs in recent years.
The following script.sh
contains the script code for packaging your system's libpng
. Note: To understande this code, make sure to review the Mason scripts section.
#!/usr/bin/env bash
MASON_NAME=libpng
MASON_VERSION=system
MASON_SYSTEM_PACKAGE=true
. ${MASON_DIR}/mason.sh
if [ ! $(pkg-config libpng --exists; echo $?) = 0 ]; then
mason_error "Cannot find libpng with pkg-config"
exit 1
fi
function mason_system_version {
mkdir -p "${MASON_PREFIX}"
cd "${MASON_PREFIX}"
if [ ! -f version ]; then
echo "#include <png.h>
#include <stdio.h>
#include <assert.h>
int main() {
assert(PNG_LIBPNG_VER == png_access_version_number());
printf(\"%s\", PNG_LIBPNG_VER_STRING);
return 0;
}
" > version.c && ${CC:-cc} version.c $(mason_cflags) $(mason_ldflags) -o version
fi
./version
}
function mason_compile {
:
}
function mason_cflags {
pkg-config libpng --cflags
}
function mason_ldflags {
pkg-config libpng --libs
}
mason_run "$@"
System packages are marked with MASON_SYSTEM_PACKAGE=true
. We're also first using pkg-config
to check whether the library is present at all. The mason_system_version
function creates a small executable which outputs the actual version. It is the only thing that is cached in the installation directory.
We have to override the mason_cflags
and mason_ldflags
commands since the regular commands return flags for static libraries, but in the case of system packages, we want to dynamically link against the package.
Releasing a package
Here is an example workflow to help get you started:
-
Create an annotated tag in git for the code you want to package.
Annotated tags can be stored, checksummed, signed and verified with GNU Privacy Guard (GPG) in Github. To create an annotated tag specify
-a
when running thetag
command, for example:git tag -a v0.1.0 -m "version 0.1.0"
-
Share your new tag.
You have to explicitly push your new tag to a shared Github server. This is the location we will share with Mason when specifying where to download the code to be packaged. Using our example above we would run:
git push origin v0.1.0
(Or you can push all tags:
git push --tags
.) -
Create a package.
We recommend working in a new branch before creating a package. For example if you want to call your new package
my_new_package
version0.1.0
you could create and checkout a branch like this:git checkout -b my_new_package-0.1.0
Now follow the instructions in the Getting started section for creating a new package.
-
Test your package.
Even though we will eventually build the package using Travis, it's a good idea to build locally to check for errors.
./mason build my_new_package 0.1.0
-
Push changes to remote.
Once you can build, push your changes up to Github remote so that Travis will know what to build and publish in the next step.
git push origin my_new_package-0.1.0
-
Build and Publish your package.
Use Mason's
trigger
command to tell Travis to build, test, and publish your new package to the S3 bucket specified inmason.sh
../mason trigger my_new_package 0.1.0
-
Check S3 to verify whether your package exists.
Using a package
Mason has two clients for installing and working with packages:
-
Mason cli - comes bundled with the Mason project, see Usage for commands
For example hpp-skel uses the Mason cli client and requires that the Mason version in scripts/setup.sh be updated in order to stay up-to-date with the latest available packages.
-
mason-js - a separate Node.js client with its own installation and usage instructions
For example node-cpp-skel uses the mason-js client and pulls packages directly from S3.
Note: The install command syntax will differ depending on the client you use.
Mason internals
Mason scripts
The script.sh
file in each package is structured like the following example:
#!/usr/bin/env bash
# This is required for every package.
MASON_NAME=libuv
MASON_VERSION=0.11.29
# This is required if Mason will need to build a static library. Specify the relative path in the
# installation directory.
MASON_LIB_FILE=lib/libuv.a
# You can specify the relative path to the pkg-config file if Mason needs to build your code before
# packaging. If the library doesn't have a pkg-config file, you will need to override the functions
# `mason_cflags` and `mason_ldflags`.
MASON_PKGCONFIG_FILE=lib/pkgconfig/libuv.pc
# This is required when you need to load the build system to build your code before packaging. You
# con't need this line if you are packaging header-only code.
. ${MASON_DIR}/mason.sh
# Overriding this Mason function is required for all pakcages so Mason knows where to obtain your
# source code. This function also caches downloaded tarballs in the mason_packages/.cache folder.
function mason_load_source {
mason_download \
https://github.com/joyent/libuv/archive/v0.11.29.tar.gz \
5bf49a8652f680557cbaf335a160187b2da3bf7f
# This unpacks the archive into the `mason_packages/.build` folder. If the tarball is BZip2
# compressed, you can also use `mason_extract_tar_bz2` instead.
mason_extract_tar_gz
# This variable contains the path to the unpacked folder inside the `.build` directory.
export MASON_BUILD_PATH=${MASON_ROOT}/.build/${MASON_NAME}-${MASON_VERSION}
}
# Override this Mason function if you need to run code before compiling, e.g. a script that
# generates configuration files.
function mason_prepare_compile {
./autogen.sh
}
# It is required to override the `mason_compile` function because it performs the actual build of
# the source code (or just copies header files into a package folder to be published later for
# header-only code). This is an example of how you would configure and make a static library.
function mason_compile {
# You must set the build system's installation prefix to `MASON_PREFIX`. For cross-platform
# builds, you have to specify the `MASON_HOST_ARG`, which is empty for regular builds and is set
# to the correct host platform for cross-compiles e.g. iOS builds use `--host=arm-apple-darwin`.
./configure \
--prefix=${MASON_PREFIX} \
${MASON_HOST_ARG} \
--enable-static \
--disable-shared \
--disable-dependency-tracking
# If the build system supports building concurrently, you can tell it do do so by providing the
# number of parallel tasks from `MASON_CONCURRENCY`.
make install -j${MASON_CONCURRENCY}
}
# Tell Mason how to clean up the build folder. This is required for multi-architecture builds. e.g.
# iOS builds perform a Simulator (Intel architecture) build first, then an iOS (ARM architecture)
# build. The results are `lipo`ed into one universal archive file.
function mason_clean {
make clean
}
# Run everything.
mason_run "$@"
Mason variables
Name | Description |
---|---|
MASON_DIR | The directory where Mason itself is installed. Defaults to the current directory. |
MASON_ROOT | Absolute path the mason_packages directory. Example: /Users/user/mason_packages . |
MASON_PLATFORM | Platform of the current invocation. Currently one of osx , ios , android , or linux . |
MASON_PLATFORM_VERSION | Version of the platform. It must include the architecture if the produced binaries are architecture-specific (e.g. on Linux). Example: 10.10 |
MASON_NAME | Name specified in the script.sh file. Example: libuv |
MASON_VERSION | Version specified in the script.sh file. Example: 0.11.29 |
MASON_SLUG | Combination of the name and version. Example: libuv-0.11.29 |
MASON_PREFIX | Absolute installation path. Example: /Users/user/mason_packages/osx-10.10/libuv/0.11.29 |
MASON_BUILD_PATH | Absolute path to the build root. Example: /Users/user/mason_packages/.build/libuv-0.11.29 |
MASON_BUCKET | S3 bucket that is used for storing pre-built binary packages. Example: mason-binaries |
MASON_BINARIES | Relative path to the gzipped tarball in the .binaries directory. Example: osx-10.10/libuv/0.11.29.tar.gz |
MASON_BINARIES_PATH | Absolute path to the gzipped tarball. Example: /Users/user/mason_packages/.binaries/osx-10.10/libuv/0.11.29.tar.gz |
MASON_CONCURRENCY | Number of CPU cores. Example: 8 |
MASON_HOST_ARG | Cross-compilation arguments. Example: --host=x86_64-apple-darwin |
MASON_LIB_FILE | Relative path to the library file in the install directory. Example: lib/libuv.a |
MASON_PKGCONFIG_FILE | Relative path to the pkg-config file in the install directory. Example: lib/pkgconfig/libuv.pc |
MASON_XCODE_ROOT | OS X specific; Path to the Xcode Developer directory. Example: /Applications/Xcode.app/Contents/Developer |
MASON_HEADER_ONLY | Set to true to specify this library as header-only, which bypasses building binaries (default false ) |
Mason functions
These are common Mason function that you might need to override in your package's script.sh
file depending on the type of library you are packaging. See https://github.com/mapbox/mason/blob/master/mason.sh to view how these functions are implemented by default. There you will find even more mason_
-functions that you might find useful to override.
mason_pkgconfig
mason_cflags
mason_ldflags
mason_static_libs
Troubleshooting
Downloaded source tarballs are cached in mason_packages/.cache
. If for some reason the initial download failed, but it still left a file in that directory, make sure you delete the partial download there.