Home

Awesome

packer-templates Build Status

Collection of Packer templates used for various infrastructure layers.

How to build stuff

To build a given template, one may use the make implicit builder, like the following for ci-stevonnie:

make ci-stevonnie

or, with a specific builder:

make ci-stevonnie BUILDER=docker

or forget about the Makefile and run with packer directly:

packer build -only=docker <(bin/yml2json < ci-stevonnie.yml)

env config bits

Most of the templates in here require some env vars. Take a look at .example.env for an example. Use of autoenv is encouraged but not required.

packer template types

There are two primary types of templates present at the top level: those intended for use as execution environment for jobs flowing through Travis CI, and those used for various backend fun in the Travis CI infrastructure. The former type all have the prefix ci-, described in more detail below:

stacks

There are two primary types of stacks: those targeting Ubuntu 14.04 (trusty), and those targeting Ubuntu 16.04 (xenial) that run on GCE and Docker.

Take a peek at what's what:

make stacks-trusty

make stacks-xenial

There may be some subtle variations, but for the most part each stack is built via the following steps.

git metadata file input

The generated files in ./tmp/git-meta/ are copied onto the provisioned machine at /var/tmp/git-meta/ for later use by the ./packer-scripts/packer-env-dump script.

purge file input

A git-tracked file in ./packer-assets is copied onto the provisioned machine at /var/tmp/purge.txt for later use by the ./packer-scripts/purge script.

packages file input

A git-tracked file in ./packer-assets is copied onto the provisioned machine at /var/tmp/packages.txt for later use by both the travis_packer_templates::default recipe and the serverspec suites via ./cookbooks/lib/support.rb.

write packer and travis env vars

The script at ./packer-scripts/packer-env-dump creates a directory on the provisioned machine at /.packer-env which is intended to be in the envdir format. Any environment variables that match ^(PACKER|TRAVIS), and (if present) the files previously written to /var/tmp/git-meta/ are copied or written into /.packer-env/.

remove default users

The script at ./packer-scripts/remove-default-users will perform a best-effort removal of users defined in ${DEFAULT_USERS} (default vagrant ubuntu). The primary reasons for this are general tidyness and to try to free up uid 2000.

pre-chef bootstrapping

The script at ./packer-scripts/pre-chef-bootstrap is responsible for ensuring the provisioned machine has all necessary packages and users for the Chef provisioning process. The steps executed include:

cloning travis-cookbooks

The script at ./packer-scripts/clone-travis-cookbooks is responsible for git clone'ing travis-cookbooks into /tmp/chef-stuff on the provisioned machine. Optional env vars supported by this script are:

Once the clone is complete, the clone directory is written to /.packer-env/TRAVIS_COOKBOOKS_DIR and the head sha is written to /.packer-env/TRAVIS_COOKBOOKS_SHA.

chef provisioning

The chef-solo provisioner will typically have no json data, but instead will leave all attribute and effective run list definition to a single wrapper cookbook located in ./cookbooks/.

chef wrapper cookbook layout

Each wrapper cookbook must contain at least a metadata.rb and a recipes/default.rb. Typically, the attributes/default.rb is defined and contains all override attribute settings. The earliest version of Chef used by either trusty or xenial stacks is 12.9, which means that all cookbook dependencies must be declared in metadata.rb, a requirement that is also enforced by the foodcritic checks.

For example, the minimal trusty image "ci-stevonnie" has a wrapper cookbook at ./cookbooks/travis-ci_stevonnie that looks like this:

cookbooks/travis_ci_stevonnie
├── README.md
├── attributes
│   └── default.rb
├── metadata.rb
├── recipes
│   └── default.rb
└── spec
    ├── ...

travis user double check

The script at ./packer-scripts/ensure-travis-user is responsible for ensuring the existence of the travis user and its home directory permissions, optionally setting the password to a random string. The list of operations is:

Optional env vars supported by this script are:

purging undesirable packages

The script at ./packer-scripts/purge is responsible for purging packages that are not desirable for the CI environment, such as the Chef that was installed prior for the Chef provisioner. Additionally, any package names present in /var/tmp/purge.txt will be purged. Optional env vars supported by this script are:

disabling apparmor

The script at ./packer-scripts/disable-apparmor is responsible for disabling apparmor if detected. This is done primarily so that services such as PostgreSQL and Docker may be used in the CI environment without first updating apparmor configuration and restaring said services.

running server specs

The script at ./packer-scripts/run-serverspecs is responsible for running the serverspec suites via the rspec executable that is part of the chefdk package. The list of operations is:

Optional env vars supported by this script are:

removing undesirable files

The script at ./packer-scripts/cleanup is responsible for removing files and directories that are unnecessary for the CI environment or otherwise add unnecessary mass to the mastered image. The list of operations is:

Optional env vars supported by this script are:

minimizing image size

The script at ./packer-scripts/minimize is responsible for reducing the size of the provisioned image by squeezing out all of the empty space into a contiguous area using the same method as bento. The list of operations is:

registering the image with job-board

The script at ./bin/job-board-register is responsible for "registering" the mastered image in a post-processing step by making an HTTP request to the job-board images API. The list of operations is:

Required env vars for this script are:

Optional env vars supported by this script are:

For more info on the relationship between a given packer build artifact and job-board, see job-board details below.

job-board details

The job-board application is responsible for tracking stack image metadata and presenting a queryable API that is used by the travis-worker API image selector. As described above, each stack image is registered with job-board along with a group, os, dist, and map of tags. When travis-worker requests a stack image identifier, it performs a series of queries with progressively lower specificity.

Of the values assigned to each stack image, the tags map is perhaps most mysterious, in part because it is so loosely defined. This is intentional, as the number of values that could be considered "tags" varies enough that maintaining them all as individual columns would result in (opinions!) too much overhead in the form of schema management and query complexity.

The implementation of the job-board-register script includes a process that converts the languages and features arrays present in /.job-board-register.yml, written from the values present in chef attributes at travis_packer_templates.job_board.{features,languages}, into "sets" represented as {key} => true. For example, if a given wrapper cookbook contains attributes like this:

override['travis_packer_templates']['job_board']['languages'] = %w(
  fribble
  snurp
  zzz
)

then the tags generated for registration with job-board would be equivalent to:

{
  "language_fribble": true,
  "language_snurp": true,
  "language_zzz": true
}

job-board tagsets

A "tagset" is the "set" (as in the type) of the "tags" applied during job-board registration of a particular stack image, including languages and features. At the time of this writing, both tagsets are used during serverspec runs, and only the languages tagset is considered during selection via the job-board API.

tagset relationships

Because the travis-worker API image selector is querying job-board for stack images that match a particular language, it is important for us to ensure reasonably consistent image selection by way of asserting the languages values do not overlap between certain stacks (an "exclusive" relationship). Additionally, it is important that we ensure certain stack features are subsets of others (an "inclusive" relationship).

Part of the CI process for this repository makes assertions about such exclusive and inclusive relationships by way of the check-job-board-tags script. The exact relationships being enforced may be viewed like so:

./bin/check-job-board-tags --list-only
exclusive relationships

An exclusive tagset relationship is equivalent to asserting that the set intersection is the empty set, e.g.:

tagset_a = %w(a b c)
tagset_b = %w(d e f)
assert (tagset_a & tagset_b).empty?
inclusive relationships

An inclusive tagset relationship is equivalent to asserting that all members of one tagset are present in another, or that a tagset's intersection with its superset is equivalent to itself, e.g.:

tagset_a = %w(a b c d e f)
tagset_b = %w(f d b)
assert (tagset_a & tagset_b).sort == tagset_b.sort

Testing cookbook changes

When submitting changes to this repository, please be aware that the top level-specs are shallow and don't include logic changes in the cookbooks.

Any cookbook specs are ran as part of the actual image building process, which is triggered when any of the ci-<image-name>.yml templates are modified.

The image build is ran as part of the packer-build repo on the branch corresponding to each template and is triggered by travis-packer-build.

This can be installed and invoked locally by running bundle install and then bundle exec travis-packer-build [options].

Example:

bundle exec travis-packer-build \
	-I ci-sardonyx.yml \
	--target-repo-slug="travis-infrastructure/packer-build" \
	--github-api-token="<your-token-here>" \
	--body-tmpl=".packer-build-pull-request-false-tmpl.yml"

You can specify the branch using -B (if you don't want to build from master).

The file .packer-build-pull-request-false-tmpl.yml here is just an example, but you can also create a different template that specifies other travis-cookbooks or packer-template branches.

Additionaly, if you just want to test a change in travis-cookbooks, you can use the shortcut script in ./bin/packer-build-cookbooks-branch:

./bin/packer-build-cookbooks-branch <travis-cookbooks-branch-name> <template-name>

Note: The above script expects the GITHUB_API_TOKEN environment variable to be set.

Once created, the images will be registered in job-board under the group: dev tag.