Awesome

swaddle

swaddle wraps up your newly built programs (with 'swaddling') as ready-to-go signed releases: archives, packages, package repositories and even release note websites, using simple data files stored in source control laid out as you would have them in a package. It's MIT-licensed, making it friendly to adopt. And it's in pure shellscript: there's no need for a working Ruby or Python or Perl installation. You can even use it directly from git, and it'll bootstrap itself. Oh, and it eats its own dog food. Those releases on GitHub were made by swaddle.

The Final Step

swaddle is your final step after build and test:-

• It creates whatever packages you want, with the best possible settings
  • Deb
  • RPM
  • tarball
  • zip
  • 7z
  • etc
• It creates debian repositories with
  • complete file contents for apt-file
  • translations
  • package pools
  • components
  • priorities
  • InRelease and Release.gpg for maximal compatibility
  • and can even import packages from other sources, so you need never use reprepro or anything else again
    • you could even use it just for that alone
  • oh, and they're versioned, too
• It tags, using a signature, your source and binaries
  • Works with gpg-agent for silent signing packages and repositories
  • Except for rpm signing, which is so badly coded that it can't use gpg-agent (workarounds welcomed)
• Automatically uploads your GPG key to a keyserver after releasing, so everyone can check it's your release
• It can create a GitHub release
  • with all files attached
  • with lots of sensible default copy
  • with links to standalone websites and versioned, signed package repositories
  • with automatic installation scripts included in your release notes tested for all major distributions
• It can create a standalone website for your packages on GitHub, referenced in the release notes
• It can push and tag, using a signature, a versioned set of repositories to GitHub pages
  • so you can always rollback
  • and never deletes older released files

All this seems complex. It isn't. Take a look at the swaddling for swaddle itself. Not much to see, is there?

Who Uses it?

• ReleaseQueue have integrated it into their upcoming product management platform.
• bish-bosh, a pure shellscript MQTT 3.1.1 client
• And, of course, we use it to release itself. Eating our own dog food and all that. See shellfire's build script, which shows how to go from a bunch of stuff in source to a complete automated release, just by typing ./build.

Differences with everything else

In many ways, we see this as the logically conclusion of fpm. It is to fpm what fpm was to RPM. Now, if only yum would die…

• it completely separates build from packaging (unlike, say, dh-*)
• source control is king: package files are just stored in source control wherever possible
• it is data-driven rather than script-driven
• all configuration data is just shell-like text files - not 200 shell scripts
• it produces valid, thorough Debian packages which use every ounce of cleverness Debian's packaging system provides
• it provides a one stop shop for creating signed, valid and versioned repositories with yours and third party packages, and even server config packages, which you can even roll back
• and, unlike everything else, it doesn't need C, Python, Perl or Ruby. It's pure shell script, built using shellfire*

* Yep, that's right. No need to have the dpkg-* or yum-utils tools installed. It'll even run on the Mac with Homebrew. The only downside is you'll need rpmbuild, because, RPM, being a brilliant format is unusable unimplementable with anything else. At least you won't have to write any more spec files, though.

How to use it

For example, image you have the shellfire application 'overdrive'. You have a git repository 'overdrive' (perhaps at GitHub), containing the following structure:-

overdrive\
	.git\
	README.md
	COPYRIGHT
	overdrive           # your shellfire application script
	swaddling\

Inside swaddling, you'll create a configuration. For example, to create a tarball, debian package and RPM, with apt and yum repositories, we might do:-

    swaddling\
	    swaddling.conf       # Essential configuration
		swaddling.conf.d\    # Any files .conf are loaded after swaddling.conf, a la Debian run-parts.
		                     # This is true for any .conf file (eg package.conf, deb.conf, etc, below) in swaddle
							 # Use it to have localized bits of sensitive configuration external to source control
        overdrive\           # name of your 'swaddle'. Usually the same as your GitHub repo name.
                             # You can have many of these (eg for multiple packages, etc) but most people need just one.
	        package.conf     # Configuration settings for all package kinds (tarball, debian, etc) built for this swaddle
			skeleton\        # Put files that never change and aren't built in here
			    any\         # For any architecture
				    etc\
					    overdrive.conf
				all\         # For packages without an architecture (Debian's 'all', RPM's 'noarch')
				amd64\       # For amd64 (and other architectures, as appropriate) - we use the Debian names (as these 
							 # are highly consistent), and convert as necessary for RPM
			body\            # Identical structure to skeleton\, but intended for files that are build outputs (so you can `.gitignore` it; often symlinked to your build folder).
			    …
            deb\             # Create this, and you're making debian packages
			    deb.conf     # Debian specific settings, if any; entirely optional
			    scripts\     # Package scripts folder
			       preinst\  # Folder containing pre installation script snippets
			       postinst\ # Folder containing post installation script snippets
			       prerm\    # Folder containing pre removal script snippets
			       postrm\   # Folder containing post removal script snippets
			    skeleton\    # As above. Merged using rsync. Allows per-package-kind, per-architecture-variant file differences
			    body\        # As above. Merged using rsync.
			        …
            rpm\             # Create this, and you're making RPMs
			    rpm.conf     # RPM specific settings, if any; entirely optional
				skeleton\    # As above.
			    …
            tar\
			    tar.conf     # Same again
			    …

If an architecture folder exists, say amd64, then a amd64 variant of a package is made. If it only exists, at, say, the level of deb, then it won't be made for a RPM or tarball. It is not allowed to have both all and another architecture (indeed, it makes no sense at all for Deb and RPM packages). So in the above example, we shouldn't have either all or amd64.

Surprisingly, there's actually very little to put in our conf files at this time. For example, the most complex is probably swaddling.conf. We might have:-

configure swaddle host_base_url 'https://raphaelcohn.github.io/swaddle/download'
configure swaddle maintainer_name 'Raphael Cohn'
configure swaddle maintainer_comment 'Package Signing Key'
configure swaddle maintainer_email 'raphael.cohn@stormmq.com'
configure swaddle vendor stormmq

Now it's possible we might not want those values to be used the same for every package. That's quite possible. A conf file deeper in the hierarchy, overrides one above it for that part. For example, we could change the vendor above for Deb overdrive packages by putting this into overdrive/deb.conf:-

configure swaddle vendor 'someone else'

Of course, using this couldn't be easier:-

swaddle --swaddling-path /path/to/swaddling --output-path /path/to/output -- overdrive

And off we go!

Education, Education, Education Configuration, Configuration, Configuration

The key to swaddle is configuration. In swaddle, there are configuration namespaces. Each namespace is useful at a different level in the hierarchy above. Some are global; some are only useful, for, say, a deb. Configuration uses the file system layout, as well, to be useful. All are designed to be source control friendly. Indeed, swaddle works best when used with git and especially GitHub.

Jargon Guide

We've tried to keep this as simple as possible.

Configuration files

All .conf files are actually shell script running inside our process, so you can (although probably shouldn't) do simple code in them. If you really, really wanted to, you could even replace our logic. Not a great idea, and not one we'd support, but we wouldn't actively try to stop you, either. Handy, nonetheless. This is a dev tool, so don't run it on configuration you don't trust, ok?

For every file like NAME.conf, there is an optional folder NAME.conf.d which can contain snippets of code. They must end .conf. These are sourced after the master NAME.conf. This folder doesn't have to exist - nor does NAME.conf. One or the other or both is allowed. Exploit this to avoid storing sensitive configuration details in source contol without sacrificing it all together for environmentally immutable config. (At this point, an aside to my fellow developers: monolithic configuration is painful for our admins. It is not source control friendly, it is not devops friendly and its not friendly to friends, period. Don't do it).

To make a configuration setting in a package, the format is:-

configure NAMESPACE KEY VALUE

Some KEYs are arrays. These can be configured as:-

configure NAMESPACE KEY VALUE1
configure NAMESPACE KEY VALUE2
…

Most settings have a default; this may be static, or it may be chosen at runtime based on your system configuration, presence of README.md, etc. Any defaults are place before the first configuration file, swaddling/swaddling.conf, is loaded.

A quick run down of the various configuration files:-

Namespaces

swaddle

This namespace is intended to be used in swaddling/swaddling.conf and swaddling/swaddle/package.conf. That said, you can override a value on a per package-variant basis by putting a setting into, say, swaddling/swaddle/deb/deb.conf.

As well as SPDX licence identifiers, we also support the values public-domain, unlicensed and licensed for non-Open Source works.

We try very hard to sign quietly and correctly. That said, getting gpg set up correctly is a beast - you might like to review Creating the perfect GPG keypair. We strongly encourage you to persist, for the benefit of us all. swaddle does not actively override any settings you put in gpg.conf (eg in $HOME/.gnupg/gpg.conf). We strongly advise you to prefer SHA512 in it, eg:-

force-mdc
default-preference-list SHA512 SHA384 SHA256 CAMELLIA256 CAMELLIA128 AES256 AES BZIP2 Uncompressed
s2k-cipher-algo CAMELLIA256
s2k-digest-algo SHA512
personal-cipher-preferences CAMELLIA256 CAMELLIA192 CAMELLIA128 AES256 AES192 AES
personal-digest-preferences SHA512
personal-compress-preferences BZIP2
cert-digest-algo SHA512

It'll make you less compatible, but, in today's Post-Snowden world, what's the value in compatible if it isn't very secure? (Aside, a rant: GPG, PGP and gnupg are just far too complex, far too brittle and far too obsessed with compatibility with 1996 to be effective security tooling. What is it with the security folks, that they create standards (PEM, TLS, etc) with incredible numbers of permutations and software tooling to match. One tiny knob of many set wrongly, and the whole is exposed. Great).

* $HOME is not necessarily the same as ~; it just usually is. Indeed, one can unset $HOME and ~ will still work.

† For those that don't know, hkp is a simple (but not REST friendly) wrapper around http. On a different port. hkps is https. Oh, and WTF? CentOS 5 is still supported until 2017 but runs a vulnerable gpg? Great one. So what is a 'critical update', then? And gpg, you're not gotting off from this scot free. You only added secure transmission of keys in 2009! I strongly suspect the keyservers are running some pretty exploitable software, but I digress… (anyone fancy writing an open source NGINX plugin to do this robustly)?

‡ Compatible with schemes expecting semantic versioning but not semantically versioned. Personally, I don't have a lot of time for semantic versioning; one man's compatible change is another man's head in his hands. What matters is that version numbers differ and monotonically increase with simple rules for knowing they're different.

swaddle_package

This namespace is intended to be used in swaddling/swaddle/package.conf. That said, you can override a value on a per package-variant basis by putting a setting into, say, swaddling/swaddle/deb/deb.conf.

swaddle_github

This namespace is intended to be used in swaddling/swaddling.conf and swaddling/swaddle/package.conf. Its settings allow the use of GitHub Releases.

If the file $HOME/.swaddle/github-personal-access-token doesn't exist, then GitHub Releases are disabled. This contains a GitHub OAUTH personal access token created from your repository settings. It is one line (no final line feed), 40 bytes in size.

* We decided not to let you specify the value directly, as even though you could put it in a conf snippet in, say swaddling/swaddling.conf.d/00-api-token.conf, and added that file to .gitignore, there's always the chance of slip up, isn't there? Been there and done that: in my case, after not copying my hidden files properly once, and then checking in what should have been excluded with a naive git add in a hurry.

swaddle_7z

This namespace is intended to be used in swaddling/swaddle/7z/7z.conf. If the folder swaddling/swaddle/7z is present, you'll get 7z archives created.

swaddle_tar

This namespace is intended to be used in swaddling/swaddle/tar/tar.conf. If the folder swaddling/swaddle/tar is present, you'll get tarballs created.

The available compressions, in rough order of compressive power, are:-

• none (.tar)
• lzop (.tar.lzo)
• gzip (turned up to 11 if possible)
• zlib (.tar.zz)
• bzip2 (.tar.bz2)
• lzma (.tar.lzma)
• xz (.tar.xz)
• lzip (.tar.lz)
• rzip (.tar.rz)
• lrzip (.tar.lrz)

swaddle_file

This namespace is intended to be used in swaddling/swaddle/file/file.conf. If the folder swaddling/swaddle/file is present, you'll get standalone files created. Obviously, these aren't really a package format, but the idea is to make it easy to create non-binary things (eg lists of dictionary words), combined patches and standalone shell scripts, etc.

If the path is empty, then the first matching path inside a skeleton or body is used.

The available compressions, in rough order of compressive power, are:-

• none
• lzop (.lzo)
• gzip (turned up to 11 if possible)
• zlib (.zz)
• bzip2 (.bz2)
• lzma (.lzma)
• xz (.xz)
• lzip (.lz)
• rzip (.rz)
• lrzip (.lrz)

swaddle_zip

This namespace is intended to be used in swaddling/swaddle/zip/zip.conf. If the folder swaddling/swaddle/zip is present, you'll get ZIP archives created.

swaddle_deb

This namespace is intended to be used in swaddling/swaddle/deb/deb.conf. If the folder swaddling/swaddle/deb is present, you'll get debian packages created.

* These values are also used by the apt repository code to supply defaults for any packages that don't have them. This is possible, because the apt repo can include packages not built by swaddle. † Refer to Debian Policy

Scripts

It is possible to create script for pre and post install actions, etc. To do this create a folder for the particular action under deb\scripts, and put a script snippet into it. There is not needed to put a shebang line (we run all scripts as #!/usr/bin/env sh). This is about the only thing one can be sure exists at install time without creating unnecessary dependencies that are user-inconvenient (eg depending on perl just to run an install script). Avoid bashisms in your scripts. Unfortunately, at this time, these script snippets can't use shellfire, but they could if there's demand for it.

Each folder is searched in glob-expansion-order for readable, non-empty regular files (or symlinks) ending in .sh. These are then concatenated together. If your script has a requirement on a particular package or program being in place before execution, use a pre_depends (see above). The folders are:-

• preinst
• postinst
• prerm
• postrm

swaddle_rpm

This namespace is intended to be used in swaddling/swaddle/rpm/rpm.conf. Its settings allow the use of GitHub Releases.

'dependencies' can be:-

• package names
• files
• expressions (such as greater than X, etc)

digest is restricted to this list:-

• sha512
• sha384
• sha256
• sha224
• sha1
• md5

RPM supports other digest types, but they're obsolete. Frankly, it's bad enough that we have to allow sha1 and md5.

compression is restricted to this list:-

• xz
• lzma
• bzip2
• gzip
• none

Interestingly, pigz -11 on a typical small RPM can often shave off another 10K…

category is restricted to this list

* By default, we also exclude everything in the list equivalent to rpm -ql filesytem, so it's unlikely you'll need to put anything in here.

Scripts

It is possible to create script for pre and post install actions, etc. To do this create a folder for the particular action, and put a script snippet into it. There is not needed to put a shebang line (we run all scripts as #!/usr/bin/env sh). This is about the only thing one can be sure exists at install time without creating unnecessary dependencies that are user-inconvenient (eg depending on perl just to run an install script). Avoid bashisms in your scripts. Unfortunately, at this time, these script snippets can't use shellfire, but they could if there's demand for it.

Each folder is searched in glob-expansion-order for readable, non-empty regular files (or symlinks) ending in .sh. These are concatenated together and inserted as a scriptlet into a RPM Spec file. If a folder is missing, no RPM scriptlet is generated. If there are readable, non-empty regular files (or symlinks) ending .depends, then these are processed in glob-expansion-order, and each line of each file becomes a scriptlet dependency of the form Requires(XXXX), where XXXX is either a package name (info) or package name predicated by version (info > 3.1). If a line is empty or starts with '#', it is ignored.

The folders are:-

Please note that the trigger-* folders are experimental and may change.

swaddle_apt

This namespace is intended to be used in swaddling/swaddling.conf. Its settings control apt repository creation.

Package Description Translations

If you want to prepare package description translations, then you can add them as PACKAGE.translation-CODE files (at outputPath/download/apt/COMPONENT/i18n). This is a semi-documented feature that might change, particularly as it not is not yet source control friendly.

When preparing apt translation files, Debian 6 and Ubuntu 10.04 (but not later versions of these distributions) require both the language code and sub-language code translations to exist. swaddle prepares these automatically for en (creating en_AU, en_CA, en_GB, en_US and en_ZA), fr (creating fr_FR), no (creating no_NB), pt (creating pt_BR) and zh (creating zh_CN, zh_HK and zh_TW). The implemented technique unfortunately overwrites any translation files you have prepared for these subcodes.

swaddle_yum

This namespace is intended to be used in swaddling/swaddling.conf. Its settings control yum repository creation.

swaddle_webs

This namespace is intended to be used in swaddling/swaddling.conf. Its settings control website creation.

digests may be any of:-

• md5
• sha1
• sha256
• sha384
• sha512

Known Bugs

• On Mac OS X, Homebrew's version of rpm (as of Sun Apr 19 20:33:06 2015 +0200 / Commit 9f9350d4f4c088f042ef4c3c4fadf9f948fb5d3b) does not work