Awesome

clodl: self-contained dynamic libraries

clodl computes the closure of a shared object. That is, given an executable or shared library, it returns a single self-contained file packing all dependencies. Think of the result as a poor man's container image. Compared to containers:

closures do not provide isolation (e.g. separate process, network, filesystem namespaces),
but closures do allow for deploying to other machines without concerns about missing dependencies.

Clodl can be used to build binary closures or library closures.

A binary closure is made from an executable or a shared library defining symbol main and can be executed. In practice, the binary closure is a zip file appended to a script that uncompresses the file to a temporary folder and has main invoked.

A library closure is a zip file containing the shared libraries in the closure, and provides one or more top-level libraries which depends on all of the others. When the closure is uncompressed, these top-level libraries can be loaded into the address space of an existing process.

Executing a closure in the address space of an existing process enables lightweight high-speed interop between the closure and the rest of the process. The closure can natively invoke any function in the process without marshalling/unmarshalling any arguments, and vice versa.

Example of binary closure

clodl is implemented as a set of Bazel build rules. It integrates with your Bazel build system, e.g. as follows:

cc_binary(
  name = "hello-cc",
  srcs = ["main.c"],
  deps = ...
)

binary_closure(
  name = "hello-closure-bin",
  src = "hello-cc",
)

With Haskell:

haskell_binary(
    name = "hello-hs",
    srcs = ["src/test/haskell/hello/Main.hs"],
	...
)

binary_closure(
  name = "hello-closure-bin",
  src = "hello-hs",
)

The test BUILD file has complete examples.

Example of library closure

clodl is useful for "jarifying" native binaries. Provided shim Java code, closures can be packed inside a JAR and then loaded at runtime into the JVM. This makes JAR's an alternative packaging format to publish and deploy native binaries.

cc_binary(
  name = "libhello.so",
  srcs = ["main.c"],
  linkshared = 1,
  deps = ...
)

library_closure(
  name = "hello-closure",
  srcs = ["libhello.so"],
)

java_binary(
  name = "hello-jar",
  classpath_resources = [":hello-closure"],
  main_class = ...,
  srcs = ...,
  runtime_deps = ...,
)

Importing clodl

In the WORKSPACE file:

http_archive(
    name = "io_tweag_clodl",
    sha256 = "1181131b0fc111a1f16f0532605e9835e308ac5bc278b62f825adb0844ff7590",
    strip_prefix = "clodl-0a7a2f93f4043a2db623f7d820578e3baea228d1",
    urls = ["https://github.com/tweag/clodl/archive/0a7a2f93f4043a2db623f7d820578e3baea228d1.tar.gz"],
)

...

In BUILD files:

load(
    "@io_tweag_clodl//clodl:clodl.bzl",
    "binary_closure",
    "library_closure",
)

...

Building it

Requirements:

The Bazel build tool;
the Nix package manager;
in Linux, the scanelf tool from the pax-utils package and the patchelf tool;
in OSX, otool and install_name_tool.

To build and test:

$ bazel build //...
$ (cd tests; bazel run hello-java)

License

clodl is free software, and may be redistributed under the terms specified in the LICENSE file.

About

clodl is maintained by Tweag I/O.

Have questions? Need help? Tweet at @tweagio.