Home

Awesome

The Bento-box meta-linker

This is the home of the bento-linker, a tool for generating glue for bento-boxes.

!! Note: This project is an expirement with no promise of support. Use at your own risk. !!

What are Bento-boxes?

Bento-boxes are independently-linked, memory-isolated pieces of code that are designed to work together. You can think of them as a light-weight alternative to processes for microcontrollers.

<br/>

bento-boxes

<br/>

Unlike processes, Bento-boxes don't require mutlithreading or virtual memory. Instead of files and pipes, boxes communicate using type-rich inter-box-communication (IBC) mechanisms that behave like familiar C functions with a few limitations.

A Bento-box is described by two things:

  1. A set of memory regions + data
  2. Function-like imports and exports

When provided by a config that describes these boxes, the bento-linker (this tool) can generate a variety of different glue that enables some pretty powerful features:

  1. Automatic handling of box state bringing up/down boxes as needed. This enables RAM-sharing, compressed boxes, or even storing boxes on external storage.

  2. Component-level firmware updates. With clearly defined memory regions and standard IBC mechanisms, it's easy to update a single part of the system. This reduces risk and bandwidth cost for rolling out bug-fixes or new features in the field.

  3. Hardware or software enforced memory isolation. No more losing the entire device because your CBOR parser had a buffer overflow. If a rogue box tries to escape its allocated memory regions it is killed and an error is politely returned to the caller.

And some less powerful but nice features:

  1. Programming language interoperability. The requirements for imports and exports are the same as the requirements for FFI and can be automatically generated.

  2. A good target for LTO, balancing optimizations and build time.

If you noticed that these sound awfully similar to WebAssembly modules, you'd be right! On of our goals with this project is to take the ideas present in WebAssembly and generalize them for other types of binaries and runtimes.

Another important thing to note, these features are all optional. Bento-boxes is a framework for generating glue based on these assumptions, but it's completely valid to generate an "insecure" box without enforced memory isolation. We've intentionally chosen rules that can be reduced to a simple table of function pointers. We've intentionally made the rules flexible to help with incremental adoption on systems with limited resources.

The tool

At a high-level, the bento-linker takes in a description of bento-boxes, and generates a set of outputs that can be used to build the final system.

Note that the bento-linker does not actually replace the build system's linker or even the build system itself. Our goal is for it to be easy to add the bento-linker to existing build systems without disruptive changes.

Commands and configuration

The bento-linker is a python3 program and can be installed using pip. Note that there has been very little testing outside of a Linux environment.

pip install -e .
bento -h

Once installed, you should be able to run bento to see a list of commands.

Really the only important command is bento build. This takes in the box's configuration and generates the requested outputs. At the time of writing, all of the other commands are only informative, listing various metadata about the evaluated bento-boxes. Feel free to play around.

So how do you actually describe the bento-box configuration?

The bento-box config is a rich set of key-value options. Each option can be specified and overwritten on the command line, however you are more likely to store a bento-box's configuration in a recipe.toml file. The bento-linker will automatically pick up recipe.toml files in each of the box's directories, or you can specify the configuration of nested boxes with additional box config section.

Storing a separate recipe.toml per box can be useful when boxes live in different repos, however we've used a single recipe.toml per system here and in the examples since it's easier to show.

You can find a full (and up to date!) list of every single configuration option by running bento options.

Recipes

Here's what a recipe.toml might look like:

name = 'box1'
runtime = 'armv7m-mpu'
stack = 0x800
heap = 0x800

output.ld = 'bb.ld'
output.h = 'bb.h'
output.c = 'bb.c'
output.mk = 'Makefile'

memory.flash = 'r-xp 0x000fe000-0x000fffff'
memory.ram   = 'rw-- 0x2003e000-0x2003ffff'

export.box1_hello = 'fn() -> err'
export.box1_add2 = 'fn(i32, i32) -> i32'
import.sys_write = 'fn(const u8 buffer[size], usize size) -> errsize'

There's a lot to unpack here, so lets take a quick look at each section:

name = 'box1'
runtime = 'armv7m-mpu'
stack = 0x800
heap = 0x800

Each box has a number of ungrouped configuration options that determine what the bento-linker actually generates. This is where you can configure which runtime or loader to use, as well as a number of parameters. You can run bento options to see what is available.

output.ld = 'bb.ld'
output.h = 'bb.h'
output.c = 'bb.c'
output.mk = 'Makefile'

Each box has a set of outputs. These are the actual files where the bento-linker generates glue.

The full list of possible outputs can be listed with the bento outputs command.

Note that every output is optional. They are intended to be consumed by the developer's build system but each output can be omitted without side-effects.

memory.flash = 'r-xp 0x000fe000-0x000fffff'
memory.ram   = 'rw-- 0x2003e000-0x2003ffff'

Each box has a set of memory regions. These are described by a set of mode flags and address range. In additional to the traditional read (r), write (w), and execute (x) flags, there is an additiona persist (p) flag used to indicate if a memory is persistant across power-cycles.

Note, if you only provide a size (r-xp 0x2000), the bento-linker will automatically allocate the appropriate memory from the containing box. However this only works if the description of the containing box is present.

Also note, you can name the memory regions anything you like.

export.box1_hello = 'fn() -> err'
export.box1_add2 = 'fn(i32, i32) -> i32'
import.sys_write = 'fn(const u8 buffer[size], usize size) -> errsize'

And last but not definitely not least, each box has a set of imports and exports. These are described by a function-like type declaration.

There are a number of argument/return types that can be used:

For more examples of recipe.tomls look at the examples! There are a number of fully functional recipe.toml files in the examples directory.

API

So once we've generated some glue, what does this give us?

The actual imports and exports are hopefully quite simple. Give the above recipe.toml, the bb.h file will be generated with this function:

ssize_t sys_write(const void *buffer, size_t size);

Additionally, bb.h will include declarations of the exports for typechecking purposes:

int box1_hello(void);
int32_t box1_add2(int32_t, int32_t);

Compile and link, and as long as you follow the rules you should be able to call between boxes without extra work.

There are also a couple extra convenience functions for interacting with the boxes.

From outside a box:

From inside a box:

By default, the bento-linker also tries to tie these into the language's stdlib so that common functionality such as printf/assert should be behave as expected.

The glue

Runtimes

The available runtimes can be listed with the bento runtimes command. These determine how each box is executed.

Loaders

The available loaders can be listed with the bento loaders command. These determine how each box is loaded before execution.

The output

Examples

You can find a number of examples in the examples directory.

Most of these examples run on an nrf52840, except the armv8m examples, which use an nrf5340.

If the example dependencies are available and a gdb-server is running, you should be able to run each example with:

bento build && make build flash reset

Example dependencies

The examples have a number of dependencies, these can be found in the extras directory.

Note, some of the extra libraries take a while to compile (think LLVM). You may consider instead downloading their recent releases here:

Testing

Currently there are only tests for configuration -> compilation. These can be found in the tests directory. You can run these with pytest:

Note that the example compilation tests require all of the example dependencies to be available.

pytest -v

Results

Unfortunately, we don't have exhaustive benchmarks across all of the runtimes. (If anyone does this, let us know).

But, to start the discussion and prove the validity of these approaches, I've put together measurements of some the above examples:

Runtime (in ns):

helloqsortmbrotmazelittlefs
native271 ns84287 ns2186562 ns3118464 ns3850311 ns
mpu463 ns84851 ns2275587 ns3203449 ns6654519 ns
awsm (wrap)9527 ns75256 ns1979281 ns5106498 ns5320550 ns
awsm8508 ns78896 ns1988887 ns6823538 ns6150274 ns
wamr (aot)28440 ns132688 ns5733677 ns12368716 ns17249899 ns
wamr93229 ns1551582 ns6608421 ns121541011 ns103575732 ns
wasm3101678 ns2079359 ns8844360 ns157946032 ns114494192 ns

runtime-comparison

Code size (in bytes):

helloqsortmbrotmazelittlefs
native7616 B5228 B8532 B16052 B22944 B
mpu7900 B5896 B9216 B16340 B23228 B
awsm (wrap)14152 B5268 B12368 B25672 B43428 B
awsm14664 B5280 B12240 B27208 B47524 B
wamr (aot)39772 B30828 B36816 B176156 B102848 B
wamr53160 B47567 B50117 B63582 B77792 B
wasm372000 B66143 B68725 B82382 B96512 B

codesize-comparison

RAM lower-bound (in bytes):

helloqsortmbrotmazelittlefs
native304 B40772 B6772 B43940 B1468 B
mpu540 B40860 B6908 B44240 B1496 B
awsm (wrap)504 B40692 B7000 B44652 B3712 B
awsm500 B40692 B7000 B44672 B3664 B
wamr (aot)15348 B44700 B17796 B43084 B77988 B
wamr19028 B43660 B19564 B48116 B91668 B
wasm318588 B45968 B19048 B49188 B94176 B

ram-comparison

Some notes:

Extending the bento-linker

At its core, the bento-linker is a framework that matches bento-box configuration to an extendable set of runtimes, loaders, and outputs.

Each piece of this framework is described by a Python class in their respective directories:

The best place to get started would be to look at the existing classes and start from the there.

Additionally, there are a set of glue classes in bento/glue, which provide some of the generic glue code that is common across all runtimes.

Related projects