Home

Awesome

Asmble

Asmble is a compiler that compiles WebAssembly code to JVM bytecode. It also contains an interpreter and utilities for working with WASM code from the command line and from JVM languages.

Features

Quick Start

WebAssembly by itself does not have routines for printing to stdout or any external platform features. For this example we'll use the test harness used by the spec. Java 8 must be installed.

Download the latest TAR/ZIP from the releases area and extract it to asmble/.

WebAssembly code is either in a binary file (i.e. .wasm files) or a text file (i.e. .wast files). The following code imports the print function from the test harness. Then it creates a function calling print for the integer 70 and sets it to be called on module init:

(module
  (import "spectest" "print_i32" (func $print (param i32)))
  (func $print70 (call $print (i32.const 70)))
  (start $print70)
)

Save this as print-70.wast. Now to run this, execute:

./asmble/bin/asmble run -testharness print-70.wast

The result will be:

70 : i32

Which is how the test harness prints an integer. See the examples directory for more examples.

CLI Usage

Assuming Java 8 is installed, download the latest release and extract it. The asmble command is present in the asmble/bin folder. There are multiple commands in Asmble that can be seen by executing asmble with no commands:

Usage:
  COMMAND options...

Commands:
  compile - Compile WebAssembly to class file
  help - Show command help
  invoke - Invoke WebAssembly function
  run - Run WebAssembly script commands
  translate - Translate WebAssembly from one form to another

For detailed command info, use:
  help COMMAND

Some of the commands are detailed below.

Compiling

Running asmble help compile:

Command: compile
Description: Compile WebAssembly to class file
Usage:
  compile <inFile> [-format <inFormat>] <outClass> [-out <outFile>]

Args:
  <inFile> - The wast or wasm WebAssembly file name. Can be '--' to read from stdin. Required.
  -format <inFormat> - Either 'wast' or 'wasm' to describe format. Optional, default: <use file extension>
  -log <logLevel> - One of: trace, debug, info, warn, error, off. Optional, default: warn
  <outClass> - The fully qualified class name. Required.
  -out <outFile> - The file name to output to. Can be '--' to write to stdout. Optional, default: <outClass.class>

This is used to compile WebAssembly to a class file. See the compilation details for details about how WebAssembly translates to JVM bytecode. The result will be a .class file containing JVM bytecode.

NOTE: There is no runtime required with the class files. They are self-contained.

Invoking

Running asmble help invoke:

Command: invoke
Description: Invoke WebAssembly function
Usage:
  invoke [-in <inFile>]... [-reg <registration>]... [-mod <module>] [<export>] [<arg>]...

Args:
  <arg> - Parameter for the export if export is present. Multiple allowed. Optional, default: <empty>
  -defmaxmempages <defaultMaxMemPages> - The maximum number of memory pages when a module doesn't say. Optional, default: 5
  <export> - The specific export function to invoke. Optional, default: <start-func>
  -in <inFile> - Files to add to classpath. Can be wasm, wast, or class file. Named wasm/wast modules here are automatically registered unless -noreg is set. Multiple allowed. Optional, default: <empty>
  -log <logLevel> - One of: trace, debug, info, warn, error, off. Optional, default: warn
  -mod <module> - The module name to run. If it's a JVM class, it must have a no-arg constructor. Optional, default: <last-in-entry>
  -noreg - If set, this will not auto-register modules with names. Optional.
  -reg <registration> - Register class name to a module name. Format: modulename=classname. Multiple allowed. Optional, default: <empty>
  -res - If there is a result, print it. Optional.
  -testharness - If set, registers the spec test harness as 'spectest'. Optional.

This can run WebAssembly code including compiled .class files. For example, put the following WebAssembly at add-20.wast:

(module
  (func (export "doAdd") (param $i i32) (result i32)
    (i32.add (get_local 0) (i32.const 20))
  )
)

This can be invoked via the following with the result shown:

asmble invoke -res -in add-20.wast doAdd 100

That will print 120. However, it can be compiled first like so:

asmble compile add-20.wast MyClass

Now there is a file called MyClass.class. Since it has a no-arg constructor because it doesn't import anything (see compilation details below), it can be invoked as well:

asmble invoke -res -in MyClass.class -reg myMod=MyClass -mod myMod doAdd 100

Note, that any Java class can be registered for the most part. It just needs to have a no-arg consstructor and any referenced functions need to be public, non-static, and with return/param types of only int, long, float, or double.

Translating

Running asmble help translate:

Command: translate
Description: Translate WebAssembly from one form to another
Usage:
  translate <inFile> [-in <inFormat>] [<outFile>] [-out <outFormat>]

Args:
  -compact - If set for wast out format, will be compacted. Optional.
  <inFile> - The wast or wasm WebAssembly file name. Can be '--' to read from stdin. Required.
  -in <inFormat> - Either 'wast' or 'wasm' to describe format. Optional, default: <use file extension>
  -log <logLevel> - One of: trace, debug, info, warn, error, off. Optional, default: warn
  <outFile> - The wast or wasm WebAssembly file name. Can be '--' to write to stdout. Optional, default: --
  -out <outFormat> - Either 'wast' or 'wasm' to describe format. Optional, default: <use file extension or wast for stdout>

Asmble can translate .wasm files to .wast or vice versa. It can also translate .wast to .wast which has value because it resolves all names and creates a more raw yet deterministic and sometimes more readable .wast. Technically, it can translate .wasm to .wasm but there is no real benefit.

All Asmble is doing internally here is converting to a common AST regardless of input then writing it out in the desired output.

Programmatic Usage

Asmble is written in Kotlin but since Kotlin is a thin layer over traditional Java, it can be used quite easily in all JVM languages.

Getting

The compiler and annotations are deployed to Maven Central. The compiler is written in Kotlin and can be added as a Gradle dependency with:

compile 'com.github.cretz.asmble:asmble-compiler:0.3.0'

This is only needed to compile of course, the compiled code has no runtime requirement. The compiled code does include some annotations (but in Java its ok to have annotations that are not found). If you do want to reflect the annotations, the annotation library can be added as a Gradle dependency with:

compile 'com.github.cretz.asmble:asmble-annotations:0.3.0'

Building and Testing

To manually build, clone the repository:

git clone --recursive https://github.com/cretz/asmble

The reason we use recursive is to clone the spec submodule we have embedded at src/test/resources/spec. Unlike many Gradle projects, this project chooses not to embed the Gradle runtime library in the repository. To assemble the entire project with Gradle installed and on the PATH (tested with 4.6), run:

gradle :compiler:assembleDist

Library Notes

The API documentation is not yet available at this early stage. But as an overview, here are some useful classes and packages:

Note, some code is not complete yet (e.g. a linker and javax.script support) but beginnings of the code still appear in the repository.

And for those reading code, here are some interesting algorithms:

Compilation Details

Asmble does its best to compile WASM ops to JVM bytecodes with minimal overhead. Below are some details on how each part is done. Every module is represented as a single class. This section assumes familiarity with WebAssembly concepts.

Constructors

Asmble creates different constructors based on the memory requirements. Each constructor created contains the imports as parameters (see imports below)

If the module does not define memory, a single constructor is created that accepts all other imports. If the module does define memory, two constructors are created: one accepting a memory instance, and an overload that instead accepts an integer value for max memory that is used to create the memory instance before sending to the first one. If the maximum memory is given for the module, a third constructor is created without any memory parameters and just calls the max memory overload w/ the given max memory value. All three of course have other imports as the rest of the parameters.

After all other constructor duties (described in sections below), the module's start function is called if present.

Memory

Memory is built or accepted in the constructor and is stored in a field. The current implementation uses a ByteBuffer. Since ByteBuffers are not dynamically growable, the max memory is an absolute max even though there is a limit which is adjusted on grow_memory. Any data for the memory is set in the constructor.

Table

In the WebAssembly MVP a table is just a set of function pointers. This is stored in a field as an array of MethodHandle instances. Any elements for the table are set in the constructor.

Globals

Globals are stored as fields on the class. A non-import global is simply a field that is final if not mutable. An import global is a MethodHandle to the getter and a MethodHandle to the setter if mutable. Any values for the globals are set in the constructor.

Imports

The constructor accepts all imports as params. Memory is imported via a ByteBuffer param, then function imports as MethodHandle params, then global imports as MethodHandle params (one for getter and another for setter if mutable), then a MethodHandle array param for an imported table. All of these values are set as fields in the constructor.

Exports

Exports are exported as public methods of the class. The export names are mangled to conform to Java identifier requirements. Function exports are as is whereas memory, global, and table exports have the name capitalized and are then prefixed with "get" to match Java getter conventions.

Exports are always separate methods instead of just changing the name of an existing method or field. This encapsulation allows things like many exports for a single item.

Types

WebAssembly has 4 types: i32, i64, f32, and f64. These translate quite literally to int, long, float, and double respectively.

Control Flow Operations

Operations such as unreachable (which throws) behave mostly as expected. Branching and looping are handled with jumps. The problem that occurs with jumping is that WebAssembly does not require compiler writers to clean up their own stack. Therefore, if the WASM ops have extra stack values, we pop it before jumping which has performance implications but not big ones. For most sane compilers, the stack will be managed stringently and leftover stack items will not be present.

Luckily, br_table jumps translate literally to JVM table switches which makes them very fast. There is a special set of code for handling really large tables (because of Java's method limit) but this is unlikely to affect most in practice.

Call Operations

Normal call operations do different things depending upon whether it is an import or not. If it is an import, the MethodHandle is retrieved from a field and called via invokeExact. Otherwise, a normal invokevirtual is done to call the local method.

A call_indirect is done via invokedynamic on the JVM. Specifically, invokedynamic specifies a synthetic bootstrap method that we create. It does a one-time call on that bootstrap method to get a MethodHandle that can be called in the future. We wouldn't normally have to use invokedynamic because we could use the index to reference a MethodHandle in the array field. However, in WebAssembly, that index is after the parameters of the call and the stack manipulation we would have to do would be far too expensive.

So we need a MethodHandle that takes the params of the target method, and then the index, to make the call. But we also need "this" because it is expected at some point in the future that the table field could be changed underneath and we don't want that field reference to be cached via the one-time bootstrap call. We do this with a synthetic bootstrap method which uses some MethodHandle trickery to manipulate it the way we want. This makes indirect calls very fast, especially on successive invocations.

Parametric Operations

A drop translates literally to a pop. A select translates to a conditional swap, then a pop.

Variable Access

Local variable access translates fairly easily because WebAssembly and the JVM treat the concept of parameters as the initial locals similarly. Granted the JVM form has "this" at slot 0. Also, WebAssembly doesn't treat 64-bit vars as 2 slots like the JVM, so some simple math is done like it is with the stack.

WebAssembly requires all locals the assume they are 0 whereas the JVM requires locals be set before use. An algorithm in Asmble makes sure that locals are set to 0 before they are fetched in any situation where they weren't explicitly set first.

Global variable access depends on whether it's an import or not. Imports call getter MethodHandles whereas non-imports simply do normal field access.

Memory Operations

Memory operations are done via ByteBuffer methods on a little-endian buffer. All operations including unsigned operations are tailored to use specific existing Java stdlib functions.

As a special optimization, we put the memory instance as a local var if it is accessed a lot in a function. This is cheaper than constantly fetching the field.

Number Operations

Constants are simply ldc bytecode ops on the JVM. Comparisons are done via specific bytecodes sometimes combined with JVM calls for things like unsigned comparison. Operators use idiomatic JVM approaches as well.

The WebAssembly spec requires a runtime check of overflow during trunc calls. This is enabled by default in Asmble. It defers to an internal synthetic method that does the overflow check. This can be programmatically disabled for better performance.

Stack

Asmble maintains knowledge of types on the stack during compilation and fails compilation for any invalid stack items. This includes the somewhat complicated logic concerning unreachable code.

In several cases, Asmble needs something on the stack that WebAssembly doesn't, such as "this" before the value of a putfield call when setting a non-import global. In order to facilitate this, Asmble does a preprocessing of the instructions. It builds the stack diffs and injects the needed items (e.g. a reference to the memory class for a load) at the right place in the instruction list to make sure they are present when needed.

As an unintended side effect of this kind of logic, it turns out that Asmble never needs local variables beyond what WebAssembly specifies. No temp variables or anything. It could be argued however that the use of temp locals might make some of the compilation logic less complicated and could even improve runtime performance in places where we overuse the stack (e.g. some places where we do a swap).

Caveats

Below are some performance and implementation quirks where there is a bit of an impedance mismatch between WebAssembly and the JVM:

FAQ

Why?

I like writing compilers and I needed a sufficiently large project to learn Kotlin really well to make a reasonable judgement on it. I also wanted to become familiar w/ WebAssembly. I don't really have a business interest for this and therefore I cannot promise it will forever be maintained.

Will it work on Android?

I have not investigated. But I do use invokedynamic and MethodHandle so it would need to be a modern version of Android. I assume, then, that both runtime and compile-time code might run there. Experiment feedback welcome.

What about JVM to WASM?

This is not an immediate goal of this project, at least not until the WASM GC proposal has been accepted. In the meantime, there is https://github.com/konsoletyper/teavm

So I can compile something in C via Emscripten and have it run on the JVM with this?

Yes, but work is required. WebAssembly is lacking any kind of standard library. So Emscripten will either embed it or import it from the platform (not sure which/where, I haven't investigated). It might be a worthwhile project to build a libc-of-sorts as Emscripten knows it for the JVM. Granted it is probably not the most logical approach to run C on the JVM compared with direct LLVM-to-JVM work.

Debugging?

Not yet, once source maps get standardized I may revisit.

TODO