Home

Awesome

lz4.js

LZ4 WASM, ASM.js and CLI for browser and Node with almost native performance. This package includes WASM, ASM.js module and variant that can be used in CLI.

Install and Use

html

<script src="node_modules/lz4/dist/lz4wasm.js"></script>

OR

<script src="node_modules/lz4/dist/lz4asm.js"></script>

npm

npm install lz4-asm

npm global

npm install lz4-asm -g
lz4-asm -h

WASM Module modes

Asynchronouse module mode ( default )

By default module will be exposed as UMD module. In browsers will be defined as window.lz4init Since WASM modules can't be easily loaded synchronously in browsers some setup for initialization have to be proceed:

In short all this flow can be described with next code example:

  const lz4module = lz4init();
  lz4module.ready.then((lz4) => {
    const lz4js = lz4.lz4js;
  })

Or even shorter with async/await and Object destructuring

  const { lz4js } = await lz4init().ready;

Synchronouse module mode

First of all to compile sources in synchronous module mode - set -s WASM_ASYNC_COMPILATION=0 option in gulp/emscripten.js file for DEV_ARGS variable.

Then in node env it can be accessed easily const lz4 = lz4init(); that's it. BUT! There is a big issue in browsers because WASM modules are restricted for synchronous load in main thread. There are two ways to load it, both require to pass argument into init function lz4init({ wasmBinary: wasmModuleAsArrayBuffer }); where wasmModuleAsArrayBuffer - preloaded file as ArrayBuffer in Uint8Array view representation:

  1. Manually load/preload module through XHR/fetch, etc. then transform to arrayBuffer and call
fetch('PATH_TO_WASM_FILE')
  .then(response => response.arrayBuffer())
  .then(wasmModuleAsArrayBuffer => lz4init({ wasmBinary: wasmModuleAsArrayBuffer }))
  .catch(err => {
    throw new Error('Failed to load WASM module');
  })
  1. Another variant - to encode .wasm module as base64, include in bundle, on runtime decode base64 to arrayBuffer and init
function _base64ToArrayBuffer(base64) {
    const binary_string = window.atob(base64);
    const binary_length = binary_string.length;
    const bytes = new Uint8Array(binary_length);
    for (var i = 0; i < binary_length; i++) {
        bytes[i] = binary_string.charCodeAt(i);
    }
    return bytes.buffer;
}

const wasmModuleAsArrayBuffer = _base64ToArrayBuffer(wasmAsBase64);
const lz4 = lz4init({ wasmBinary: wasmModuleAsArrayBuffer });

ASM.js Module modes

Both async (default) and sync modes available for ASM.js module but with one important difference. For sync variant there NO NEED for any extra setup. Set -s WASM_ASYNC_COMPILATION=0 option in gulp/emscripten.js and simply use as:

const lz4 = lz4init();

It is much easier operate with ASM.js variant is sync mode, but for some cases it operate slower than WASM variant. You can see difference in tests if target different variants in top require statement.

API

static values

lz4.compress(source, options)

compress to a lz4 buffer.

More about options you can find at LZ4 frame description: https://github.com/lz4/lz4/blob/master/doc/lz4_Frame_format.md

lz4.decompress(source)

decompress a lz4 buffer.

lz4.createCompressStream(options)

create a nodejs transform stream.

More about options you can find at LZ4 frame description: https://github.com/lz4/lz4/blob/master/doc/lz4_Frame_format.md

lz4.createDecompressStream()

create a nodejs transform stream.

Development

Require

Init

Clone the repo.

git clone https://github.com/ukyo/lz4.js.git

Install the dev dependencies.

cd path/to/lz4.js
npm install

Download the original LZ4 repo and compile for development.

npx gulp init

Write code

Watch for code updates and run tests.

npx gulp watchDev

Run development tests

Release

Release build.

npx gulp release

List of available gulp commands

buildLib
cleanTest
cleanRelease
compileDev
compileAsmRelease
compileWasmRelease
concatDev
concatAsmRelease
concatWasmRelease
debugTests
fetchLib
rollup
runTests
watchDev

initTask
release
releaseAsm
releaseWasm
testDev