Home

Awesome

BriefLZ - small fast Lempel-Ziv

Version 1.3.0

Copyright (c) 2002-2020 Joergen Ibsen

http://www.ibsensoftware.com/

BriefLZ CI codecov

About

BriefLZ is a small and fast open source implementation of a Lempel-Ziv style compression algorithm. The main focus is on speed and code footprint, but the ratios achieved are quite good compared to similar algorithms.

Why BriefLZ?

Two widely used types of Lempel-Ziv based compression libraries are those that employ entropy encoding to achieve good ratios (like Brotli, zlib, Zstd), and those that forgo entropy encoding to favor speed (like LZ4, LZO).

BriefLZ attempts to place itself somewhere between these two by using a universal code, which may improve compression ratios compared to no entropy encoding, while requiring little extra code.

Not counting the optional lookup tables, the compression function blz_pack is 147 LOC, and the decompression function blz_depack is 61 LOC (and can be implemented in 103 bytes of x86 machine code).

If you do not need the extra speed of libraries without entropy encoding, but want reasonable compression ratios, and the footprint of the compression or decompression code is a factor, BriefLZ might be an option.

Benchmark

Here are some results from running lzbench on the Silesia compression corpus on a Core i5-4570 @ 3.2GHz:

Compressor nameCompressionDecompress.Compr. sizeRatio
brotli 2019-10-01 -95.80 MB/s432 MB/s56,697,01526.75
zstd 1.4.3 -929 MB/s848 MB/s60,185,63728.40
brieflz 1.3.0 -91.95 MB/s415 MB/s63,947,75130.17
zstd 1.4.3 -3199 MB/s776 MB/s66,681,18231.46
brieflz 1.3.0 -617 MB/s400 MB/s67,208,42031.71
brotli 2019-10-01 -395 MB/s382 MB/s67,369,24431.79
zlib 1.2.11 -911 MB/s320 MB/s67,644,54831.92
zlib 1.2.11 -363 MB/s313 MB/s72,967,04034.43
brotli 2019-10-01 -1220 MB/s338 MB/s73,499,22234.68
zstd 1.4.3 -1345 MB/s880 MB/s73,508,82334.68
brieflz 1.3.0 -396 MB/s369 MB/s75,550,73635.65
zlib 1.2.11 -191 MB/s295 MB/s77,259,02936.45
lz4hc 1.9.2 -928 MB/s3417 MB/s77,884,44836.75
brieflz 1.3.0 -1164 MB/s362 MB/s81,138,80338.28
lzo1x 2.10 -1531 MB/s702 MB/s100,572,53747.45
lz4 1.9.2550 MB/s3444 MB/s100,880,80047.60

Please note that this benchmark is not entirely fair because BriefLZ has no window size limit.

Usage

The include file include/brieflz.h contains documentation in the form of doxygen comments. A configuration file is included, so you can simply run doxygen to generate documentation in HTML format.

If you wish to compile BriefLZ on 16-bit systems, make sure to adjust the constants BLZ_HASH_BITS and DEFAULT_BLOCK_SIZE.

When using BriefLZ as a shared library (dll on Windows), define BLZ_DLL. When building BriefLZ as a shared library, define both BLZ_DLL and BLZ_DLL_EXPORTS.

The example folder contains a simple command-line program, blzpack, that can compress and decompress a file using BriefLZ. For convenience, the example comes with makefiles for GCC and MSVC.

BriefLZ uses CMake to generate build systems. To create one for the tools on your platform, and build BriefLZ, use something along the lines of:

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --config Release

You can also simply compile the source files and link them into your project. CMake just provides an easy way to build and test across various platforms and toolsets.

License

This projected is licensed under the zlib License (Zlib).