Home

Awesome

(this project is no longer maintained, please move to http://github.com/richox/orz)

libzling

fast and lightweight compression library and utility.

Introduction

Libzling is an improved lightweight compression utility and library. libzling uses fast order-1 ROLZ (16MB block size and 10MB dictionary size) followed with Huffman encoding, making it 3 times as fast as gzip on compressing, while still getting much better compression ratio and decompression speed.

Simple benchmark with enwik8(100,000,000 bytes), also on Large Text Compression Benchmark (thanks to Matt Mahoney)

CPU: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
MEM: 65726592 kB
OS:  Linux 2.6.32.43-tlinux-1.0.14-default
ToolEncodeDecodeCompressed Sizestatus
xz1m23.838s0m2.265s26375764PASS
bzip20m10.680s0m4.633s29008758PASS
libzling e40m4.058s0m0.989s29721410PASS
libzling e30m3.383s0m1.007s29999672PASS
libzling e20m2.860s0m1.021s30477912PASS
libzling e10m2.504s0m1.021s30855731PASS
libzling e00m2.208s0m1.043s31456189PASS
gzip0m6.502s0m1.089s36518322PASS

Build & Install

You can build and install libzling automatically by cmake with the following command:

cd ./build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/path/to/install
make
make install

Usage

Libling provides simple and lightweight interface. here is a simple program showing the basic usage of libzling. (compiled with g++ -Wall -O3 zling_sample.cpp -o zling_sample -lzling)

#include "libzling/libzling.h"

int main() {
    // compress
    {
        const int level = 0;  // valid levels: 0, 1, 2, 3, 4
        FILE* fin = fopen("./1.txt", "rb");
        FILE* fout = fopen("./1.txt.zlng", "wb");

        baidu::zling::FileInputter  inputter(fin);
        baidu::zling::FileOutputter outputter(fout);

        baidu::zling::Encode(&inputter, &outputter, level);
        fclose(fin);
        fclose(fout);
    }

    // decompress
    {
        FILE* fin = fopen("./1.txt.zlng", "rb");
        FILE* fout = fopen("./2.txt", "wb");

        baidu::zling::FileInputter  inputter(fin);
        baidu::zling::FileOutputter outputter(fout);

        baidu::zling::Decode(&inputter, &outputter);
        fclose(fin);
        fclose(fout);
    }
    return 0;
}

However libzling supports more complicated interface, see ./demo/zling.cpp for details.