Awesome
LibDeflate.jl
This package provides Julia bindings for libdeflate.
Libdeflate is a heavily optimized implementation of the DEFLATE compression algorithm used in the zip, bgzip and gzip formats. Unlike libz or gzip, libdeflate does not support streaming, and so is intended for use in of files that fit in-memory or for block-compressed files like bgzip. But it is significantly faster than either libz or gzip.
This package provides simple functionality for working with raw DEFLATE payloads, zlib and gzip data. It is intended for internal use by other packages, not to be used directly by users. Hence, its interface is somewhat small.
:warning: This package ONLY works with in-memory buffers, and ONLY buffers with a length < 2^32 bytes :warning:
Interface
Many functions have a "safe" and an "unsafe" variant. The unsafe works with pointers, the safe attempts to convert Julia objects to ReadableMemory
or WriteableMemory
, which are simply structs containing pointers.
When possible, use the safe variants as the overhead is rather small.
For more details on these functions, read their docstrings which define their API. Functions and types without a docstring are internal.
No functions here are expected to throw errors. On error, they return a LibDeflateError
object.
Common exported types
Decompressor
: Create an object that decompresses using DEFLATE.Compressor(N)
: Create an object that compresses using DEFLATE levelN
.LibDeflateError
: An enum will all LibDeflate errors. Functions are either successful or return this.ReadableMemory
: A pointer and a length. Constructable from types that are pointer-readable.WriteableMemory
: A pointer and a length. Constructable from types that are pointer-writeable.
Working with DEFLATE payloads
(unsafe_)decompress!
: DEFLATE decompress payload.(unsafe_)compress!
: DEFLATE compress payload
Working with gzip files
-
(unsafe_)gzip_decompress!
: Decompress gzip data. -
(unsafe_)gzip_compress!
: Compress gzip data and/or metadata -
(unsafe_)parse_gzip_header
: Parse out gzip header -
is_valid_extra_data
: Check if some bytes are valid metadata for the gzip "extra" field.
Working with Libz files
(unsafe_)zlib_decompress!
: Decompress zlib data.(unsafe_)zlib_compress!
: Compress zlib data
Miscellaneous
(unsafe)_crc32
: Compute the crc32 checksum of the bytes atdata
. Note that this is not the same algorithm ascrc32c
.(unsafe)_adler32
: Compute the Adler32 checksum of the bytes atdata
.