Awesome

zarrs_tools

Various tools for creating and manipulating Zarr v3 data with the zarrs rust crate.

A changelog can be found here.

Tools

All tools support input and output of Zarr V3 data. Some tools additionally support input of a V3 compatible subset of Zarr V2.

• zarrs_reencode: reencode an array. Manipulate the chunk size, shard size, codecs, fill value, chunk key encoding separator, and attributes.
• zarrs_filter (feature filter): apply simple image filters (transformations) to an array.
• zarrs_ome (feature ome): convert an array to an OME-Zarr multi-scale image.
  • Supports OME-Zarr 0.5-dev (as Zarr V3) and 0.5-dev1. The first is recognised by Neuroglancer.
• zarrs_info (feature info): return metadata related info or the range/histogram of an array.
• zarrs_validate (feature validate): validate that two arrays are equivalent.
• zarrs_binary2zarr (feature binary2zarr): create an array from piped binary data.
• zarrs_ncvar2zarr (feature ncvar2zarr): convert a netCDF variable to an array.

See docs/ for tool documentation.

zarrs Benchmarking

• zarrs_reencode: suitable for round trip benchmarking.
• zarrs_benchmark_read_sync (feature benchmark): benchmark the zarrs sync API.
• zarrs_benchmark_read_async (feature benchmark): benchmark the zarrs async API.

See the LDeakin/zarr_benchmarks repository for benchmarks of zarrs against other Zarr V3 implementations.

Install

Prebuilt Binaries

# Requires cargo-binstall https://github.com/cargo-bins/cargo-binstall
cargo binstall zarrs_tools

Prebuilt binaries are not available on all platforms.

From crates.io

cargo install --all-features zarrs_tools

From source

cargo install --all-features --path .
# cargo install --all-features --git https://github.com/LDeakin/zarrs_tools

Enabling SIMD intrinsics

Encoding and decoding performance may be improved with avx2/sse2 enabled (if supported).

This can be enabled by compiling with either of:

• RUSTFLAGS="-C target-cpu=native"
• RUSTFLAGS="-C target-feature=+avx2,+sse2"

Enabling non-default zarrs codecs

Non-default zarrs codecs (see zarrs crate features) can be enabled by passing them as feature flags. For example:

cargo install zarrs_tools --all-features --features zarrs/bitround,zarrs/zfp,zarrs/bz2,zarrs/pcodec

Licence

zarrs_tools is licensed under either of

• the Apache License, Version 2.0 LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0 or
• the MIT license LICENSE-MIT or http://opensource.org/licenses/MIT, at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.