Home

Awesome

Leveled - An Erlang Key-Value Store

Introduction

Build Status

Leveled is a simple Key-Value store based on the concept of Log-Structured Merge Trees, with the following characteristics:

The store has been developed with a <b>focus on being a potential backend to a Riak KV</b> database, rather than as a generic store. It is intended to be a fully-featured backend - including support for secondary indexes, multiple fold types and auto-expiry of objects.

An optimised version of Riak KV has been produced in parallel which will exploit the availability of HEAD requests (to access object metadata including version vectors), where a full GET is not required. This, along with reduced write amplification when compared to leveldb, is expected to offer significant improvement in the volume and predictability of throughput for workloads with larger (> 4KB) object sizes, as well as reduced tail latency.

There may be more general uses of Leveled, with the following caveats:

More Details

For more details on the store:

Feedback

Please create an issue if you have any suggestions. You can ping me <b>@masleeds</b> if you wish

Running Leveled

Unit and current tests in leveled should run with rebar3.

A new database can be started by running

{ok, Bookie} = leveled_bookie:book_start(StartupOptions)   

This will start a new Bookie. It will start and look for existing data files, under the RootPath, and start empty if none exist. Further information on startup options can be found here here.

The book_start method should respond once startup is complete. The leveled_bookie module includes the full API for external use of the store.

Running in Riak requires Riak 2.9 or beyond, which is available from January 2019.

There are two main branches under active development:

develop-3.4 - default: Target for the Riak 3.4 release with support for OTP 24 and OTP 26;

develop-3.1: Target for the Riak 3.2 release with support for OTP 22 and OTP 24.

There are two legacy branches, used in older versions of Riak:

develop-3.0: Used in the Riak 3.0 release with support for OTP 20 and OTP 22;

develop-2.9: Used in the Riak 2.9 release with support for OTP R16 through to OTP 20.

Contributing

In order to contribute to leveled, fork the repository, make a branch for your changes, and open a pull request. The acceptance criteria for updating leveled is that it passes rebar3 dialyzer, xref, eunit, and ct with 100% coverage.

To have rebar3 execute the full set of tests, run:

./rebar3 do xref, dialyzer, cover --reset, eunit --cover, ct --cover, cover --verbose

For those with a Quickcheck license, property-based tests can also be run using:

./rebar3 as eqc do eunit --module=leveled_simpleeqc, eunit --module=leveled_statemeqc