Home

Awesome

level-bench

Benchmark abstract-level databases. Currently only suitable for use in Node.js.

level badge npm Node version Test Coverage Standard Common Changelog Donate

Example

npm i level-bench classic-level
npx level-bench run batch-put classic-level
npx level-bench run batch-put classic-level --b [ --keys seq ]
npx level-bench run batch-put classic-level --b [ --keys seqReverse ]
npx level-bench plot batch-put

Yields the following (showing that writing keys in reverse sequential order is fastest):

Example plot

Highlights

Usage

level-bench run <benchmark> [target]

Run a benchmark. The benchmark argument must be one of the named benchmarks listed below.

The target argument should be a path or an npm package name that is installed nearby (for example level-bench run put classic-level). It defaults to the current working directory. A package.json must exist alongside the resolved target. If the module doesn't have a default export and it has more than one named export, pass a --class or -c option to use the named export by that name (for example level-bench run put classic-level -c ClassicLevel).

If target does not create persistent databases (like memory-level) you must pass --mem.

Options for the db can be provided via --db <subargs>. For example --db [ --cacheSize 16mb ] or --db [ --valueEncoding json ]. Note that the brackets must be surrounded by spaces.

Benchmark-specific options can be provided via -b <subargs>. For example -b [ -n 1e6 --concurrency 1 ]. These options are listed below.

Results are by default written to .benchmarks/<benchmark>.<time>.csv and an accompanying JSON file for metadata. To write results to a custom file specify --out example.csv (-o for short). The metadata is used to derive a distinct benchmark name. When this doesn't suffice (for example because you're benchmarking a spinning disk versus an SSD, a fact that isn't included in the metadata) or when labels in the plot become too long, you can specify a custom name with --name example.

Examples

We can compare the performance of two git branches:

git checkout main && npm i
level-bench run put

git checkout wip && npm i
level-bench run put

Or check the overhead of a specific encoding:

level-bench run put level --db [ --valueEncoding utf8 ]
level-bench run put level --db [ --valueEncoding json ]

Or compare the effect of options:

level-bench run put classic-level
level-bench run put classic-level --db [ --no-compression ]

Then plot both (or more) runs with:

level-bench plot put

Options

Yet to document.

level-bench plot <benchmark> [files]

Plot the results using gnuplot (which must be installed and available in PATH). The files argument should be (glob patterns resolving to) CSV files as generated by level-bench run. If not provided, defaults to .benchmarks/<benchmark>.*.csv.

The plot is written to .benchmarks/<benchmark>.<time>.png by default. This can be overridden with --out <filename> (-o for short).

Options

Yet to document.

Benchmarks

put

Perform concurrent put() operations. Records the Simple Moving Average (SMA) of the duration of the last 1000 writes, as well as the Cumulative Moving Average (CMA) of the throughput in MB/s. Options:

Tips:

batch-put

Perform concurrent batch() operations. Same as put, but in batches rather than singular puts. Options:

get

Perform get() operations. Inserts -n sequential keys into the database, then reads them (in random order by default). Records the Simple Moving Average (SMA) of the duration of the last 1000 reads, as well as the Cumulative Moving Average (CMA) of the throughput in MB/s. Options:

iterate

Yet to document.

stream

Yet to document.

self-distribution

Not a benchmark, but a temporary cheat to reuse the tooling we have here to test (and visualize) some of the internals. Needs a valid target argument, same as real benchmarks, although that argument is not actually used.

Generate keys with a certain order and probability distribution. Options:

Example:

level-bench run self-distribution memory-level -b [ --distribution zipfian --skew 1 ]
level-bench run self-distribution memory-level -b [ --distribution zipfian --skew=-1 ]
level-bench run self-distribution memory-level -b [ --keys seq ]
level-bench plot self-distribution
<!-- ### Other ideas - Write batches in different sizes (feature: define a matrix) - Write while increasing the number of open iterators (and thus snapshots) -->

Limitations

The target abstract-level implementation must take a location as its first argument (if persistent) or ignore that argument (if transient). Options are passed to both the constructor with the signature (location, options) and to db.open(options, callback).

Contributing

Level/bench is an OPEN Open Source Project. This means that:

Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.

See the Contribution Guide for more details.

Donate

Support us with a monthly donation on Open Collective and help us continue our work.

License

MIT