Home

Awesome

classic-level

An abstract-level database backed by LevelDB. The successor to leveldown with builtin encodings, sublevels, events, hooks and support of Uint8Array. If you are upgrading, please see UPGRADING.md.

:pushpin: What is abstract-level? Head on over to Frequently Asked Questions.

level badge npm Node version Test Coverage Standard Common Changelog Donate

Usage

const { ClassicLevel } = require('classic-level')

// Create a database
const db = new ClassicLevel('./db', { valueEncoding: 'json' })

// Add an entry with key 'a' and value 1
await db.put('a', 1)

// Add multiple entries
await db.batch([{ type: 'put', key: 'b', value: 2 }])

// Get value of key 'a': 1
const value = await db.get('a')

// Iterate entries with keys that are greater than 'a'
for await (const [key, value] of db.iterator({ gt: 'a' })) {
  console.log(value) // 2
}

Usage from TypeScript requires generic type parameters.

<details><summary>TypeScript example</summary>
// Specify types of keys and values (any, in the case of json).
// The generic type parameters default to ClassicLevel<string, string>.
const db = new ClassicLevel<string, any>('./db', { valueEncoding: 'json' })

// All relevant methods then use those types
await db.put('a', { x: 123 })

// Specify different types when overriding encoding per operation
await db.get<string, string>('a', { valueEncoding: 'utf8' })

// Though in some cases TypeScript can infer them
await db.get('a', { valueEncoding: db.valueEncoding('utf8') })

// It works the same for sublevels
const abc = db.sublevel('abc')
const xyz = db.sublevel<string, any>('xyz', { valueEncoding: 'json' })
</details>

Supported Platforms

We aim to support Active LTS and Current Node.js releases, Electron >= 30, as well as any future Node.js and Electron releases thanks to Node-API.

The classic-level npm package ships with prebuilt binaries for popular 64-bit platforms as well as ARM, M1, Android, Alpine (musl), Windows 32-bit, Linux flavors with an old glibc (2.28) and is known to work on:

When installing classic-level, node-gyp-build will check if a compatible binary exists and fallback to compiling from source if it doesn't. In that case you'll need a valid node-gyp installation.

If you don't want to use the prebuilt binary for the platform you are installing on, specify the --build-from-source flag when you install:

npm install classic-level --build-from-source

If you are working on classic-level itself and want to recompile the C++ code, run npm run rebuild.

API

The API of classic-level follows that of abstract-level with a few additional methods and options that are specific to LevelDB. The documentation below only covers the differences.

db = new ClassicLevel(location[, options])

Create a database or open an existing database. The location argument must be a directory path (relative or absolute) where LevelDB will store its files. If the directory does not yet exist (and options.createIfMissing is true) it will be created recursively. Options are the same as in abstract-level except for the additional options accepted by db.open() and thus by this constructor.

A classic-level database obtains an exclusive lock. If another process or instance has already opened the underlying LevelDB store at the same location then opening will fail with error code LEVEL_LOCKED.

Opening

The db.open([options]) method has additional options:

It also has the following options for advanced performance tuning, only to be modified if you can prove actual benefit for your particular application.

<details> <summary>Click to expand</summary> </details>

Closing

The db.close() method has an additional behavior: it waits for any pending operations to finish before closing. For example:

// close() will wait for the put() to finish.
const promise1 = db.put('key', 'value')
const promise2 = db.close()

Reading

The db.get(key[, options]), db.getMany(keys[, options]) and db.iterator([options]) methods have an additional option:

A classic-level database supports snapshots (as indicated by db.supports.snapshots) which means db.get(), db.getMany() and db.iterator() read from a snapshot of the database, created synchronously at the time that db.get(), db.getMany() or db.iterator() was called. This means they will not see the data of simultaneous write operations, commonly referred to as having snapshot guarantees.

The db.iterator([options]) method also accepts:

While iterator.nextv(size) is reading entries from LevelDB into memory, it sums up the byte length of those entries. If and when that sum has exceeded highWaterMarkBytes, reading will stop. If nextv(2) would normally yield two entries but the first entry is too large, then only one entry will be yielded. More nextv(size) calls must then be made to get the remaining entries.

If memory usage is less of a concern, increasing highWaterMarkBytes can increase the throughput of nextv(size). If set to 0 then nextv(size) will never yield more than one entry, as highWaterMarkBytes will be exceeded on each call. It can not be set to Infinity. On key- and value iterators (see below) it applies to the byte length of keys or values respectively, rather than the combined byte length of keys and values.

Optimal performance can be achieved by setting highWaterMarkBytes to at least size multiplied by the expected byte length of an entry, ensuring that size is always met. In other words, that nextv(size) will not stop reading before size amount of entries have been read into memory. If the iterator is wrapped in a Node.js stream or Web Stream then the size parameter is dictated by the stream's highWaterMark option. For example:

const { EntryStream } = require('level-read-stream')

// If an entry is 50 bytes on average
const stream = new EntryStream(db, {
  highWaterMark: 1000,
  highWaterMarkBytes: 1000 * 50
})

Side note: the "watermark" analogy makes more sense in Node.js streams because its internal highWaterMark can grow, indicating the highest that the "water" has been. In a classic-level iterator however, highWaterMarkBytes is fixed once set. Getting exceeded does not change it.

The highWaterMarkBytes option is also applied to an internal cache that classic-level employs for next() and for await...of. When next() is called, that cache is populated with at most 1000 entries, or less than that if highWaterMarkBytes is exceeded by the total byte length of entries. To avoid reading too eagerly, the cache is not populated on the first next() call, or the first next() call after a seek(). Only on subsequent next() calls.

Writing

The db.put(key, value[, options]), db.del(key[, options]) and db.batch(operations[, options]) and chainedBatch.write([options]) methods have an additional option:

Additional Methods

The following methods and properties are not part of the abstract-level interface.

db.location

Read-only getter that returns the location string that was passed to the constructor (as-is).

db.approximateSize(start, end[, options])

Get the approximate number of bytes of file system space used by the range [start..end). The result might not include recently written data. The optional options object may contain:

Returns a promise for a number.

db.compactRange(start, end[, options])

Manually trigger a database compaction in the range [start..end]. The optional options object may contain:

Returns a promise.

db.getProperty(property)

Get internal details from LevelDB. When issued with a valid property string, a string value is returned synchronously. Valid properties are:

ClassicLevel.destroy(location)

Completely remove an existing LevelDB database directory. You can use this method in place of a full directory removal if you want to be sure to only remove LevelDB-related files. If the directory only contains LevelDB files, the directory itself will be removed as well. If there are additional, non-LevelDB files in the directory, those files and the directory will be left alone.

Returns a promise for the completion of the destroy operation.

Before calling destroy(), close a database if it's using the same location:

const db = new ClassicLevel('./db')
await db.close()
await ClassicLevel.destroy('./db')

ClassicLevel.repair(location)

Attempt a restoration of a damaged database. It can also be used to perform a compaction of the LevelDB log into table files. From LevelDB documentation:

If a DB cannot be opened, you may attempt to call this method to resurrect as much of the contents of the database as possible. Some data may be lost, so be careful when calling this function on a database that contains important information.

Returns a promise for the completion of the repair operation.

You will find information on the repair operation in the LOG file inside the database directory.

Before calling repair(), close a database if it's using the same location.

Development

Getting Started

This repository uses git submodules. Clone it recursively:

git clone --recurse-submodules https://github.com/Level/classic-level.git

Alternatively, initialize submodules inside the working tree:

cd classic-level
git submodule update --init --recursive

Contributing

Level/classic-level is an OPEN Open Source Project. This means that:

Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.

See the Contribution Guide for more details.

Publishing

  1. Increment the version: npm version ..
  2. Push to GitHub: git push --follow-tags
  3. Wait for CI to complete
  4. Download prebuilds into ./prebuilds: npm run download-prebuilds
  5. Optionally verify loading a prebuild: npm run test-prebuild
  6. Optionally verify which files npm will include: canadian-pub
  7. Finally: npm publish

Donate

Support us with a monthly donation on Open Collective and help us continue our work.

License

MIT