Awesome
flash-store
<!-- markdownlint-disable MD013 -->FlashStore is Key-Value persistent storage with easy to use ES6 Map-like API(both Async and Sync support), powered by LevelDB and TypeScript.
Requirements
- Node.js v10 or above
Examples
Try the following command
npm install
npm run demo
The basic function as follows:
import { FlashStore } from 'flash-store'
const flashStore = new FlashStore('flashstore.workdir')
await flashStore.set(1, 'a')
console.log(`key: 1, value: ${await flashStore.get(1)}`)
// Output: 'a'
await flashStore.del(1)
console.log(`key: 1, value: ${await flashStore.get(1)}`)
// Output: undefined
Supported Backend
Backend | Flash Store | Install NPM Command |
---|---|---|
LevelDB | v1.0 | npm install flash-store@1 |
SQLite | v0.20 | npm install flash-store@0.20 |
Medea | v0.18 | |
SnapDB | v0.16 | |
RocksDB | v0.14 | npm install flash-store@0.14 |
LevelDB | v0.12 | npm install flash-store@0.12 |
SnapDB & Medea were all deprecated because of lots of unknown bugs.
API Reference
FlashStore
FlashStore implements the Standard ES6 Map API with Async modification, powered by async-map-like
/**
* ES6 Map API with Async
*/
export interface AsyncMap<K = any, V = any> {
[Symbol.asyncIterator]() : AsyncIterableIterator<[K, V]>
size : Promise<number>
clear () : Promise<void>
delete (key: K) : Promise<void>
entries() : AsyncIterableIterator<[K, V]>
get (key: K) : Promise<V | undefined>
has (key: K) : Promise<boolean>
keys () : AsyncIterableIterator<K>
set (key: K, value: V) : Promise<void>
values () : AsyncIterableIterator<V>
}
class FlashStore<K, V> implements AsyncMap<K, V> {}
FlashStoreSync
FlashStoreSync implements the Standard ES6 Map API:
class FlashStoreSync<K, V> implements Map<K, V> {}
- You get a sync API at the cost of all your data have to be kept in memory.
- The data will be async writing back to disk for persistent storage in background.
- The performance of
FlashStoreSync
can be expected high because it's all in memory.
Document
See Also
- Node.js databases: an embedded database using LevelDB
- How to Cook a Graph Database in a Night - LevelGraph
- Graph database JS style for Node.js and the Browser. Built upon LevelUp and LevelDB.
- 浅析 BigTable 和 LevelDB 的实现
Known Issues
- The
gte
andlte
inoptions
do not work property. (#4)
Version History
master v1.3 (Sep 12, 2021)
- ES Modules support (#93)
- Remove default
workdir
inFlashStore
constructor.
v1.0 (Aug 16, 2021) LevelDB v7.0
LevelDB v7.0.0 support.
v0.20 Apr 2020 SQLite as Backend
- We hardcoded the key type to be
string
only in this version. - We decide to try better-sqlite3 as it claim is very fast.
- The other alternates (would love to try in the future if necessary):
v0.18 Feb 2019 - Medea as Backend
DEPRECATED: Due to #79 #74 and also it is very unstable in production as my experiences. (e.g. memory leak & block event loop)
- Switch from SnapDB to MedeaDown
Medea is a persistent key-value storage library that runs everywhere Node runs.
"It is a pure JS implementation of leveldown and it's almost as fast." — @Raynos link
"The LevelDOWN-compatible wrapper for Medea really opens up the possibility to reuse the modules that have already been created around the LevelUP ecosystem." — @kevinswiber link
Known Issues: FlashStore 会写满磁盘的问题 #155
async function compact (store: FlashStore): Promise<void> {
await store.size
const db = (store as any).levelDb.db.db.db
await new Promise((resolve, reject) => {
db.compact((err: any) => {
if (err) {
return reject(err)
}
resolve()
})
})
}
v0.16 May 2019 - SnapDB as Backend
- Switch from RocksDB to SnapDB #45
- #50 has been fixed.
WARN: Do not use this version because it has known issues
v0.14 May 2019 - RocksDB as Backend
v0.12 Jan 2019 - LevelDB as Backend
- Use LevelDB as backend to skip the compiling when install.
- Using leveldb official typings from
@types/
v0.7 Aug 2018 - Nosql-LevelDB as Backend
- Use nosql-leveldb as backend to prevent segfault.
v0.6 Jul 2018
- Upgrade to TypeScript 3.0
v0.4 Jun 2018
1. Refactor API to implement ES6 Map
interface
- Update the API to ES6
Map
-like, the difference is that FlashStore is all async.
2. Add FlashStoreSync
as a in-memory Write-back Cache for Flash-Store
Add a new class FlashStoreSync
which is a in-memory full loaded Write-back Cache for Flash-Store:
- Writes directly to
cache
, lazy writes tostore
. - Reads from cache, never read-miss because cache have the full data of the store which will never expire.
- API of
FlashStoreSync
is the same as the ES6Map
v0.2 Sep 2017
Init version, API is LevelDB-like.
FAQ
Q: What's the difference between the flash-store
and memory-card
Short answer:
flash-store
is for save data to local filesystem.memory-card
is for save data to a distributed network storage, it can be serialized/deserialized safely with the minimum payload transfered by design.
Long answer:
flash-store
and memory-card
are all built by @huan, and they are all follow the ES6 Map API.
flash-store
is using a no-sql local file database to maximum the performance, it can be used as a local database, or a local cache for whatever you want to cache from other API.
memory-card
is using a local file to store data in JSON format by default, however, it supports more distributed methods and it can be serialized/deserialized safely with the minimum payload for transfer the data between networks (for example, it can serialize itself to only include the redis
config and then deserialize it to restore a MemoryCard instance with the same data backend). Learn more from it's repository at here
Author
Copyright & License
- Code & Docs © 2017-now Huan (李卓桓) <zixia@zixia.net>
- Code released under the Apache-2.0 License
- Docs released under Creative Commons