Awesome
WASI Key-Value Store
A proposed WebAssembly System Interface API.
Current Phase
WASI Key-Value is currently in Phase 2.
Champions
- Dan Chiarlone
- David Justice
- Jiaxiao Zhou
- Taylor Thomas
Portability Criteria
wasi:keyvalue
must have at least two implementations in terms of relevant mainstream key-value
stores in each of the following categories:
- Open-source key-value stores like Redis, Memcached, Etcd etc.
- Proprietary key-value stores like AWS DynamoDB, Azure CosmosDB, Google Cloud Firestore etc.
The implementations must be able to run on Linux, MacOS, and Windows.
A testsuite that passes on the platforms and implementations mentioned above.
Table of Contents
Introduction
This wasi:keyvalue
proposal defines a collection of interfaces for interacting with
key-value stores. It additionally defines a world, wasi:keyvalue/imports
, that
groups together common interfaces including
- Store operations on single key-value pairs
- Atomic operations on single key-value pairs
- Batch operations on multiple key-value pairs
There is also an additional world called wasi:keyvalue/handle-watch
, which includes the full
imports
world as well as a set of interfaces for watching for changes to a key-value store.
The API is designed to hide data-plane differences between different key-value stores. The control-plane behavior/functionality (e.g., including cluster management, data consistency, replication, and sharding) are not specified, and are provider specific.
Goals
The primary goal of this API is to allow WASI programs to access a wide variety of key-value stores.
This means the wasi:keyvalue
interfaces should be implementable by a wide variety of key-value
stores, including but not limited to: in-memory key-value stores, on-disk key-value stores, document
databases, relational databases, and either single-node or distributed key-value stores.
The second goal of wasi:keyvalue
interfaces is to abstract away the network stack, allowing
applications that need to access key-value stores to not worry about what network protocol is used
to access the key-value store.
A third design goal of wasi:keyvalue
interfaces is to allow performance optimizations without
compromising the portability of the API. This includes minimizing the number of copies of data,
minimizing the number of requests and round-trips, and support for asynchronous operations.
Non-goals
- The
wasi:keyvalue
interfaces only focus on the most common use cases for key-value stores. It does not attempt to cover all possible use cases. - The
wasi:keyvalue
interfaces do not guarantee data consistency except as defined in the interface documentation. Data consistency models (eventual, strong, casual etc.) are diverse and are specific to each key-value store's architecture. The implication is that components importingwasi:keyvalue
interfaces would need to be aware of the data consistency model of the key-value store they are using. - The
wasi:keyvalue
interfaces do not handle data replication and sharding. These are control-plane behaviors that are specific to each key-value store's architecture. Replication needs and sharding management are left to the outside of thewasi:keyvalue
interfaces. - No cluster management is provided. Operational aspects like scaling, node management, and cluster
health are to be managed separately from the wasi:keyvalue interface
wasi:keyvalue
interfaces. - No built-in monitoring. Monitoring of key-value store performance and health should be done using
external tools and not be expected as part of the
wasi:keyvalue
interfaces.
API walk-through
The proposal can be understood by first reading the comments of world.wit
,
store.wit
, atomic.wit
, and
batch.wit
.
Working with the WIT
Bindings can be generated from the wit
directory via:
wit-bindgen c wit/ --world imports
and can be validated and otherwise manipulated via:
wasm-tools component wit wit/ ...
References & acknowledgements
This proposal was inspired by Dapr's [State API]
Change log
- 2024-03-29:
- Simplified interfaces to 3 key pieces of functionality: store, atomics, and batch. Also removed the use of streams to simplify the starting point of the API
- 2024-01-16:
- Changed the
readwrite
andbatch
interface names toeventual
andeventual-batch
. See more details here.
- Changed the
- 2023-12-19:
- Changed the
size
toincoming-value-size
and it's signature
- Changed the
- 2023-11-30:
- Changed the
get
andget-many
andkeys
signatures - Updated comments in all the interfaces.
- Renamed
wasi-cloud-error
towasi-keyvalue-error
- Changed the
- 2023-05-17: Updated batch example to use one interface instead of 2
- 2023-05-25: Change the WITs to the newest syntax.
- 2023-02-13: The following changes were made to the API:
- Added
bucket
type to thetypes
interface. - Uses pseudo-stream and pseudo-resource types inspired from wasi-io
- Added *.wit files for the interfaces and worlds, which are verified by the wasm-tools
- Added a inbound-keyvalue interface that handles the
watch
operation.
- Added
- 2023-01-28: The following changes were made to the API:
- 2022-11-29: Renamed
bulk-get
toget-multi
andbulk-set
toset-multi
to be consistent with the naming convention of the other interfaces. - 2022-10-31: Initial draft