Awesome
<div align="center"> <h1>Openraft</h1> <h4> Advanced <a href="https://raft.github.io/">Raft</a> in 🦀 Rust using <a href="https://tokio.rs/">Tokio</a>. Please ⭐ on <a href="https://github.com/datafuselabs/openraft">github</a>! </h4> </div>This project intends to improve raft as the next-generation consensus protocol for distributed data storage systems (SQL, NoSQL, KV, Streaming, Graph ... or maybe something more exotic).
Currently, openraft is the consensus engine of meta-service cluster in databend.
-
🚀 Get started:
- Openraft guide is the best place to get started,
- Openraft docs for more in-depth details,
- Openraft FAQ explains some common questions.
-
💡 Example Applications:
- Examples with Openraft 0.9 require Openraft 0.9 on crate.io/openraft
- Examples with Openraft 0.10 require Openraft 0.10, which is not yet published on crate.io;
-
🙌 Questions?
- Why not take a peek at our FAQ? You might find just what you need.
- Wanna chat? Come hang out with us on Discord!
- Or start a new discussion over on GitHub.
- Or join our Feishu group.
- And hey, if you're on WeChat, add us:
drmingdrmer
. Let's get the conversation started!
Whatever your style, we're here to support you. 🚀 Let's make something awesome together!
- Openraft is derived from async-raft with several bugs fixed: Fixed bugs.
Status
- The features are almost complete for building an application.
- Performance: Supports 70,000 writes/sec for single writer, and 1,000,000 writes/sec for 256 writers. See: Performance
- Unit test coverage stands at 92%.
- The chaos test has not yet been completed, and further testing is needed to ensure the application's robustness and reliability.
API status
-
Openraft API is not stable yet. Before
1.0.0
, an upgrade may contain incompatible changes. Check our change-log. A commit message starts with a keyword to indicate the modification type of the commit:DataChange:
on-disk data types changes, which may require manual upgrade.Change:
if it introduces incompatible changes.Feature:
if it introduces compatible non-breaking new features.Fix:
if it just fixes a bug.
Versions
-
Branch main has been under active development. The main branch is for the release-0.10.
-
Branch release-0.9: Latest: ( v0.9.0 | Change log ); Upgrade guide: ⬆️ 0.8 to 0.9;
release-0.9
Won't accept new features but only bug fixes. -
Branch release-0.8: Latest: ( v0.8.8 | Change log ); Upgrade guide: ⬆️ 0.7 to 0.8, ⬆️ 0.8.3 to 0.8.4;
release-0.8
Won't accept new features but only bug fixes. -
Branch release-0.7: Latest: ( v0.7.6 | Change log ); Upgrade guide: ⬆️ 0.6 to 0.7;
release-0.7
Won't accept new features but only bug fixes. -
Branch release-0.6: Latest: ( v0.6.8 | Change log );
release-0.6
won't accept new features but only bug fixes.
Roadmap
- 2022-10-31 Extended joint membership
- 2023-02-14 Minimize confliction rate when electing;
See: Openraft Vote design;
Or use standard raft mode with feature flag
single-term-leader
. - 2023-04-26 Goal performance is 1,000,000 put/sec.
- Reduce the complexity of vote and pre-vote: get rid of pre-vote RPC;
- Support flexible quorum, e.g.: Hierarchical Quorums
- Consider introducing read-quorum and write-quorum, improve efficiency with a cluster with an even number of nodes.
Performance
The benchmark is focused on the Openraft framework itself and is run on a minimized store and network. This is NOT a real world application benchmark!!!
clients | put/s | ns/op |
---|---|---|
256 | 1,014,000 | 985 |
64 | 730,000 | 1,369 |
1 | 70,000 | 14,273 |
For benchmark detail, go to the ./cluster_benchmark folder.
Features
- Async and Event-Driven: Operates based on Raft events without reliance on periodic ticks, optimizing message batching for high throughput.
- Extensible Storage and Networking: Customizable via
RaftLogStorage
,RaftStateMachine
andRaftNetwork
traits, allowing flexibility in choosing storage and network solutions. - Unified Raft API: Offers a single
Raft
type for creating and interacting with Raft tasks, with a straightforward API. - Cluster Formation: Provides strategies for initial cluster setup as detailed in the cluster formation guide.
- Built-In Tracing Instrumentation: The codebase integrates tracing for logging and distributed tracing, with the option to set verbosity levels at compile time.
Functionality:
- ✅ Leader election: by policy or manually(
trigger_elect()
). - ✅ Non-voter(learner) Role: refer to
add_learner()
. - ✅ Log Compaction(snapshot of state machine): by policy or manually(
trigger_snapshot()
). - ✅ Snapshot replication.
- ✅ Dynamic Membership: using joint membership config change. Refer to dynamic membership
- ✅ Linearizable read:
ensure_linearizable()
. - ⛔️ Wont support: Single-step config change. <!-- TODO: explain why -->
- ✅ Toggle heartbeat / election:
enable_heartbeat
/enable_elect
. - ✅ Trigger snapshot / election manually:
trigger_snapshot()
/trigger_elect()
. - ✅ Purge log by policy or manually:
purge_log()
.
Who use it
- Databend - The Next-Gen Cloud [Data+AI] Analytics
- CnosDB - A cloud-native open source distributed time series database.
- yuyang0/rrqlite - A rust implementation of rqlite.
- raymondshe/matchengine-raft - A example to demonstrate how openraft persists snapshots/logs to disk.
- Helyim - SeaweedFS implemented in pure Rust.
- RobustMQ - Next generation cloud-native converged message queue.
Contributing
Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.
Contributors
<a href="https://github.com/datafuselabs/openraft/graphs/contributors"> <img src="https://contrib.rocks/image?repo=datafuselabs/openraft"/> </a>Made with contributors-img.
License
Openraft is licensed under the terms of the MIT License or the Apache License 2.0, at your choosing.