Awesome
<img width="529" alt="arbiter" src="https://user-images.githubusercontent.com/20118821/236929861-2a1fe071-0053-453c-ac86-224b32febcd6.png">Expanding the EVM tooling ecosystem.
Overview
Arbiter is a blazing-fast Ethereum sandbox that lets developers orchestrate event-driven simulations.
The framework allows for fine-grained control over a (Rust) Ethereum Virtual Machine (EVM) to provide stateful Ethereum smart-contract interactions and the creation of behaviors that can be coalesced into complex scenarios or automation.
We use ethers-rs
middleware on top of revm, which is used in ETH clients such as reth
as well as foundry
.
This gives us speed, configurability, and modularity that feels like a lightweight custom Ethereum node.
The primary use of Arbiter is to probe the mechanism security of smart contracts. If this interests you, please see the Vulnerability Corpus.
The Arbiter workspace has five crates:
arbiter
: The bin that exposes a command line interface for forking and binding contracts.arbiter-core
: A lib containing the core logic for the Arbiter framework, including theArbiterMiddleware
discussed before, and theEnvironment
, our sandbox.arbiter-engine
: A lib that provides abstractions for building simulations, agents, and behaviors.arbiter-macros
: A lib crate that contains the macros used to simplify development with Arbiter.arbiter-bindings
: A lib crate containing bindings for utility smart contracts used for testing and development.
Book
Here you can find the Arbiter Documentation. This is an mdbook that provides higher level understanding of how to use the entirety of the Arbiter framework.
Motivation
Arbiter was built to allow you to work with smart contracts in a stateful sandbox and design agents that can be used alongside the contracts. This gives you many capabilities. For instance, smart contract engineers must test their contracts against various potentially adversarial environments and parameters and not rely on static stateless testing.
In Decentralized Finance (DeFi), a wide array of complex decentralized applications can use the testing described above. Still, implicit financial strategies also encompass many agents and parameterizations. A financial engineer may want to test their strategies against thousands of market conditions, contract settings, shocks, and autonomous or random AI agents while ensuring their approach isn't vulnerable to bytecode-level exploits. Likewise, the same engineer may want to develop searcher agents, solver agents, or other autonomous agents that can be run on the blockchain.
Working with the Arbiter Framework
To work with Arbiter, you must have Rust installed on your machine.
You can install Rust by following the instructions here.
It will also be helpful to get the cargo-generate
package, which you can install by doing:
cargo install cargo-generate
Examples
We have an example that will run what we have set up in a template. To run this, you can clone the repository and update the submodules:
git clone https://github.com/primitivefinance/arbiter.git
cd arbiter
git submodule update --init --recursive
From here, you can now try running the following from the clone's root directory:
cargo run --example template
This command will enter the template CLI and show you the commands and flags.
To run the ModifiedCounter.sol
example and see some logging, try:
cargo run --example template simulate examples/template/configs/example.toml -vvv
This sets the log level to debug
so you can see what's happening internally.
Initialization
To create your own Arbiter project off of our template arbiter-template, you can run the following:
cd <your/chosen/directory>
cargo generate https://github.com/primitivefinance/arbiter-template.git
You'll be prompted to provide a project name, and the rest will be set up for you!
Binary
To install the Arbiter binary, run:
cargo install arbiter
This will install the Arbiter binary on your machine. You can then run arbiter --help
to see that Arbiter was correctly installed and see the help menu.
Bindings
You can load or write your own smart contracts in the contracts/
directory of your templated project and begin writing your own simulations.
Arbiter treats Rust smart-contract bindings as first-class citizens.
The contract bindings are generated via Foundry's forge
command.
arbiter bind
wraps forge
with convenience features that generate all your bindings to src/bindings
as a Rust module.
Foundry power-users can use forge
directly.
Forking
To fork a state of an EVM network, you must first create a fork config file.
An example is provided in the examples/fork
directory.
Essentially, you provide your storage location for the data, the network you want, the block number you want, and metadata about the contracts you want to fork.
arbiter fork <fork_config.toml>
This will create a fork of the network you specified in the config file and store it in your specified location.
It can then be loaded into an arbiter-core
Environment
using the Fork::from_disk()
method.
Forking is done this way to ensure that all emulation does not require a constant connection to an RPC endpoint.
You may find that Anvil has a more accessible forking interface. However, an online forking mechanism makes RPC calls to update the state as necessary.
Arbiter Environment
forking is for creating a state, storing it locally, and being able to initialize a simulation from that state when desired.
We plan to allow arbiter-engine
to integrate with other network types, such as Anvil, in the future!
Optional Arguments
You can run arbiter fork <fork_config.toml> --overwrite
to overwrite the fork if it already exists.
Cargo Docs
To see the Cargo docs for the Arbiter crates, please visit the following:
You will find each of these on crates.io.
Benchmarks
In arbiter-core
, we have a a small benchmarking suite that compares the ArbiterMiddleware
implementation over the Environment
to the Anvil local testnet chain implementation.
The biggest reasons we chose to build Arbiter was to gain more control over the EVM environment and to have a more robust simulation framework. Still, we also wanted to gain speed, so we chose to build our own interface over revm
instead of using Anvil (which uses revm
under the hood).
For the following, Anvil was set to mine blocks for each transaction instead of setting an enforced block time. The Environment
was configured with a block rate of 10.0.
Preliminary benchmarks of the ArbiterMiddleware
interface over revm
against Anvil are given in the following table.
To run the benchmarking code yourself, you can run:
cargo bench --package arbiter-core
Operation | ArbiterMiddleware | Anvil | Relative Difference |
---|---|---|---|
Deploy | 238.975µs | 7712.436µs | ~32.2729x |
Lookup | 565.617µs | 17880.124µs | ~31.6117x |
Stateless Call | 1402.524µs | 10397.55µs | ~7.413456x |
Stateful Call | 2043.88µs | 154553.225µs | ~75.61756x |
The above can be described by:
-
Deploy: Deploying a contract to the EVM. In this method, we deployed both
ArbiterToken
andArbiterMath
, so you can divide the time by two to estimate the time it takes to deploy a single contract. -
Lookup: Look up the
balanceOf
for a client's address forArbiterToken
. In this method, we calledArbiterToken
'sbalanceOf
function 100 times. Divide by 100 to get the time to look up a single balance. -
Stateless Call: Calling a contract that does not mutate state. In this method, we called
ArbiterMath
'scdf
function 100 times. Divide by 100 to get the time to call a single stateless function. -
Stateful Call: Calling a contract that mutates state. In this call, we called
ArbiterToken
'smint
function 100 times. Divide by 100 to get the time to call a single stateful function.
The benchmarking code can be found in the arbiter-core/benches/
directory, and these specific times were achieved over a 1000 run average.
The above was achieved by running cargo bench --package arbiter-core
, which will automatically run with the release profile.
Times were achieved on an Apple Macbook Pro M1 Max with 8 performance and 2 efficiency cores and 32GB of RAM.
Of course, the use cases of Anvil and the ArbiterMiddleware
can be different.
Anvil represents a more realistic environment with networking and mining. At the same time, the ArbiterMiddleware
is a simpler environment with the bare essentials to running stateful simulations.
Anvil also mines blocks for each transaction, while the ArbiterMiddleware
does not.
Please let us know if you need any help with these benchmarks or suggestions for improving them!
Testing
If you contribute, please write tests for any new code you write. To run the tests, you can run the following:
cargo test --all --all-features
Contributing
See our Contributing Guidelines