Home

Awesome

Caliper Introduction

Caliper is a blockchain performance benchmark framework, which allows users to test different blockchain solutions with predefined use cases, and get a set of performance test results.

Currently supported blockchain solutions:

Currently supported performance indicators:

See [to add the link to PSWG] to find out the definitions and corresponding measurement methods.

##Achitecture See Architecture introduction.

Build

Pre-requisites

Make sure following tools are installed

Run npm install in caliper folder to install dependencies locally

Install blockchain SDKs

Run benchmark

All predefined benchmarks can be found in caliper/benchmark folder. To start a benchmark, just run node main.js -c yourconfig.json -n yournetwork.json in the folder of the benchmark.

# start the simple benchmark, default config.json is used
cd ~/caliper/benchmark/simple
node main.js

Some example SUTs are provided in caliper/network folder, they can be launched automatically before the test by setting the bootstrap commands in the configuration file, e.g

{
  "command" : {
    "start": "docker-compose -f network/fabric/simplenetwork/docker-compose.yaml up -d",
    "end" : "docker-compose -f network/fabric/simplenetwork/docker-compose.yaml down;docker rm $(docker ps -aq)"
  }
}

The scripts defined in command.start will be called before the test, and the scripts defined in command.end will be called after the finish of all tests. You can use them to define any preparation or clean-up works.

You can also run the test with your own blockchain network, a network configuration should be provided and corresponding file path should be specified in configuration file's blockchain.config.

Note:

Alternative

You can also use npm scripts to run a benchmark.

$ npm run list

> caliper@0.1.0 list /home/hurf/caliper
> node ./scripts/list.js

Available benchmarks:
drm
simple

$ npm test -- simple -c ./benchmark/simple/config.json -n ./benchmark/simple/fabric.json

> caliper@0.1.0 test /home/hurf/caliper
> node ./scripts/test.js "simple" "-c" "./benchmark/simple/config.json" "-n" "./benchmark/simple/fabric.json"
......

Run benchmark with distributed clients (experimental)

In this way, multiple clients can be launched on distributed hosts to run the same benchmark.

  1. Start the ZooKeeper service

  2. Launch clients on target machines separately by running node ./src/comm/client/zoo-client.js zookeeper-server or npm run startclient -- zookeeper-server . Time synchronization between target machines should be executed before launching the clients.

    Example:

    $ npm run startclient -- 10.229.42.159:2181

> caliper@0.1.0 startclient /home/hurf/caliper
> node ./src/comm/client/zoo-client.js "10.229.42.159:2181"

Connected to ZooKeeper
Created client node:/caliper/clients/client_1514532063571_0000000006
Created receiving queue at:/caliper/client_1514532063571_0000000006_in
Created sending queue at:/caliper/client_1514532063571_0000000006_out
Waiting for messages at:/caliper/client_1514532063571_0000000006_in......

  3. Modify the client type setting in configuration file to 'zookeeper'.

    Example:

    "clients": {
  "type": "zookeeper",
  "zoo" : {
    "server": "10.229.42.159:2181",
    "clientsPerHost": 5
  }
}

  4. Launch the benchmark on any machine as usual.

Note:

Write your own benchmarks

Caliper provides a set of nodejs NBIs (North Bound Interfaces) for applications to interact with backend blockchain system. Check the src/comm/blockchain.js to learn about the NBIs. Multiple Adaptors are implemented to translate the NBIs to different blockchain protocols. So developers can write a benchmark once, and run it with different blockchain systems.

Generally speaking, to write a new caliper benchmark, you need to:

Directory Structure

DirectoryDescription
/benchmarkSamples of the blockchain benchmarks
/docsDocuments
/networkBoot configuration files used to deploy some predefined blockchain network under test.
/srcSouce code of the framework
/src/contractSmart contracts for different blockchain systems