Home

Awesome

Server Benchmarks

A benchmark suite which, transparently, stress-tests web servers and generates a report in markdown. It measures the requests per second, data transferred and time between requests and responses.

Benchmarks: Jun 20, 2020 at 8:17pm (UTC)

Why YABS (Yet Another Benchmark Suite)

It's true, there already enough of benchmark suites to play around. However, most of them don't even contain real-life test applications to benchmark, therefore the results are not always accurate e.g. a route handler executes SQL queries or reads and sends JSON. This benchmark suite is a fresh start, it can contain any type of tests as the tests are running as self-executables and the measuring is done by a popular and trusted 3rd-party software which acts as a real HTTP Client (one more reason of transparency). Contributions and improvements are always welcomed here.

Use case

Measure the performance of application(s) between different versions or implementations (or web frameworks).

This suite can be further customized, through its tests.yml file, in order to test personal or internal web applications before their public releases.

Installation

The only requirement for the benchmark tool is the Go Programming Language.

$ go get github.com/kataras/server-benchmarks@master
$ go install github.com/codesenberg/bombardier@master

Depending on your test cases you may want to install Node.js and .NET Core too.

How to run

  1. Navigate to your tests directory, the one which includes a tests.yml file
  2. Open a terminal and execute: server-benchmarks
  3. Wait for the executable server-benchmarks (or server-benchmarks.exe for windows) to finish
  4. That's all, now open the the results from the generated RESULTS.md file.

Advanced usage

$ server-benchmarks --wait-run=3s -i ./tests.dev.yml -o ./dev -g-spreadsheet $GoogleSpreadsheetID -g-secret client_secret.json

Run using Docker

The only requirement is Docker.

$ docker run -v ${PWD}:/data kataras/server-benchmarks

Benchmarks

The following generated README contains benchmark results from builtin tests between popular HTTP/2 web frameworks as of 2022.

Note: it's possible that the contents of this file will be updated regularly to accept even more tests cases and frameworks.

System

ProcessorAMD Ryzen 9 4900HS with Radeon Graphics
RAM15.42 GB
OSMicrosoft Windows 11 Pro
Bombardierv1.2.4
Gogo1.19beta1
.Net Core6.0.300

| Node.js | v18.2.0 |

Last updated: Jun 20, 2022 at 8:17pm (UTC)

Terminology

Name is the name of the framework(or router) used under a particular test.

Reqs/sec is the avg number of total requests could be processed per second (the higher the better).

Latency is the amount of time it takes from when a request is made by the client to the time it takes for the response to get back to that client (the smaller the better).

Throughput is the rate of production or the rate at which data are transferred (the higher the better, it depends from response length (body + headers).

Time To Complete is the total time (in seconds) the test completed (the smaller the better).

Results

Test:Static

📖 Fires 1000000 requests, receives a static message as response.

NameLanguageReqs/secLatencyThroughputTime To Complete
IrisGo284059438.34us49.58MB3.52s
ChiGo275525451.01us48.18MB3.62s
EchoGo267815466.16us46.64MB3.74s
KestrelC#263479472.72us44.68MB3.80s
GinGo263399472.70us45.98MB3.80s
MartiniGo233051534.43us40.68MB4.29s
KoaJavascript1312740.93ms29.24MB7.50s
BuffaloGo789631.58ms13.78MB12.66s
ExpressJavascript410783.02ms11.54MB24.22s

Test:Parameterized

📖 Fires 550000 requests with a dynamic parameter of string, receives a hello text based on the parameter as response.

NameLanguageReqs/secLatencyThroughputTime To Complete
IrisGo277099449.55us53.07MB1.99s
ChiGo272434456.62us52.21MB2.02s
EchoGo261467476.01us50.14MB2.10s
GinGo259308480.32us49.70MB2.12s
KestrelC#233843534.73us43.34MB2.36s
MartiniGo225790551.37us43.29MB2.44s
KoaJavascript1146671.08ms27.21MB4.78s
BuffaloGo767471.63ms14.71MB7.17s
ExpressJavascript371103.32ms11.11MB14.69s

Test:REST

📖 Fires 200000 requests with a dynamic parameter of int, sends JSON as request body and receives JSON as response.

NameLanguageReqs/secLatencyThroughputTime To Complete
IrisGo238954521.69us64.15MB0.84s
GinGo229665541.96us62.86MB0.87s
ChiGo228072545.78us62.61MB0.88s
EchoGo224491553.84us61.70MB0.89s
MartiniGo198166627.46us54.47MB1.01s
KestrelC#163486766.90us47.42MB1.23s
BuffaloGo1024781.22ms28.14MB1.95s
KoaJavascript484252.56ms15.39MB4.14s
ExpressJavascript236225.25ms9.04MB8.41s

License

This project is licensed under the MIT License.