Home

Awesome

Last update: 22/06/2018. TLDR: Aiohttp is now using uvloop, Sanic slighly improved .

Before bashing this please open an issue and help us improve it.

Python Web Benchmarks

A simple project to benchmark Python web frameworks against Vibora in a few use cases (more cases will be added soon).

Running Instructions

  1. Install Docker (sudo apt-get install docker.io)

  2. Clone the project (git clone https://github.com/vibora-io/benchmarks)

  3. Build the image (sudo docker build . -t vibora_benchmarks)

  4. Run the benchmark (sudo docker run vibora_benchmarks)

You can configure the benchmark by editing config.json.

Disclaimer

Results (AWS c5.2xlarge - 8 CPU cores - Dedicated Tenancy)


Infamous Hello World

Testing how fast they can answer a static response. Not much useful but it sets the high bar for each framework.

Sanic does not send the "Date" http header which is a violation of the HTTP protocol (actually you could argue that this header is not mandatory but since it's used every where by cache engines (and the RFC says unless you don't have a clock you must send) I wouldn't even lose my time arguing).


FrameworksRequests/SecVersion
Tornado14,1975.02
Django22,8232.0.6
Flask37,4871.0.2
Aiohttp61,2523.3.2
Sanic119,7640.7.0
Vibora368,4560.0.6

Validate JSON

Testing how fast they receive a JSON, parse and validate it. Frameworks that do not have a validation engine (or it's too slow) are using Marshmallow, at least for now, unless a fast and tested library appears on the wild.

Vibora/Aiohttp have "streaming enabled" by default which means the user can receive the response in chunks (or even reject it) before sending the response... So beware that Sanic, for example, has a streaming mode which makes the route slower than normal and could affect performance.


FrameworksRequests/SecVersion
Tornado12,1265.02
Flask18,3261.0.2
Django18,3172.0.6
Aiohttp36,2993.3.2
Sanic69,9210.7.0
Vibora130,1970.0.6

Forms Parsing

Testing how fast they receive (and parse) a POST with a multipart-form inside it containing a few parameters and one single file.

Most frameworks with the exception of Vibora and Aiohttp do not have a streaming multipart parser which means they must load the entire form in-memory (big file uploads are impracticable) before letting the user interact with it (unless of course you read the stream yourself and parse it).


FrameworksRequests/SecVersion
Aiohttp2,3153.3.2
Django7,3112.0.6
Tornado10,2925.02
Flask11,9231.0.2
Sanic47,5200.7.0
Vibora76,6120.0.6

Redis API

Testing how fast they receive a POST with a JSON, validate it and do a simple GET in Redis.


FrameworksRequests/SecVersion
Tornado12,1265.02
Django18,3172.0.6
Flask18,3261.0.2
Aiohttp28,5213.3.2
Sanic70,1320.7.0
Vibora131,6140.0.6