Awesome
A set of tools to help with testing the scalability and load behaviour of MQTT environments.
Status
This project has not been used internally since around 2015 and is not in active development. We will attempt to review and merge bugfixes, but any large feature work we simply don't have time to help with.
At the time of writing, we are not aware of any forks that you should track instead, nor have we kept up with the state of the art in this field.
Install
Requires python2, paho-mqtt python library 1.1 or greater, and fusepy.
virtualenv .env
. .env/bin/activate
pip install .
The tools are in multiple layers:
- Python modules for message generation and sending/receiving (Documented as pydoc)
- command line tools for sending/receiving with stats on a single host (Documented here)
- fabric scripts for running those tools on multiple hosts (Documented in README-swarm.md)
malaria publish
The publisher can mimic multiple separate clients, publishing messages of a known size, or with time of flight tracking information, at a known rate, or simply as fast as possible.
Note, for higher values of "processes" you will need to modify ulimit settings!
The messages themselves are provided by generators that can be easily plugged in, see beem.msgs
usage: malaria publish [-h] [-c CLIENTID] [-H HOST] [-p PORT]
[-q {0,1,2}] [-n MSG_COUNT] [-s MSG_SIZE] [-t]
[-T MSGS_PER_SECOND] [-P PROCESSES]
Publish a stream of messages and capture statistics on their timing
optional arguments:
-h, --help show this help message and exit
-c CLIENTID, --clientid CLIENTID
Set the client id of the publisher, can be useful for
acls Default has pid information appended (default:
beem.loadr-8015)
-H HOST, --host HOST MQTT host to connect to (default: localhost)
-p PORT, --port PORT Port for remote MQTT host (default: 1883)
-q {0,1,2}, --qos {0,1,2}
set the mqtt qos for messages published (default: 1)
-n MSG_COUNT, --msg_count MSG_COUNT
How many messages to send (default: 10)
-s MSG_SIZE, --msg_size MSG_SIZE
Size of messages to send. This will be gaussian at (x,
x/20) (default: 100)
-t, --timing Message bodies will contain timing information instead
of random hex characters. This overrides the --msg-
size option, obviously (default: False)
-T MSGS_PER_SECOND, --msgs_per_second MSGS_PER_SECOND
Each publisher should target sending this many msgs
per second, useful for simulating real devices.
(default: 0)
-P PROCESSES, --processes PROCESSES
How many separate processes to spin up
(multiprocessing) (default: 1)
Examples
To fire up 8 processes each sending 10000 messages of ~100 bytes each, sending as fast as the code allows.
malaria publish -P 8 -n 10000 -H mqtt.example.org -s 100
To fire up 500 processes, each sending 5 messages per second, each sending 1000 messages, with time in flight tracking information
malaria publish -t -n 1000 -P 500 -T 5
Example output
$ ./malaria publish -t -n 100 -P 4
Still waiting for results from 4 process(es)
INFO:beem.load:beem.loadr-9850-1:Finished publish 100 msgs at qos 1
INFO:beem.load:beem.loadr-9850-1:Waiting for 100 messages to be confirmed still...
Still waiting for results from 4 process(es)
INFO:beem.load:beem.loadr-9850-0:Finished publish 100 msgs at qos 1
INFO:beem.load:beem.loadr-9850-0:Waiting for 100 messages to be confirmed still...
INFO:beem.load:beem.loadr-9850-2:Finished publish 100 msgs at qos 1
INFO:beem.load:beem.loadr-9850-2:Waiting for 100 messages to be confirmed still...
Still waiting for results from 4 process(es)
INFO:beem.load:beem.loadr-9850-3:Finished publish 100 msgs at qos 1
INFO:beem.load:beem.loadr-9850-3:Waiting for 100 messages to be confirmed still...
Still waiting for results from 4 process(es)
Still waiting for results from 4 process(es)
Still waiting for results from 4 process(es)
Clientid: beem.loadr-9850-0
Message succes rate: 100.00% (100/100 messages)
Message timing mean 298.03 ms
Message timing stddev 13.85 ms
Message timing min 267.18 ms
Message timing max 304.76 ms
Messages per second 49.78
Total time 2.01 secs
Clientid: beem.loadr-9850-1
Message succes rate: 100.00% (100/100 messages)
Message timing mean 915.68 ms
Message timing stddev 15.22 ms
Message timing min 883.67 ms
Message timing max 924.15 ms
Messages per second 49.81
Total time 2.01 secs
Clientid: beem.loadr-9850-2
Message succes rate: 100.00% (100/100 messages)
Message timing mean 95.74 ms
Message timing stddev 15.99 ms
Message timing min 65.17 ms
Message timing max 104.73 ms
Messages per second 49.77
Total time 2.01 secs
Clientid: beem.loadr-9850-3
Message succes rate: 100.00% (100/100 messages)
Message timing mean 731.22 ms
Message timing stddev 18.25 ms
Message timing min 711.27 ms
Message timing max 748.39 ms
Messages per second 49.77
Total time 2.01 secs
Clientid: Aggregate stats (simple avg) for 4 processes
Message succes rate: 100.00% (400/400 messages)
Message timing mean 510.17 ms
Message timing stddev 15.83 ms
Message timing min 481.82 ms
Message timing max 520.51 ms
Messages per second 49.78
Total time 3.35 secs
malaria subscribe
The subscriber side can listen to a broker and print out stats as messages are received. It needs to be told how many messages and how many virtual clients it should expect, and at least at present, requires the publisher to be operating in "time of flight tracking" mode. (Instrumented payloads)
This works in single threaded mode (at present) modelling the use case of a central data processor. It aborts if it ever detects messages being dropped.
usage: malaria subscribe [-h] [-c CLIENTID] [-H HOST] [-p PORT]
[-q {0,1,2}] [-n MSG_COUNT] [-N CLIENT_COUNT]
[-t TOPIC]
Listen to a stream of messages and capture statistics on their timing
optional arguments:
-h, --help show this help message and exit
-c CLIENTID, --clientid CLIENTID
Set the client id of the listner, can be useful for
acls Default has pid information appended. (default:
beem.listr-8391)
-H HOST, --host HOST MQTT host to connect to (default: localhost)
-p PORT, --port PORT Port for remote MQTT host (default: 1883)
-q {0,1,2}, --qos {0,1,2}
set the mqtt qos for subscription (default: 1)
-n MSG_COUNT, --msg_count MSG_COUNT
How many messages to expect (default: 10)
-N CLIENT_COUNT, --client_count CLIENT_COUNT
How many clients to expect. See docs for examples of
how this works (default: 1)
-t TOPIC, --topic TOPIC
Topic to subscribe to, will be sorted into clients by
the '+' symbol (default: mqtt-malaria/+/data/#)
Examples
To monitor a publisher of 500 processes, 1000 msgs per process:
malaria subscribe -n 1000 -N 500
Example output:
$ ./malaria subscribe -n 1000 -N 500
INFO:beem.listen:beem.listr-8518:Listening for 500000 messages on topic mqtt-malaria/+/data/# (q1)
DEBUG:beem.listen:beem.listr-8518:Storing initial drop count: 62491
INFO:beem.listen:beem.listr-8518:Still waiting for 500000 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 500000 messages
----snip-----
INFO:beem.listen:beem.listr-8518:Still waiting for 16997 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 14550 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 12031 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 9608 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 7130 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 4626 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 2247 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 744 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 41 messages
INFO:beem.listen:beem.listr-8518:Still waiting for 0 messages
Clientid: beem.listr-8518
Total clients tracked: 500
Total messages: 500000
Total time: 203.94 secs
Messages per second: 2451 (0.407888 ms per message)
Messages duplicated: []
Flight time mean: 991.66 ms
Flight time stddev: 414.76 ms
Flight time min: 2.29 ms
Flight time max: 2076.39 ms
Similar Work
Bees with Machine guns was the original inspiration, and I still intend to set this up such that the publisher components can be run as "bees"
Also