Awesome
api-gateway-zmq-adaptor
A ZMQ adaptor for the API-Gateway to facilitate a high performant async messaging queue.
It runs along the api-gateway process on each node where the gateway is installed.
Status
This module is under development and it's considered production ready, being used without any problems with the API Gateway.
How it works
This adapter establishes a communication channel with the API Gateway.
The API Gateway sends messages to this adaptor via inproc://, the ZMQ adaptor making them available for consumption further through the XPUB socket.
--------------------------------------------------------------------
| API-GATEWAY BOX |
| |
| +---------------+ |
| | API Gateway | |
| |---------------| |
| | PUB | |
| +---------------+ |
| (connect) |
| | |
| v |
| (bind) |
| +------------------------------------------------------------+ |
| | XSUB | |
| | | |
| | ( default: ipc:///tmp/nginx_queue_listen ) | |
| |------------------------------------------------------------| |
| | ZMQ ADAPTOR | |
| |------------------------------------------------------------| |
| | XPUB | |
| | | |
| | ( default: tcp://0.0.0.0:6001 ) | |
| +------------------------------------------------------------+ |
| (bind) |
| | |
--------------------------------------------------------------------
^
|
|
|
-----------------------------
^ ^
| |
(connect) (connect)
+--------------------+ +--------------------+
| SUB | | SUB |
|--------------------| |--------------------|
| Gateway Tracking | | Gateway Tracking |
| Service | | Service |
+--------------------+ +--------------------+
For better performance, the adapter can bind to a separate NIC for the external communication; this allows the Gateway to use all the ports for the regular API traffic, making the message queue as less intrusive as possible.
Capturing and sending usage data
The ZMQ Adapter gets usage data from the API Gateway by opening a listening socket via IPC.
The API Gateway publishes the messages to that address ( i.e. ipc:///tmp/nginx_queue_listen ).
The ZMQ Adapter makes the messages available on a specific port, binding it to all IPs 0.0.0.0 by default, on the API Gateway Node.
There are 2 flags that control where to listen and where to publish messages as the following example shows:
api-gateway-zmq-adaptor -p tcp://0.0.0.0:6001 -b ipc:///tmp/nginx_queue_listen
-pflag defines the publishing address; consumers connect to this address in order to receive messages from the API Gateway-bflag defines the internal address the adapter binds to in order to listen for messages sent by the Gateway
Debugging
Start the adapter with the -d flag to see all the messages published by the API Gateway and flowing through the adapter.
Usages
- Performant logging mechanism
- Report usage and tracking
Developer guide
Requirements
zeromq , czmq
On a MAC you can install them using Brew:
brew install zeromq
brew install czmq
To build the adaptor use:
make install
To run the unit tests you can execute:
make test
Unit tests require the check library.
For another quick test you can also run the adaptor with the -t flag using ^C to stop it:
$ /usr/local/api-gateway-zmq-adaptor -t
ZeroMQ version 4.1.0 (czmq 2.2.0)
RUNNING IN TEST MODE & DEBUG MODE for XPUB -> XSUB
Starting SUB->PUSH Proxy [tcp://0.0.0.0:5000] -> [ipc:///tmp/nginx_queue_push]
Starting Debug receiver thread [ipc:///tmp/nginx_queue_push] ...
Starting XPUB->XSUB Proxy [ipc:///tmp/nginx_queue_listen] -> [tcp://0.0.0.0:6001]
Starting Test publisher thread [ipc:///tmp/nginx_queue_listen] ...
... sending:PUB-I-39438
Starting Debug subscriber thread [tcp://0.0.0.0:6001] ...
... sending:PUB-H-79844
> got: PUB-H-79844
... sending:PUB-J-19755
> got: PUB-J-19755