Home

Awesome

Nginx stream server traffic status module

License

Nginx stream server traffic status module

Table of Contents

Version

This document describes nginx-module-sts v0.1.1 released on 04 Jul 2018.

Dependencies

Compatibility

Earlier versions does not work.

Screenshots

nginx-module-sts screenshot

Installation

  1. Clone the git repository.
shell> git clone git://github.com/vozlt/nginx-module-sts.git

shell> git clone git://github.com/vozlt/nginx-module-stream-sts.git
  1. Add the module to the build configuration by adding
--with-stream
--add-module=/path/to/nginx-module-sts
--add-module=/path/to/nginx-module-stream-sts

  1. Build the nginx binary.

  2. Install the nginx binary.

Synopsis

http {
    stream_server_traffic_status_zone;

    ...

    server {

        ...

        location /status {
            stream_server_traffic_status_display;
            stream_server_traffic_status_display_format html;
        }
    }
}

stream {
    server_traffic_status_zone;

    ...

    server {
        ...
    }
}

Description

This is an Nginx module that provides access to stream server traffic status information. This is a porting version of the nginx-module-vts to the NGINX "stream" subsystem so as to support the same features in nginx-module-vts. It contains the current status such as servers, upstreams, user-defined filter.

First of all, It is required both the directive server_traffic_status_zone in stream block and stream_server_traffic_status_zone in http block, and then if the directive stream_server_traffic_status_display is set, can be access to as follows:

JSON document contains as follows:

{
    "hostName": ...,
    "nginxVersion": ...,
    "loadMsec": ...,
    "nowMsec": ...,
    "connections": {
        "active":...,
        "reading":...,
        "writing":...,
        "waiting":...,
        "accepted":...,
        "handled":...,
        "requests":...
    },
    "sharedZones": {
        "name":...,
        "maxSize":...,
        "usedSize":...,
        "usedNode":...
    },
    "streamServerZones": {
        "...":{
            "port":...,
            "protocol":...,
            "connectCounter":...,
            "inBytes":...,
            "outBytes":...,
            "responses":{
                "1xx":...,
                "2xx":...,
                "3xx":...,
                "4xx":...,
                "5xx":...,
            },
            "sessionMsecCounter":...,
            "sessionMsec":...,
            "sessionMsecs":{
                "times":[...],
                "msecs":[...]
            },
            "sessionBuckets":{
                "msecs":[...],
                "counters":[...]
            }
        }
        ...
    },
    "streamFilterZones": {
        "...":{
            "...":{

                "port":...,
                "protocol":...,
                "connectCounter":...,
                "inBytes":...,
                "outBytes":...,
                "responses":{
                    "1xx":...,
                    "2xx":...,
                    "3xx":...,
                    "4xx":...,
                    "5xx":...,
                },
                "sessionMsecCounter":...,
                "sessionMsec":...,
                "sessionMsecs":{
                    "times":[...],
                    "msecs":[...]
                },
                "sessionBuckets":{
                    "msecs":[...],
                    "counters":[...]
                }
            },
            ...
        },
        ...
    },
    "streamUpstreamZones": {
        "...":[
            {
                "server":...,
                "connectCounter":...,
                "inBytes":...,
                "outBytes":...,
                "responses":{
                    "1xx":...,
                    "2xx":...,
                    "3xx":...,
                    "4xx":...,
                    "5xx":...
                },
                "sessionMsecCounter":...,
                "sessionMsec":...,
                "sessionMsecs":{
                    "times":[...],
                    "msecs":[...]
                },
                "sessionBuckets":{
                    "msecs":[...]
                    "counters":[...]
                },
                "uSessionMsecCounter":...,
                "uSessionMsec":...,
                "uSessionMsecs":{
                    "times":[...],
                    "msecs":[...]
                },
                "uSessionBuckets":{
                    "msecs":[...]
                    "counters":[...]
                },
                "uConnectMsecCounter":...,
                "uConnectMsec":...,
                "uConnectMsecs":{
                    "times":[...],
                    "msecs":[...]
                },
                "uConnectBuckets":{
                    "msecs":[...]
                    "counters":[...]
                },
                "uFirstByteMsecCounter":...,
                "uFirstByteMsec":...,
                "uFirstByteMsecs":{
                    "times":[...],
                    "msecs":[...]
                },
                "uFirstByteBuckets":{
                    "msecs":[...]
                    "counters":[...]
                },
                "weight":...,
                "maxFails":...,
                "failTimeout":...,
                "backup":...,
                "down":...
            }
            ...
        ],
        ...
    }
}

The directive stream_server_traffic_status_display_format sets the default ouput format that is one of json,jsonp,html,prometheus. (Default: json)

Traffic calculation as follows:

All calculations are working in log processing phase of Nginx.

Caveats: this module relies on nginx logging system(NGX_STREAM_LOG_PHASE:last phase of the nginx stream), so the traffic may be in certain cirumstances different that real bandwidth traffic. Websocket, canceled downloads may be cause of inaccuracies. The working of the module doesn't matter at all whether the access_log directive "on" or "off". Again, this module works well on "access_log off".

Control

It is able to reset or delete traffic zones through a query string. The request responds with a JSON document.

http {

    stream_server_traffic_status_zone;

    ...

    server {

        server_name example.org;

        ...


        location /status {
            stream_server_traffic_status_display;
            stream_server_traffic_status_display_format html;
        }
    }                                                                                                                                                                                           }
}

stream {
    geoip_country    /usr/share/GeoIP/GeoIP.dat;

    server_traffic_status_zone;

    server_traffic_status_filter_by_set_key $geoip_country_code country::*;

    server {

        ...

    }

    ...

}

If it set as above, then the control uri is like example.org/status/control.

The available request arguments are as follows:

To get status of traffic zones on the fly

This is similar to the status/format/json except that it can get each zones.

To get fully zones

To get group zones

To get each zones

To reset traffic zones on the fly

It reset the values of specified zones to 0.

To reset fully zones

To reset group zones

To reset each zones

To delete traffic zones on the fly

It delete the specified zones in shared memory.

To delete fully zones

To delete group zones

To delete each zones

JSON

The following status information is provided in the JSON format:

Json used by status

/{status_uri}/format/json

/{status_uri}/control?cmd=status&...

Json used by control

/{status_uri}/control?cmd=reset&...

/{status_uri}/control?cmd=delete&...

Variables

The following embedded variables are provided in stream block:

Limit

It is able to limit total traffic per each server by using the directive server_traffic_status_limit_traffic. It also is able to limit all traffic by using the directive server_traffic_status_limit_traffic_by_set_key. When the limit is exceeded, the server will return the 503 (Service Temporarily Unavailable) error in reply to a request. The return code can be changeable.

To limit traffic for server

stream {

    server_traffic_status_zone;

    ...

    server {

        listen 1981;

        server_traffic_status_limit_traffic in:64G;
        server_traffic_status_limit_traffic out:1024G;

        ...
    }
}

To limit traffic for filter

stream {
    geoip_country /usr/share/GeoIP/GeoIP.dat;

    server_traffic_status_zone;

    ...

    server {

        listen 1981;

        server_traffic_status_filter_by_set_key $geoip_country_code country::$server_addr;
        server_traffic_status_limit_traffic_by_set_key FG@country::$server_addr@US out:1024G;
        server_traffic_status_limit_traffic_by_set_key FG@country::$server_addr@CN out:2048G;

        ...

    }
}

To limit traffic for upstream

stream {

    server_traffic_status_zone;

    ...

    upstream backend {
        server 10.10.10.17:80;
        server 10.10.10.18:80;
    }

    server {

        listen 1981;

        server_traffic_status_limit_traffic_by_set_key UG@backend@10.10.10.17:80 in:512G;
        server_traffic_status_limit_traffic_by_set_key UG@backend@10.10.10.18:80 in:1024G;
        proxy_pass backend;

        ...

    }
}

Caveats: Traffic is the cumulative transfer or counter, not a bandwidth.

Use cases

It is able to calculate the user defined individual stats by using the directive server_traffic_status_filter_by_set_key.

To calculate traffic for individual country using GeoIP

stream {
    geoip_country /usr/share/GeoIP/GeoIP.dat;

    server_traffic_status_zone;
    server_traffic_status_filter_by_set_key $geoip_country_code country::*;

    ...

    server {

        ...

        server_traffic_status_filter_by_set_key $geoip_country_code country::$server_addr:$server_port;

    }
}

Basically, country flags image is built-in in HTML. The country flags image is enabled if the country string is included in group name which is second argument of server_traffic_status_filter_by_set_key directive.

Customizing

To customize after the module installed

  1. You need to change the {{uri}} string to your status uri in status.template.html as follows:
shell> vi share/status.template.html

var vtsStatusURI = "yourStatusUri/format/json", vtsUpdateInterval = 1000;
  1. And then, customizing and copy status.template.html to server root directory as follows:
shell> cp share/status.template.html /usr/share/nginx/html/status.html
  1. Configure nginx.conf
   server {
       server_name example.org;
       root /usr/share/nginx/html;

       # Redirect requests for / to /status.html
       location = / {
           return 301 /status.html;
       }

       location = /status.html {}

       # Everything beginning /status (except for /status.html) is
       # processed by the status handler
       location /status {
           stream_server_traffic_status_display;
           stream_server_traffic_status_display_format json;
       }
   }
  1. Access to your html.
http://example.org/status.html

To customize before the module installed

  1. Modify share/status.template.html (Do not change {{uri}} string)

  2. Recreate the ngx_http_stream_server_traffic_status_module_html.h as follows:

shell> cd util
shell> ./tplToDefine.sh ../share/status.template.html > ../src/ngx_http_stream_server_traffic_status_module_html.h
  1. Add the module to the build configuration by adding
--add-module=/path/to/nginx-module-sts
--add-module=/path/to/nginx-module-stream-sts

  1. Build the nginx binary.

  2. Install the nginx binary.

Directives

stream_server_traffic_status

--
Syntaxstream_server_traffic_status <on|off>
Defaultoff
Contexthttp, server, location

Description: Enables or disables the module working. If you set stream_server_traffic_status_zone directive, is automatically enabled.

stream_server_traffic_status_zone

--
Syntaxstream_server_traffic_status_zone [shared:name]
Defaultshared:stream_server_traffic_status
Contexthttp

Description: Sets parameters for a shared memory zone specified by server_traffic_status_zone directive in stream block. Caveats: The name must be same as specified by server_traffic_status_zone.

stream_server_traffic_status_display

--
Syntaxstream_server_traffic_status_display
Default-
Contexthttp, server, location

Description: Enables or disables the module display handler.

stream_server_traffic_status_display_format

--
Syntaxstream_server_traffic_status_display_format <json|html|jsonp|prometheus>
Defaultjson
Contexthttp, server, location

Description: Sets the display handler's output format. If you set json, will respond with a JSON document. If you set html, will respond with the built-in live dashboard in HTML. If you set jsonp, will respond with a JSONP callback function(default: ngx_http_stream_server_traffic_status_jsonp_callback). If you set prometheus, will respond with a prometheus document.

stream_server_traffic_status_display_jsonp

--
Syntaxstream_server_traffic_status_display_jsonp callback
Defaultngx_http_stream_server_traffic_status_jsonp_callback
Contexthttp, server, location

Description: Sets the callback name for the JSONP.

stream_server_traffic_status_average_method

--
Syntaxstream_server_traffic_status_average_method <AMM|WMA> [period]
DefaultAMM 60s
Contexthttp, server, location

Description: Sets the method which is a formula that calculate the average of response processing times. The period is an effective time of the values used for the average calculation.(Default: 60s) If period set to 0, effective time is ignored. In this case, the last average value is displayed even if there is no requests and after the elapse of time. The corresponding values are sessionMsec, uSessionMsec, uConnectMsec, uFirstByteMsec in JSON.

server_traffic_status

--
Syntaxserver_traffic_status <on|off>
Defaultoff
Contextstream, server

Description: Enables or disables the module working. If you set server_traffic_status_zone directive, is automatically enabled.

server_traffic_status_zone

--
Syntaxserver_traffic_status_zone [shared:name:size]
Defaultshared:stream_server_traffic_status:1m
Contextstream

Description: Sets parameters for a shared memory zone that will keep states for various keys. The cache is shared between all worker processes.

server_traffic_status_filter

--
Syntaxserver_traffic_status_filter <on|off>
Defaulton
Contextstream, server

Description: Enables or disables the filter features.

server_traffic_status_filter_by_set_key

--
Syntaxserver_traffic_status_filter_by_set_key key [name]
Default-
Contextstream, server

Description: Enables the keys by user defined variable. The key is a key string to calculate traffic. The name is a group string to calculate traffic. The key and name can contain variables such as $host, $server_addr, $server_port. The name's group belongs to streamFilterZones if specified. The key's group belongs to streamServerZones if not specified second argument name. The example with geoip module is as follows:

stream {

      ...

      server {
          listen 1981;
          server_traffic_status_filter_by_set_key $geoip_country_code country::$server_addr:$server_port;

          ...

      }
}

  ...
  "streamServerZones": {
  ...
  },
  "streamFilterZones": {
      "country::example.org": {
          "KR": {
              "port":...,
              "protocol":...,
              "connectCounter":...,
              "inBytes":...,
              "outBytes":...,
              "responses":{
                  "1xx":...,
                  "2xx":...,
                  "3xx":...,
                  "4xx":...,
                  "5xx":...,
              },
              "sessionMsec":...
              "sessionMsecs":{
                  "times":[...],
                  "msecs":[...]
              },
            },
          },
          "US": {
          ...
          },
          ...
      },
      ...
  },
  ...

server_traffic_status_filter_check_duplicate

--
Syntaxserver_traffic_status_filter_check_duplicate <on|off>
Defaulton
Contextstream, server

Description: Enables or disables the deduplication of server_traffic_status_filter_by_set_key. It is processed only one of duplicate values(key + name) in each directives(stream, server) if this option is enabled.

server_traffic_status_limit

--
Syntaxserver_traffic_status_limit <on|off>
Defaulton
Contextstream, server

Description: Enables or disables the limit features.

server_traffic_status_limit_traffic

--
Syntaxserver_traffic_status_limit_traffic member:size [code]
Default-
Contextstream, server

Description: Enables the traffic limit for specified member. The member is a member string to limit traffic. The size is a size(k/m/g) to limit traffic. The code is a code to return in response to rejected requests.(Default: 503)

The available member strings are as follows:

server_traffic_status_limit_traffic_by_set_key

--
Syntaxserver_traffic_status_limit_traffic_by_set_key key member:size [code]
Default-
Contextstream, server

Description: Enables the traffic limit for specified key and member. The key is a key string to limit traffic. The member is a member string to limit traffic. The size is a size(k/m/g) to limit traffic. The code is a code to return in response to rejected requests.(Default: 503)

The key syntax is as follows:

The available group strings are as follows:

The available member strings are as follows:

The member is the same as server_traffic_status_limit_traffic directive.

server_traffic_status_limit_check_duplicate

--
Syntaxserver_traffic_status_limit_check_duplicate <on|off>
Defaulton
Contextstream, server

Description: Enables or disables the deduplication of server_traffic_status_limit_by_set_key. It is processed only one of duplicate values(member | key + member) in each directives(stream, server) if this option is enabled.

server_traffic_status_average_method

--
Syntaxserver_traffic_status_average_method <AMM|WMA> [period]
DefaultAMM 60s
Contextstream, server

Description: Sets the method which is a formula that calculate the average of response processing times. The period is an effective time of the values used for the average calculation.(Default: 60s) If period set to 0, effective time is ignored. In this case, the last average value is displayed even if there is no requests and after the elapse of time. The corresponding value is only $sts_session_time variable.

Caveats: The $sts_session_time variable is the value calculated at the time of the last request. It is not calculated when using variables.

server_traffic_status_histogram_buckets

--
Syntaxserver_traffic_status_histogram_buckets second ...
Default-
Contextstream

Description: Sets the observe buckets to be used in the histograms. By default, if you do not set this directive, it will not work. The second can be expressed in decimal places with a minimum value of 0.001(1ms). The maximum size of the buckets is 32. If this value is insufficient for you, change the NGX_STREAM_SERVER_TRAFFIC_STATUS_DEFAULT_BUCKET_LEN in the nginx-mdule-stream-sts/src/ngx_stream_server_traffic_status_node.h and the NGX_HTTP_STREAM_SERVER_TRAFFIC_STATUS_DEFAULT_BUCKET_LEN in the nginx-module-sts/src/ngx_http_stream_server_traffic_status_node.h.

For examples:

Caveats: By default, if you do not set this directive, the histogram statistics does not work.

See Also

TODO

Donation

License

Author

YoungJoo.Kim(김영주) [vozltx@gmail.com]