Awesome

Nginx log parser and Prometheus exporter

This service parses incoming syslog messages from Nginx sent over UDP and converts them into Prometheus metrics exported through the built-in HTTP server.

If the prefix-list is specified then per-URI statistics are generated. Using this feature without URI-prefix list is dangerous because it leads to an unbounded memory usage. It's therefore enabled only with a limited prefix list.

The URI prefix list is a plain-text file with a single prefix per line, e.g.
```
/api/call1.json
/api/call2.json
/api/call3.json
```
URIs received from nginx are stripped of any query parameters - only the part before '?' is used.
It optionally supports country lookup for client IPs using MaxMind GeoIP database.

Nginx configuration snippet

log_format collector '$remote_addr|$scheme|$host|$request_method|$server_protocol|$request_uri|$status|$request_time|$request_length|$bytes_sent';
access_log syslog:server=1.1.1.1:1514,tag=nginx collector;

Getting it

Get RPM & DEB directly from releases, only linux-amd64 build currently available.
Build youself:

go get github.com/blind-oracle/nginx-prometheus
cd $GOPATH/src/github.com/blind-oracle/nginx-prometheus
go build

Usage example

If you're using RPM/DEB packages then options are set in /etc/default/nginx-prometheus

./nginx-prometheus \
-debug false \
-listenSyslog 0.0.0.0:1514 \
-listenHTTP 0.0.0.0:1514 \
-geoipCountryDB /etc/nginx-prometheus/country.mmdb \
-uriPrefixFile /etc/nginx-prometheus/uriPrefixes.txt

debug - prints every syslog message received if true (defaults to false)
listenSyslog - ip:port on which to listen for UDP Syslog messages (defaults to 0.0.0.0:1514)
listenHTTP - ip:port on which to listen for incoming HTTP requests from Prometheus (defaults to 0.0.0.0:11080)
geoipCountryDB - path to MaxMind country GeoIP database (optional)
uriPrefixFile - path to a URI prefix list (optional)

Grafana dashboard

Add Prometheus source to Grafana and import grafana-dashboard.json for most common graphs.