Home

Awesome

nsq-logger

Build Status Build Status NPM version

Nsq service to read messages from all topics listed within a list of nsqlookupd services.|

NPM

INFO: all examples are written in coffee-script

Install

  npm install nsq-logger

Initialize

var logger = new NsqLogger( config );

Example:

var NsqLogger = require( "nsq-logger" );

var config = {
    clientId: "myFooClient"
};

// create the logger
var logger = new NsqLogger( config );

// create a writer instance
/*
var NsqWriter = require( "nsq-logger/writer" );
var writer = new NsqLogger( config );
*/
// or just grab the one used inside the logger
var writer = logger.Writer;

logger.on( "message", function( topic, data, done ){
    // process your topic
    // Example response: -> topic="topic23" data ="Do some Stuff!"
    
    // mark message as done
    done();
});

writer.connect();
writer.publish( "topic23", "Do some Stuff!" );

Config

<a name="config"></a>

Methods

.activate()

Activate the module

Return

( Boolean ): true if it is now activated. false if it was already active

.deactivate()

Deactivate the module

Return

( Boolean ): true if it is now deactivated. false if it was already inactive

.active()

Test if the module is currently active

Return

( Boolean ): Is active?

.destroy( cb )

This will stop all readers and disconnect every connection from nsq

Arguments

Events

message

The main event to catch and process messages from all topics.

Arguments

Example:

logger.on( "message", function( topic, data, done ){
    // process your message.
    // E.g: writing the data to a db with the topic as tablename
    myDB.write( "INSERT INTO " + topic + " VALUES ( " + data + " )", done );
});

ready

Emitted once the list of topics was received and the readers are created and connected. This is just an internal helper. The Method list will also wait for the first response. The events add, remove and change are active after this first response. Example:

topics.on( "ready", function( err ){
    // handle the error
});

Properties

logger.config

<a name="logger-prop-config"></a>

Type: ( Config )

This is the internaly used configuration.

Attributes

See logger config

logger.Writer

Type: ( NsqWriter )

To write messages you can use the internal writer instance.

Details see Writer

logger.Topics

Type: ( NsqTopics )

The logger uses a module called nsq-topics to sync the existing topics and generate the readers for each topic. You can grab the internal used insatnce with logger.Topics

Details see nsq-topics

logger.ready

Type: ( Boolean )

The logger is ready to use

Writer

<a name="writer"></a>

Initialize

var NsqWriter = require( "nsq-logger/writer" )

Example:

var NsqWriter = require( "nsq-logger/writer" );

var config = {
    clientId: "myFooClient",
    host: "127.0.0.1",
    port: 4150
};

// create the writer
var writer = new NsqWriter( config );

writer.connect();
writer.publish( "topic23", "Do some Stuff!" );

Config

<a name="writer-config"></a>

Methods

.connect()

You have to connect the writer before publishing data

Return

( Writer ): retuns itself for chaining

.disconnect()

disconnect the client

Return

( Writer ): retuns itself for chaining

.publish()

You have to connect the writer before publishing data

Arguments

Return

( Writer ): retuns itself for chaining

Example:

writer
  .connect()
  .publish(
  	"hello", // the topic
  	JSON.strinigify( { to: [ "nsq-logger" ] } ) // the data to send
  );

.activate()

Activate the module

Return

( Boolean ): true if it is now activated. false if it was already active

.deactivate()

Deactivate the module

Return

( Boolean ): true if it is now deactivated. false if it was already inactive

.active()

Test if the module is currently active

Return

( Boolean ): Is active?

.destroy( cb )

Disconnect and remove all event listeners

Arguments

Events

message

The main event to catch and process messages from all topics.

Arguments

Example:

logger.on( "message", function( topic, data, done ){
    // process your message.
    // E.g: writing the data to a db with the topic as tablename
    myDB.write( "INSERT INTO " + topic + " VALUES ( " + data + " )", done );
});

ready

Emitted once the list of topics was received and the readers are created and connected. This is just an internal helper. The Method list will also wait for the first response. The events add, remove and change are active after this first response. Example:

topics.on( "ready", function( err ){
    // handle the error
});

Properties

writer.ready

Type: ( Boolean )

The writer is ready to use

writer.connected

Type: ( Boolean )

The writer is connected to nsqd

Reader

<a name="reader"></a>

Initialize

var NsqReader = require( "nsq-logger/reader" )
var reader = NsqReader( topic, channel, config )

Example:

var NsqReader = require( "nsq-logger/reader" );

var config = {
    clientId: "myFooClient",
    lookupdTCPAddresses: "127.0.0.1:4160",
    lookupdHTTPAddresses: "127.0.0.1: 4161",
};

// create the reader
var reader = new NsqReader( "topic23", "channel42", config );


reader.on( "message", function( topic, data, done ){
    // process your topic
    // Example response: -> data ="Do some Stuff!"
    
    // mark message as done
    done();
});
reader.connect();

Paramater

NsqReader( topic, channel, config )

Config

<a name="reader-config"></a>

Methods

.connect()

You have to connect the writer before publishing data

Return

( Writer ): retuns itself for chaining

.disconnect()

disconnect the client

Return

( Writer ): retuns itself for chaining

.activate()

Activate the module

Return

( Boolean ): true if it is now activated. false if it was already active

.deactivate()

Deactivate the module

Return

( Boolean ): true if it is now deactivated. false if it was already inactive

.active()

Test if the module is currently active

Return

( Boolean ): Is active?

.destroy( cb )

Disconnect and remove all event listeners

Arguments

Events

message

The main event to catch and process messages from the defined topic.

Arguments

Example:

logger.on( "message", function( data, done ){
    // process your message.
    // E.g: writing the data to a db
    myDB.write( "INSERT INTO mylogs VALUES ( " + data + " )", done );
});

ready

Emitted once the list of topics was received and the readers are created and connected. This is just an internal helper. The Method list will also wait for the first response. The events add, remove and change are active after this first response. Example:

topics.on( "ready", function( err ){
    // handle the error
});

Properties

writer.ready

Type: ( Boolean )

The writer is ready to use

writer.connected

Type: ( Boolean )

The writer is connected to nsqd

TODO's / IDEAS

Release History

VersionDateDescription
2.2.12019-02-27fixed package lock file
2.2.02019-02-27added option to autoconnect logger and writer on init
2.1.02019-01-25compatibility with nsq < and > 1.x
2.0.22019-01-25finally a working travis test
2.0.12019-01-25fixed broken build
2.0.02019-01-25update to nsq 1.x by updating nsqjs and use nsq-topics 1.x
1.0.02019-01-08Make the module compatible with node:10 but no longer compatible with node < 5
0.1.32017-07-25Small fix to catch JSON stringify errors within Writer.publish
0.1.22016-07-22Fixed a stupid error with the host config of the writer
0.1.12016-07-19Removed debugging output
0.1.02016-07-15Updated dependencies Issue#2 and optimized activate Issue#3
0.0.72016-01-20Added raw nsqjs Message as last argument to the message event
0.0.62015-12-04Bugfix on setting an array configuration; added code banner
0.0.52015-12-03Added namespaces and made multiple parallel logger instances possible.
0.0.42015-12-03configuration bugfix
0.0.32015-12-02updated object tests
0.0.22015-12-02Internal restructure and docs
0.0.12015-12-02Initial version

NPM

Initially Generated with generator-mpnodemodule

Other projects

NameDescription
nsq-topicsNsq helper to poll a nsqlookupd service for all it's topics and mirror it locally.
nsq-nodesNsq helper to poll a nsqlookupd service for all it's nodes and mirror it locally.
node-cacheSimple and fast NodeJS internal caching. Node internal in memory cache like memcached.
nsq-watchWatch one or many topics for unprocessed messages.
rsmqA really simple message queue based on redis
redis-heartbeatPulse a heartbeat to redis. This can be used to detach or attach servers to nginx or similar problems.
systemhealthNode module to run simple custom checks for your machine or it's connections. It will use redis-heartbeat to send the current state to redis.
rsmq-clia terminal client for rsmq
rest-rsmqREST interface for.
redis-sessionsAn advanced session store for NodeJS and Redis
connect-redis-sessionsA connect or express middleware to simply use the redis sessions. With redis sessions you can handle multiple sessions per user_id.
redis-notificationsA redis based notification engine. It implements the rsmq-worker to safely create notifications and recurring reports.
hyperrequestA wrapper around hyperquest to handle the results
task-queue-workerA powerful tool for background processing of tasks that are run by making standard http requests
soyerSoyer is small lib for server side use of Google Closure Templates with node.js.
grunt-soy-compileCompile Goggle Closure Templates ( SOY ) templates including the handling of XLIFF language files.
backlunrA solution to bring Backbone Collections together with the browser fulltext search engine Lunr.js
domelA simple dom helper if you want to get rid of jQuery
obj-schemaSimple module to validate an object by a predefined schema

The MIT License (MIT)

Copyright © 2015 M. Peter, http://www.tcs.de

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.