Home

Awesome

level-jobs

Job Queue in LevelDB for Node.js

Build Status

Install

$ npm install level-jobs --save

Use

Create a levelup database

var levelup = require('levelup');
var db = levelup('./db')

Require level-jobs

var Jobs = require('level-jobs');

Define a worker function

This function will take care of a work unit.

function worker(id, payload, cb) {
  doSomething(cb);
}

This function gets 3 arguments:

This callback function accepts an error as the first argument. If an error is provided, the work unit is retried.

Wrap the database

var queue = Jobs(db, worker);

This database will be at the mercy and control of level-jobs, don't use it for anything else!

(this database can be a root levelup database or a sublevel)

You can define a maximum concurrency (the default is Infinity):

var maxConcurrency = 2;
var queue = Jobs(db, worker, maxConcurrency);

More Options

As an alternative the third argument can be an options object with these defaults:

var options = {
  maxConcurrency: Infinity,
  maxRetries:     10,
  backoff: {
    randomisationFactor: 0,
    initialDelay: 10,
    maxDelay: 300
  }
};

var queue = Jobs(db, worker, options);

Push work to the queue

var payload = {what: 'ever'};

var jobId = queue.push(payload, function(err) {
  if (err) console.error('Error pushing work into the queue', err.stack);
});

or in batch:

var payloads = [
  {what: 'ever'},
  {what: 'ever'}
];

var jobIds = queue.pushBatch(payloads, function(err) {
  if (err) console.error('Error pushing works into the queue', err.stack);
});

Delete pending job

(Only works for jobs that haven't started yet!)

queue.del(jobId, function(err) {
  if (err) console.error('Error deleting job', err.stack);
});

or in batch:

queue.delBatch(jobIds, function(err) {
  if (err) console.error('Error deleting jobs', err.stack);
});

Traverse jobs

queue.pendingStream() emits queued jobs. queue.runningStream() emits currently running jobs.

var stream = queue.pendingStream();
stream.on('data', function(d) {
  var jobId = d.key;
  var work = d.value;
  console.log('pending job id: %s, work: %j', jobId, work);
});

Events

A queue object emits the following event:

Client isolated API

If you simply want a pure queue client that is only able to push jobs into the queue, you can use level-jobs/client like this:

var QueueClient = require('level-jobs/client');

var client = QueueClient(db);

client.push(work, function(err) {
  if (err) throw err;
  console.log('pushed');
});

License

MIT