Home

Awesome

simpledb

If you're using this library, feel free to contact me on twitter if you have any questions! :) @rjrodger

NOTE: this project follows the Open-Open policy - if you submit a pull request or an issue, you get commit rights, so feel free to merge yourself after asking for feedback from the other contribs.

IMPORTANT: YOUR CODE CONTRIBUTIONS (if any) ARE MADE UNDER THE MIT LICENSE. By submitting a pull request or issue you indicate agreement with this condition.

Please open an issue to indicate a release should be published to NPM, and we can discuss.

Current Version: 0.2.0

Tested on: node 0.12.4

A user-friendly fault-tolerant library for Amazon AWS SimpleDB access. The core SimpleDB actions are mapped to functions:

var simpledb = require('simpledb')
var sdb      = new simpledb.SimpleDB({keyid:'YOUR_AWS_KEY_ID',secret:'YOUR_AWS_SECRET_KEY'})

sdb.createDomain( 'yourdomain', function( error ) {

  sdb.putItem('yourdomain', 'item1', {attr1:'one', attr2:'two'}, function( error ) {

    sdb.getItem('yourdomain', 'item1', function( error, result ) {
      console.log( 'attr1 = '+result.attr1 )
      console.log( 'attr2 = '+result.attr2 )
    })
  })
})

Any given SimpleDB request has a non-trivial chance of failing. This library implements the exponential back-off retry algorithm as recommended in the SimpleDB developer guide.

This library depends on the excellent aws-lib module: aws-lib

Key Features:

Core Functions:

This is still an early version so there's probably some wierdness - use at your risk. Secure connections are not supported on node 0.3.x.

Installation

npm install simpledb

And in your code:

var simpledb = require('simpledb')

Or clone the git repository: git clone git://github.com/rjrodger/simpledb.git

The simpledb module depends on the aws-lib module. npm will install this automatically.

Usage

This module uses the node.js-style callback convention. All functions take a callback function as their last argument. This callback function should accept three arguments:

callback( error, result, meta )

Where error is an object ({Code:'...',Message:'...'}) describing any errors that occured. If the function was successful then error is null.

So, you check if error is null to see if you can continue working:

sdb.listDomains( function( error, result, meta ) {
  if( error ) {
    console.log('listDomains failed: '+error.Message )
  }
  else {
    // do stuff with result, an array of domain names
  }
})

The result parameter contains the results of a successful action and what the result parameter is depends on the action. It could be a string, an array or an object.

The meta parameter contains a description of the request, including the underlying details from Amazon. Take a look with:

console.log( JSON.stringify(meta) )

Conventions

Where possible, the SimpleDB naming style is preserved: CamelCaseBaby. Names of functions and their parameters also mostly match SimpleDB.

The simpledb.SimpleDB wrapper options (maxtry, secure, etc) are not directly related to Amazon, and so have their own names.

It is sometimes necessary to embed meta-directives into the Amazon query or result objects. These non-Amazon attributes always begin with the $ character, but are in CamelCase. For example: $AsArrays.

This wrapper is based on the REST API. I looked at the SOAP API but... yeah. No X.509 for you. Yet.

API

For the API examples, assume the following lines of code at the top of your source code file:

var simpledb = require('simpledb')

var sdb = new simpledb.SimpleDB(
  {keyid:'YOUR_AWS_KEY_ID',secret:'YOUR_AWS_SECRET_KEY'},
  simpledb.debuglogger
)

This gives you the standard wrapper, with a basic debugger that prints to STDOUT.

You should really also read the Amazon SimpleDB documentation so that you understand how SimpleDB works: Amazon SimpleDB Developer Guide

As a get-out-of-jail, you can provide request attribute overrides. You supply these in an optional override argument just before the callback argument. You can use an override on any of the SimpleDB action wrapper functions.

sdb.getItem('domain','itemname', {ConsistentRead:'false'} ,function(err,res,meta){ ... })

In the above code, {ConsistentRead:"false"} is the optional override argument.

simpledb.SimpleDB: simpledb.SimpleDB( options, logger )

Create a new SimpleDB wrapper. The options argument sets general options for the requests. The logger argument receives logging events so that you can debug and/or record SimpleDB interactions.

options: required

For further options, see the section on options below

logger: optional

See the section on logging below

createDomain: sdb.createDomain(domain,override,callback)

Create a domain. A domain is like a SQL table, sort of.

sdb.createDomain('<domain>',function(err,res,meta){
  if( !err ) {
    console.log('Mul-ti-pass!')
  }
}

Where <domain> is the name of your domain.

domainMetadata: sdb.domainMetadata(domain,override,callback)

Get some statistics about your domain, such as a count of items and how much storage it is using (you pay Amazon for this!).

sdb.domainMetadata('<domain>',function(err,res,meta){
   console.log('Mmm, floor pie! '+JSON.stringify(res) )
}

Where <domain> is the name of your domain.

listDomains: sdb.listDomains(override,callback)

Returns a list of your domain names as an array of strings. Restricted to the specified SimpleDB host (default=sdb.amazonaws.com). See the simpledb.SimpleDB options to change this.

sdb.listDomains(function(err,res,meta){
   console.log('You hear that? That's market bacon hitting the pan: '+JSON.stringify(res) )
}

deleteDomain: sdb.deleteDomain(domain,override,callback)

Delete a domain. Cannot be undone!

sdb.deleteDomain('<domain>',function(err,res,meta){
  if( !err ) {
    console.log('God made the world, but we made the field.')
  }
}

Where <domain> is the name of your domain.

putItem: sdb.putItem(domain,itemname,attrs,override,callback)

Store an item in SimpleDB.

sdb.putItem('<domain>','<itemname>',
  {
    <attr>:'<value>',
    ...
  },
  function(err,res,meta){
    console.log("Memories, you're talking about memories: "+JSON.stringify(res))
  })

Where <itemname> is the unique name of your item, and <attr>:"<value>" are the attribute-value pairs for your item. The value must be either a string or an array of strings.

If you want to use conditional puts, you'll need to add some override values:

sdb.putItem('<domain>','<itemname>',
  {
    <attr1>:'<value>',
    <attr2>:['<value1>','<value2>',...]
    ...
  },
  {
    'Expected.1.Name':'VersionNumber',
    'Expected.1.Value':'1'
  },
  function(err,res,meta){
    console.log("Nobody expects the spanish inquistion! "+JSON.stringify(res))
  })

batchPutItem: sdb.batchPutItem( domain, items, override, callback )

Store multiple items in the same request. This is more efficient. The items argument is an array of item objects. Each item object must have a $ItemName meta-attribute that specifies the name of the item.

sdb.batchPutItem('<domain>',
  [
    { $ItemName:'<itemname1>', <attr>:'<value>', ...},
    { $ItemName:'<itemname2>', <attr>:'<value>', ...}
  ],function(err,res,meta){
    console.log("And what was your ownership share diluted down to?"+JSON.stringify(res))
  })

batchDeleteItem: sdb.batchDeleteItem( domain, items, override, callback )

Delete multiple items in one request. This is more efficient. The items argument is an array of item objects. Each item object must have a $ItemName meta-attribute that specifies the name of the item.

sdb.batchDeleteItem('<domain>',
  [
    { $ItemName:'<itemname1>', <attr>:'<value>', ...},
    { $ItemName:'<itemname2>', <attr>:'<value>', ...}
  ],function(err,res,meta){
    console.log("Done"+JSON.stringify(res))
  })

getItem: sdb.getItem( domain, itemname, override, callback )

Get an item from SimpleDB using the item's unique name. The values of the item's attributes are returned as strings. You can provide an $AsArrays meta-directive in the override argument. When true, all attribute values are returned as arrays. As SimpleDb is schemaless, it is not possible to know in advance if an attribute is multi-valued. In the default case, {$AsArrays:false}, multiple values are returned as string, with the value list comma-separated. SimpleDB returns multiple values in alphabetical order.

sdb.getItem('<domain>','<itemname>',function( error , result, meta ){
  console.log("Those are good burgers, Walter: "+JSON.stringify(res))
})

sdb.getItem('<domain>','<itemname>',{$AsArrays:true},function( error, result, meta ){
  console.log("I've been watching television so much the shows are starting to run together: "+JSON.stringify(res))
})

By default, simpledb uses consistent reads. For improved performance, if this is suitable for your application, you can set the consistent option to false when creating simpledb.SimpleDB. Or you can set it on a case-by-case basis, using an override: {ConsistentRead:"false"}

deleteItem: sdb.deleteItem( domain, itemname, attrs, override, callback )

Delete an item from SimpleDB. The attrs argument is optional, and can be:

If no attributes are specified, the item is completely removed. If present, only the specified attributes are removed. If all the attributes of an item are removed, then it will also be completely deleted.

sdb.deleteItem('<domain>','<itemname>',function( error, result, meta ){
  console.log("Well, Ted, like I said the last time: it won't happen again: "+JSON.stringify(res))
})

sdb.deleteItem('<domain>','<itemname>',[ '<attr>', ... ], function( error, result, meta ){
  console.log("I felt like destroying something beautiful. "+JSON.stringify(res))
})

sdb.deleteItem('<domain>','<itemname>',
  { '<attr1>': '<value1>', 'attr2': ['<value2>, ... ], ... },
  function( error, result, meta ){
    console.log("I don't know what to write about. "+JSON.stringify(res))
  }
)

select: sdb.select( query, override, callback )

Perform a SELECT-style query on a SimpleDB domain. The syntax is almost-but-not-quite SQL. You should read the Amazon documentation: Using Select

The results are returned as an array of items. Each item contains an $ItemName meta-attribute providing you with the name of the item.

If you need to handle NextToken you'll need to do this manually with the override argument. You can get the NextToken from the meta parameter to your callback.

sdb.select("select * from <domain> where <attribute> = '<value>'",function( error, result, meta ){
  console.log("I'll get you, my pretty, and your little dog too! "+JSON.stringify(result)+" "+JSON.stringify(meta))
})

request: sdb.request( action, attrs, callback )

Make a direct request to SimpleDB. You're on your own! Again, read Amazon SimpleDB Developer Guide Unlike the other functions above, the request function is not a SimpleDB action wrapper. Use it when the wrapper functions have painted themselves into a corner.

sdb.request("<action>",
  {
    <attribute>:"<value>",
    ...
  },
  function( error, result, meta ){
    console.log("Gotta keep 'em separated: "+JSON.stringify(res))
  })

Where <action> is the SimpleDB action, such as GetItem, and <attribute>:"<value>" are the SimpleDB REST request attribute pairs.

client: sdb.client

The aws-lib client object. Use this to send raw requests. Go hardcore.

handle: sdb.handle( start, action, query, tryIndex, last, response, stop, callback, )

Replace this with your own implementation to change the handling of SimpleDB responses. Most useful is to modify the response in some way and then call the original function. Also good for testing.

This example counts the number of requests made:

var resultcount = 0

var orighandle = sdb.handle
sdb.handle = function(start,action,query,tryIndex,last,response,stop,callback){
  res.$ResultCount = resultcount++
  orighandle(start,action,query,tryIndex,last,response,stop,callback)
}

Options

The additional options that can be given to simpledb.SimpleDB are:

Logging

You can provide a logger callback when you are creating the simpledb.SimpleDB object to get notifications of request processing events. A simple logger that prints to STDOUT is provided by simpledb.debuglogger:

var sdb = new simpledb.SimpleDB( {...}, simpledb.debuglogger )

The logger callback accepts the following arguments: logger( type, date, ... )

For type=create, fired when the simpledb.SimpleDB object is created, the arguments are:

For type=request, fired just before a request is made to SimpleDB, the arguments are:

For type=handle, fired after each response from SimpleDB, the arguments are:

For type=error, fired after any response with an error, the arguments are:

For type=status, fired after each retry, the arguments are:

Testing

The unit tests use expresso

npm install expresso
npm install eyes

To configure your keys, edit the test/keys.js file. The tests are in test/simpledb.test.js

Amazon AWS SimpleDB

Here's some more information on SimpleDB:

Amazon AWS SimpleDB Developer Guide