Home

Awesome

Lotte

Lotte is a headless, automated testing framework built on top of PhantomJS and inspired by Ghostbuster. It adds jQuery-like methods and chaining, more assertion logic and an extensible core. Tests can be written in either JavaScript or CoffeeScript.

Lotte comes with tools for accessing the DOM, evaluating arbitrary code, simulating mouse and keyboard input.

Tests are sandboxed and run asynchronously. Blocking methods are available to simulate dependencies and to control the flow of execution.

This project is still highly experimental. Using it may cause your computer to blow up. Seriously.

Prerequisites

Optional Dependencies

Installation

$ npm -g install lotte

-global is preferred so you can run lotte from any directory.

Usage

Create a new file lotte_github.js (preferably in an empty directory) and copy the following code:

this.open('https://github.com', function() {
  this.describe('Sign Up button', function() {
    this.assert.ok(this.$('.signup-button').length, 'expects button to be in the DOM');
    this.success();
  });
});

Run lotte from within the directory and you should see the following output:

/tmp/lotte_github.js
  @ https://github.com
      ✓ Sign Up button

Command-Line Options

You can customise many aspects of Lotte's behaviour either on the command-line on through Lottefiles. The following options are available:

$ lotte --help
Usage: lotte [OPTION...] PATH

Options:
  --help, -h        give this help page
  --version, -v     print program version
  --concurrent, -c  limit of files to run in parallel                       [default: 4]
  --timeout, -t     timeout for individual files (in milliseconds)          [default: 30000]
  --include, -I     glob pattern to match files in PATH          [string]   [default: "**/lotte_*.js"]
  --exclude, -E     glob pattern to remove included files        [string]
  --lottefile, -f   look for 'lottefile' in PATH                 [string]   [default: "Lottefile"]
  --verify          verify PhantomJS version (expected <2.0.0)   [boolean]  [default: true]
  --phantom         executable for PhantomJS                     [string]
  --coffee          executable for CofeeScript                   [string]

There are four key options you would want to customise while the rest should work with their defaults.

Lottefile

In order to avoid having to type the full lotte command-line each time, you can use Lottefiles to store your settings per project.

Lottefiles are regular JavaScript files where each global variable maps to a command-line option. For example, the following command:

$ lotte --include '**/*.coffee' --include '**/*.js' --concurrent 1 tests

can be stored in a Lottefile as this:

path       = 'tests'
include    = ['**/*.coffee', '**/*.js']
concurrent = 1

Running lotte from the project directory will then read the Lottefile and scan the tests directory for all files matching **/*.{coffee,js}.

Writing Tests

Tests can be written in either JavaScript or CoffeeScript. In the sections below substitute @ with this. if you are using JavaScript. Arguments wrapped in square brackets [ and ] are optional. Arguments ending in ... can be used more than once.

At the top-level, the following functions are available:

Putting it all together, a test file could look like this:

@title 'Github'
@base  'https://github.com'

@open '/', 'the homepage', ->
  # ...body of test...

Cases & Grouping

Once you have successfully requested an URI, you can start writing test cases against the page.

The following functions are available:

Putting it all together a test file could now look like this:

@title 'Github'
@base  'https://github.com'

@open '/', 'the homepage', ->
  @describe 'counter shows number of repositories', ->
    # ...assertion logic...
  @group 'Sign Up button', ->
    @describe 'is in place', ->
      # ...assertion logic...
    @describe 'takes you to /plans', ->
      # ...assertion logic...

Flow of Execution

Each test case is executed in the order in which it is defined:

@describe 'I run first', ->
@describe 'I run second', ->
# etc.

If a test case contains an asynchronous function call, the next test case is executed without waiting for the function to finish:

@describe 'I run first', ->
@describe 'I run second', ->
  setTimeout ( -> ), 2500
@describe 'I run third in parallel with second still running', ->

Be extremely careful when dealing with asynchronous function. For example, using .click() to follow an anchor could change the page while another test case is running.

If a test case fails, any remaining test cases are skipped:

@describe 'I run first', ->
  throw 'Whoops!'
@describe 'I should run second, but I never will', ->

To simulate dependencies and control the flow of execution, you can use the following functions:

The earlier example can now be rewritten as follows to make it synchronous again:

@describe 'I run first', ->
@describe 'I run second', ->
  setTimeout ( -> ), 2500
@describe 'I run third', ->
  @wait 'I run second', ->
    # ...assertion logic...

Environments

Lotte uses PhantomJS to execute tests. While you may be writing tests in JavaScript and expect to be able to access the DOM of a page directly, this is not the case.

Each test file runs in its own sandboxed environment. Each page you request also runs in a sandbox. You cannot access variables across environments, i.e., you cannot define a variable in your test file and access it within the page you have just requested:

@open 'https://github.com', ->
  @describe 'Sandbox', ->
    val = 'value'
    # following line throws an exception
    console.log(@page.evaluate( -> return val))
    throw 'exit'

In the above code snippet @page.evaluate runs the function as if it were defined on the page you just requested, i.e, github.com. In order to do so, PhantomJS serializes the function, but it does not include the context in which it was defined. When the function is executed, val is missing in the new context causing it to throw an exception.

Another limitation of PhantomJS is the fact you cannot return complex types from the page. Objects are serialized before they leave the page sandbox and unserialized back in the parent (test case) environment:

@open 'https://github.com', ->
  @describe 'serialize/unserialize', ->
    h1 = @page.evaluate( -> return document.querySelector('h1'))
    # prints 'H1' correctly
    console.log h1.tagName
    # prints 'undefined' as functions cannot be serialized/unserialized
    console.log h1.focus
    throw 'exit'

Lotte comes with a workaround which allows you to pass variables to the PhantomJS environment. This hack has the same limitations as outlined above (it uses JSON.stringify internally):

@open 'https://github.com', ->
  @describe '@using(..)', ->
    expected = 'git'
    @$('h2').first @using { expected }, (element) -> element.innerHTML.indexOf(expected) is 0
    throw 'exit'

Document Queries

There is a lot of boilerplate code required to access the DOM of a page. Lotte comes with a jQuery-like query function to abstract some of the most common operations:

The earlier example can now be rewritten as follows:

@open 'https://github.com', ->
  @describe 'Document Queries', ->
    # prints 'H1' correctly
    console.log @$('h1').tagName
    # prints 'undefined'
    console.log @$('h1').focus
    throw 'exit'
Additional Methods

A DocumentQuery object has the following methods to deal with the DOM:

See DOM Assertions below for additional methods.

Assertions

Lotte comes with two types of assertion logic:

Generic Assertions

If you have used Node's built-in assert module, these functions will be familiar:

DOM Assertions

DocumentQuery (see above) comes with additional methods to deal with assertions:

A general note which applies to all functions above: you cannot access variables from scope within block (see Environments).

An example test file that performs DOM assertions could look like this:

@open 'https://github.com', ->
  @describe 'Navigation and children have classes', ->
    @$('.nav').first   (element) -> element.classList.contains('logged_out')
    @$('.nav li').each (element) -> !! element.className
    throw 'exit'

Passing Tests

So far we have ended tests with throw 'exit'. To pass a test case, use the following functions:

If you don't end a test case either by failing any of the asserts or calling @success(), the entire test file will hang until timeout is reached at which point it is recorded as failed.

FAQs

Contributing

The goal of this project is to provide an awesome tool for developers to test their websites or apps in their favourite language with minimum effort.

Commit and code reviews, ideas and documentation improvements are welcomed.

Changelog

Copyright

Copyright (c) 2011 Stan Angeloff. See LICENSE.md for details.