Home

Awesome

Puffing Billy Gem Version Build Status

A rewriting web proxy for testing interactions between your browser and external sites. Works with ruby + rspec.

Puffing Billy is like webmock or VCR, but for your browser.

Overview

Billy spawns an EventMachine-based proxy server, which it uses to intercept requests sent by your browser. It has a simple API for configuring which requests need stubbing and what they should return.

Billy lets you test against known, repeatable data. It also allows you to test for failure cases. Does your twitter (or facebook/google/etc) integration degrade gracefully when the API starts returning 500s? Well now you can test it!

it 'should stub google' do
  proxy.stub('http://www.google.com/').and_return(text: "I'm not Google!")
  visit 'http://www.google.com/'
  expect(page).to have_content("I'm not Google!")
end

You can also record HTTP interactions and replay them later. See caching below.

Installation

Add this line to your application's Gemfile:

gem 'puffing-billy', group: :test

And then execute:

$ bundle

Or install it yourself as:

$ gem install puffing-billy

Setup for Capybara

In your rails_helper.rb:

require 'billy/capybara/rspec'

# select a driver for your chosen browser environment
Capybara.javascript_driver = :selenium_billy # Uses Firefox
# Capybara.javascript_driver = :selenium_headless_billy # Uses Firefox in headless mode
# Capybara.javascript_driver = :selenium_chrome_billy
# Capybara.javascript_driver = :selenium_chrome_headless_billy
# Capybara.javascript_driver = :apparition_billy
# Capybara.javascript_driver = :webkit_billy
# Capybara.javascript_driver = :poltergeist_billy
# Capybara.javascript_driver = :cuprite_billy

Note: :poltergeist_billy doesn't support proxying any localhosts, so you must use :webkit_billy, :apparition_billy, or a custom headless selenium registration for headless specs when using puffing-billy for other local rack apps. See this phantomjs issue for any updates.

Setup for Watir

In your rails_helper.rb:

require 'billy/watir/rspec'

# select a driver for your chosen browser environment
@browser = Billy::Browsers::Watir.new :firefox
# @browser = Billy::Browsers::Watir.new = :chrome
# @browser = Billy::Browsers::Watir.new = :phantomjs

Setup for Cucumber

An example feature:

Feature: Stubbing via billy

  @javascript @billy
  Scenario: Test billy
    And a stub for google

Setup for Cucumber + Capybara

In your features/support/env.rb:

require 'billy/capybara/cucumber'

After do
  Capybara.use_default_driver
end

And in steps:

Before('@billy') do
  Capybara.current_driver = :poltergeist_billy
end

And /^a stub for google$/ do
  proxy.stub('http://www.google.com/').and_return(text: "I'm not Google!")
  visit 'http://www.google.com/'
  expect(page).to have_content("I'm not Google!")
end

It's good practice to reset the driver after each scenario, so having an @billy tag switches the drivers on for a given scenario. Also note that stubs are reset after each step, so any usage of a stub should be in the same step that it was created in.

Setup for Cucumber + Watir

In your features/support/env.rb:

require 'billy/watir/cucumber'

After do
  @browser.close
end

And in steps:

Before('@billy') do
  @browser = Billy::Browsers::Watir.new :firefox
end

And /^a stub for google$/ do
  proxy.stub('http://www.google.com/').and_return(text: "I'm not Google!")
  @browser.goto 'http://www.google.com/'
  expect(@browser.text).to eq("I'm not Google!")
end

Setup remote Chrome

In the case you are using a Chrome instance, running on another machine, or in another Docker container, you need to :

WebSockets

Puffing billy doesn't support websockets, so if you are using them, or ActionCable for the Ruby On Rails developers, you can tell Chrome to bypass the proxy for websockets by adding the flag --proxy-bypass-list=ws://* to your remote chrome intance or Docker container.

Minitest Usage

Please see this link for details and report back to Issue #49 if you get it fully working.

Examples

# Stub and return text, json, jsonp (or anything else)
proxy.stub('http://example.com/text/').and_return(text: 'Foobar')
proxy.stub('http://example.com/json/').and_return(json: { foo: 'bar' })
proxy.stub('http://example.com/jsonp/').and_return(jsonp: { foo: 'bar' })
proxy.stub('http://example.com/headers/').and_return(
  headers: { 'Access-Control-Allow-Origin' => '*' },
  json: { foo: 'bar' }
)
proxy.stub('http://example.com/wtf/').and_return(body: 'WTF!?', content_type: 'text/wtf')

# Stub redirections and other return codes
proxy.stub('http://example.com/redirect/').and_return(redirect_to: 'http://example.com/other')
proxy.stub('http://example.com/missing/').and_return(code: 404, body: 'Not found')

# Even stub HTTPS!
proxy.stub('https://example.com:443/secure/').and_return(text: 'secrets!!1!')

# Pass a Proc (or Proc-style object) to create dynamic responses.
#
# The proc will be called with the following arguments:
#   params:  Query string parameters hash, CGI::escape-style
#   headers: Headers hash
#   body:    Request body string
#   url:     The actual URL which was requested
#   method:  The HTTP verb which was requested
proxy.stub('https://example.com/proc/').and_return(Proc.new { |params, headers, body, url, method|
  {
    code: 200,
    text: "Hello, #{params['name'][0]}"
  }
})

# You can also use Puffing Billy to intercept requests and responses. Just pass
# a Proc and use the pass_request method. You can manipulate the request
# (headers, URL, HTTP method, etc) and also the response from the upstream
# server. The scope of the delivered callable is the user scope where
# it was defined. Setting method to 'all' will intercept requests regardless of
# the method.
proxy.stub('http://example.com/', method: 'all').and_return(Proc.new { |*args|
  response = Billy.pass_request(*args)
  response[:headers]['Content-Type'] = 'text/plain'
  response[:body] = 'Hello World!'
  response[:code] = 200
  response
})

# Stub out a POST. Don't forget to allow a CORS request and set the method to 'post'
proxy.stub('http://example.com/api', method: 'post').and_return(
  headers: { 'Access-Control-Allow-Origin' => '*' },
  code: 201
)

# Stub out an OPTIONS request. Set the headers to the values you require.
proxy.stub('http://example.com/api', method: 'options').and_return(
  headers: {
    'Access-Control-Allow-Methods' => 'GET, PATCH, POST, PUT, OPTIONS',
    'Access-Control-Allow-Headers' => 'X-Requested-With, X-Prototype-Version, Content-Type',
    'Access-Control-Allow-Origin'  => '*'
  },
  code: 200
)

Stubs are reset between tests. Any requests that are not stubbed will be proxied to the remote server.

If for any reason you'd need to reset stubs manually you can do it in two ways:

# reset a single stub
example_stub = proxy.stub('http://example.com/text/').and_return(text: 'Foobar')
proxy.unstub example_stub

# reset all stubs
proxy.reset

Caching

Requests routed through the external proxy are cached.

By default, all requests to localhost or 127.0.0.1 will not be cached. If you're running your test server with a different hostname, you'll need to add that host to puffing-billy's whitelist.

In your rails_helper.rb:

Billy.configure do |c|
  c.whitelist = ['test.host', 'localhost', '127.0.0.1'] # To replace the default whitelist, OR
  c.whitelist << 'mynewhost.local' # to append a host without overriding the defaults.
end

If you would like to cache other local rack apps, you must whitelist only the specific port for the application that is executing tests. If you are using Capybara, this can be accomplished by adding this in your rails_helper.rb:

server = Capybara.current_session.server
Billy.config.whitelist = ["#{server.host}:#{server.port}"]

If you would like to cache whitelisted URLs, you can define them in c.cache_whitelist. This is useful for scenarios where you may want to set c.non_whitelisted_requests_disabled to true to only allow whitelisted URLs to be accessed, but still allow specific URLs to be treated as if they were non-whitelisted.

If you want to use puffing-billy like you would VCR you can turn on cache persistence. This way you don't have to manually mock out everything as requests are automatically recorded and played back. With cache persistence you can take tests completely offline.

The cache works with all types of requests and will distinguish between different POST requests to the same URL.

Params

Billy.configure do |c|
  c.cache = true
  c.cache_request_headers = false
  c.ignore_params = ["http://www.google-analytics.com/__utm.gif",
                     "https://r.twimg.com/jot",
                     "http://p.twitter.com/t.gif",
                     "http://p.twitter.com/f.gif",
                     "http://www.facebook.com/plugins/like.php",
                     "https://www.facebook.com/dialog/oauth",
                     "http://cdn.api.twitter.com/1/urls/count.json"]
  c.path_blacklist = []
  c.merge_cached_responses_whitelist = []
  c.persist_cache = true
  c.ignore_cache_port = true # defaults to true
  c.non_successful_cache_disabled = false
  c.non_successful_error_level = :warn
  c.non_whitelisted_requests_disabled = false
  c.cache_path = 'spec/req_cache/'
  c.certs_path = 'spec/req_certs/'
  c.proxy_host = 'example.com' # defaults to localhost
  c.proxy_port = 12345 # defaults to random
  c.proxied_request_host = nil
  c.proxied_request_port = 80
  c.cache_whitelist = []
  c.record_requests = true # defaults to false
  c.cache_request_body_methods = ['post', 'patch', 'put'] # defaults to ['post']
end

Example usage:

require 'table_print' # Add this dependency to your gemfile

Billy.configure do |c|
  c.record_requests = true
end

RSpec.configure do |config|
  config.prepend_after(:example, type: :feature) do
    puts "Requests received via Puffing Billy Proxy:"

    puts TablePrint::Printer.table_print(Billy.proxy.requests, [
      :status,
      :handler,
      :method,
      { url: { width: 100 } },
      :headers,
      :body
    ])
  end
end

This will generate a human readable list of all requests after each test run:

Requests received via Puffing Billy Proxy:
STATUS   | HANDLER | METHOD  | URL                                     | HEADERS                        | BODY
---------|---------|---------|-----------------------------------------|--------------------------------|-----
complete | proxy   | GET     | http://127.0.0.1:56692/                 | {"Accept"=>"text/html,appli... |
complete | proxy   | GET     | http://127.0.0.1:56692/assets/appl...   | {"Accept"=>"text/css,*/*;q=... |
complete | proxy   | GET     | http://127.0.0.1:56692/assets/app...    | {"Accept"=>"*/*", "Referer"... |
complete | proxy   | GET     | http://127.0.0.1:56692/javascript/index | {"Accept"=>"text/html,appli... |
complete | stubs   | OPTIONS | https://api.github.com:443/             | {"Access-Control-Request-Me... |
complete | stubs   | GET     | https://api.github.com:443/             | {"Accept"=>"*/*", "Referer"... |
inflight |         | GET     | http://127.0.0.1:56692/example          | {"Referer"=>"http://127.0.0... |
.

Finished in 1.98 seconds (files took 2.11 seconds to load)
1 example, 0 failures

The handler column indicates how Puffing Billy handled your request:

If your status is set to inflight this request has not yet been handled fully. Either puffing billy crashed internally on this request, or your test ended before it could complete successfully.

Cache Scopes

If you need to cache different responses to the same HTTP request, you can use cache scoping.

For example, an index page may return zero or more items in a list, with or without pagination, depending on the number of entries in a database.

There are a few different ways to use cache scopes:

# If you do nothing, it uses the default cache scope:
it 'defaults to nil scope' do
  expect(proxy.cache.scope).to be_nil
end

# You can change context indefinitely to a specific cache scope:
context 'with a cache scope' do
  before do
    proxy.cache.scope_to "my_cache"
  end

  # Remember to set the cache scope back to the default in an after block
  # within the context it is used, and/or at the global rails_helper level!
  after do
    proxy.cache.use_default_scope
  end

  it 'uses the cache scope' do
    expect(proxy.cache.scope).to eq("my_cache")
  end

  it 'can be reset to the default scope' do
    proxy.cache.use_default_scope
    expect(proxy.cache.scope).to be_nil
  end

  # Or you can run a block within the context of a cache scope:
  # Note: When using scope blocks, be sure that both the action that triggers a
  #       request and the assertion that a response has been received are within the block
  it 'can execute a block against a named cache' do
    expect(proxy.cache.scope).to eq("my_cache")
    proxy.cache.with_scope "another_cache" do
      expect(proxy.cache.scope).to eq "another_cache"
    end
    # It
    expect(proxy.cache.scope).to eq("my_cache")
  end
end

If you use named caches it is highly recommend that you use a global hook to set the cache back to the default before or after each test.

In Rspec:

RSpec.configure do |config|
  config.before :each { proxy.cache.use_default_scope }
end

Separate Cache Directory for Each Test

If you want the cache for each test to be independent, i.e. have it's own directory where the cache files are stored, you can do so.

in Cucumber

use a Before tag:

Before('@javascript') do |scenario, block|
  Billy.configure do |c|
    feature_name = scenario.feature.name.underscore
    scenario_name = scenario.name.underscore
    c.cache_path = "features/support/fixtures/req_cache/#{feature_name}/#{scenario_name}/"
    FileUtils.mkdir_p(Billy.config.cache_path) unless File.exist?(Billy.config.cache_path)
  end
end

in Rspec

use a before(:each) block:

RSpec.configure do |config|
  base_cache_path = Billy.config.cache_path
  base_certs_path = Billy.config.certs_path
  config.before :each do |x|
    feature_name = x.metadata[:example_group][:description].underscore.gsub(' ', '_')
    scenario_name = x.metadata[:description].underscore.gsub(' ', '_')
    Billy.configure do |c|
      cache_scenario_folder_path = "#{base_cache_path}/#{feature_name}/#{scenario_name}/"
      FileUtils.mkdir_p(cache_scenario_folder_path) unless File.exist?(cache_scenario_folder_path)
      c.cache_path = cache_scenario_folder_path

      certs_scenario_folder_path = "#{base_certs_path}/#{feature_name}/#{scenario_name}/"
      FileUtils.mkdir_p(certs_scenario_folder_path) unless File.exist?(certs_scenario_folder_path)
      c.certs_path = certs_scenario_folder_path
    end
  end
end

Stub requests recording

If you want to record requests to stubbed URIs, set the following configuration option:

Billy.configure do |c|
  c.record_stub_requests = true
end

Example usage:

it 'should intercept a GET request' do
  stub = proxy.stub('http://example.com/')
  visit 'http://example.com/'
  expect(stub.has_requests?).to be true
  expect(stub.requests).not_to be_empty
end

Proxy timeouts

By default, the Puffing Billy proxy will use the EventMachine::HttpRequest timeouts of 5 seconds for connect and 10 seconds for inactivity when talking to downstream servers.

These can be configured as follows:

Billy.configure do |c|
  c.proxied_request_connect_timeout = 20
  c.proxied_request_inactivity_timeout = 20
end

Customising the javascript driver

If you use a customised Capybara driver, remember to set the proxy address and tell it to ignore SSL certificate warnings. See lib/billy.rb to see how Billy's default drivers are configured.

Working with VCR and Webmock

If you use VCR and Webmock elsewhere in your specs, you may need to disable them for your specs utilizing Puffing Billy. To do so, you can configure your rails_helper.rb as shown below:

RSpec.configure do |config|
  config.around(:each, type: :feature) do |example|
    WebMock.allow_net_connect!
    VCR.turned_off { example.run }
    WebMock.disable_net_connect!
  end
end

As an alternative if you're using VCR, you can ignore requests coming from the browser. One way of doing that is by adding to your rails_helper.rb the excerpt below:

VCR.configure do |config|
  config.ignore_request do |request|
    request.headers.include?('Referer')
  end
end

Note that this approach may cause unexpected behavior if your backend sends the Referer HTTP header (which is unlikely).

Raising errors from stubs

By default Puffing Billy suppresses errors from stub-blocks. To make it raise errors instead, add this test initializers:

EM.error_handler { |e| raise e }

SSL usage

Unfortunately we cannot setup the runtime certificate authority on your browser at time of configuring the Capybara driver. So you need to take care of this step yourself as a prepartion. A good point would be directly after configuring this gem.

Google Chrome Headless example

Google Chrome/Chromium is capable to run as a test browser with the new headless mode which is not able to handle the deprecated --ignore-certificate-errors flag. But the headless mode is capable of handling the user PKI certificate store. So you just need to import the runtime Puffing Billy certificate authority on your system store, or generate a new store for your current session. The following examples demonstrates the former variant:

# Install the fabulous `os` gem first
# See: https://rubygems.org/gems/os
# gem install os
#
# --

# Overwrite the local home directory for chrome. We use this
# to setup a custom SSL certificate store.
ENV['HOME'] = "#{Dir.tmpdir}/chrome-home-#{Time.now.to_i}"

# Clear and recreate the Chrome home directory.
FileUtils.rm_rf(ENV['HOME'])
FileUtils.mkdir_p(ENV['HOME'])

# Setup a new pki certificate database for Chrome
if OS.linux?
  system <<~SCRIPT
    cd "#{ENV['HOME']}"
    curl -s -k -o "cacert-root.crt" "http://www.cacert.org/certs/root.crt"
    curl -s -k -o "cacert-class3.crt" "http://www.cacert.org/certs/class3.crt"
    echo > .password
    mkdir -p .pki/nssdb
    CERT_DIR=sql:$HOME/.pki/nssdb
    certutil -N -d .pki/nssdb -f .password
    certutil -d ${CERT_DIR}  -A -t TC \
      -n "CAcert.org" -i cacert-root.crt
    certutil -d ${CERT_DIR} -A -t TC \
      -n "CAcert.org Class 3" -i cacert-class3.crt
    certutil -d sql:$HOME/.pki/nssdb -A \
      -n puffing-billy -t "CT,C,C" -i #{Billy.certificate_authority.cert_file}
  SCRIPT
end

# Setup the macOS certificate store
if OS.mac?
  prompt = 'Add Puffing Billy root certificate authority ' \
           'to system certificate store'
  system <<~SCRIPT
    sudo -p "# #{prompt}`echo $'\nPassword: '`" \
      security find-certificate -a -Z -c 'Puffing Billy' \
      | grep 'SHA-1 hash' | cut -d ':' -f2 | xargs -n1 \
      sudo security delete-certificate -Z >/dev/null 2>&1 || true
    sudo security add-trusted-cert -d -r trustRoot \
      -k /Library/Keychains/System.keychain \
      #{Billy.certificate_authority.cert_file}
  SCRIPT
end

Mind the reset of the HOME environment variable. Fortunately Chrome takes care of the users home, so we can setup a new temporary directory for the test run, without messing with potential user configurations.

The macOS support requires the input of your password to manipulate the system certificate store. If you are lazy you can turn off sudo password prompt for the security command, but it's strongly advised against. (You know passwordless security, is no security in this case) Further, the macOS handling here cleans up old Puffing Billy root certificate authorities and put the current one into the system store. So after a run of your the suite only one certificate will be left over. If this is not enough you can handling the cleanup again with a custom on-after hook.

TLS hostname validation

em-http-request was modified to emit a warning if being used without the TLS verify_peer option. Puffing Billy defaults to specifying verify_peer: false but you can now modify configuration to do peer verification. So if you've gone to the trouble of setting up your own certificate authority and self-signed certs you can enable it like so:

Billy.configure do |c|
  c.verify_peer = true
end

Resources

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

TODO

  1. Integration for test frameworks other than RSpec.
  2. Show errors from the EventMachine reactor loop in the test output.