Home

Awesome

reg-suit

CircleCI npm version License: MIT

reg-suit is a command line interface for visual regression testing.

Getting Started

$ npm install -g reg-suit
$ cd path-to-your-project
$ reg-suit init
# Answer a few questions...

$ reg-suit run

asciicast

If you want more details, this sample repository may help you.

Plugins

reg-suit has it's own plugin system. Plugins integrate various functions and services into your project.

The following plugins are available:

For example, installing keygen-git-hash and publish-s3 plugins, you can get regression testing workflow according with GitHub flow as shown in the figure below.

GitHub workflow

CLI Usage

reg-suit [options] <command>

run command

Run visual testing, publish the current snapshot images, and send notifications. It's equivalent to reg-suit sync-expected && reg-suit compare && reg-suit publish -n.

sync-expected command

Fetch images published already as the expected snapshot data into working directory. The expected key is detected with the installed key-generator plugin, and installed publisher-plugin fetches images data.

compare command

Compare images in the actualDir with images fetched at sync-expected and create an HTML report.

publish command

Publish the compare result and actual images to external storage with publisher-plugin with the actual snapshot key generated by installed key-generator plugin.

init command

Install and configure reg-suit and plugins into your project.

prepare command

Configure the installed plugin(s). It's useful to configure reg-suit and plugins.

Global options

If you want more details, please exec reg-suit -h or reg-suit <command> -h.

Configuration

To configure reg-suit, put regconfig.json under the project root directory. regconfig.json should be JSON file such as:

{
  "core": {
    "workingDir": ".reg",
    "actualDir": "images",
    "thresholdRate": 0.05
  },
  "plugins": {
    "reg-keygen-git-hash-plugin": {},
    "reg-publish-s3-plugin": {
      "bucketName": "your-aws-s3-bucket"
    }
  }
}

The core section contains reg-suit core setting and the plugins section contains plugin specific options.

core

{
  actualDir: string;
  workingDir?: string;        // default ".reg"
  thresholdRate?: number;     // default 0
  thresholdPixel?: number;    // default 0
  enableAntialias?: boolean;  // default false
  ximgdiff?: {
    invocationType: "none" | "client";  // default "client"
  };
  concurrency?: number;       // default 4
}

plugins

Entries of plugins section are described as key-value pairs. Each key should be plugin name. If you want configurable value, see README.md under the each plugin package(e.g. packages/reg-publish-s3-plugin/README.md).

Embed environment values

reg-suit replaces embedded placeholders in plugins section to environment values at runtime. For example:

  "plugins": {
    "reg-publish-s3-plugin": {
      "bucketName": "$S3_BUCKET_NAME"
    }
  }

export S3_BUCKET_NAME="my-bucket"
reg-suit run

# reg-publish-s3-plugin is configured with the following value:
#
# {
#   "bucketName": "my-bucket"
# }

Smart difference detection

If you turn core.ximgdiff option on in regconfig.json, reg-suit outputs a report with x-img-diff-js.

x-img-diff-js is a difference detection engine which calculates more structural information than naive pixel based comparison result. reg-suit use this to display which parts of testing image were inserted or moved.

If invocationType is set to "client", x-img-diff-js works on your web browser (It uses Web Assembly and Web Workers, so you need "modern" browser).

Run with CI service

A working demonstration is here.

Workaround for Detached HEAD

reg-suit(git-hash-plugin) needs the current branch name to identify the base-commit hash. However, under some CI services' environment(e.g. TravisCI, WerckerCI), the HEAD is detached. So you should attach it explicitly.

For example:

# .github/workflows/reg.yml

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Use Node.js v10
        uses: actions/setup-node@v1
        with:
          node-version: "10.x"
      - name: npm install, build, and test
        run: |
          npm i
      - name: workaround for detached HEAD
        run: |
          git checkout ${GITHUB_REF#refs/heads/} || git checkout -b ${GITHUB_REF#refs/heads/} && git pull
      - name: run reg-suit
        run: |
          npx reg-suit run

# .travis.yml

script:
  - git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" # This line is necessary to disable --single-branch option to fetch all remote branches on TravisCI.
  - git fetch origin # Ditto
  - git checkout $TRAVIS_BRANCH || git checkout -b $TRAVIS_BRANCH
  - npx reg-suit run

# wercker.yml

build:
  steps:
    - script:
      name: Attach HEAD
      code: |
        git checkout $WERCKER_GIT_BRANCH || git checkout -b $WERCKER_GIT_BRANCH
    - script:
      name: Run reg-suit
      code: |
        npx reg-suit run

# appveyor.yml

environment:
  nodejs_version: "8"

install:
  - ps: Install-Product node $env:nodejs_version
  - git checkout %APPVEYOR_REPO_BRANCH%

test_script:
  - npx reg-suit run

# .gitlab-ci.yml

test:
  script:
    - git checkout $CI_COMMIT_REF_NAME || git checkout -b $CI_COMMIT_REF_NAME && git pull
    - npx reg-suit run

Examples

The following repositories using reg-suit. These repos can help you to introduce visual snapshot testing.

If you use reg-suit, let us know your repository. We'll list it at the above :)

Contribute

PRs are welcome!

Bootstrap

git clone https://github.com/reg-viz/reg-suit.git; cd reg-suit
yarn
yarn run bootstrap

Test

yarn run test

Remarks

License

MIT. See LICENSE.txt.

reg-viz