Awesome

elgentos/magento2-cypress-testing-suite

A community-driven Cypress testing suite for Magento 2

This testing suite was announced in Peter Jaap his talk about Cypress testing in Magento 2 at Reacticon 4. Watch the talk for more information about Cypress testing.

You can see all runs of this testing suite in the public Cypress Dashboard for this project. See for example this succesfull run.

Table of Contents

• Prerequisites
• Limitations
• Progress
• Setup
• Running
• Videos
• Contributing

Prerequisites

• Magento 2.3.x / 2.4.x
• Hyvä (1.1.17) or Luma-based theme
• npm
• An admin bearer token (see Setup)

Assumptions & limitations

Assumptions

• Magento 2 runs in Single Store Mode

• Default language is English

• Viewport is 1920x1280, with support for mobile viewports

• Sample Data is installed

• Multi Source Inventory is not used

Not plug & play

This test suite is not plug & play for your store. A number of tests rely on Magento's default sample data. These tests will fail when you don't have the sample data. It is up to you to change the fixtures/selectors/tests to make them pass for your store.

Open source-only right now

We don't do Commerce builds over at elgentos, so we haven't spent time creating tests for Commerce-only functionality.

No 100% test coverage

We do not particularly strive for 100% test coverage. We have identified the most common and most revenue-dependent scenarios. For example, we do test viewing products, filtering categories, adding products to the cart etc, but we (currently) do not test the Email a Friend or Compare Products feature since these are rarely used in an average Magento store. We are perfectly willing to merge a PR with these tests of course.

No extensibility / inheritance of tests

You need to copy the whole suite into your project. We are open to suggestions on how to solve this, see Discussions.

The Hyvä checkout tests assume the React Checkout is used

The Hyvä checkout tests assume the Hyvä React Checkout. To skip checkout related Hyvä tests, set the environment variable CYPRESS_MAGENTO2_SKIP_CHECKOUT.
PR's are definitely welcome to improve the checkout related tests.

Progress

We are at 80%; 69 out of the proposed 86 tests are done.

Wording & naming are subject to change.

Installation

First, install Cypress and the dependencies in the root of your Magento 2 project:

npm ci

The easiest way to install the tests is to clone this repository and move the cypress folder into your project. As of right now, we do not provide a fallback mechanism for customizations to the tests, see Limitations.

git clone git@github.com:elgentos/magento2-cypress-testing-suite.git
mv magento2-cypress-testing-suite/cypress .
mv magento2-cypress-testing-suite/cypress.config.js .
rm -rf magento2-cypress-testing-suite
npm install cypress@^12.2.0 cypress-localstorage-commands@^2.2.2 cypress-tags@^1.1.2 typescript@^4.8.3

If you only need the Hyvä tests:

rm -rf cypress/{fixtures,page-objects,integration}/luma
rm -rf cypress/{integration}/luma-accessibility

If you only need the Luma tests;

rm -rf cypress/{fixtures,page-objects,integration}/hyva

Then edit the cypress.config.js file in the root of your project to update your baseUrl, the projectId and possibly some other defaults we've set:

const baseUrl =
  process.env.NODE_ENV === "develop"
    ? "http://cypress.magento2.localhost"
    : "https://example.com/";

    projectId: "8vuidn"

Also add these lines to your .gitignore to avoid cluttering your Git repo;

node_modules
cypress/screenshots
cypress/videos
cypress.env.json

Accessibility test installation

The accessibility test specs use cypress-axe and as such need extra items installed via npm.

npm install --save-dev cypress-axe
npm install --save-dev axe-core

Setup

Some tests are dependent on making changes in the database. This is done through the Magento 2 REST API. You will need to create an admin token for these tests. This is easily done using magerun2.

Get a list of all the admin users: magerun2 admin:user:list

If you don't have any admin users, create one; magerun2 admin:user:create --admin-user=john --admin-firstname=John --admin-lastname=Doe --admin-password=JohnDoe123 --admin-email="john@example.com"

Then create a token: magerun2 admin:token:create username_goes_here

You then can add the token to cypress.env.json as an environment variable:

{
  "MAGENTO2_ADMIN_TOKEN": "token_goes_here"
}

Overwrite the base url

{
  "MAGENTO2_BASE_URL": "https://demoshops.splendid-internet.de/magento/demoshop-magento2-daily"
}

Alternatively you can set it in your CI/CD variables by prefixing the environment variable name with CYPRESS_: CYPRESS_MAGENTO2_ADMIN_TOKEN: <token_goes_here>.

Tags

We use tags to discern between hot tests and cold tests. If you followed the installation instructions above the cypress-tags module is already installed.
Note: it is used in the cypress.config.js file in the setupNodeEvents callback.

Running

npx cypress run
# npx cypress open # if you want to use the GUI

Individual specs can be run using the following command:

npx cypress run --spec ./cypress/integration/path/to/some.spec.js

Hot and cold tests based on tags can be run using the following command:

CYPRESS_INCLUDE_TAGS=hot npx cypress run

Running against local environment

Set up your local base URL in cypress.config.js, or export CYPRESS_MAGENTO2_BASE_URL.
Then run Cypress with NODE_ENV=develop; npx cypress run.

Environment variables

Even though the test suite is intended to become part of a project, it is possible to change some behavior using environment variables.
This is useful for running the suite in different environments, for example, development, CI, or against production.

• NODE_ENV if set to develop the development base URL configured in cypress.config.js will be used, and the default timeout is set to 10 seconds
• CYPRESS_MAGENTO2_BASE_URL If set, this value will be used as the Magento 2 base_url. Otherwise, the base URL from cypress.config.js will be used.
• CYPRESS_MAGENTO2_SPEC_PATTERN If set, only tests matching this glob pattern will be executed. Otherwise, the tests configured in cypress.config.js will be used.
• CYPRESS_MAGENTO2_EXPORT_PATTERN If set, tests matching this glob pattern will be excluded.
• CYPRESS_MAGENTO2_DEFAULT_TIMEOUT If set, used as the default timeout. Otherwise, the timeout defaults to 10 seconds if NODE_ENV is set to develop, or 4 seconds otherwise.
• CYPRESS_MAGENTO2_ADMIN_TOKEN Used to authenticate against the Magento 2 API for setting up test fixtures.
• CYPRESS_MAGENTO2_SKIP_CHECKOUT Set to a truthy value to skip any Hyvä tests that assume a Checkout is installed.
• CYPRESS_MAGENTO2_SPEC_SUITE Set the test suite to run, if not set defaults to luma or hyva depending on response headers.

Any of these can also be configured in a cypress.env.json file without the CYPRESS_ prefix.

Running tests from modules

Spec files in extensions will be found by the default spec pattern at

• app/code/**/Test/Cypress/hyva/**/*.spec.js or app/code/**/Test/Cypress/luma/**/*.spec.js
• vendor/**/Test/Cypress/hyva/**/*.spec.js or vendor/**/Test/Cypress/luma/**/*.spec.js

Tests for different frontends can be supplied by using a different folder in Test/Cypress and setting the MAGENTO2_SPEC_SUITE config to that name. For example: app/code/Example/Module/Test/Cypress/vue/user/account.spec.js would be found by

CYPRESS_MAGENTO2_SPEC_SUITE=vue npx cypress run

If you do not want all tests to be run, regardless of the folder names, set MAGENTO2_SPEC_SUITE to an empty string.

Running accessibility tests

Accessibility spec files will not be found by the default spec pattern. This was desired as they should be seen as optional.

To run the accessibility tests locally you can update your cypress.env.json to include the following

{
    "MAGENTO2_SPEC_PATTERN": "cypress/integration/luma-accessibility/**/*.spec.js"
}

Videos

https://user-images.githubusercontent.com/431360/193906592-2859ce76-c889-4377-afa0-a5d01ee06919.mp4

https://user-images.githubusercontent.com/431360/193906756-a4f384c8-c6e2-422b-bf16-38134d34af25.mp4

https://user-images.githubusercontent.com/431360/193906780-8e2e62ce-23db-406c-82f1-080d17409934.mp4

https://user-images.githubusercontent.com/431360/193906808-f0a04467-72d4-4ef4-a1f4-e05de8f16252.mp4

Contributing

We are very open to contributions! We would love to have mobile viewport support for Hyvä, tests for Commerce functionality, additional tests, code improvements, a fallback mechanism, etcetera etcetera. See the Issues tab for issues to pick up.

We will be updating this readme soon with extensive contribution guidelines, but here is a short summary:

• Avoid creating global cy functions (Custom Commands), instead put functions in utils/helpers and import them
• Avoid creating aliases that are only used once
• Use cy.get() as much as possible, only use cy.contains() in specific cases - try to avoid it
• Do not add assertions to page objects, move those to the spec files. Red flag; should() in a page object
• Every test (an it() function) has to be able to run stand-alone; it should not depend on any other test. You can test this by add .only (see Cypress docs).