This living document serves to prescribe instructions and best practices for writing end-to-end (E2E) tests with Playwright in the Gutenberg project.

Running tests

# Run all available tests.
npm run test:e2e

# Run in headed mode.
npm run test:e2e -- --headed

# Run tests with specific browsers (`chromium`, `firefox`, or `webkit`).
npm run test:e2e -- --project=webkit --project=firefox

# Run a single test file.
npm run test:e2e -- <path_to_test_file> # E.g., npm run test:e2e -- site-editor/title.spec.js

# Debugging.
npm run test:e2e -- --debug

If you’re developing in Linux, it currently requires testing Webkit browsers in headed mode. If you don’t want to or can’t run it with the GUI (e.g. if you don’t have a graphic interface), prepend the command with xvfb-run to run it in a virtual environment.

# Run all available tests.
xvfb-run npm run test:e2e

# Only run webkit tests.
xvfb-run -- npm run test:e2e -- --project=webkit

If you’re already editing in VS Code, you may find the Playwright extension helpful for running, writing and debugging tests.

Best practices

Read the best practices guide for Playwright.

Forbid $, use locator instead

In fact, any API that returns ElementHandle is discouraged. This includes $, $$, $eval, $$eval, etc. Locator is a much better API and can be used with playwright’s assertions. This also works great with Page Object Model since that locator is lazy and doesn’t return a promise.

Use accessible selectors

Use getByRole to construct the query wherever possible. It enables us to write accessible queries without having to rely on internal implementations.

// Select a button which includes the accessible name "Hello World" (case-insensitive).
page.getByRole( 'button', { name: 'Hello World' } );

It can also be chained to perform complex queries:

// Select an option with a name "Buttons" under the "Block Library" region.
page.getByRole( 'region', { name: 'Block Library' } )
    .getByRole( 'option', { name: 'Buttons' } )

See the official documentation for more info on how to use them.

Selectors are strict by default

To encourage better practices for querying elements, selectors are strict by default, meaning that it will throw an error if the query returns more than one element.

Inline simple utility helper function

Most utility functions are simple enough to be inlined directly in tests. With the help of accessible selectors, simple utils are easier to write now. For utils that only take place on a certain page, use Page Object Model instead. Otherwise, only create an util if the action is complex and repetitive enough.

Favor Page Object Model over utils

As mentioned above, Page Object Model is the preferred way to create reusable utility functions on a certain page.

The rationale behind using a POM is to group utils under namespaces to be easier to discover and use. In fact, PageUtils in the e2e-test-utils-playwright package is also a POM, which avoids the need for global variables, and utils can reference each other with this.

Restify actions to clear or set states

It’s slow to set states manually before or after tests, especially when they’re repeated multiple times between tests. It’s recommended to set them via API calls. Use requestUtils.rest and requestUtils.batchRest instead to call the REST API (and add them to requestUtils if needed). We should still add a test for manually setting them, but that should only be tested once.

Avoid global variables

In the previous E2E setup, page and browser were global variables, which made working with multiple pages or parallel tests harder.

@playwright/test uses fixtures to inject page, browser, and other parameters into tests.

Make explicit assertions

We can insert as many assertions in one test as needed. It’s better to make explicit assertions whenever possible. For instance, if we want to assert that a button exists before clicking on it, we can do expect( locator ).toBeVisible() before performing locator.click(). This makes the tests flow better and easier to read

Common pitfalls

Overusing snapshots

Cross-browser testing

By default, tests are only run in chromium. You can tag tests to run them in different browsers. Use @browser anywhere in the test title to run it in that browser. Tests will always run in chromium by default, append -chromium to disable testing in chromium. Available browsers are chromium, firefox, and webkit.

test( 'I will run in @firefox and @webkit (and chromium by default)', async ( { page } ) => {
    // ...
} );

test( 'I will only run in @firefox but not -chromium', async ( { page } ) => {
    // ...
} );

test.describe( 'Grouping tests (@webkit, -chromium)', () => {
    test( 'I will only run in webkit', async ( { page } ) => {
        // ...
    } );
} );