Skip to main content

Our tool helps product teams create and maintain Automated Browser Tests. Interact with your website through Reflow, and we record reproducible sequences of browser interactions.

When you release a new version of your application, reflow can replay these interactions against your new version to validate all your configured user flows.

It can be run locally or accessed at app.reflow.io. It supports Chromium(Chrome/Edge), Firefox, and WebKit (Safari)

It requires zero programming knowledge to use. However, Reflow wraps the Playwright end-to-end testing framework, and can be optionally extended with snippets to execute arbitrary functionality.

Capabilities​

  • It enables browser sequences to be recorded in a no-code web UI, and executed either locally or in a cloud-based test runner.
  • To record tests, users:
  • To replay tests, users either:
    • Execute the test in the Web UI locally, which runs the test on the local machine
    • Execute the tests via the CLI, optionally using an API key for CI servers.
    • Execute the test in the cloud web UI, which will run the test in the reflow cloud.
    • Execute the test as part of a Pipeline, which will execute a set of tests concurrently, optionally on a schedule.
  • A test can be parameterized with:
    • Key/Value string pairs, which can be used to:
      • Configure actions with ES6 template variables
      • Conditionally execute actions using snippets
  • A test can be:
  • Visual assertions allow:
    • Tests executions to be inspected against previous runs
    • Specific elements to be asserted on against recorded screenshots, either raising warnings or stopping the test when a mismatch occurs.
    • An output diff image demonstrated the differences between the two executions.
    • Differences are both compared and highlighted via the SSIM algorithm.
  • Snippets allow:
    • Users to add custom typescript handler functions, executed inline during the test in an isolated VM.
    • These handler functions can access and invoke functions in the Playwright page context.
    • Have access to all variables, as well as custom parameters configured by the QA team.
    • Set/Change variables, which can be used in subsequent steps.

Visual Assertions​

Visual Assertions are a technique to verify that a page element appears to the same as you intend. This is useful when:

  • You do not expect anything to change, hence any change should raise either a warning or an error upon execution.
  • You want to provide an Approval workflow to validate changes on their look-and-feel.
  • You want to author your tests fast, and do not care for configuring specific element attribute matches

They can also be used for process automation workflows, such as producing and regenerating themed images using the CLI.

To create a visual assertion, open the Custom Action Dialog and select the Assertion action. The assertion type "Matches Visually" represents a visual assertion.

In the developer tools panel, select the element you want to visually assert on. Once this is done, the reference image should shortly be shown in the custom action form.

Finally, chose "Raise Warning" or "Raise Error" to determine the error condition of the visual assertion.

Auto Healing​

Auto healing is a technique which both minimizes the impact of flaky tests, and reduces maintenance effort to fix tests when they change.

Auto healing is possible for standard element interactions, such as Click, Fill, and Assert.

It works by a using combination of techniques to find an element on a page by comparing it to the element that was used when the test was last successful.

If these techniques result in either ambiguity (i.e. more than one element matches the fuzzy selector), or no element found, reflow will rank all possible elements on the page to find the closest match to the originally recorded element and fail the test. When a user looks at the failed test, reflow will immediately offer an "Auto Heal" button, that will visually demonstrate the closest match reflow found, and allow a user to approve a change to the original test that will replace the element selector with the new found one.

If the element comes from an assertion with type "Raise Warning", then multiple auto healing targets may be available within the test. Actions which can be auto-healed are colour coded yellow when viewing a test run.

Snippets​

A snippet is a reusable piece of code that can be used to automate a specific task. Snippets are defined by two exports.

  1. A handler function, which is called when the snippet is executed.
  2. A dependsOn array, which is a list of parameters that this snippet depends on.

Snippets must have a team-unique name. They will be referenced by the name in the test configuration.

If a snippet changes, all tests that depend on that snippet by name implicitly change too.

To create a snippet, start by navigating to the folder where you want to store your snippets, and execute reflowio pull snippet. This will create a snippet folder if it doesn't exist, and pull all snippets that exist under your team into that local directory.

import type { Page } from 'playwright-core';

interface Event {
   page: Page;
   variables: Record<string, string>;
}

export async function handler(event: Event): Promise<Event> {
   const page = event.page;
   await page.click('[placeholder="Email"]');
   await page.fill('[placeholder="Email"]', event.variables.email);
   await page.click('[placeholder="Password"]');
   await page.fill('[placeholder="Password"]', event.variables.password);
   await page.click(':text("EmailPasswordSign in") >> button:has-text("Sign in")');

   return event;
}

export const dependsOn = ['email', 'password'];

The snippet name is defined based on the filename, with all - characters replaced by .

The version of playwright defined is the same as in the reflowio dependency in npm. This may change over time, but if there is a breaking change we will introduce a runtime variable.

Snippets are compiled with esbuild and run in a sandbox environment on the local server. This means they do not have access to the filesystem, but can do anything they want in their VM. They can import other common utility functions into themselves: they will be compiled into a bundle before being pushed to the server. For instance, you can import a file upload directly into the bundle, then use it to interact with the page file upload API.

Please always use methods that eventually resolve or reject. If the handler function does not reject on a failure, it can execute for a long time: until the test eventually times out or the server is killed by the Runner UI.

To push snippets into reflow, execute reflowio push snippet. This will push all snippets in the local directory to the server, overriding upon any changes. To pull from reflow, execute reflowio pull snippets. This will overwrite the local directory's snippets if they already exist. Please make sure to commit any changes to version control to avoid the loss of your work.

Mobile Device Emulation​

There are three browser engines supported: Chromium(Chrome / Edge), Firefox and WebKit(Safari).

Android devices can be effectively emulated via the Chromium browser engine. iOS devices (iPhone / iPad) can be emulated via the WebKit browser engine.

The reflow web UI and CLI enable one test configuration to be executed against different device configurations. The device configuration effects the viewport, screen, userAgent, deviceScaleFactor (pixel density), isMobile and hasTouch flags.

Further emulation conditions, such as network latency/failure rates, locale/timezone emulation, geolocation, and permission options can be set using the Emulation API

Pipelines​

Pipelines are the composition model within Reflow.

A pipeline can have many stages. Each stage in a pipeline executes sequentially.

A stage can have many actions. Each action in a stage executes in parallel.

An action represents one test execution. When all actions in a stage are successful, the next pipeline stage is executed. If one action in a stage fails, the next stage does not execute.

Who is this for / Why was this built?​

Reflow is built to reduce the cost of test automation for product teams working on web applications. We saw lots of organisations with broken QA, and internalised the belief that reducing the cost of creating and maintaining automated testing would be valuable. So we built this.

More details in our blog.

Quick Start​

Install reflow via npm: npm install -g reflowio

The reflow dashboard is spawned via:

reflowio dashboard

To access the dashboard, you will need to create an account or login to the site at http://localhost:3100. Login credentials will be securely communicated via cognito.

Pricing​

Free​

No credit card is required for free user features:

  • All local storage, execution, and team features (up to 5 users) are free.
  • Cloud storage of anything needed to run your tests, including visual selector images and baseline screenshot comparisons is free.
  • Cloud storage of Test Execution Screenshots is subject to a 3 day retention policy. I.e. after 3 days, they will be deleted. This will not affect anything run locally, as it is cached on the local machine.
  • Cloud execution up to 30 minutes / month / user is free. I.e. for a 5 person team, 2.5 hours / month of cloud execution is included for free.

Licensing is via AWS Marketplace: an hourly charge will be added to your AWS bill based on your license usage. Full details within our pricing calculator. Includes:

  • Indefinite storage of Run Execution Screenshots / Traces
  • Unlimited Cloud Execution. A browser grid of up to 100 concurrent browsers is available. Concurrency limits can be adjusted.
  • Premium features such as API keys and warm hosts.
  • Cost Management options to ensure cloud bills remain within expected limits
  • High Priority Support via Email / Video Call

Support​

Support is available via live chat and support@reflow.io

This site uses cookies to enhance your user experience and conduct analytics to understand how our services are used. By continuing to use our site, you consent to the placement and use of cookies as described in our Privacy Policy.