Reflow v4.18.1

Changelog

When the reflow CLI is executed, it will compare its version against the API server running. It will warn upon any minor version increases, and fail on any major version increases.

This tool generally follows semantic versioning. Given a version number [MAJOR.MINOR.PATCH]:

A MAJOR version increase is made when the API has changed in such a way that the CLI is now incompatible
A MINOR version increase is made when a backwards compatible functionality is added
A PATCH version increase is made when a backwards compatible fix/chore is implemented.

For stability, it is always recommended to always run the latest version of the reflow CLI.

This file is also available at https://reflow.io/changelog

June 07, 2022

Fix

Add missing dependency (esbuild) into npm package

June 07, 2022

Feature

Unlimited size snippets. Snippets now bundle dependencies. Snippets can bundle dependencies as text via "!inline!" as a prefix to the import path statement.

June 06, 2022

Fix

Fix element psuedo-selectors from being un-interactable: propagate interactions to parent elements.

May 17, 2022

Fix

Introduce low-interval graphql polling on the client to handle appsync subscription failures

May 16, 2022

Fix

Fix a recording edge case where the selected element wouldn't be correctly picked when there were multiple aria-label elements with the same value

Chore

Documentation when opening the test recorder in the cloud to recommend its run locally

May 13, 2022

Feature

Additional --stable option in CLI to optionally enable CLI to do a stable run.

Speed

Speed up CLI by reducing unnecessary data load when making test runs

May 11, 2022

Feature

Allow extraction of more element attributes, including input values.

May 3, 2022

Feature

Track application releases via sourcemap content hashing

Feature

Capability to download sourcemap content.

Feature

Capability to associate multiple browser sequences with an application

Feature

Capability to disable screenshot stabilisation for faster action execution.

Speed

Optimized replay pathways: reducing all wait times on a per-action basis. Approximately 70% speedup.

Speed

Move screenshot comparison into a worker thread

Speed

Move screenshot comparison outside hot pathway: do not wait for comparison before progression

Speed

Do not load chatwoot unless button is pressed

Fix

Install browsers on reflow install

Fix

Remove duplicate page load on loading up a recorder instance.

April 19, 2022

Feature

Embed support widget (chatwoot)

April 15, 2022

Fix

Automatically install browsers after installing reflow locally

April 9, 2022

Fix

Improve mobile view of recording pane: make panel persistent, fix padding

April 8 2022

Fix

Expire images in frontend S3 cache after 15 minutes : this should prevent stale images from being shown on client, but still allow browser to cache the presigned image.

April 5 2022

Speed

Save test actions in parallel.

Fix

Sync theme between devtools pane and reflow main app

Fix

Auto reload chrome when DNS propagation appears to be an issue whilst waiting for recorder

Fix

Fix image comparison assertions inside deep iframe

Fix

Adjust backup to screencast frames to run more often

Fix

Increase expiry timeout on presigned images in UI to avoid them expiring after browser cache expires but before refresh.

Mar 25 2022

Feature

Action timeouts are now configurable. Apply timeouts to snippets / execute javascript functions.

Mar 23 2022

Feature

CLI command to run a single image comparison. Local usage only.

Feature

CLI commands without API keys cache credentials on local machine for 15 minutes -- eliminating the need for re-login with sequential commands

Feature

New Action: "Execute Snippet" and corresponding "Push Snippets" "Pull Snippets" CLI commands to syncronize them between the server and a local folder.

Feature

Run Metric Graphs showing all run results and duration at the top of the run history.

Mar 17 2022

Fix

Handle additional image cropping edge cases where targeted element is hidden with width outside of the image width

Mar 17 2022

Fix

Increase Apollo Timeout in API Key CLI.

Mar 17 2022

Fix

Increase Apollo Timeout in JWT CLI to account for lower internet connection speeds.

Mar 17 2022

Fix

Fix edge case in image comparison when cropping an image to a size bigger than the destination; could happen upon significant visual changes.

Mar 16 2022

Fix

Ensure CLI-passed variables are also saved in test metadata for CLI filtering.

Mar 16 2022

Feature

Capability to pass device-emulation-profile name to CLI

Mar 16 2022

Fix

More resilient authentication via force refreshing session every 20 minutes.

Fix

Fix a bug in executing custom browser javascript causing some interactions to not execute.

Mar 16 2022

Feature

New action for executing custom javascript in browser.

Feature

No longer require API key for CLI pull command -- default to login if not available

Mar 14 2022

Feature

CLI command to download all images for a given test run. Requires API key.

Mar 13 2022

Feature

Capability to add an API key that allows cli/api access without login. Available to AWS Marketplace users only

Mar 08 2022

Feature

Capability to add a description to a Test Action

Mar 07 2022

Feature

Implement test name/description/device changes via UI

Fix

Handle ambiguity better when performing the "best selector" search.

Feb 28 2022

Chore

Documentation when opening the test dashboard with no pre-existing tests

Feb 28 2022

Feature

Auto healing button now also prompts to auto heal visual regressions

Feb 23 2022

Feature

Capability to send a kill command to a cloud runner to stop an in-progress run

Feb 22 2022

Speed

Preload the recorder whilst waiting for VM to start

Feb 21 2022

Feature

Always show picture of "best effort" element selector upon a failed-to-find element interaction

Feature

Add Auto heal button to rewrite the test action with that "best effort" selector.

Feature

More stability improvements in element match

Speed

Frontend optimization around realtime subscription updates

Feb 17 2022

Feature

Sort customer usage records

Feature

Enable sorting of more test/run grid columns

Feb 16 2022

Feature

Expose customer usage records for AWS Marketplace customers in the billing page.

Fix

Remove overlay of element dimensions on captured screenshots

Fix

Better handling of record page interactions where device scale does not match viewport scale.

Feb 14 2022

Feature

Serverless recorders in preparation for multi-region / custom runners push

Security

Isolate runs to not run on shared VMs; each team now on an isolated virtual cluster. Breaking change due to API protocol change.

Feature

Expose more runner information in a Runners UI.

Feature

Expose more details about the Run's runner in API response

Feb 03 2022

Feature

Track speed of DOM changes for action element relative to the prior action finish time and use that to power element waiting loops. Speeds up replayed element interactions without sacrificing reliability.

Fix

Backend upgraded (now uses appsync pipeline resolvers), with some non-breaking graphql api changes introduced

Fix

Increased reliability around page frame changes; more retry loops whilst waiting for page frame to load.

January 16 2022

Feature

Software Licenses / AWS Marketplace integration

Fix

Integration with new backend infrastructure stack; adjusted API URL

Fix

When an initial URL is invalid, the recorder still opens and provides an appropriate error

November 23 2021

Feature

Single click oauth login

Fix

Dark mode now also applies to authenticator component

November 05 2021

Feature

Capability to use an emulated mobile device to record / run tests.

Feature

Capability to execute a test against a different device to the one configured in the test.

Feature

Capability to compose pipeline actions with different devices to the one configured in the test.

Feature

Visual Diff (Percentage) of the associated action page screenshot against its baseline is rendered on the test run action.

Feature

Capability to explicitly use a given baseline in the execute run UI.

Feature

When running a run without pre-defining a baseline, the most recent successful run for the same device/test version is used. If it is the first run of that device, the most recent successful run of that test version is used. If it is the first run of that test version, then it is not compared with any other run.

November 01 2021

Fix

Tab width of watch recording tabs works better on mobile

Speed

Lowered datastore overhead

October 29 2021

Feature

Nodes now register/heartbeat against a manager process that points them towards things to execute. Avoids run contention and enables higher parallelism, usage based pricing. Breaking change, as polling for runs now unsupported

Feature

Screenshot stability now powered by SSIM. Custom Mix of SSIM and pixeldiff used for visualistation of diff.

Fix

Playwright version now pinned to avoid playwright screenshot bug

Fix

First / last actions now show post/prior screenshot respectively (previously were just a blank screenshot)

Fix

Retry loops added to page stability functions

Fix

Frame injection ensured before executing findElement calls; fixing failure caused by frame too slow.

Fix

Fix capturing click/drag events when viewport is scrolled

Speed

Screenshots of selected nodes are only done when the custom action form is open.

October 20 2021

Feature

Assertions can now be raised as either warnings or errors (previously, only errors). Assertions configured as warnings do not block the flow, but are highlighted when viewing results.

October 16 2021

Fix

Dark mode now default

October 15 2021

Feature

Highlight the elements that reflow interacts with in UI

Feature

Highlight mouse coordinates on selector / page images for mouse click or drag events

Feature

Dark mode

Fix

Completed elements now link appropriately to old test actions

Fix

Exponential-Backoff retry loops around element node screenshotting to improve reliability

Fix

Consolidation between <component>:has-text(<text>) and <component>=<text> selector scores.

October 12 2021

Fix

Better support for refreshing devtools by creating a cdp session per connected one, instead of sharing a global cdp session

Fix

Better support for iframe actions where frame url doesn't exactly match

Fix

Better support for click/drag by tracking mouse location statefully. CDP event for mouse release could, under some conditions, have incorrect X/Y params so instead "most recent" mouse move event is now used

Fix

Add a backup method to CDP screencast frame/frame acks. CDP only has frame/frame-ack as a best-effort API. This adds an interval check so that if we know about DOM updates but haven't had a new frame since before DOM updates and a timeout (2.5s) the CDP screencast session is refreshed via stop/start APIs.

Feature

Show visual comparisons of elements after assert step.

Feature

Never show again option for action delete confirmations.

Security

Enable chromium sandbox when run on local machines to reduce untrusted test execution risk.

October 4 2021

Speed

Only use S3 to store non-draft images

Feature

Visual screenshot assert: capability to asserts a given element matches the exact screenshot of an element when the test was first recorded

Feature

In custom action selector form UI, the selected element will now show a screenshot of the captured element just below the preferred selector.

Fix

Find typescript more reliably when invoked in a non-npm directory.

September 31 2021

Fix

Remove browser event memoization layer; recalculate selectors just before supplementing action details.

Speed

Ensure reflow only gets evaluated once per frame via stripping the magic symbol and replacing it with a hardcoded text constant on document root.

Speed

Increase performance inside browser by memoizing events and only recalculating selectors once per event.

Speed

Increase performance by only sending mouse move events when mouse is down.

September 30 2021

Feature

Fill actions with a selector containing input[type="password"] get their text title replaced with '*'s.

Feature

Show number of found elements and associated score for each selector in selector layout UI.

Feature

Support for clicking / dragging elements.

This site uses cookies to enhance your user experience and conduct analytics to understand how our services are used. By continuing to use our site, you consent to the placement and use of cookies as described in our Privacy Policy.