Skip to main content

Reflow v4.17.0

Changelog

When the reflow CLI is executed, it will compare its version against the API server running. It will warn upon any minor version increases, and fail on any major version increases.

This tool generally follows semantic versioning. Given a version number [MAJOR.MINOR.PATCH]:

  1. A MAJOR version increase is made when the API has changed in such a way that the CLI is now incompatible
  2. A MINOR version increase is made when a backwards compatible functionality is added
  3. A PATCH version increase is made when a backwards compatible fix/chore is implemented.

For stability, it is always recommended to always run the latest version of the reflow CLI.

This file is also available at https://reflow.io/changelog

May 17, 2022

Fix
Introduce low-interval graphql polling on the client to handle appsync subscription failures

May 16, 2022

Fix
Fix a recording edge case where the selected element wouldn't be correctly picked when there were multiple aria-label elements with the same value
Chore
Documentation when opening the test recorder in the cloud to recommend its run locally

May 13, 2022

Feature
Additional --stable option in CLI to optionally enable CLI to do a stable run.
Speed
Speed up CLI by reducing unnecessary data load when making test runs

May 11, 2022

Feature
Allow extraction of more element attributes, including input values.

May 3, 2022

Feature
Track application releases via sourcemap content hashing
Feature
Capability to download sourcemap content.
Feature
Capability to associate multiple browser sequences with an application
Feature
Capability to disable screenshot stabilisation for faster action execution.
Speed
Optimized replay pathways: reducing all wait times on a per-action basis. Approximately 70% speedup.
Speed
Move screenshot comparison into a worker thread
Speed
Move screenshot comparison outside hot pathway: do not wait for comparison before progression
Speed
Do not load chatwoot unless button is pressed
Fix
Install browsers on reflow install
Fix
Remove duplicate page load on loading up a recorder instance.

April 19, 2022

Feature
Embed support widget (chatwoot)

April 15, 2022

Fix
Automatically install browsers after installing reflow locally

April 9, 2022

Fix
Improve mobile view of recording pane: make panel persistent, fix padding

April 8 2022

Fix
Expire images in frontend S3 cache after 15 minutes : this should prevent stale images from being shown on client, but still allow browser to cache the presigned image.

April 5 2022

Speed
Save test actions in parallel.
Fix
Sync theme between devtools pane and reflow main app
Fix
Auto reload chrome when DNS propagation appears to be an issue whilst waiting for recorder
Fix
Fix image comparison assertions inside deep iframe
Fix
Adjust backup to screencast frames to run more often
Fix
Increase expiry timeout on presigned images in UI to avoid them expiring after browser cache expires but before refresh.

Mar 25 2022

Feature
Action timeouts are now configurable. Apply timeouts to snippets / execute javascript functions.

Mar 23 2022

Feature
CLI command to run a single image comparison. Local usage only.
Feature
CLI commands without API keys cache credentials on local machine for 15 minutes -- eliminating the need for re-login with sequential commands
Feature
New Action: "Execute Snippet" and corresponding "Push Snippets" "Pull Snippets" CLI commands to syncronize them between the server and a local folder.
Feature
Run Metric Graphs showing all run results and duration at the top of the run history.

Mar 17 2022

Fix
Handle additional image cropping edge cases where targeted element is hidden with width outside of the image width

Mar 17 2022

Fix
Increase Apollo Timeout in API Key CLI.

Mar 17 2022

Fix
Increase Apollo Timeout in JWT CLI to account for lower internet connection speeds.

Mar 17 2022

Fix
Fix edge case in image comparison when cropping an image to a size bigger than the destination; could happen upon significant visual changes.

Mar 16 2022

Fix
Ensure CLI-passed variables are also saved in test metadata for CLI filtering.

Mar 16 2022

Feature
Capability to pass device-emulation-profile name to CLI

Mar 16 2022

Fix
More resilient authentication via force refreshing session every 20 minutes.
Fix
Fix a bug in executing custom browser javascript causing some interactions to not execute.

Mar 16 2022

Feature
New action for executing custom javascript in browser.
Feature
No longer require API key for CLI pull command -- default to login if not available

Mar 14 2022

Feature
CLI command to download all images for a given test run. Requires API key.

Mar 13 2022

Feature
Capability to add an API key that allows cli/api access without login. Available to AWS Marketplace users only

Mar 08 2022

Feature
Capability to add a description to a Test Action

Mar 07 2022

Feature
Implement test name/description/device changes via UI
Fix
Handle ambiguity better when performing the "best selector" search.

Feb 28 2022

Chore
Documentation when opening the test dashboard with no pre-existing tests

Feb 28 2022

Feature
Auto healing button now also prompts to auto heal visual regressions

Feb 23 2022

Feature
Capability to send a kill command to a cloud runner to stop an in-progress run

Feb 22 2022

Speed
Preload the recorder whilst waiting for VM to start

Feb 21 2022

Feature
Always show picture of "best effort" element selector upon a failed-to-find element interaction
Feature
Add Auto heal button to rewrite the test action with that "best effort" selector.
Feature
More stability improvements in element match
Speed
Frontend optimization around realtime subscription updates

Feb 17 2022

Feature
Sort customer usage records
Feature
Enable sorting of more test/run grid columns

Feb 16 2022

Feature
Expose customer usage records for AWS Marketplace customers in the billing page.
Fix
Remove overlay of element dimensions on captured screenshots
Fix
Better handling of record page interactions where device scale does not match viewport scale.

Feb 14 2022

Feature
Serverless recorders in preparation for multi-region / custom runners push
Security
Isolate runs to not run on shared VMs; each team now on an isolated virtual cluster. Breaking change due to API protocol change.
Feature
Expose more runner information in a Runners UI.
Feature
Expose more details about the Run's runner in API response

Feb 03 2022

Feature
Track speed of DOM changes for action element relative to the prior action finish time and use that to power element waiting loops. Speeds up replayed element interactions without sacrificing reliability.
Fix
Backend upgraded (now uses appsync pipeline resolvers), with some non-breaking graphql api changes introduced
Fix
Increased reliability around page frame changes; more retry loops whilst waiting for page frame to load.

January 16 2022

Feature
Software Licenses / AWS Marketplace integration
Fix
Integration with new backend infrastructure stack; adjusted API URL
Fix
When an initial URL is invalid, the recorder still opens and provides an appropriate error

November 23 2021

Feature
Single click oauth login
Fix
Dark mode now also applies to authenticator component

November 05 2021

Feature
Capability to use an emulated mobile device to record / run tests.
Feature
Capability to execute a test against a different device to the one configured in the test.
Feature
Capability to compose pipeline actions with different devices to the one configured in the test.
Feature
Visual Diff (Percentage) of the associated action page screenshot against its baseline is rendered on the test run action.
Feature
Capability to explicitly use a given baseline in the execute run UI.
Feature
When running a run without pre-defining a baseline, the most recent successful run for the same device/test version is used. If it is the first run of that device, the most recent successful run of that test version is used. If it is the first run of that test version, then it is not compared with any other run.

November 01 2021

Fix
Tab width of watch recording tabs works better on mobile
Speed
Lowered datastore overhead

October 29 2021

Feature
Nodes now register/heartbeat against a manager process that points them towards things to execute. Avoids run contention and enables higher parallelism, usage based pricing. Breaking change, as polling for runs now unsupported
Feature
Screenshot stability now powered by SSIM. Custom Mix of SSIM and pixeldiff used for visualistation of diff.
Fix
Playwright version now pinned to avoid playwright screenshot bug
Fix
First / last actions now show post/prior screenshot respectively (previously were just a blank screenshot)
Fix
Retry loops added to page stability functions
Fix
Frame injection ensured before executing findElement calls; fixing failure caused by frame too slow.
Fix
Fix capturing click/drag events when viewport is scrolled
Speed
Screenshots of selected nodes are only done when the custom action form is open.

October 20 2021

Feature
Assertions can now be raised as either warnings or errors (previously, only errors). Assertions configured as warnings do not block the flow, but are highlighted when viewing results.

October 16 2021

Fix
Dark mode now default

October 15 2021

Feature
Highlight the elements that reflow interacts with in UI
Feature
Highlight mouse coordinates on selector / page images for mouse click or drag events
Feature
Dark mode
Fix
Completed elements now link appropriately to old test actions
Fix
Exponential-Backoff retry loops around element node screenshotting to improve reliability
Fix
Consolidation between <component>:has-text(<text>) and <component>=<text> selector scores.

October 12 2021

Fix
Better support for refreshing devtools by creating a cdp session per connected one, instead of sharing a global cdp session
Fix
Better support for iframe actions where frame url doesn't exactly match
Fix
Better support for click/drag by tracking mouse location statefully. CDP event for mouse release could, under some conditions, have incorrect X/Y params so instead "most recent" mouse move event is now used
Fix
Add a backup method to CDP screencast frame/frame acks. CDP only has frame/frame-ack as a best-effort API. This adds an interval check so that if we know about DOM updates but haven't had a new frame since before DOM updates and a timeout (2.5s) the CDP screencast session is refreshed via stop/start APIs.
Feature
Show visual comparisons of elements after assert step.
Feature
Never show again option for action delete confirmations.
Security
Enable chromium sandbox when run on local machines to reduce untrusted test execution risk.

October 4 2021

Speed
Only use S3 to store non-draft images
Feature
Visual screenshot assert: capability to asserts a given element matches the exact screenshot of an element when the test was first recorded
Feature
In custom action selector form UI, the selected element will now show a screenshot of the captured element just below the preferred selector.
Fix
Find typescript more reliably when invoked in a non-npm directory.

September 31 2021

Fix
Remove browser event memoization layer; recalculate selectors just before supplementing action details.
Speed
Ensure reflow only gets evaluated once per frame via stripping the magic symbol and replacing it with a hardcoded text constant on document root.
Speed
Increase performance inside browser by memoizing events and only recalculating selectors once per event.
Speed
Increase performance by only sending mouse move events when mouse is down.

September 30 2021

Feature
Fill actions with a selector containing input[type="password"] get their text title replaced with '*'s.
Feature
Show number of found elements and associated score for each selector in selector layout UI.
Feature
Support for clicking / dragging elements.
This site uses cookies to enhance your user experience and conduct analytics to understand how our services are used. By continuing to use our site, you consent to the placement and use of cookies as described in our Privacy Policy.