Selectors
When recording test actions, reflow collects large numbers of data points on the underlying element that's been interacted with.
When replaying a test action, reflow will intelligently look for the appropriate element to interact with by combining these data points, using multiple strategies together to beat non-determinism. It will:
- Wait until the network is idle (or a timeout)
- Wait until the page looks similar to how it looked when the test action was captured (or a timeout)
- Wait until the Preferred Selector is found, or a timeout based on how long it took for the element to become visible during the most recent successful run/recording
- If the preferred selector is found, unique, and appropriate (e.g. a "click" action will not work on elements with the attribute "disabled"), select that element
- Else, within a score-limit, rank all possible selectors for that element and its parents, evaluate these selectors based on their specificity, and if a statistically likely element comes out, and it visually looks like the recorded element, pick that one.
- If no statistically likely common element can be found within the score limit, increase the score limit and try again, caching prior responses and appending prior results until a clear winner comes out.
- If no clear winner comes out by a maximum score, fail the test, and propose an auto-healing fix with the best of the available options.
With this process, Reflow is highly likely to always select the same element between tests on an arbitrary website. If there are issues, then high specificity attributes are recommended to be added to the element or its parents to help reflow always select the same one.
In order, reflow scores specificity roughly as follows in the table below (lower score means higher specificity). This scoring is applied recursively from an element until the root of its DOM/Shadow DOM node. nth-child is used to combine parent and child selectors, should it not be a uniquely identified subelement when combined with a parent selector.
| Element Attribute | Score |
|---|---|
data-testid or data-test-id or data-test | 1 |
placeholder | 10 |
aria-label | 10 |
alt | 10 |
role | 50 |
name | 50 |
type | 50 |
id | 100 |
| Element Inner text | 100 * MULT(length of text) |
| Element Node Name | 200 |
| Minimal Combined Class Name that's unique in page | 200 |
These scores are multiplied up/down depending on the number of similiar elements in the page frame. I.e. if a page element with placeholder="enter text" is found five times, the score for that selector will be multiplied by 5 (50).
Auto Healing
When the web page updates in such a way that the element changes, the new element selector hierarchy is collected during the test run using the specificity rules above. Should the test succeed, subsequent tests will use the most recent successful run to guide element selection.
Due to this, as long as changes are relatively small over time, the test will auto-heal interactions where a manually written test would require additional work to fix.
For example, if a software release changed an anchor node (link) into a button, but one of the following is true: [kept in the same element hierarchy, kept the same text, kept the same [data-test-id] etc], the test would continue working, and update itself appropriately without user interaction.
If the element has changed by a significant amount, the test will fail and require user interaction. Significant is defined to be if all of the following are true:
- More than 10% of the element selectors are no longer valid
- The element visually looks different to a degree greater than 1% of all baseline images for that element
To repair the test, there are two options: 1) The test can be debugged in the edit screen, by clicking "Play", then waiting until the error is replicated. At this point, delete the original step and re-record it, then save the test. 2) If reflow finds an element that looked similar to the original element, but it was not confident enough to select it, it will offer it as an auto-healing option when the action is opened in the view recording user interface.