Repair and rerun
Reflow runs a flow one way: reflow plan, then reflow apply. No model runs
inside the runner. A failed step stays failed: Reflow does not retry it another
way, rewrite it or propose a rewrite. The coding agent that calls the CLI
repairs it, using Reflow’s retained evidence and execution plan, then reruns the
affected checks.
Use Reflow to inspect this failure. Determine whether the application regressed or the test needs updating, make the smallest justified repair, then plan and rerun the affected checks. Show me the before-and-after evidence and unresolved failures before preparing the PR.
Tests can be saved in Reflow or in your repository. Use a slug or test ID with plan/apply for either location; the paths below show repository examples. See test storage for selection and authoring.
The repair loop
Section titled “The repair loop”- Read the failed step, its inputs and retained evidence. Compare the result with the intended behavior and current application knowledge.
- If the recorded error and evidence do not explain the failure, hold the failed run and inspect its live environment.
- Fix the application or update an obsolete test within the requested scope. Preserve assertions that describe behavior the application must keep.
- Keep the same target and run
reflow plan. Changed flow steps, covered source files and dependency outputs invalidate affected evidence automatically. A flow with a Browser step then runs again from its first step, unless its failed run is held (what reruns). - Run
reflow apply, inspect the actual outcomes and snapshots, and prepare the relevant delta for the PR. A successful command still needs its business result checked. Apply also lists the snapshots that differ from a reference run of the same flow; a difference never fails the run, so state for each whether it is an expected change or a regression, and why.
The Reflow skill guides your connected agent through MCP and the same tools as the CLI. Reflow needs no model configuration: the agent uses its own model and permissions.
Hold a failed run and inspect it
Section titled “Hold a failed run and inspect it”Normally a failed step ends the execution and closes its browser and plugins. Apply with a hold to keep them:
reflow apply --hold .reflow/flows/checkout.mdThrough MCP, pass hold: true to manage_execution with action: "apply".
A hold exists only for an apply started this way, so apply again with the hold
after a failure. A passing execution is never held.
When a step fails, the result is recorded as usual: the failed step, the
skipped steps and the evidence. The worker then keeps the browser page and
every plugin the flow opened as the failed step left them. reflow apply
prints the execution ID, the failed step and when the hold ends, and keeps
waiting.
The agent inspects the held run with the debug_execution MCP tool and that
execution ID:
action | What it does |
|---|---|
act | Runs one RFL line and answers passed or failed. A provider line such as db.query sql="select 1" goes to that plugin binding. Any other line, such as expect role=alert, goes to the Browser page. |
snapshot | Returns the page URL, title and accessibility outline. |
screenshot | Captures the viewport. |
release | Ends the hold and tears the environment down. Not needed before applying a fix. |
Nothing done in a hold is recorded. The lines are not steps, and the failed result and its evidence do not change. The lines do act on the real application and test data, so prefer checks that change nothing.
You do not release the hold before trying a fix. Edit the flow, then apply again in another terminal or tool call, for that flow or for the same selection as before:
reflow apply .reflow/flows/checkout.mdThe new execution of the held flow continues on the held browser and plugins
from the failed step. The steps that passed are reused, not replayed. The
failed execution stays failed, and the second apply makes a new execution with
its own result. The first apply prints hold ended and its failed report, and
exits 1. What act changed during the hold stays on the page for the continued
step. If the second apply fails and was given --hold, the same environment is
held again.
The same loop works for a whole collection. Run reflow apply --hold with no
selector. The flows before the failing one finish, the failing flow is held,
and the flows after it wait behind the hold. After the edit, run
reflow apply --hold again: it continues the held flow, reuses the flows that
passed and runs the flows that had not started. A waiting flow that the second
apply does not select is stopped and not run; plan and apply name it.
Sometimes apply cannot continue: a step that had already passed must run again
(it was edited, or the frontmatter, bindings or inputs changed), a flow the
held flow needs must run again, --golden is set, the flow reads its mailbox,
the worker is near its time limit, or the hold already ended. Apply then ends
the hold, runs in a fresh environment and says why. A --coverage-session
apply never continues either: it is refused until the flow is tainted, then
runs every step. reflow plan tells you beforehand which of the two will
happen. With a Reflow service or device service that does not support this
yet, the second apply is refused while the execution is held, and you release
first.
Release the hold when you give up or will not apply again. The hold also ends
on Ctrl-C or cancellation, or by itself after at most 30 minutes. --hold
cannot be combined with --no-wait or --coverage-session. See
debugging a failed execution
for the full list of reasons, the printed lines, command IDs and uncertain
replies.
Changes already select what reruns
Section titled “Changes already select what reruns”Suppose checkout needs seed-customer. Changing the seed step means the next plan
runs that step and the checkout work that depends on it. Unrelated matching work
can reuse its evidence.
A flow with a Browser step is different. All its steps share one browser, and
that browser closes when the run ends. So when any step of the flow must run,
for example because you edited its last step, the whole flow replays from its
first step. reflow plan lists every step as run. The edited step shows its
own reason. Every other step shows Reconstruct stateful session. Plan also
prints a note that says the flow replays.
- An unchanged flow reuses all its steps.
- A flow without a Browser step reuses the unchanged steps before the first step that must run.
- A flow with a Browser step replays from its first step, unless its failed run is held.
To repair a failing Browser flow without the replay, apply with
reflow apply --hold, inspect the held run with debug_execution, fix the
failed step or a later one, and apply again. That apply continues on the held
browser from the failed step (hold a failed run).
A hold exists only for a failed run, and editing a step that already passed
ends it.
covers narrows which source files affect a flow. Without it, the known Git
working tree is covered conservatively. Planning observes staged, unstaged and
untracked non-ignored source changes; it does not execute steps or poll external
systems for changes. Apply checks the current plan again before admission.
Source coverage through symbolic links
Section titled “Source coverage through symbolic links”Links to files or directories inside the checkout can reuse evidence when both
the link and its covered target contents are unchanged. Reflow fingerprints the
resolved contents as well as the link, including executable permissions. A target
edit, addition, deletion or changed link invalidates the affected coverage.
covers can name the link path or its descendants, for example
covers: ["shared-skills/**"].
Links resolved outside the checkout, broken or cyclic links, unreadable targets,
submodules and targets outside the Git source inventory
remain uncertain. Within a covered directory target, Git-ignored sibling files
are excluded just as they are elsewhere in the checkout. A link directly to an
ignored or wholly unrepresented target remains uncertain. Reflow does not read
ignored target contents.
Source inspection runs during planning and again before execution. Large checkouts
with many links can take several seconds to inspect.
Planning stops if link expansion exceeds its inspection bounds. Unresolved coverage keeps the
affected steps selected for execution; narrowing covers is appropriate only
when it accurately describes the source that affects that flow.
reflow plan .reflow/flows/checkout.mdreflow apply .reflow/flows/checkout.mdWhen to request fresh evidence
Section titled “When to request fresh evidence”If a database, feature flag or running application changed without a corresponding
source or input change, the agent can use taint to request another observation:
reflow taint checkoutreflow plan .reflow/flows/checkout.mdreflow apply .reflow/flows/checkout.mdUse checkout#Confirm order to select a named step, quoted when it contains spaces.
The plan includes its affected dependents. Taint does not undo external writes or
clear live ownership. Named mailboxes and generated test values stay the same.
Use reflow reset only when you intend to replace the selected group’s test data,
then plan again. Reset does not delete application records or resolve uncertain
execution. A step that reached a time limit failed, and the next apply runs it
again without a taint. Completion is unknown when an answer was lost (a provider or
its connection dropped, or the worker stopped without reporting), or when a command
that can change something answered with an error, as a PostgreSQL db.execute does
when the server refuses a statement.
Then inspect the existing run and reconcile its effects before deliberately
retrying. After an uncertain apply
response, keep its request identity; a new request can repeat an external write.
Keep the repair meaningful
Section titled “Keep the repair meaningful”A selector can become obsolete; a missing role selector can be a regression. The agent must distinguish them using the requested change and application rules. It should stop when the intended behavior is unclear, access is missing, completion is uncertain, or the same failure repeats without progress. It reports the remaining problem instead of weakening assertions or accepting a different screenshot to make the run green.
Every repair is an ordinary, reviewable change to the flow or the application. Reflow changes neither by itself.
What planning refuses
Section titled “What planning refuses”No model runs inside the runner, so nothing carries out a step without RFL, judges a natural-language invariant, validates a run or repairs a failed step. Reading, listing and syncing a flow that asks for one of these works. Planning and execution refuse it. The message names the flow and the replacement, and ends in a code:
| Flow uses | Message |
|---|---|
| A step with an empty fence, or only comments | Flow "checkout": Step "Pay" (line 12) has no RFL. Reflow no longer executes plain-language steps with a model: write the step as RFL statements (the calling agent can author them; see https://reflow.io/docs/reflow-language/ section "The execution rule"). [prose_step_removed] |
| An invariant that is not one RFL assertion | Flow "checkout" declares a natural-language invariant ("no errors are shown"). Reflow no longer runs a model inside the runner: write each invariant as one RFL assertion, or let your agent judge the run's evidence. [model_invariant_removed] |
validate: true | Flow "checkout" sets validate: true. Reflow no longer runs a model inside the runner: write the check as RFL assertions, or let your agent judge the run's evidence. [model_validation_removed] |
self_heal: true | Flow "checkout" sets self_heal: true. Reflow no longer runs a model inside the runner: remove the key; when a step fails, your agent holds the run (reflow apply --hold), inspects it with debug_execution, edits the flow and applies again. [self_heal_removed] |
repair_proposals: true | The same message for repair_proposals: true, ending in [repair_proposals_removed] |
Plan and apply is the one way to run a flow. Reflow keeps no accepted
expectation sets and does not build or serve a target. A flow that declares
one is refused the same way, and so is a private Browser binding that sets
managedStatic:
| Flow or binding declares | Message |
|---|---|
expectations or expectation_set | Flow "checkout" declares expectations. Reflow no longer keeps accepted expectation sets: remove the key, assert in RFL, and let your agent compare the run's evidence with the last passing run. [expectations_removed] |
managed_static or managed_build | Flow "checkout" declares managed_static. Reflow no longer builds or serves a managed target: remove the key, start your application yourself (or with a LocalBash or Docker plugin step) and point the flow at its URL. [managed_target_removed] |
managedStatic or managedBuild in a binding the flow uses | Binding "site" sets managedStatic. followed by the same replacement and [managed_target_removed] |
RFL has no bare snapshot <name> statement, and Reflow stores no approved
baseline. reflow plan, reflow apply and reflow taint refuse a flow that
uses the statement and print the replacement for each occurrence, one per line:
| Flow uses | Message |
|---|---|
The bare statement snapshot <name> | Flow "checkout": `snapshot <name>` was removed (1 use in this flow). Stored approved baselines were removed: visual differences are now found by comparing this run with a reference run. Write the Browser provider's `snapshot` command instead.- Step "Pay" (line 12): replace `snapshot checkout` with `ui.snapshot name="checkout"`.See https://reflow.io/docs/reflow-language/ section "Visual checkpoints". [bare_snapshot_removed] |
threshold has no replacement, because a visual difference never fails a run;
settle="2s" becomes settle=2000. Saving a flow that uses the statement
is accepted, and the save receipt carries removed_feature_warnings naming this
plan-time refusal. A snapshot is the Browser snapshot command, and your
agent judges the differences reflow apply reports against a reference run: see
Snapshots and review.
To fix a refused flow, ask your agent to write each step as RFL, write each
invariant as one RFL expect assertion such as expect text="Ready", and
remove the keys.
A flow that sets self_heal, validate or repair_proposals to false is
accepted: see the
frontmatter reference.
See Snapshots and review for browser, terminal and database evidence, and Write a flow for dependencies and variables.