Measuring e2e code coverage locally
Reflow can record how much of a Go application, or of a Swift, C, C++ or Objective-C application built with LLVM source-based coverage, your end-to-end flows exercise. You build the application with coverage instrumentation, run it while you apply flows, stop it gracefully, and ingest the profile it writes. Reflow stores one canonical document per run window, tied to the exact commit and tree it was measured against. For Go, that is checked against the build stamp in the binary. For LLVM, it is the worktree you bind, because LLVM binaries carry no commit stamp. Go and LLVM programs produce separate documents; see Swift and other LLVM languages and Coverage from a MacOSApp run. To query stored documents, see Reading coverage as an agent.
This page covers the local, process-level workflow (reflow coverage). It
measures whole processes over a run window. Per-flow numbers and an automatic
counter-reset surface are not available yet.
What you need
Section titled “What you need”- For Go: a Go module whose server or worker runs from your own
mainpackage. For Swift and other LLVM languages, see their section for the requirements. - For Go: Go 1.20 or later (the examples were verified with Go 1.26.1). The
goon yourPATHmust be the same version that built the binary. - A Reflow login and a selected collection for that login: either
reflow collections select <id>or a target selected withreflow target <url> --group <name> --collection <id>. Otherwise pass--collection <id>to each command.
Why package main must be instrumented
Section titled “Why package main must be instrumented”Go only starts its coverage runtime from an instrumented main package. If
main is excluded from -coverpkg, the build still succeeds, but coverage is
silently inactive: no counters are written when the process exits. Always build
the binary whose main you run with -cover, and include main in the
instrumented packages.
Injecting coverage with go build -overlay is not supported. Go’s cover tool
does not apply overlays, which forces main out of the instrumented set.
Set up the build
Section titled “Set up the build”Choose one of the two supported setups. Both must build with
-buildvcs=true (the default inside a Git checkout) so that the binary records
the commit it was built from. reflow coverage bind refuses binaries without
that stamp.
Setup (a): a build-tagged adapter in package main
Section titled “Setup (a): a build-tagged adapter in package main”Commit this file next to your main.go. The build tag keeps it out of normal
builds.
//go:build reflowcoverage
package main
// Reflow coverage startup check (setup a). Runs from init(); `reflow coverage verify` is the backstop.import ( "fmt" "os" "path/filepath" "runtime/coverage")
func init() { dir := os.Getenv("GOCOVERDIR") if dir == "" { fmt.Fprintln(os.Stderr, "reflow coverage: GOCOVERDIR is not set") os.Exit(70) } if err := coverage.WriteMetaDir(dir); err != nil { fmt.Fprintln(os.Stderr, "reflow coverage: coverage is not active:", err) os.Exit(70) } marker := filepath.Join(dir, fmt.Sprintf("reflow-start.%d", os.Getpid())) if err := os.WriteFile(marker, nil, 0o600); err != nil { fmt.Fprintln(os.Stderr, "reflow coverage: marker:", err) os.Exit(70) }}Save it as zz_reflow_coverage.go. At startup it checks that coverage is active,
exits with status 70 if it is not, and writes a reflow-start.<pid> marker that
reflow coverage verify counts.
Build from inside the checkout:
go build -buildvcs=true -tags reflowcoverage -cover -covermode=atomic \ -coverpkg=./... -o /tmp/app-cov/app .Setup (b): a harness module committed inside the app repository
Section titled “Setup (b): a harness module committed inside the app repository”If you cannot add a file to your main package, commit a small nested module,
for example internal/reflowcov/, whose own main calls your application’s
command package and includes the same adapter file:
module example.com/app/internal/reflowcov
go 1.26
require example.com/app v0.0.0
replace example.com/app => ../..package main
import "example.com/app/cmd/server"
func main() { server.Main() }Run go mod tidy in the harness directory and commit its go.sum. Build it
with your application’s packages instrumented; the pattern also covers the
harness’s own main:
cd internal/reflowcovgo build -buildvcs=true -tags reflowcoverage -cover -covermode=atomic \ -coverpkg=example.com/app/... -o /tmp/app-cov/app .The harness must live inside the application repository. Go stamps the VCS
revision from the repository that contains the main module, so a harness kept
elsewhere carries a foreign or missing stamp, which bind refuses. Reflow maps
Go import paths to files through every go.mod in the committed tree, so
nested modules resolve to their real paths.
Make SIGTERM return from main
Section titled “Make SIGTERM return from main”Go writes coverage counters only when main returns or the program calls
os.Exit. A process killed by an unhandled signal writes nothing. Handle
SIGTERM and SIGINT so that main returns after a graceful shutdown:
func main() { ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt) defer stop()
srv := &http.Server{Addr: ":8080", Handler: routes()} go func() { if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { log.Fatal(err) } }()
<-ctx.Done() shutdown, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel() _ = srv.Shutdown(shutdown) // Returning from main flushes the coverage counters.}Run the built binary directly. go run starts a separate child process and is
not supported.
Keep coverage files outside the worktree
Section titled “Keep coverage files outside the worktree”Put the cover directory and the instrumented binary
outside the repository checkout, for example under /tmp/app-cov/. bind
records whether the worktree is clean with git status; files it can see make
every document dirty, and Go stamps later builds with vcs.modified=true. If
they must live inside the checkout, git-ignore them. bind refuses a cover
directory inside the worktree that is not git-ignored
(cover_dir_in_worktree) and warns about such a binary.
Measure a run window
Section titled “Measure a run window”-
Build the binary with setup (a) or (b), from a committed tree. Go records
vcs.modified=truefor untracked files as well as modified ones, so build from a clean checkout or worktree:git status --porcelainmust print nothing.bindrefuses a binary stampedvcs.modified=truewhen the worktree is clean, and warns when it binds a dirty worktree. -
Bind a fresh, empty cover directory to the binary and the current commit:
Terminal window reflow coverage bind --binary /tmp/app-cov/app --cover-dir /tmp/app-cov/data \--exclude '**/testenv/**' --exclude '**/*_gen.go' --jsonbindcreates the directory (mode 700) if needed and refuses one that is not empty, or one inside the worktree that is not git-ignored. It records the commit, tree, object format and dirty state, the binary’s SHA-256 and itsgo version -mbuild information, and writesreflow-coverage-binding.json(mode 600) in the cover directory. It refuses a binary that has no VCS stamp, was built from a different commit, was built from a modified tree while the worktree is clean, or was not built with-cover. The repository label comes from theoriginremote ashost/owner/repo; pass--repository <label>to set it explicitly. Optional--excludeglobs are recorded in the binding file, andbindprints how many tracked non-test.gofiles each one matches; see Excluding code from the denominator. TheNext:line namesverifywhen the binary was built with-tags reflowcoverage(the adapter is compiled in), and the exit-only steps otherwise. -
Start every instrumented process (for example a server and a worker running the same binary) with the cover directory in its environment, using your usual process manager:
Terminal window GOCOVERDIR=/tmp/app-cov/data /tmp/app-cov/app serve &GOCOVERDIR=/tmp/app-cov/data /tmp/app-cov/app worker & -
Verify that coverage is active in exactly the processes you expect:
Terminal window reflow coverage verify --cover-dir /tmp/app-cov/data --processes 2Within
--timeout-ms(default 10000), exactly onecovmeta.*file and exactly that manyreflow-start.<pid>markers must exist. Reflow then reads the cover mode withgo tool covdata debugdump;go env GOVERSIONmust match the binary’s Go version. The mode and process IDs are recorded in the binding file. A build that leavesmainuninstrumented fails here.When several processes start at the same moment, one may occasionally print
coverage meta-data emit failed ... rename .... The processes share the same meta-data, andverifyconfirms the singlecovmeta.*file. Ifverifypasses, the run is complete; it fails when the meta-data is missing. -
Apply your flows as usual. Keep each apply’s request identity (
request.idempotency_keyinreflow apply --json) if you want the document to name the batches it covers.Terminal window reflow apply --json --result-file /tmp/app-cov/apply.json -
Stop the processes gracefully with SIGTERM, so each one returns from
mainand writes itscovcounters.*file. -
Ingest the cover directory:
Terminal window reflow coverage ingest --cover-dir /tmp/app-cov/data \--batch <idempotency-key> --jsoningestrequires the unmodified binding file and a completedverify; no process still writing (counters_in_progress); exactly onecovmeta.*file, matching the verified one;reflow-start.*markers for exactly the verified processes (process_countif one started afterverify); and exactly one completecovcounters.*file for each verified process and for no other (missing_counters,unverified_counters,duplicate_pid,counter_incomplete). It then derives the profile itself: you no longer rungo tool covdata textfmt.ingestchecks thatgo env GOVERSIONequals the binary’s Go version (go_version_mismatch), copies exactly the checked files into a private temporary directory, checks the copies again (cover_dir_changedif a file was replaced, grew or became a link after the checks), runsgo tool covdata textfmton that copy only, removes the directory afterwards, and requires the profile’smode:to equal the verified mode. Deriving the profile from a copy of the checked files means the document cannot describe a profile from another run. It maps import paths to repository files through thego.modfiles of the bound commit’s tree (not your current worktree), builds one canonical document, uploads it and prints the document ID, totals, batch counts and how many files each exclude glob matched. Byte-identical canonical documents (same profile, binding, exclude globs, producer and scope, including batch keys) dedupe: an identical document already stored in the collection is reused rather than stored twice. Batch keys are part of the document, so the same profile ingested with different--batchkeys, or once with and once without, is stored as separate documents. Profiles larger than 64 MiB or 2,000,000 blocks are refused astoo_large, never truncated.--batchis repeatable, up to 64 keys.Pass every batch that ran during the window, whatever its outcome: a batch whose runs failed, were skipped or ended unknown still drove traffic that is in the counters, and Reflow accepts any key that has runs in the collection (up to 1,000 runs per batch). For each key the document reports
sameBatchReusedFlows(runs in that batch with at least one step, every one of them reused) andexecutedFlows(the batch’s other runs: at least one step ran rather than being reused, or the run has no steps). A flow that the plan reused whole from an earlier batch creates no run in the new batch, so it is in neither count; pass the earlier batch’s key too if its traffic belongs to the window.reusedFlowsis a deprecated alias ofsameBatchReusedFlows.You may still pass a profile, as
[--format go-textfmt] <file>, for example one your own tooling already produced. It must be byte-identical to the derived profile, oringestrefuses it asprofile_mismatchand reports both sizes and SHA-256 digests.go tool covdata textfmtsorts packages and blocks, so the output is the same every time for the same cover directory. -
Inspect the stored result:
Terminal window reflow coverage status --latestreflow coverage status --latest --clean --top 20reflow coverage status --document <document-id> --json--latestselects the newest finalized document;--cleanlimits it to documents bound to a clean worktree. The output shows statement coverage first, then line coverage, the instrumented-package denominator and any exclusions, followed by the packages with the most uncovered statements (--top N, default 10, at most 100). The attribution line shows the tier:process (verified)orprocess (exit-only). Ties are ordered by package name. Packages beyond the 2,000-row limit are folded into one row; its uncovered statements are reported separately and never ranked. With--json, the result addspercent(statements,lines; after exclusions, two decimals,nullwhen there is nothing to count),rawPercent(the same figures without exclusions) andtopUncovered(package,uncoveredStmts,stmtsFound,stmtsHit,percent), plusexcludeMatches(see Excluding code from the denominator).
Use a new, empty cover directory and a new bind for each run window.
No app changes (exit-only)
Section titled “No app changes (exit-only)”If you cannot add the startup adapter or a harness module, you can still
measure a run window with a plain -cover build. This path is weaker than the
verified path above (setup (a) or (b)), which remains the recommended setup:
nothing checks the processes when they start, so Reflow checks the run only against
the files Go writes: the covmeta.* file written when an instrumented process
starts, and the covcounters.* file written when it exits. Documents ingested this way are marked exit-only.
-
Build from a clean, committed tree, with no adapter and no build tag. Go records
vcs.modified=truefor untracked files as well as modified ones, so build from a clean checkout or worktree:git status --porcelainmust print nothing.Terminal window go build -buildvcs=true -cover -covermode=atomic -coverpkg=./... \-o /tmp/app-cov/app . -
Bind a fresh, empty cover directory before running anything. Keep it outside the worktree, or git-ignore it:
Terminal window reflow coverage bind --binary /tmp/app-cov/app --cover-dir /tmp/app-cov/dataWithout
-tags reflowcoveragein the binary’s build information, theNext:line describes this path: run withGOCOVERDIR, stop with SIGTERM, theningest --processes N. -
Start every process with
GOCOVERDIR=/tmp/app-cov/data, then apply your flows. Skipreflow coverage verify; it needs the adapter’s startup markers. -
Stop every process and wait for all of them to exit. A plain Go binary writes its counters only when
mainreturns or the program callsos.Exit. Go’s default SIGTERM handling kills the process without writing anything, so exit-only cannot rescue an application that does not return frommainon SIGTERM. Handle the signal as shown in Make SIGTERM return from main. -
Ingest, giving the number of processes that ran.
ingestderives the profile itself, exactly as on the verified path:Terminal window reflow coverage ingest --cover-dir /tmp/app-cov/data --processes 2 --jsonIf
ingestreportscounter_countwith no counter files at all, the processes may still be running: stop them with SIGTERM (mainmust return), then ingest again.
ingest --processes checks, in order: the binary still has the SHA-256
recorded at bind (binary_changed, binary_missing); no process is still
writing (counters_in_progress); exactly one covmeta.* file exists
(no_covmeta: the binary was not built with -cover, GOCOVERDIR was
wrong or unset, or no process started; multiple_covmeta); it is a valid Go coverage meta-data file
whose hash matches its contents and its name (covmeta_version,
covmeta_corrupt, invalid_mode, covmeta_hash_mismatch); every package it
describes is embedded in the bound binary (covmeta_not_from_binary); there
are exactly that many complete counter files from distinct processes
(counter_hash_mismatch, counter_count, duplicate_pid,
counter_incomplete). Only then does it derive the profile, with the same
go env GOVERSION check (go_version_mismatch), the same private copy of
exactly the checked files (cover_dir_changed) and the same optional
byte-identical <file> comparison (profile_mismatch) as the verified path,
and require the profile’s mode to match the covmeta (mode_mismatch).
A cover directory that was verified refuses --processes; ingest it without
the flag.
Both tiers derive the profile from a copy of exactly the files they checked,
so neither document can include counters from outside the run window that
ingest checked. The tiers differ in what those checks can prove. What
exit-only cannot prove, compared with the verified path:
- No startup check. Nothing confirms that coverage was active in each
process when it started. A process that never started, crashed or was killed
without returning from
mainis detected only as a missing counter file (counter_count). - Process IDs are not registered in advance. Reflow counts the counter files that exist; it cannot tell that they came from the processes you meant to run.
- Another build of the same source is not detected when it differs only
in
-covermodeor uses a narrower-coverpkg: its packages are still embedded in the bound binary. Compare the document’s instrumented packages with the packages you expect;reflow coverage statusprints the count for exit-only documents (instrumentedPackagesin--json).
Reading the numbers
Section titled “Reading the numbers”- Statement coverage is the primary figure.
stmtsHit / stmtsFounduses Go’s own statement counts and matches the percentagego tool cover -funcreports for the same profile. - Line coverage (Reflow) is shown second. It counts source lines covered by Go’s statement blocks; a line is hit when any block on that line ran. It differs from Go’s statement percentage, so Reflow labels it explicitly.
- Denominator: instrumented packages. Only packages compiled with
coverage appear. Code you excluded from
-coverpkgis absent, not counted as uncovered, and every percentage is shown with that note. - Refused entries. Profile entries with absolute paths, or import paths that do not belong to a module in the bound tree (such as dependencies), are counted as refused and never stored.
Excluding code from the denominator
Section titled “Excluding code from the denominator”Generated code, test environments or vendored helpers can dominate the
denominator without being meaningful targets for end-to-end flows. Pass
--exclude <glob> to bind, once per glob:
reflow coverage bind --binary /tmp/app-cov/app --cover-dir /tmp/app-cov/data \ --exclude '**/testenv/**' --exclude '**/gen/**' --exclude 'internal/k8s/**' \ --exclude '**/*_gen.go'Globs are anchored at the repository root: they match the whole
repository-relative path of a file, with / separators. testenv/** matches
only a top-level testenv/ directory, not server/internal/testenv/env.go; to
match a directory at any depth, start the glob with **/, as in
**/testenv/**.
| Pattern | Matches |
|---|---|
** (a whole segment) | Zero or more whole path segments |
* | Any run of characters within one segment, never / |
? | Exactly one character other than / |
| anything else | Itself, literally (including ., [, { and spaces) |
So **/*_gen.go matches x_gen.go and a/b/x_gen.go, testenv/** matches
everything under testenv/, and *.go matches only files at the repository
root. Globs must be printable ASCII, 1 to 256 characters, without a leading
/, a .. segment or \; at most 64 are allowed. They are recorded in the
binding file and covered by its digest.
Each command reports how many files every glob matched, so a glob that misses is visible at once:
bindcounts tracked repository files: the tracked non-test.gofiles in the bound commit’s tree.ingestcounts instrumented profile files: the files in the canonical document, which only lists files of instrumented packages.statusprints the instrumented profile file counts the server recorded when it stored the document; documents stored before these counts existed show “counts unavailable”. Run inside a checkout that has the bound commit, it also prints the tracked repository file counts.
The two numbers differ whenever the build did not instrument every package a
glob covers (for example generated packages left out of -coverpkg), or an
instrumented package has files with no statements, which go tool covdata textfmt does not list (type-, const- or var-only files are common in generated
code). Files refused as absolute or outside the repository are not counted
either. Each line says which number it is.
A glob that matches nothing produces a warning that suggests the **/ form.
Exclusion is applied only when the numbers are aggregated. The uploaded
document still contains every file and the complete totals, plus the list of
globs. Package rows, the headline percentages and the top uncovered packages
use only files that no glob matches, and the output lists the globs with the
number of excluded files, statements and lines. rawPercent keeps the figures
before exclusion.
Expected sizes
Section titled “Expected sizes”A real Go service measured during design (Gram) had a denominator of 232,404 statements and 366,326 lines across 2,510 files in 463 package directories. Each process wrote about 1.9 MB of counters, a counter flush took about 3 ms on average, and ingest took 0.6 seconds. That is well inside the limits: 20,000 files, 2,000,000 ranges, a 64 MiB profile and a 16 MiB compressed document.
The document also lists the instrumented packages (their import paths). At
most 4,096 are allowed, each 1 to 256 bytes and 60 KiB (61,440 bytes) in total;
beyond that the document is refused as too_large. The list is stored complete,
before any --exclude, in a database field whose size cannot grow without
downtime, so these caps leave room for every pattern plus its JSON quoting. The
byte cap is usually the one reached first: Gram’s 655 packages take about
45 KB, and a repository whose import paths average 68 bytes reaches the cap at
about 850 packages. Import paths may contain only ASCII letters, digits and
-._~+!@, separated by single slashes (Go itself accepts only -._~+ besides letters and digits), so no
pattern grows when it is written as JSON; a path outside that set is refused
as invalid_canonical. Narrow -coverpkg if a very large repository exceeds
the caps.
Deleting a document
Section titled “Deleting a document”reflow coverage delete --document <document-id>delete retires one finalized document. Its uploader can delete it, and so can
a team owner or admin using an admin-scoped token (forbidden otherwise). The
document disappears from status at once, and its stored object is removed in
the background. Repeating the command while the document is still being
deleted succeeds; a repeat after it has been removed reports not_found. A
reservation that was never finalized is refused as not_finalized and expires
on its own. Otherwise
retention removes documents: each collection keeps its latest 50 clean and,
separately, its latest 20 dirty documents.
Swift and other LLVM languages
Section titled “Swift and other LLVM languages”Apps built with LLVM source-based coverage (Swift, C, C++, Objective-C) use
--format llvm-cov. Their documents are separate from Go documents: they report
line and region coverage, have no statement coverage, and are never blended with
a Go helper’s numbers.
- Build the instrumented binary, for example with SwiftPM:
swift build -c release -Xswiftc -profile-generate -Xswiftc -profile-coverage-mapping -Xswiftc -file-prefix-map -Xswiftc "$PWD=.". In Xcode, pass the same compiler flags (-profile-generate,-profile-coverage-mapping,-file-prefix-map <root>=.) to the target that builds the app. The prefix map keeps recorded source paths relative, but the result depends on the directory you build from (a package in a subdirectory can record a doubled prefix). Check one recorded file name, for example withllvm-cov export -format=lcov ... | grep -m1 ^SF:, and set--path-stripto match, or""when it is already repository-relative. - Bind a fresh, empty directory outside the worktree to the exact binary that
will run and the prefix to strip from recorded paths. For an app bundle that
is the Mach-O executable in
<App>.app/Contents/MacOS/. Bind the copy that actually launches: re-signing or copying from another build changes its bytes, and ingest refusesbinary_changed.reflow coverage bind --format llvm-cov --binary <binary> --cover-dir <dir> --path-strip <prefix>. If the recorded paths are already relative to the repository root (for example a build with-file-prefix-map <root>=.), pass an explicitly empty prefix:--path-strip ""(or--path-strip=). Every stripped path must be a file in the bound commit; other paths are counted as refused, not stored. - Run each instrumented process with
LLVM_PROFILE_FILE=<dir>/<name>-%p.profraw(%pgives one file per process;%malso works; do not use%ccontinuous mode; a fixed file name lets later processes overwrite earlier profiles). Children inherit the variable when the parent passes its environment through. Quit each process normally (for a Cocoa app, its own Quit command). By default the LLVM profile runtime writes the profile from its exit handler, so only a normal exit writes it: SIGKILL and a crash never do, and a signal the program does not handle (for example SIGTERM sent to an ordinary Cocoa app) ends it without writing. An app that handles SIGTERM or SIGINT itself and then exits normally does write its profile; rely on that only for apps whose handler you know. Wait until every process has exited before ingesting. - Ingest with the exact process count:
reflow coverage ingest --format llvm-cov --cover-dir <dir> --processes <n> [--batch <key>]....nis the number of instrumented LLVM processes you launched in the window, counted from what you started, not from the files you find. With%peach of them must leave one.profraw. Ingest refuses (profile_count_mismatch) unless exactlynprofiles exist, and refuses any other file in the directory. A missing profile is missing evidence, not 0%: never lowernto match the files, which would silently accept a process that was killed or crashed; fix the stop and start a new window.
Ingest merges the raw profiles with llvm-profdata, exports them with
llvm-cov (through xcrun on macOS) and refuses unless the per-line result
equals llvm-cov’s lcov line records and the regions equal its region summary.
It also refuses if the binary or the llvm tool versions changed since bind.
Line counts are LLVM’s per-line view; llvm-cov’s per-file line summary adds up
per-function counts and can exceed it. These documents are always exit-only:
nothing verifies at startup that profiling is active, and the binary carries no
commit stamp, so the source binding is the worktree you bound.
Coverage from a MacOSApp run
Section titled “Coverage from a MacOSApp run”The MacOSApp plugin launches an isolated copy of a macOS app for each step. To measure that app (and any Go helper it starts), set up one run window per producer and give the app the coverage variables through its binding.
-
One directory and one bind per producer. Create a fresh, empty directory outside the worktree for each instrumented program, for example
/tmp/myapp-cov/swiftfor the app and/tmp/myapp-cov/gofor a Go helper. Bind each one to the exact binary that will run:--format llvm-covwith the Mach-O executable inside the isolated copy named by the binding’sapplication, and the Go helper binary with a plainbind. The LLVM document is always exit-only. Go and LLVM documents are separate; never add their counts or quote one blended figure. -
Pass the variables through
environmentFile. In a Reflow run, the binding’s privateenvironmentFileis the only way to give the appGOCOVERDIRorLLVM_PROFILE_FILE: Reflow starts plugins with a fixed minimal environment, so variables exported in your shell or in the device service’s environment do not reach the app. The file is read on the device that runs the step, so use absolute paths on that machine:{"LLVM_PROFILE_FILE": "/tmp/myapp-cov/swift/myapp-%p.profraw","GOCOVERDIR": "/tmp/myapp-cov/go"}Keep it
chmod 600. The app’s children inherit these variables when the app passes its environment through, which is how a Go helper started by the app receivesGOCOVERDIR. Launching through LaunchServices oropeninstead may not propagate them. The step’s state records the file’s SHA-256 (inputs.environmentFileSha256), not its values. -
Apply your flows and keep each apply’s request identity as a batch key:
request.idempotency_keyfromreflow apply --json, or therequest_idyou pass tomanage_executionaction: "apply"over MCP. -
Count actual launches, not flows. Count each instrumented runtime separately, from the processes the window started:
- App (LLVM): every MacOSApp
runandinspectstep that launches the app starts one app process; add any other process of the same bound executable started in the window. Steps reused from earlier evidence and steps refused before launch start nothing. That total is the app’sn. A different instrumented binary, such as a Go helper or another LLVM helper, is a separate document with its own directory, bind and count. - Go helper: every Go process started with that
GOCOVERDIR: each helper the app started, each child that inherited the variable, and any other run of the same binary in the window.
Every counted process must write its file. MacOSApp quits the app through its own Quit command when it can, and the step’s state records how it stopped (
lifecycle.appStop,lifecycle.forced).graceful_quitwithlifecycle.forced: falseis a normal exit through the app’s own Quit, the safe path for a Cocoa app.terminatedmeans SIGTERM: whether a profile was written depends on how the app handles it; for an ordinary Cocoa app expect none.killedmeans SIGKILL, which writes nothing. A Go process writes counters whenmainreturns or it callsos.Exit; a crash or SIGKILL writes nothing, and SIGTERM writes only if the program handles it and returns. If a counted process left no file, the ingest refuses: do not lowern; fix the stop and run a new window. - App (LLVM): every MacOSApp
-
Ingest each directory with the same batch keys, after every process has exited:
Terminal window reflow coverage ingest --format llvm-cov --cover-dir /tmp/myapp-cov/swift --processes <app-n> --batch <key>reflow coverage ingest --cover-dir /tmp/myapp-cov/go --processes <go-n> --batch <key>A missing profile or counter file refuses the ingest. It is missing evidence, never 0%: start a new window rather than editing the count.
Process attribution
Section titled “Process attribution”A document describes everything the instrumented processes executed during the run window: startup, background jobs and requests from anything else that talked to them, as well as your flows. Batch keys record which applies ran in the window and how many flows each executed or reused. They do not mean every covered line was caused by those flows.
Clean and dirty documents
Section titled “Clean and dirty documents”A document is clean only when git status --porcelain=v2 --untracked-files=normal --ignore-submodules=none printed nothing at bind time.
Otherwise it is dirty and records a digest of that status output. Dirty
documents are for local status only. They are never review or pull request
evidence. Build and bind from a committed, clean tree for results you want to
share.
Reading coverage as an agent
Section titled “Reading coverage as an agent”Agents and scripts read stored documents with six read-only commands. Each
returns one bounded page with --json; pass nextCursor back unchanged with
--cursor to continue. In the local MCP connection (reflow mcp) the
query_code_coverage tool runs the same reads; the remote HTTP MCP connection
does not offer it (operation: documents, show, files, lines, gaps
or diff).
| Read | What it answers |
|---|---|
documents [--limit <n>] | Which finalized documents exist, newest first, with their exact IDs. |
show --document <id> | Source provenance (repository, commit, tree, clean or dirty), producer and tool versions, tier, batch evidence and metrics. |
files --document <id> [--query <substring>] | Files ranked by missed lines. |
gaps --document <id> | Non-excluded files with missed line ranges [start, end], largest gaps first. |
lines --document <id> --path <file> [--with-source] | One file’s instrumented line ranges [start, end, hits] and its missed runs. |
diff --document <after> --base <before> | What changed between two documents. |
Every read names an exact document ID. Copy it from documents; there is no
mutable “latest” alias in these reads. A document removed by retention or
delete answers not_found.
A metric that a producer does not measure is null and shown as “not
measured”, never as 0%. Go measures statements but not regions or branches;
LLVM (Swift) measures regions but not statements.
Source text
Section titled “Source text”The server stores no source. show, gaps and lines give the exact
provenance instead, so you can open the same file in your own checkout.
lines --with-source adds the text only when the local Git repository has the
document’s commit, that commit’s tree equals the recorded tree, and the
document was built from a clean tree. The text then comes from Git’s object at
that commit (not from the working tree) and is labelled local-checkout with
its blob ID. Otherwise localSource names why it is unavailable (for example
built_dirty, commit_missing or tree_mismatch). The repository label in
the provenance is not verified against your remote.
Before and after a new test
Section titled “Before and after a new test”Coverage is process-wide: nothing attributes lines to a single flow. To learn
what one new test adds without running the existing suite again, run only the
new test in its own window and compare cumulative documents built from saved
raw files: see cumulative coverage and test gain.
That page also covers how diff classifies a pair (likeForLike or
descriptive) and the refusals to expect.
When to stop adding tests
Section titled “When to stop adding tests”- Zero added coverage is advisory, not proof that a test is redundant. A new assertion, an important boundary value, a regression case or a different oracle on the same path can matter without running new lines. Keep such a test and note why (“zero-delta, kept because …”).
- Stop duplicating equivalent scenarios: the same class of inputs, the same path and the same assertions as an existing test. Coverage alone cannot show that; compare the tests’ stated purposes and oracles too.
- Never skip, disable or delete a test from coverage data alone. Flag it for review with its purpose beside the delta.
- Do not add exclusions to make numbers pass, and do not chase excluded files.
- Prefer gaps that carry risk over raising a percentage.
Command reference
Section titled “Command reference”| Command | Purpose |
|---|---|
reflow coverage bind --binary <path> --cover-dir <dir> [--collection <id>] [--repository <label>] [--exclude <glob>]... | Tie a fresh cover directory to the binary and current commit. |
reflow coverage bind --format llvm-cov --binary <mach-o> --cover-dir <dir> --path-strip <prefix> | Same for an LLVM (Swift, C, C++, Objective-C) binary; --path-strip "" when recorded paths are already relative. |
reflow coverage verify --cover-dir <dir> --processes <n> [--timeout-ms <ms>] | Check that exactly n instrumented processes started with coverage active. |
reflow coverage ingest --cover-dir <dir> [--processes <n>] [--batch <key>]... [[--format go-textfmt] <file>] | Derive the profile and upload one canonical document for the run window (--processes for exit-only). |
reflow coverage ingest --format llvm-cov --cover-dir <dir> --processes <n> [--batch <key>]... | Merge exactly n LLVM profiles and upload one exit-only document. |
reflow coverage status [--document <id> | --latest [--clean]] [--top <n>] [--collection <id>] | Show statement and line coverage and the most uncovered packages. |
reflow coverage delete --document <id> [--collection <id>] | Retire one finalized document. |
reflow coverage documents [--limit <n>] [--cursor <c>] | List finalized documents and their IDs. |
reflow coverage show --document <id> | One document with provenance, evidence and metrics. |
reflow coverage files --document <id> [--query <substring>] [--limit <n>] [--cursor <c>] | Files ranked by missed lines. |
reflow coverage gaps --document <id> [--limit <n>] [--cursor <c>] | Missed line ranges in non-excluded files. |
reflow coverage lines --document <id> --path <file> [--with-source] | One file’s line ranges and missed runs. |
reflow coverage diff --document <after> --base <before> [--limit <n>] [--cursor <c>] | Like-for-like gain or a descriptive change between two documents. |
Every command accepts --json. Refusals are named (for example
unstamped_binary, vcs_mismatch, process_count, missing_counters,
profile_mismatch or mode_mismatch) and exit non-zero.