Critical Journey Coverage

Critical Journey coverage measures whether the product outcomes that matter to a release have current evidence, representative state coverage, attributable Decisions, and connected downstream explanations. It is not the number of automated tests or the percentage of application code executed.

Coverage should expose uncertainty. A single composite score can hide the exact gap that makes a release risky, so this model keeps four ratios separate.

Why test count is an unreliable coverage proxy

One Journey may have many scripts for minor variations and still lack evidence for the actual customer outcome. Another may have one well-designed Run that verifies the complete consequence across the few contexts that change product meaning.

Raw counts do not answer:

• Which customer or business outcomes are release-critical?
• Is the latest Journey Version the one that was run?
• Did the intended result occur or did the script merely finish?
• Were the relevant roles, plans, flags, locales, or viewports covered?
• Was the evidence reviewed and decided?
• Which support, documentation, training, demos, or marketing outputs became stale?

Start with an outcome inventory, then measure the gaps around it.

Step 1: define the critical Journey inventory

List the outcomes whose failure would materially affect users, access, money, trust, support, privacy, or release readiness. Keep the inventory small enough that every item can have an owner and explicit consequence.

For each Product Journey, record:

• actor and owner;
• intended outcome checkpoint;
• release consequence;
• representative context policy;
• dependencies and external side effects;
• downstream documentation and media;
• current Journey Version.

Do not start by importing every test and calling the resulting count coverage.

Ratio 1: outcome coverage

Outcome coverage is the proportion of critical Journeys with current evidence that the intended result occurred.

Journeys with current outcome evidence ÷ critical Journeys

Current evidence must resolve to the active Journey Version and a completed checkpoint. An old Run against a superseded outcome does not count merely because the URL or test name is unchanged.

Gap interpretation

• A Journey with no Run has an execution gap.
• A Journey with a Run but no outcome evidence has a checkpoint or evidence gap.
• A Journey whose latest evidence belongs to an older version has a freshness gap.

Ratio 2: context coverage

Context coverage is the proportion of declared context executions with current verified evidence.

Verified declared contexts ÷ planned declared contexts

A context is deliberate product state, not an arbitrary browser permutation. Relevant axes can include actor, role, permission, plan, data, feature flag, authentication, locale, timezone, theme, motion, viewport, clock, and allowed side effects.

Use State Capsules so the denominator represents meaningful states rather than a ceremonial cross-product.

Gap interpretation

• Missing a critical role or plan branch is a product-meaning gap.
• Skipping an irrelevant theme permutation should not lower coverage.
• A context that ran with unreconstructed state is not verified context coverage.

Ratio 3: Decision coverage

Decision coverage is the proportion of critical Journeys whose current evidence has an attributable Decision.

Journeys with a current attributable Decision ÷ critical Journeys

Passing automation is not automatically an approval. Review may need to accept an intentional change, reject a regression, authorize a repair, document an exception, or apply a release consequence.

A comment, assignment, or AI recommendation does not count as a Decision. The record needs actor, authority, rationale, evidence, policy, and supersession semantics.

Gap interpretation

• High outcome coverage with low Decision coverage means evidence exists but release authority is unresolved.
• An expired exception reopens the gap.
• A Decision against superseded evidence is not current coverage.

Ratio 4: downstream lineage coverage

Downstream lineage coverage is the proportion of affected customer-facing outputs connected to their owning Journey evidence.

Affected outputs with Journey lineage ÷ affected outputs

Outputs include documentation screenshots, step guides, support procedures, walkthroughs, in-product guidance, support clips, training, demos, marketing visuals, and captioned product video.

If a release affects no outward outputs, this ratio is not applicable—not automatically 100%.

Gap interpretation

• An output with no Journey link can become stale without detection.
• A linked Rendition may still require editorial Review after a semantic or material visual change.
• A Publication without a current receipt or freshness state has a delivery gap.

Do not average the four ratios blindly

Consider this example:

Coverage axis	Result
Outcome	100%
Context	80%
Decision	20%
Downstream lineage	100%

An average can look acceptable while most critical Journeys lack a current attributable Decision. The correct next action is to close the Decision gap, not to create more tests.

The Journey Coverage Calculator therefore reports each ratio and its missing count separately. It identifies the lowest applicable axis but does not manufacture a release-readiness score.

How to select representative contexts

Prioritize contexts that change:

• permission or recipient;
• product entitlement or capacity;
• checkpoint semantics;
• branching behavior;
• privacy or side-effect policy;
• layout, direction, or interaction usability;
• external provider behavior;
• release consequence.

Avoid multiplying contexts whose product meaning is identical. Add a context when evidence shows a distinct risk or contract branch.

Evidence freshness rules

Coverage becomes stale when:

• a new Journey Version changes intent, branch, checkpoint, target, or state policy;
• the product dependency graph changes;
• the runner or environment no longer represents the release candidate;
• evidence retention or privacy policy invalidates the usable Observation;
• an exception expires;
• a downstream output changes without a new approved revision;
• a Publication is revoked, replaced, or loses delivery integrity.

Do not refresh a date without materially refreshing evidence.

Example: team invitation coverage

Critical Journey inventory

One Journey: invite a teammate with the intended workspace and role.

Planned contexts

1. owner with capacity;
2. admin with permission;
3. owner at plan limit;
4. existing member as recipient.

Current evidence

The first three contexts have current Observations; the existing-member branch has not run.

Coverage

• Outcome: 1 of 1 Journeys = 100%.
• Context: 3 of 4 contexts = 75%.
• Decision: 0 of 1 Journeys = 0% until Review records the consequence.
• Downstream lineage: 2 of 3 affected support/docs outputs = 67%.

The next action is not “add another test.” It is to record the Decision, then close the missing context and output-lineage gaps.

Use a release book to preserve the result

A Release Book presents changed Journeys, verified contexts, Decisions, accepted risks, downstream updates, and approved outputs from the same identities. It can show incomplete coverage without translating it into a false green status.

Different teams may receive different views, but they reconcile the same object IDs and hashes.

Coverage review checklist

• Critical Journey inventory is outcome-based and owned.
• Every Journey points to its current immutable version.
• Outcome checkpoints prove customer or business consequences.
• Planned contexts represent distinct product meaning.
• Current Runs retain usable evidence.
• Failures are classified before retry or repair.
• Current evidence has attributable Decisions.
• Exceptions retain scope and expiry.
• Affected outputs have Journey lineage and freshness state.
• Publications retain receipts and active/revoked status.
• Gaps remain visible in release reporting.

Improve the lowest applicable axis first

If outcome coverage is lowest, run or repair the missing critical Journeys. If context coverage is lowest, verify the missing meaningful states. If Decision coverage is lowest, move current evidence through Review. If lineage coverage is lowest, connect the affected outputs and Publications to source evidence.

Use the Journey Coverage Calculator, apply the method through Critical User Journey Testing, or explore the QA solution.