Accessibility Testing in CI for Product Journeys

Accessibility Testing in CI for Product Journeys connects automated and manual accessibility evidence to complete outcomes, declared contexts, components, owners, barriers, Decisions, and release consequences. CI runs repeatable checks; it does not replace expert judgment or certify the whole product.

Choose critical Journeys

Prioritize authentication, checkout, publishing, invitations, account recovery, destructive actions, complex forms, dialogs, dynamic updates, and high-impact customer/operator outcomes. Add component checks for shared primitives, but retain the Journey context where the component’s behavior matters.

Pin rule and protocol versions

Record engine, automated rule, WCAG/internal requirement, manual protocol, and component versions. A changing scanner can change results without product change. Update rule versions through Review and preserve historical evidence.

Declare contexts

Roles, permissions, plan, locale, viewport, zoom, theme, reduced motion, data, assistive technology, input method, and feature flags can alter accessibility. Select meaningful contexts and record skipped/unsupported cells.

Automated evidence

Playwright documents integration with @axe-core/playwright and warns automation cannot detect every violation. Store rule ID/version, target, occurrence, component/owner, context, checkpoint, and evidence hash.

Avoid snapshotting the entire raw violations array. Preserve bounded fingerprints and durable barrier identity.

ARIA snapshots

Use ARIA snapshots for meaningful accessible-tree contracts such as heading structure, dialog name, status, and control relationships. Do not make giant brittle whole-page snapshots when only a critical region matters.

Manual protocols

Schedule required protocols for keyboard sequence, focus entry/return, announcements, error comprehension, alternative text quality, zoom/reflow, screen reader behavior, and cognitive usability. Record evaluator and exact context.

CI can require a current manual result without pretending the runner performed it.

Barriers and ownership

Connect occurrences to a stable barrier and component version. Assign an owner. Record remediation and verification. Shared-component fixes become candidates across affected Journeys and require evidence in each relevant context.

Exceptions

An exception requires authorized Decision, rationale, scope, criteria, Journeys/contexts, owner, grant time, and expiry. Keep the finding visible. Expiry or material change reopens Review.

Failure policy

New serious barriers, missing required manual evidence, expired exceptions, privacy violations, or failed remediation can block release under policy. Infrastructure failure retries separately. Do not suppress rules or mark not-tested as pass.

Evidence alongside function and visuals

A functional pass can remain inaccessible. A visual match can hide a lost name. A remediation can intentionally change pixels. Store peer evidence on the same Run timeline and decide subjects independently.

Conformance boundary

Bind statements to exact Journey, version, contexts, criteria, automated/manual evidence, exceptions, policy, and time. State “supports,” “partially supports,” “not tested,” or “not applicable” honestly. Do not claim universal compliance from CI.

Privacy

Accessibility artifacts can contain DOM snippets, text, screenshots, user data, or URLs. Redact and bound public evidence. Keep private Runs under workspace authorization and retention.

Component and Journey layers

Run focused component checks for shared states and complete Journey checks for interaction/outcome. Component evidence can identify broad impact; Journey evidence verifies the component inside actor, data, layout, and surrounding workflow.

When a shared fix lands, discover affected Journeys, select representative contexts, and create new verification events rather than closing all barriers by association.

Baselines and known debt

If existing barriers cannot be fixed immediately, preserve stable fingerprints, owners, scope, evidence, and expiring Decisions. Avoid snapshots of verbose tool output that churns on unrelated HTML. Do not exclude entire containers to keep CI green.

Trend open/verified/excepted barriers without treating accepted debt as a pass.

CI artifacts and retention

Store machine-readable results, bounded human views, engine logs, relevant DOM/ARIA snapshots, and manual evidence references. Hash and retain them according to release, privacy, and legal-hold policy. Public conformance statements use sanitized authorized evidence only.

Pull requests and mainline

PR checks should identify affected Journeys, new/changed barriers, expired evidence, and owner. Scheduled/mainline Runs catch environment or dependency drift. Production-safe Runs require explicit side-effect and privacy policy.

Failure-class boundaries

Scanner crash is infrastructure. Missing target can be target or product. Wrong authenticated role is state. A rule violation is accessibility evidence. Leaked private content is privacy/policy. Preserve the distinction for correct retry and ownership.

Common anti-patterns

• “zero automated violations” presented as compliance;
• manual testing referenced without version/date/evaluator;
• permanent rule suppression;
• exceptions with no expiry;
• component fix closing unrelated Journey occurrences;
• accessibility score averaged with functional/visual status;
• public artifacts built from private customer state.

Integration boundaries

Keep axe, Playwright, manual-testing tools, issue trackers, design-system catalogs, and conformance systems authoritative for their native records. A Reshot adapter must preserve rule/protocol version, target, component, occurrence, owner, evidence, and source identity. A pasted summary is not a verified integration.

Release communication

Publish accessibility impact only from reviewed scoped evidence. Explain remediations, remaining barriers, accepted risk, and affected contexts without exposing private details. Link the current statement and immutable release evidence where authorized.

Example pipeline

1. build production-like target;
2. materialize declared state;
3. run functional Journey;
4. capture visual and automated accessibility evidence at checkpoints;
5. validate ARIA contracts;
6. attach current manual protocols;
7. update barriers and ownership;
8. evaluate exceptions/expiry;
9. record Review Decisions;
10. include results and outward impact in Release Book.

Metrics

Track Journeys/contexts with current automated and manual evidence, open barriers by owner/component, remediation verification, exception expiry, release blocks, and affected outward instructions. Avoid a single accessibility score.

First pilot

Choose one dialog or multi-step Journey. Add axe evidence, a focused ARIA snapshot, keyboard/focus protocol, barrier ownership, and an expiring exception test. Confirm expiration reopens Review and the Release Book does not call accepted risk a pass.

Continue with Accessibility Evidence Across Product Journeys, Accessibility Testing Across User Journeys, and Product Journey Runs.