Ivan Ball-llovera

↑↓ to navigate Enter to open "…" all these words ANDOR to combine

Architecture Decision Record

ADR-063: WCAG 2.1 AA Accessibility as a Shipped Test Contract and CI Gate

Status

Accepted (2026-08-01). Revised 2026-08-14: refreshed the E2ETestBase helper line anchors (explanatory comments were added above ScanGridAsync), the two consumer suite scan counts (ADC now 31, Store now 23), and the MMCA.Helpdesk pin location. Revised 2026-08-23: the forgot-password vertical (ADR-091) added a fourth asserting Identity workflow base and three gallery assertions, so the base count, the gallery count (now eleven assertions across seven classes), and both consumer subclass counts are updated, along with a correction to ADC's grid/strict split; no decision changed. Revised 2026-08-31: a signed-in devices/sessions page scan joined the gallery (now twelve assertions across eight classes), and the PageExtensions, ADC/Store deploy.yml, and MMCA.Helpdesk pin line anchors are refreshed; no decision changed. Revised 2026-09-01: the gallery count is re-measured from source (fifteen assertions across nine classes), which adds the virtualized-grid page scan in GridPageE2ETests (a class the record had never listed) and the two notification inbox deep-link scans, corrects the signed-in set (the notification scans seed the fake auth cookie too, not the sessions scan alone), and refreshes the E2ETestBase helper, ADC deploy.yml, and Store deploy.yml deploy-job line anchors; no decision changed. Revised 2026-09-19: all three suite counts are re-measured from source (ADC now 45 scans, Store now 32, the gallery seventeen assertions across eleven classes), and the two shell-level gallery classes the record had never listed, ShellPagesE2ETests and MobileNavKeyboardE2ETests, are added to the narrative; no decision changed. Revised 2026-10-01: citations re-anchored to current source; no decision changed. Revised 2026-10-06: the deploy-gate narrative now records the cross-browser-freshness gate that enforces firefox and webkit recency on both deploys, and ADC's ProfileManagementTests base is named precisely (AdcE2ETestBase); no decision changed. Revised 2026-10-07: the CI workflow prune of that day folded cross-browser-freshness into a single freshness job that the deploy needs list names, so the record now cites it as a step of that job, and the CI and deploy anchors are refreshed after that prune; no decision changed. Revised 2026-10-21: the gallery count is re-measured (nineteen assertions across twelve classes, adding the role administration scans), the gallery no longer uses the pager exception, and the CI, deploy, theme and Helpdesk anchors are refreshed; no decision changed. Revised 2026-10-26: anchors refreshed after the CI and deploy workflow edits that moved the Common ui-e2e and ADC/Store deploy.yml gate lines; no decision changed. Revised 2026-10-28: anchors refreshed for the profile-page base scan and the Common ui-e2e E2E_BROWSER assignment, which the body still cited at stale lines; no decision changed.

Context

Accessibility was documented before it was enforced. The narrative guide (common-ACCESSIBILITY.md, rubric section 21) named WCAG 2.1 AA as the target for the shared MMCA.Common.UI surface and listed the pages that had been checked, but a guide cannot fail a build. That left three open problems.

First, the target itself was ambiguous. "Run axe" is not a contract: axe ships advisory "best-practice" rules alongside the WCAG rule sets, so two repos running the same tool with different options measure different things and disagree about whether a page passes.

Second, conformance is a property of rendered pixels and the accessibility tree, not of code structure. ADR-015's fitness functions gate structure (layer references, purity) and ADR-058's conformance bases gate a booted host's runtime contracts (ProblemDetails, OpenAPI, security headers). Neither can see a contrast ratio or a missing accessible name. This tier needs a real browser engine, which means it belongs in the E2E package rather than in either of those.

Third, the regressions are silent and come from shared values. A theme token changed for visual reasons can drop a filled button's label under the 4.5:1 floor across every consuming app at once, and nobody notices until a user reports it. Accessibility only stays true if the check runs on the same schedule as the code that can break it.

Decision

Ship WCAG 2.1 AA as a named, versioned test contract in MMCA.Common.Testing.E2E, assert it from the package's own workflow bases, and wire it as a required merge check and a deploy gate.

  • One named target, shipped in the package. AxeOptions.Wcag21Aa (MMCA.Common/Source/Hosting/MMCA.Common.Testing.E2E/Infrastructure/AxeOptions.cs:17) pins every scan to a RunOnly tag list of exactly wcag2a, wcag2aa, wcag21a, wcag21aa (:22): levels A and AA across WCAG 2.0 and 2.1. axe's "best-practice" rules are deliberately outside the target (:12-15), so the gate fails only on conformance violations, never on advisories.
  • A violation is a thrown test failure, not a report. AssertNoAccessibilityViolationsAsync(AxeRunOptions?) (.../Testing.E2E/Infrastructure/PageExtensions.cs:316) runs axe and, on any violation, throws AccessibilityViolationException (:355; the type is .../Testing.E2E/Infrastructure/AccessibilityViolationException.cs:7) carrying each rule's impact, id, help text, and the offending markup compacted to one line per node (:345-353, the per-node CompactHtml line at :351), so a red gate points at the element rather than at a dashboard.
  • The package's Identity workflow bases assert it, so a consumer inherits the scan. Four bases carry the assertion, so a subclass gets a login-page, register-page, profile-page, forgot-password-page, and reset-password-page a11y test with no test code of its own: .../Testing.E2E/Workflows/Identity/UserLoginTestsBase.cs:82, UserRegistrationTestsBase.cs:99, ProfileManagementTestsBase.cs:204, and PasswordResetTestsBase.cs (class at :17, the two scans asserting at :88 and :99, added with the password-recovery flow of ADR-091) each call the assert with AxeOptions.Wcag21Aa.
  • Consumer page scans go through two helpers that make strictness explicit. On .../Testing.E2E/Infrastructure/E2ETestBase.cs, ScanAsync() (:539) waits for any loading bar to clear and asserts the strict options (:542); ScanGridAsync() (:529) additionally waits for a seeded data row before scanning and asserts with the one recorded exception (:534). Which helper a page uses is the declaration of which rule set applies to it.
  • Exactly one recorded exception exists, and it is a value, not a switch. AxeOptions.Wcag21AaExceptMudPagerCombobox (AxeOptions.cs:35) carries the same four WCAG tags (:40) and disables a single rule, aria-input-field-name (:44). It exists for pages whose only combobox is MudBlazor's own MudTablePager "rows per page" select: MudBlazor 9.6.0 mirrored combobox semantics onto the MudSelect presenter but gives the pager's select no accessible name, and MudTablePager exposes no Label/aria-label parameter, so it is not fixable from app markup (:26-33). Every other WCAG 2.1 AA rule still runs on those pages.
  • Common gates it cross-browser as a required merge check. The ui-e2e job (MMCA.Common/.github/workflows/ci.yml:223) builds the out-of-slnx gallery plus E2E project and runs the axe scans across a chromium, firefox, webkit matrix (:232), one engine per leg via E2E_BROWSER (:376), with fail-fast: false so each engine reports independently (:230). All three contexts block merges (MMCA.Common/CONTRIBUTING.md:63-64, enumerated in the branch-protection payload at :202-204).
  • Both deployed apps gate the deploy on it. The e2e-gate jobs (MMCA.ADC/.github/workflows/deploy.yml:808, MMCA.Store/.github/workflows/deploy.yml:754) call the reusable e2e.yml workflow (ADC :820, Store :765) chromium-only (ADC :822, Store :767) against the full Aspire stack, and the deploy job waits on that gate (ADC :1161, Store :1099). The same deploy needs list also carries the freshness job (ADC :1161, Store :1099), whose cross-browser-freshness step (ADC :925, Store :859) launches no browser: it asserts that the scheduled firefox and webkit e2e.yml legs (Monday firefox, Thursday webkit) each have a recent successful run, so engine coverage beyond chromium gates the deploy by recency rather than per commit (ADC rationale at :909-924, Store at :851-858).
  • The gate already owns design-token decisions. Contrast values in the shared theme are set to what the scan will accept, with the ratio recorded in place: light-palette WarningContrastText (MMCA.Common/Source/Presentation/MMCA.Common.UI/Theme/MMCATheme.cs:36, rationale at :28-34, caught by a gated admin-order-list scan on a chip), dark-palette PrimaryContrastText (:71, rationale at :68-70, caught by the gated dark-mode scan), dark WarningContrastText (:90) and ErrorContrastText (:98), and the brand secondary itself, moved to Teal 700 for a 5.3:1 light surface ratio (.../MMCA.Common.UI/Theme/BrandColors.cs:26, rationale at :22-24).

Adoption differs per repo and is uneven on purpose. MMCA.Common scans its own backend-less gallery: nineteen assertions across twelve classes over GalleryAxeTestBase (MMCA.Common/Tests/Presentation/MMCA.Common.UI.E2E.Tests/Infrastructure/GalleryAxeTestBase.cs:14), every one of them under the strict AxeOptions.Wcag21Aa (the gallery does not use the pager exception), covering login (LoginPageE2ETests.cs:34), register (RegisterPageE2ETests.cs:36), the primitives showcase (ComponentsPageE2ETests.cs:80), both dark-mode states (DarkModeE2ETests.cs:30,:40), the notification pages, whose five scans cover the history table (NotificationPagesE2ETests.cs:31), the inbox (:43), its two deep-link states (an unknown notification id at :60, a known one at :73), and the compose page (:85), the role administration list and editor of ADR-116 (Administration/RoleAdminPagesE2ETests.cs:31 and :43, class at :15), the virtualized grid page, whose windowed markup is scanned under the strict options like every other gallery page (GridPageE2ETests.cs:82, class at :16), and the password-recovery pages added with ADR-091: forgot-password in both its form state and its confirmation state, which replaces the form and is therefore a distinct rendering (ForgotPasswordPageE2ETests.cs:46 and :60), plus reset-password (ResetPasswordPageE2ETests.cs:48). Two shell-level classes cover states no page-specific test reaches: Layout/ShellPagesE2ETests.cs:31 (class at :16) asserts the strict options over the three framework-owned shell routes in one theory (/, /not-found, /forbidden), so the nineteen assertions scan twenty-one page states in total, and Layout/MobileNavKeyboardE2ETests.cs:36 (class at :16) scans the home page at a 390x844 phone viewport with the hamburger menu open, the one state the rest of the suite misses because it runs at desktop width. The notification and sessions scans are the signed-in ones: both the notification pages and the shared devices/sessions page (SessionsPageE2ETests.cs:34, class at :13) carry a real [Authorize], so those tests seed the gallery's cookie-toggled fake authentication scheme before navigating (NotificationPagesE2ETests.cs:91, SessionsPageE2ETests.cs:39) while the login, register, components, role administration, grid, shell and mobile-nav scans stay deliberately anonymous. The sessions scan covers the populated table, its current-device marker, and the per-device sign-out buttons under the strict options. MMCA.Store subclasses all four Identity bases (Tests/E2E/MMCA.Store.E2E.Tests/Workflows/Identity/UserLoginTests.cs:5, UserRegistrationTests.cs:5, ProfileManagementTests.cs:5, PasswordResetTests.cs:5). MMCA.ADC subclasses three of the four (Tests/E2E/MMCA.ADC.E2E.Tests/Workflows/Identity/UserLoginTests.cs:5, UserRegistrationTests.cs:5, PasswordResetTests.cs:5); its ProfileManagementTests derives the app's own AdcE2ETestBase wrapper rather than ProfileManagementTestsBase (ProfileManagementTests.cs:8) because the ADC profile page supports only password change and account deletion, so it does not inherit the base's profile scan. Beyond the Identity bases each app carries a dedicated suite: ADC's Tests/E2E/MMCA.ADC.E2E.Tests/Workflows/AccessibilityTests.cs:29 holds 45 page scans (13 through ScanGridAsync, 32 strict) and Store's Tests/E2E/MMCA.Store.E2E.Tests/Workflows/AccessibilityTests.cs:18 holds 32 (9 grid, 23 strict). MMCA.Helpdesk adopts none of it: it pins the package version (MMCA.Helpdesk/Directory.Packages.props:93) but no project references it, and the repo has no E2E test project at all (Tests/ holds only Architecture and Modules), so the seed has no browser accessibility gate today.

Rationale

  • A named constant is the contract. Putting the rule set in a shipped, referenced symbol rather than in each repo's test setup means "what WCAG 2.1 AA means here" has exactly one definition, it versions with the package, and a change to it is a reviewable diff in one file instead of drift discovered by comparing CI logs.
  • Excluding best-practice rules is what makes the gate blockable. A gate that can red on advisory findings gets demoted to advisory itself. Scoping to the four WCAG tags means every failure is a real conformance violation, which is the precondition for making the check required rather than informational.
  • Assert in the base, not in the consumer. The same push-left posture as ADR-058: shipping the assertion inside UserLoginTestsBase / UserRegistrationTestsBase / ProfileManagementTestsBase / PasswordResetTestsBase means an app that adopts the Identity workflow tests cannot accidentally adopt them without the a11y scan. Opting out is visible (it means not subclassing the base), the way ADC's profile page is.
  • A real engine, not reflection. Contrast, accessible names, and live-region announcement exist only in a rendered accessibility tree. That is why this sits in the E2E package with Playwright and not in the NetArchTest tier (ADR-015) or the booted-host conformance tier (ADR-058).
  • Cross-browser where it is cheap, chromium where it is expensive. The gallery is backend-less, so all three engines run there and catch the cross-engine class of defect at low cost. A full Aspire-stack E2E leg costs roughly twenty minutes, so the deploy path runs chromium only and leans on the gallery matrix, plus the cross-browser-freshness recency check over the scheduled firefox and webkit legs, for engine coverage.
  • Exceptions as distinct option values, not as suppression flags. A second constant with a different name forces the caller to say which contract it is scanning under, and it keeps the exception's justification attached to the thing being excepted rather than buried in a config file.

Trade-offs

  • Best-practice rules are out of scope, deliberately. Findings axe would classify as best practice (and anything WCAG AAA) are not measured at all, so the gate can be green on a page a specialist would still improve. This is the price of a blockable gate, and it is recorded on the constant itself (AxeOptions.cs:12-15).
  • One accepted exception, with real blast radius. Wcag21AaExceptMudPagerCombobox disables aria-input-field-name for the whole page scan, not just for the pager node. A grid page that later gains a genuinely unnamed combobox of its own would pass ScanGridAsync (E2ETestBase.cs:534). The exception is documented as "use only where the sole combobox is a pager" (AxeOptions.cs:33), which is a convention the compiler cannot enforce. It is upstream-owned: it stands until MudBlazor labels the pager select.
  • Automation covers only what axe can judge. Focus order, reading order, and announcement quality are not machine-checkable and are covered by the manual screen-reader checklist in common-ACCESSIBILITY.md, which is a periodic human pass, not a gate.
  • The deploy gate is ui-scoped and may legitimately skip. Both apps gate e2e-gate on a ui change filter (ADC deploy.yml:819, Store deploy.yml:764), and deploy accepts success or skipped for it (ADC :1198, Store :1132), so a backend-only or infra-only deploy ships without a browser scan. That is the intended cost trade (a backend change cannot alter rendered markup) with the post-deploy smoke gate as backstop, but it does mean "deployed" does not always mean "axe ran on this commit".
  • Consumer breadth is hand-maintained. Nothing forces a new page into AccessibilityTests, so coverage grows by discipline: 45 scans in ADC and 32 in Store today, against far larger page inventories.
  • MMCA.Helpdesk has no accessibility gate. The reference app demonstrates the framework's layers but not this contract, so a reader following the seed sees no worked example of adopting the scan.

Revision (2026-10-01)

No decision or rationale changed. Citations are re-anchored to current source: the Identity base assertions (UserLoginTestsBase.cs:82, UserRegistrationTestsBase.cs:95, ProfileManagementTestsBase.cs:191), the E2ETestBase helpers (ScanGridAsync :355/:360, ScanAsync :365/:368), the Common ui-e2e job (ci.yml:248, :255, :257, :321) and its branch-protection payload (CONTRIBUTING.md:198-200), the ADC and Store deploy.yml gate, filter, needs and acceptance lines (ADC :827, :838, :841, :1185, :1230; Store :799, :810, :813, :1136, :1177), the MMCATheme.cs contrast tokens (:36, :66, :85, :93), both AccessibilityTests class declarations (ADC :29, Store :18), and the MMCA.Helpdesk pin (Directory.Packages.props:94). The suite counts (ADC 45 scans, 13 grid and 32 strict; Store 32, 9 grid and 23 strict) are re-measured and unchanged.

Revision (2026-10-06)

No decision or rationale changed.

  • The deploy-gate bullet and the cross-browser rationale now record the cross-browser-freshness gate in both apps' deploy needs (ADC deploy.yml:1000, Store deploy.yml:954), which enforces recency of the scheduled firefox and webkit E2E legs; the earlier text implied the gallery matrix was the only engine coverage behind a deploy.
  • ADC's ProfileManagementTests is described as deriving the app's AdcE2ETestBase wrapper (ProfileManagementTests.cs:8), not E2ETestBase literally; the substance (it does not inherit the base's profile scan) is unchanged.
  • Anchors re-verified against current source: PageExtensions.cs (:316, :355, :345-353), UserRegistrationTestsBase.cs:99, the E2ETestBase helpers (ScanGridAsync :529/:534, ScanAsync :539/:542), and the ADC and Store deploy.yml gate, reusable-workflow, browser, filter, needs and acceptance lines (ADC :874, :886, :888, :885, :1242, :1287; Store :841, :853, :855, :852, :1187, :1228).

Revision (2026-10-07)

Re-verified against current source. No decision or rationale changed: Common still gates the gallery scans across all three engines, both apps still gate the deploy on a chromium-only e2e-gate that may legitimately skip, and the scheduled firefox and webkit legs still gate it by recency. What moved is how that recency check sits in the deploy graph, plus the line anchors.

  1. The 2026-10-07 CI workflow prune merged the separate freshness jobs, cross-browser-freshness among them, into one freshness job, so the check is no longer its own entry in the deploy needs list (it was when the 2026-10-06 revision was written). The needs list now names the freshness job (ADC MMCA.ADC/.github/workflows/deploy.yml:1145, Store MMCA.Store/.github/workflows/deploy.yml:1083), and cross-browser-freshness is one step inside that job (ADC :909, Store :856), with its rationale comment at ADC :893-908 and Store :848-855. The Monday firefox / Thursday webkit recency semantics are unchanged (ADC :899, Store :850).
  2. Anchors re-verified against current source: the Common ui-e2e job (ci.yml:223), its fail-fast: false (:230), browser matrix (:232) and E2E_BROWSER (:291); the ADC and Store e2e-gate jobs (ADC deploy.yml:792, Store deploy.yml:751), their ui filter (ADC :803, Store :761), reusable e2e.yml call (ADC :804, Store :762) and chromium-only browsers (ADC :806, Store :764); the deploy needs lists (ADC :1145, Store :1083) and their success-or-skipped acceptance of e2e-gate (ADC :1182, Store :1116).

Revision (2026-10-21)

Re-verified against current source. No decision or rationale changed: the target is still AxeOptions.Wcag21Aa, Common still requires all three engines (MMCA.Common/CONTRIBUTING.md:63-64), and both apps still gate the deploy on a chromium-only e2e-gate plus the cross-browser-freshness recency step. What moved is the gallery's inventory, plus the line anchors.

  1. The gallery count is re-measured from source: nineteen assertions across twelve classes (was seventeen across eleven), scanning twenty-one page states. The new class is Administration/RoleAdminPagesE2ETests.cs (class at :15), whose two anonymous strict scans cover the role administration list (:31) and editor (:43) of ADR-116.
  2. The notification history scan no longer uses the pager exception: all five notification scans, history included, assert the strict AxeOptions.Wcag21Aa (NotificationPagesE2ETests.cs:31), and no gallery test uses Wcag21AaExceptMudPagerCombobox, so the exception's only callers are now the consumers' ScanGridAsync pages. The narrative no longer calls it "the gallery's one use".
  3. Anchors re-verified against current source: the branch-protection payload (CONTRIBUTING.md:202-204); the notification scans (NotificationPagesE2ETests.cs:43, :60, :73, :85) and their cookie seeding (:91); the ADC and Store e2e-gate jobs (MMCA.ADC/.github/workflows/deploy.yml:807, MMCA.Store/.github/workflows/deploy.yml:753), their ui filter (ADC :818, Store :763), reusable e2e.yml call (ADC :819, Store :764) and chromium-only browsers (ADC :821, Store :766); the deploy needs lists naming freshness and e2e-gate (ADC :1160, Store :1098) and their success-or-skipped acceptance (ADC :1197, Store :1131); the cross-browser-freshness step (ADC :924, Store :858) with its rationale comment (ADC :908-923, Store :850-857); the dark-palette MMCATheme.cs contrast tokens (PrimaryContrastText :71, rationale :68-70; WarningContrastText :90; ErrorContrastText :98); and the MMCA.Helpdesk pin (Directory.Packages.props:93), still referenced by no project.

Revision (2026-10-26)

Re-verified against current source. No decision or rationale changed: Common still runs the gallery scans across a chromium, firefox, webkit matrix with fail-fast: false, both apps still gate the deploy on a chromium-only e2e-gate that may legitimately skip, and the cross-browser-freshness step of the freshness job still gates it by recency. Only line anchors moved.

  1. Anchors re-verified against current source: the Common E2E_BROWSER assignment (MMCA.Common/.github/workflows/ci.yml:344; the ui-e2e job :223, fail-fast :230 and matrix :232 are unchanged); the ADC and Store e2e-gate jobs (MMCA.ADC/.github/workflows/deploy.yml:808, MMCA.Store/.github/workflows/deploy.yml:754), their ui filter (ADC :819, Store :764), reusable e2e.yml call (ADC :820, Store :765) and chromium-only browsers (ADC :822, Store :767); the deploy needs lists naming freshness and e2e-gate (ADC :1161, Store :1099) and their success-or-skipped acceptance (ADC :1198, Store :1132); the cross-browser-freshness step (ADC :925, Store :859) with its rationale comment (ADC :909-924, Store :851-858).

Revision (2026-10-28)

Re-verified against current source. No decision or rationale changed: the four Identity bases still assert AxeOptions.Wcag21Aa, and Common still runs the gallery scans across a chromium, firefox, webkit matrix with fail-fast: false. Two body anchors that earlier revisions never corrected are re-pointed.

  1. Anchors re-verified against current source: the profile-page scan in MMCA.Common/Source/Hosting/MMCA.Common.Testing.E2E/Workflows/Identity/ProfileManagementTestsBase.cs:204 (inside ProfilePage_ShouldHaveNoAccessibilityViolations, method at :193; the body cited :191), and the Common E2E_BROWSER assignment (MMCA.Common/.github/workflows/ci.yml:376, inside the ui-e2e job at :223, whose fail-fast :230 and matrix :232 are unchanged; the body cited :344).

ADR-015 (architecture fitness functions: the structural tier this parallels at the browser tier, and the same invariant-over-discipline posture), ADR-058 (runtime conformance suites shipped as subclassable bases in a package: the pattern this record applies to accessibility, one tier further out), ADR-028 (dark theme: the dark palette's contrast tokens, MMCATheme.cs:71,:90,:98, exist in the form they do because the gated dark-mode scan rejected the defaults), and the how-to companion common-ACCESSIBILITY.md (scanned-state inventory, manual screen-reader checklist, and the tracked limitations).

← Back to Architecture Decision Records View source on GitHub ↗