Method · Quality gates

Quality gates every change passes before merge

A quality gate is a check that must pass before a change can merge. Every change we ship passes six: its acceptance tests, security, privacy, accessibility, a performance budget and a senior review. A change that fails any of them does not merge.

Updated 22 November 2026

What a quality gate is

A quality gate is an automated or human check that blocks a merge until it passes. Gates run on every pull request, in the same order, for every change. Their job is to make the rules in the spec and the constitution enforceable, so quality does not depend on anyone remembering to check.

A spec says what must be true. On its own it cannot make anyone, or any coding agent, follow it. Gates close that gap. Each rule we care about is either a gate or a criterion tested inside one. If a rule matters and nothing checks it, we add the check.

Gate What it checks How
1. Acceptance tests Every criterion the change touches Playwright and Vitest, by criterion ID
2. Security Dependencies, secrets, headers Dependency audit, secret scan, header test
3. Privacy Storage keys and outgoing requests Inventory diff in a real browser
4. Accessibility WCAG 2.2 AA axe in the test suites, manual pass on new UI
5. Performance Core Web Vitals and bundle budget Lab run against the constitution’s budget
6. Senior review Intent, scope, maintainability A person who did not write the change

Gate 1: acceptance tests

The first gate runs the automated tests for every acceptance criterion, plus the full suite to catch regressions. A change passes only if every criterion it touches has a passing test and no earlier criterion breaks. A new criterion without a test fails the gate.

Tests are named after the criteria they prove, which is how the trace report is built later. How we write those criteria is covered in acceptance criteria for frontends.

Gate 2: security

The security gate checks the dependency tree, scans for secrets and verifies security headers. A known vulnerable dependency, a committed key or a weakened Content Security Policy fails the build. These checks run on every change, so problems are found when they are introduced rather than in a later review.

  • Dependencies. An audit of direct and transitive packages against known advisories. New dependencies need a reason in the pull request.
  • Secrets. A scan of the diff and history for keys, tokens and credentials.
  • Headers. A test that requests the built pages and checks the Content Security Policy, HSTS, X-Content-Type-Options, Referrer-Policy and frame protections against the constitution.
  • Sessions. Tests that session tokens are httpOnly cookies and never readable by scripts.

More detail is in frontend security.

Gate 3: privacy

The privacy gate loads the built pages in a real browser and records every storage write and every outgoing request. It compares them with the inventory the spec allows. A new cookie, local-storage key or third-party domain that the spec does not list and justify fails the gate.

This turns “no tracking without consent” into a check that runs on every change. It catches the common leak: a library or embed that quietly calls a third party. The gate also checks the order of events, so nothing non-essential is stored or sent before the consent choice.

# privacy-inventory.yaml (excerpt) · the gate fails on anything not listed
storage:
  - key: consent_choice
    purpose: Remember the visitor's consent decision
    set_before_consent: true
  - key: session
    purpose: Keep the member signed in (httpOnly cookie)
    set_before_consent: true
third_party_requests: []   # none allowed on any page

See privacy-first frontends for how the inventory is built.

Gate 4: accessibility

The accessibility gate runs axe inside the end-to-end tests for every page and state the change touches. Accessibility criteria from the spec run as tests too: keyboard paths, focus order, announcements. Automated checks catch only part of WCAG 2.2 AA, so new interactive UI also gets a manual keyboard and screen-reader pass.

Testing states matters as much as testing pages. A form can pass on load and fail in its error state, where contrast, focus and announcements usually break. Because the spec lists every state, the gate checks every state.

Gate 5: performance budget

The performance gate checks the change against the budget in the constitution. We use Core Web Vitals: Largest Contentful Paint, Interaction to Next Paint and Cumulative Layout Shift, plus a JavaScript budget per route. A change that pushes a route over budget fails, even if it passes everything else.

Lab measurements in CI catch regressions on each change. Field data from real users, at the 75th percentile, shows whether the budget holds in practice after release. Both are needed: lab data is repeatable, field data is real.

Metric Typical budget we set Measured
LCP ≤ 2.5 s Lab per change, field after release
INP ≤ 200 ms Lab interaction tests, field after release
CLS ≤ 0.1 Lab per change, field after release
JS per route Set per product in the constitution Build output per change

The numbers are set per product in the constitution. These are the defaults we start from, which match Google’s thresholds for “good”.

Gate 6: senior review

The last gate is a person. A senior engineer who did not write the change, and did not operate the agent that wrote it, reads the diff against its task and criteria. They check intent, unasked-for scope, architecture and maintainability, which no automated check can judge.

The review comment records which criteria were checked. When a review finds a gap in the spec, the fix is a spec delta, not a quiet patch. More on how this works with coding agents is in agent-assisted delivery.

Why gates instead of a final QA phase

Gates find problems when they are cheapest to fix: in the change that introduced them. A final QA phase finds the same problems weeks later, tangled with other changes and close to a deadline. Gates also scale with agent-assisted delivery. More changes per week means more checks, run automatically, at the same bar.

Gates on an existing frontend

For an existing codebase, an Audit runs these same checks once and reports the findings as spec deltas. The gates can then be added to the existing CI one at a time, starting with the ones that find the most. From that point, nothing new can make the frontend worse.

Start with a Spec Sprint

One to two weeks to a versioned spec you own, whoever builds it.

Start a Spec Sprint