Fail the build and the tests on a silently broken design token #220

Merged
robbertbos merged 1 commit from deps/nldd-guardrails into main 2026-08-05 13:36:28 +00:00
Owner

Follow-up to #219. Independent of it: both branch off main, and only CHANGELOG.md conflicts between them (both add a bullet in the same spot), so whichever merges second keeps both lines.

Why

Every risk found in the 0.8.77 upgrade passed typecheck, eslint, vitest, the build and the Playwright e2e suite while the app rendered in Times at 16px. A var() pointing at a custom property nobody defines is invalid at computed-value time: the whole declaration is dropped, silently, with no warning anywhere in the toolchain. NLDD retires tokens inside 0.8.x patch releases, so the names we read move under us - --shadow-md and --color-focus have already gone this way.

Three checks

Each one is proven to fail on input that should fail it. A check whose reassuring answer is "nothing found" is worth nothing until you have seen it say something.

1. frontend/src/styles/nlddTokens.test.ts - resolves every var(--...) in src/** against the custom properties defined by the installed @nldd/design-system plus our own tokens.css. Fails on a reference that has neither a definition nor a fallback.

  • Runs in ~50 ms, no browser, no flake.
  • Green today, so it needs no cleanup to adopt. Four dangling references exist (--color-danger, --font-weight-bold, --semantics-buttons-primary-tinted-background-color, --toast-duration) and all four carry a fallback, so they still render. They are untidy, not broken.
  • Self-tested two ways. Against the pre-fix tokens.css on 0.8.77 it names both offending lines. With a bogus reference added to base.css it reports --dit-token-bestaat-niet at src/styles/base.css:204.
  • A second assertion guards the guard: it fails if fewer than 500 definitions or 100 references were found, so the check cannot pass by reading nothing at all.

2. frontend/tests/e2e/document-typography.spec.ts - asserts the rendered result rather than the cause: system-ui at 18px/27px, and no RijksSans in the computed stack or in any @font-face. The face list must still contain JetBrains Mono, so the negative assertion cannot pass on a page that loaded no stylesheets.

3. frontend/vite.config.ts - fontTuningPlugin now checks its own output instead of its removal counter.

The comment already said "fail loudly rather than shipping the font because a rename broke a regex", but the code only called this.warn. A reshaped @font-face leaves the rule in the CSS while removedFaces reads zero, and the build would have shipped the licence-restricted font on a warning nobody reads in CI. Testing the stripped CSS for a surviving RijksSans face makes that an error.

Self-tested by neutering RIJKS_FACE so it matches nothing: the build fails with a RijksSans @font-face survived the strip in [assets/index-xEuRwH19.css]. Restored, it builds clean with only the two JetBrains Mono woff2 emitted.

The counter check stays a warning on purpose: upstream may legitimately stop bundling RijksSans, and that should not break the build.

An earlier version of this also re-scanned the bundle for surviving RijksSans assets. That was a false positive - rolldown still enumerates entries the first pass deleted, so it failed a build that was in fact correct. Dropped, because every matching asset is deleted in the same pass and the check could never fire meaningfully.

What this does not cover

Honest scope: of the ten confirmed risks in the 0.8.77 analysis, this catches one. It happens to be the only breaking one.

The rest - the card shadow default going to none, the corner radius going 0px to 10px, the divider colour shifting, the banner frame moving into the shadow DOM - are value and structure changes. The token name still exists; its meaning changed. No name check sees that. The only net that covers that class is Playwright screenshot baselines on a few surfaces, which costs baseline maintenance on every deliberate style change. Not proposed here.

Verification

vue-tsc typecheck, eslint (0 errors), vitest 572 passed in 78 files (was 570/77), vite build, Playwright e2e 65 passed / 3 skipped (was 63), pre-commit run --all-files.

Follow-up to #219. Independent of it: both branch off `main`, and only `CHANGELOG.md` conflicts between them (both add a bullet in the same spot), so whichever merges second keeps both lines. ## Why Every risk found in the 0.8.77 upgrade passed typecheck, eslint, vitest, the build and the Playwright e2e suite while the app rendered in `Times` at 16px. A `var()` pointing at a custom property nobody defines is invalid at computed-value time: the whole declaration is dropped, silently, with no warning anywhere in the toolchain. NLDD retires tokens inside 0.8.x patch releases, so the names we read move under us - `--shadow-md` and `--color-focus` have already gone this way. ## Three checks Each one is proven to fail on input that should fail it. A check whose reassuring answer is "nothing found" is worth nothing until you have seen it say something. **1. `frontend/src/styles/nlddTokens.test.ts`** - resolves every `var(--...)` in `src/**` against the custom properties defined by the installed `@nldd/design-system` plus our own `tokens.css`. Fails on a reference that has neither a definition nor a fallback. - Runs in ~50 ms, no browser, no flake. - Green today, so it needs no cleanup to adopt. Four dangling references exist (`--color-danger`, `--font-weight-bold`, `--semantics-buttons-primary-tinted-background-color`, `--toast-duration`) and all four carry a fallback, so they still render. They are untidy, not broken. - Self-tested two ways. Against the pre-fix `tokens.css` on 0.8.77 it names both offending lines. With a bogus reference added to `base.css` it reports `--dit-token-bestaat-niet at src/styles/base.css:204`. - A second assertion guards the guard: it fails if fewer than 500 definitions or 100 references were found, so the check cannot pass by reading nothing at all. **2. `frontend/tests/e2e/document-typography.spec.ts`** - asserts the rendered result rather than the cause: `system-ui` at 18px/27px, and no RijksSans in the computed stack or in any `@font-face`. The face list must still contain JetBrains Mono, so the negative assertion cannot pass on a page that loaded no stylesheets. **3. `frontend/vite.config.ts`** - `fontTuningPlugin` now checks its own output instead of its removal counter. The comment already said "fail loudly rather than shipping the font because a rename broke a regex", but the code only called `this.warn`. A reshaped `@font-face` leaves the rule in the CSS while `removedFaces` reads zero, and the build would have shipped the licence-restricted font on a warning nobody reads in CI. Testing the stripped CSS for a surviving RijksSans face makes that an error. Self-tested by neutering `RIJKS_FACE` so it matches nothing: the build fails with `a RijksSans @font-face survived the strip in [assets/index-xEuRwH19.css]`. Restored, it builds clean with only the two JetBrains Mono woff2 emitted. The counter check stays a warning on purpose: upstream may legitimately stop bundling RijksSans, and that should not break the build. An earlier version of this also re-scanned the bundle for surviving RijksSans *assets*. That was a false positive - rolldown still enumerates entries the first pass deleted, so it failed a build that was in fact correct. Dropped, because every matching asset is deleted in the same pass and the check could never fire meaningfully. ## What this does not cover Honest scope: of the ten confirmed risks in the 0.8.77 analysis, this catches **one**. It happens to be the only breaking one. The rest - the card shadow default going to `none`, the corner radius going 0px to 10px, the divider colour shifting, the banner frame moving into the shadow DOM - are value and structure changes. The token name still exists; its meaning changed. No name check sees that. The only net that covers that class is Playwright screenshot baselines on a few surfaces, which costs baseline maintenance on every deliberate style change. Not proposed here. ## Verification `vue-tsc` typecheck, eslint (0 errors), vitest 572 passed in 78 files (was 570/77), `vite build`, Playwright e2e 65 passed / 3 skipped (was 63), `pre-commit run --all-files`.
robbertbos force-pushed deps/nldd-guardrails from 01980f0e31
All checks were successful
CI / release-scripts (pull_request) Successful in 7s
security-scan / SBOM (trivy) (pull_request) Successful in 11s
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 14s
security-scan / Python SAST (bandit) (pull_request) Successful in 20s
security-scan / JS SCA (npm audit) (pull_request) Successful in 21s
CI / pre-commit (pull_request) Successful in 31s
security-scan / Python SCA (pip-audit) (pull_request) Successful in 41s
CI / backend-test (pull_request) Successful in 55s
test-build / build (backend) (pull_request) Successful in 1m1s
test-build / build (frontend) (pull_request) Successful in 1m5s
test-build / build (pull_request) Successful in 0s
CI / frontend-test (pull_request) Successful in 1m13s
CI / e2e (pull_request) Successful in 3m13s
CI / backend-test-postgres (pull_request) Successful in 4m16s
to 4a948c80af
All checks were successful
CI / release-scripts (pull_request) Successful in 6s
security-scan / SBOM (trivy) (pull_request) Successful in 8s
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 13s
security-scan / Python SAST (bandit) (pull_request) Successful in 21s
security-scan / JS SCA (npm audit) (pull_request) Successful in 21s
CI / pre-commit (pull_request) Successful in 30s
security-scan / Python SCA (pip-audit) (pull_request) Successful in 39s
CI / backend-test (pull_request) Successful in 52s
test-build / build (backend) (pull_request) Successful in 1m1s
test-build / build (frontend) (pull_request) Successful in 1m6s
test-build / build (pull_request) Successful in 0s
CI / frontend-test (pull_request) Successful in 1m10s
CI / e2e (pull_request) Successful in 3m9s
CI / backend-test-postgres (pull_request) Successful in 4m27s
2026-08-05 13:20:09 +00:00
Compare
robbertbos force-pushed deps/nldd-guardrails from 4a948c80af
All checks were successful
CI / release-scripts (pull_request) Successful in 6s
security-scan / SBOM (trivy) (pull_request) Successful in 8s
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 13s
security-scan / Python SAST (bandit) (pull_request) Successful in 21s
security-scan / JS SCA (npm audit) (pull_request) Successful in 21s
CI / pre-commit (pull_request) Successful in 30s
security-scan / Python SCA (pip-audit) (pull_request) Successful in 39s
CI / backend-test (pull_request) Successful in 52s
test-build / build (backend) (pull_request) Successful in 1m1s
test-build / build (frontend) (pull_request) Successful in 1m6s
test-build / build (pull_request) Successful in 0s
CI / frontend-test (pull_request) Successful in 1m10s
CI / e2e (pull_request) Successful in 3m9s
CI / backend-test-postgres (pull_request) Successful in 4m27s
to 2a4fd4b326
Some checks failed
CI / release-scripts (pull_request) Successful in 6s
security-scan / SBOM (trivy) (pull_request) Successful in 12s
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 13s
security-scan / Python SAST (bandit) (pull_request) Successful in 21s
security-scan / JS SCA (npm audit) (pull_request) Successful in 21s
CI / pre-commit (pull_request) Successful in 30s
security-scan / Python SCA (pip-audit) (pull_request) Successful in 42s
CI / backend-test (pull_request) Successful in 1m5s
test-build / build (backend) (pull_request) Successful in 1m10s
test-build / build (frontend) (pull_request) Successful in 1m13s
test-build / build (pull_request) Successful in 0s
CI / frontend-test (pull_request) Successful in 1m23s
CI / e2e (pull_request) Successful in 3m22s
CI / backend-test-postgres (pull_request) Successful in 4m27s
security-scan / Filesystem scan (trivy fs) (push) Successful in 15s
CI / release-scripts (push) Successful in 6s
security-scan / SBOM (trivy) (push) Successful in 8s
security-scan / JS SCA (npm audit) (push) Successful in 17s
security-scan / Python SAST (bandit) (push) Successful in 21s
CI / pre-commit (push) Successful in 30s
security-scan / Python SCA (pip-audit) (push) Successful in 40s
publish-main / build (push) Has been cancelled
publish-main / build (frontend) (push) Has been cancelled
CI / frontend-test (push) Has been cancelled
CI / backend-test-postgres (push) Has been cancelled
publish-main / build (backend) (push) Has been cancelled
CI / e2e (push) Has been cancelled
CI / backend-test (push) Has been cancelled
2026-08-05 13:31:54 +00:00
Compare
robbertbos deleted branch deps/nldd-guardrails 2026-08-05 13:36:29 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
robbertbos/waggle!220
No description provided.