Cache and log hardening - phase 0 of the #189 data-minimization track #313

Merged

robbertbos merged 1 commit from phase-0-leak-hygiene into main 2026-08-15 20:39:55 +00:00

Conversation 0 Commits 1 Files changed 46 +1016 -212

robbertbos commented 2026-08-15 12:42:20 +00:00

Owner

Copy link

Research for #189 identified twelve cache/log leak surfaces that are independent of any architecture choice. This PR closes them (phase 0); the direction itself (refs-only cards, content on demand) stays in #189.

At the transport edge:

• Every /api response defaults to Cache-Control: no-store; the byte-proxy caches only inline raster images, for an hour; avatars and emoji keep their cache.
• Typed text no longer travels in request lines: mention-candidates (card + channel), resolve-mentions and people search are POST with the text in the body (resolve-mentions and people now take real lists). Accepted exception: the PDF preview keeps the real filename as its path segment, because the browser viewer titles itself after it (see security.md).
• The production image runs uvicorn with --no-access-log; the ZAD ingress keeps the single copy of request lines. The pod writes its own scrubbed request log instead (status, duration, pseudonymous user/session ids; no IPs or query strings), and oidc_sessions.last_seen_at marks session liveness once a minute.

In the application:

• SQLAlchemy errors stringify without bind parameters (hide_parameters), so a failed card insert cannot put message text in a traceback.
• 422 responses carry type/loc/msg but never the submitted input; import validation details name field paths, not values.
• The rate limiter refusal log carries ip: instead of a raw client IP (same treatment as audit_events).
• sync_logs rows age out after 90 days (WAGGLE_SYNC_LOG_RETENTION_DAYS), pruned on the audit-purge tick.

In the browser:

• Logout sends Clear-Site-Data: "cache", "storage" (theme rehydrates from the settings table) and sweeps localStorage traces; the 401 path sweeps too.

Docs: security.md gains a section on response caching, browser storage, access logs and the shared-workstation scenario; deployment.md gains WAGGLE_LOG_LEVEL (never DEBUG in production) and WAGGLE_SYNC_LOG_RETENTION_DAYS.

Gates: 1970 backend tests at 100% coverage, 1177 vitest, vue-tsc + build, pinned pre-commit across all files, full e2e suite locally green after rebase.

Research for #189 identified twelve cache/log leak surfaces that are independent of any architecture choice. This PR closes them (phase 0); the direction itself (refs-only cards, content on demand) stays in #189. At the transport edge: - Every /api response defaults to Cache-Control: no-store; the byte-proxy caches only inline raster images, for an hour; avatars and emoji keep their cache. - Typed text no longer travels in request lines: mention-candidates (card + channel), resolve-mentions and people search are POST with the text in the body (resolve-mentions and people now take real lists). Accepted exception: the PDF preview keeps the real filename as its path segment, because the browser viewer titles itself after it (see security.md). - The production image runs uvicorn with --no-access-log; the ZAD ingress keeps the single copy of request lines. The pod writes its own scrubbed request log instead (status, duration, pseudonymous user/session ids; no IPs or query strings), and oidc_sessions.last_seen_at marks session liveness once a minute. In the application: - SQLAlchemy errors stringify without bind parameters (hide_parameters), so a failed card insert cannot put message text in a traceback. - 422 responses carry type/loc/msg but never the submitted input; import validation details name field paths, not values. - The rate limiter refusal log carries ip:<hash> instead of a raw client IP (same treatment as audit_events). - sync_logs rows age out after 90 days (WAGGLE_SYNC_LOG_RETENTION_DAYS), pruned on the audit-purge tick. In the browser: - Logout sends Clear-Site-Data: "cache", "storage" (theme rehydrates from the settings table) and sweeps localStorage traces; the 401 path sweeps too. Docs: security.md gains a section on response caching, browser storage, access logs and the shared-workstation scenario; deployment.md gains WAGGLE_LOG_LEVEL (never DEBUG in production) and WAGGLE_SYNC_LOG_RETENTION_DAYS. Gates: 1970 backend tests at 100% coverage, 1177 vitest, vue-tsc + build, pinned pre-commit across all files, full e2e suite locally green after rebase.

robbertbos added 15 commits 2026-08-15 12:42:20 +00:00

Default Cache-Control: no-store on every /api response ... ed5f424697

Authenticated JSON (card snapshots, threads, drafts) was served without any
cache directive, so browsers could write it to the disk cache and replay it
on back/forward navigation, surviving logout. The middleware now sets
no-store as a default; routes that cache deliberately keep their own header
via setdefault.

Cache proxied attachments only when they are inline raster images ... dff8688e7f

The file proxy sent private, max-age=86400 for every MIME type, parking PDF,
text, video and forced downloads in the browser's disk cache for a day,
beyond logout. Only inline png/jpeg/gif/webp keep the day cache; everything
else, plus ?disposition=attachment and ?preview=html, is no-store now.
_AVATAR_MIMES is renamed _RASTER_IMAGE_MIMES because the same set now backs
the cache rule.

Stop reflecting submitted values in validation errors ... 34b48bfdbf

Pydantic puts the offending input in every 422 error dict and in
str(ValidationError); on the import route that reflects a whole export
bundle (card text, notes) back in the response. 422 errors now carry only
type/loc/msg, and the import problem details name field paths instead of
values.

Hide SQL bind parameters in database error text ... 1ec74ad047

StatementError.__str__ appends [parameters: ...] and several background
loops log str(exc); a failed card or draft insert therefore wrote the full
message snapshot into the log. hide_parameters=True removes the values at
the source.

Hash the client IP in the rate-limit bucket key ... 14038d86ca

slowapi logs the bucket key on every refusal, which put the raw client IP
in the application log while audit_events deliberately stores only a hash.
The IP fallback now hashes the address with the same helper; per-user keys
are unchanged and bucketing behaviour is identical.

Shorten the inline-image attachment cache to an hour ... 3ae898b3e1

Attachment viewing is infrequent enough that a day-long disk cache buys
little; an hour keeps scroll performance while bounding how long bytes
outlive a session on a shared workstation. Avatars and emoji keep their
longer cache.

Purge sync_logs on the same nightly retention tick as audit_events ... 14d9fd1fb2

sync_logs rows were never removed: no content, but an unbounded per-cycle
account of when each user's client was active. They now age out after
WAGGLE_SYNC_LOG_RETENTION_DAYS (default 90), pruned in the audit purge
loop's tick.

Move typed text out of query strings on the lookup endpoints ... f82a394b92

Mention-candidates (card and channel), resolve-mentions and people search
carried what the user types - and for resolve-mentions up to 2048 chars of
usernames naming who appears in a saved message - as GET query parameters,
which land verbatim in the uvicorn access log and in the ZAD ingress logs
we cannot scrub. All four are POST now, with the same validation moved into
request bodies; resolve-mentions and people take real lists instead of
comma-joined strings. Targets keeps GET (no user text).

Stop putting the attachment filename in the PDF-preview URL ... f7c88d2852

The trailing path segment exists to title the browser's PDF viewer, but a
path lands verbatim in the uvicorn and ingress access logs, and filenames
are content. The URL now carries a fixed generic segment; the modal header
keeps showing the real name.

Sweep personal localStorage traces on logout and session expiry ... d3234f33ad

Reply-draft text, per-card visibility keys, and the recent-mention and
emoji lists survived logout in the browser profile; only the mention cache
was wiped. One sweep now runs next to it on both the logout and the 401
path. Theme and section-collapse stay: preferences, not personal traces.

Disable the uvicorn access log in the production image ... 9ae3a56874

The ZAD HAProxy ingress already records every request line at the edge; the
in-pod copy also carried query strings and real client IPs that the
redaction chain does not scrub. Dev and preview keep their access log.

Document the client-side leak surfaces in the threat model ... b862ecf974

New security.md section on response caching, browser storage, access logs
and the shared-workstation scenario; deployment.md gains WAGGLE_LOG_LEVEL
(never DEBUG in production) and WAGGLE_SYNC_LOG_RETENTION_DAYS rows.

Cover the validation-detail truncation and drop its dead branch ... 1e7aaebfd7

A ValidationError always carries at least one error, so the empty-locs
fallback could never run; the >5-paths truncation now has the test it was
missing.

Changelog for the cache/log hardening round 0acd4ec602

Ask the browser to drop the HTTP cache on logout ... 04d7582faa

The proxied attachment bytes live in the browser's disk cache for up to an
hour; Clear-Site-Data: "cache" purges them on the deliberate logout in
Chromium and Firefox. "cache" only, so theme and layout preferences keep
surviving logout on purpose.

robbertbos force-pushed phase-0-leak-hygiene from 04d7582faa to 746cd3c0fe 2026-08-15 12:53:56 +00:00 Compare

robbertbos force-pushed phase-0-leak-hygiene from 746cd3c0fe to 65cda0de63 2026-08-15 13:07:54 +00:00 Compare

robbertbos force-pushed phase-0-leak-hygiene from 65cda0de63 to 44367ba486 2026-08-15 20:01:27 +00:00 Compare

robbertbos referenced this pull request 2026-08-15 20:07:52 +00:00

Cut v2026.8.15 #319

robbertbos force-pushed phase-0-leak-hygiene from 44367ba486 to bc7fea65c7 2026-08-15 20:24:50 +00:00 Compare

robbertbos force-pushed phase-0-leak-hygiene from bc7fea65c7 to 53f612b964 2026-08-15 20:30:29 +00:00 Compare

robbertbos force-pushed phase-0-leak-hygiene from 53f612b964 to ff2ceeabfa 2026-08-15 20:36:00 +00:00 Compare

robbertbos merged commit ff2ceeabfa into main 2026-08-15 20:39:55 +00:00

robbertbos deleted branch phase-0-leak-hygiene 2026-08-15 20:39:55 +00:00

robbertbos referenced this pull request from a commit 2026-08-15 21:25:26 +00:00

Channel typeahead: ~ links a channel

robbertbos referenced this pull request 2026-08-15 21:32:54 +00:00

Channel typeahead: ~ links a channel #320

robbertbos referenced this pull request from a commit 2026-08-15 21:47:35 +00:00

Channel typeahead: ~ links a channel

robbertbos referenced this pull request 2026-08-16 12:47:37 +00:00

Explore: fetch Mattermost data directly from the frontend (backend data minimization) #189

robbertbos referenced this pull request from a commit 2026-08-16 13:16:46 +00:00

Channel typeahead: ~ links a channel

robbertbos referenced this pull request from a commit 2026-08-16 13:57:29 +00:00

Channel typeahead: ~ links a channel

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

help wanted

Need some help

  

invalid

Something is wrong

  

nldd ds

  

question

More information is needed

  

wontfix

This won't be fixed

No labels

bug

duplicate

enhancement

help wanted

invalid

nldd ds

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

robbertbos/waggle!313

No description provided.