Attachments end-to-end: drafts, replies and scheduled posts (fase 2b) #164

Merged
robbertbos merged 8 commits from fase-2b-kabel-v2 into main 2026-07-24 07:00:49 +00:00
Owner

Fase 2b van "nieuw bericht opstellen + plannen". Base: main.

Bijlagen reizen nu end-to-end als geparkeerde refs (owned uuids) door concept, direct antwoord en gepland bericht, en uploaden pas naar Mattermost op het moment van verzenden (dat ontwijkt de 7-daagse vervaltermijn van MM voor niet-gekoppelde bestanden). De oude POST /cards/{id}/files-route (MM file_id-currency) is geschrapt ten gunste van de kaartloze POST /outgoing-files (ref-currency). Een dagelijkse orphan-sweep ruimt geparkeerde bestanden op die na 7 dagen door geen enkele rij meer worden gerefereerd.

Uit de adversariele review-pass (security / correctness / WCAG)

  • Concept-bijlagen rehydrateren bij het laden, zodat een heropend concept ze niet stil loskoppelt.
  • Een bijlage die tijdens uploaden wordt verwijderd lekt niet meer; een lopende concept-save wordt geflusht bij unmount.
  • Upload-fouten worden in een aria-live-regio aangekondigd en met tekst gemarkeerd (niet alleen kleur).
  • De runner verwijdert geparkeerde bestanden pas nadat STATE_SENT is vastgelegd.
  • _delete_orphan_refs is per-eigenaar gescoped; outgoing_files faalt dicht op een eigenaarloze sidecar.

BIO/DPIA/NeRDS-review-fixes (in deze PR verwerkt)

  • Bijlage-inhoud versleuteld at rest (eigen HKDF-domein waggle-parkedfile-v1).
  • Retry plaatst geen dubbele bijlage meer (mm_file_id-memo persistent).
  • Account loskoppelen ruimt de geparkeerde bijlagen op.
  • Toegankelijke upload-status + disabled-reden (WCAG 4.1.3 / 1.4.1).
  • Audit-events op de vertrouwensgrens: verzending naar MM + quota-weigering.
  • Docs: quota-grens, volume-encryptie/DPIA, bewaartermijn.

Poorten

Backend 1398 tests, 100% coverage; frontend tsc + vitest + build + eslint; pre-commit schoon. Postgres-parity + e2e via CI.


Vervangt #145 (die PR liep vast in Forgejos mergeability-cache na meerdere force-pushes/reopens; branch en inhoud zijn identiek en lokaal geverifieerd - schone merge + alle gates groen).

Fase 2b van "nieuw bericht opstellen + plannen". Base: main. Bijlagen reizen nu end-to-end als geparkeerde refs (owned uuids) door concept, direct antwoord en gepland bericht, en uploaden pas naar Mattermost op het moment van verzenden (dat ontwijkt de 7-daagse vervaltermijn van MM voor niet-gekoppelde bestanden). De oude `POST /cards/{id}/files`-route (MM file_id-currency) is geschrapt ten gunste van de kaartloze `POST /outgoing-files` (ref-currency). Een dagelijkse orphan-sweep ruimt geparkeerde bestanden op die na 7 dagen door geen enkele rij meer worden gerefereerd. ## Uit de adversariele review-pass (security / correctness / WCAG) - Concept-bijlagen rehydrateren bij het laden, zodat een heropend concept ze niet stil loskoppelt. - Een bijlage die tijdens uploaden wordt verwijderd lekt niet meer; een lopende concept-save wordt geflusht bij unmount. - Upload-fouten worden in een aria-live-regio aangekondigd en met tekst gemarkeerd (niet alleen kleur). - De runner verwijdert geparkeerde bestanden pas nadat STATE_SENT is vastgelegd. - `_delete_orphan_refs` is per-eigenaar gescoped; `outgoing_files` faalt dicht op een eigenaarloze sidecar. ## BIO/DPIA/NeRDS-review-fixes (in deze PR verwerkt) - Bijlage-inhoud versleuteld at rest (eigen HKDF-domein `waggle-parkedfile-v1`). - Retry plaatst geen dubbele bijlage meer (mm_file_id-memo persistent). - Account loskoppelen ruimt de geparkeerde bijlagen op. - Toegankelijke upload-status + disabled-reden (WCAG 4.1.3 / 1.4.1). - Audit-events op de vertrouwensgrens: verzending naar MM + quota-weigering. - Docs: quota-grens, volume-encryptie/DPIA, bewaartermijn. ## Poorten Backend 1398 tests, 100% coverage; frontend tsc + vitest + build + eslint; pre-commit schoon. Postgres-parity + e2e via CI. --- _Vervangt #145 (die PR liep vast in Forgejos mergeability-cache na meerdere force-pushes/reopens; branch en inhoud zijn identiek en lokaal geverifieerd - schone merge + alle gates groen)._
Attachments now travel as parked refs (owned uuids) through draft, immediate
reply and scheduled send, and upload to Mattermost only at send-time, dodging
MM's 7-day unattached-file expiry. The old POST /cards/{id}/files route is
retired in favour of the card-less POST /outgoing-files; the reply route
validates refs at request-time and again at send-time. A daily orphan sweep
reaps parked files no row references after a 7-day TTL, wired into the
lifespan and reported in get_task_statuses.

Fixes from the adversarial review pass (security / correctness / WCAG):

- Draft attachments now rehydrate on load. The save side was wired but not the
  load side, so reopening a draft showed no chips and the first keystroke PUT an
  empty file_refs list, detaching and reaping the bytes. useFileUpload.seedDone
  restores parked files as done entries and useServerDraftSync seeds
  lastSavedRefs from the server so no empty list is ever sent.
- A file removed mid-upload no longer leaks: the finished upload's ref is
  deleted instead of lingering to the sweep and eating the user's quota.
- A pending draft-save is flushed on unmount, so closing the card modal inside
  the 500ms debounce no longer drops an edit (notably an attachment removal).
- An immediate reply now drops the server draft, so its content and now-sent
  refs do not linger and reload later.
- Upload failures are announced in an aria-live region and marked with text, not
  colour alone; a failed attachment gives an actionable reason instead of the
  misleading "wait until uploads finish".
- An upload response without a usable ref is treated as a failure.
- scheduled_runner._settle_sent deletes parked files only after STATE_SENT has
  committed; a crash in the old window turned a real send into a FAILED row
  whose bytes were already gone.
- _delete_orphan_refs is scoped to the owner: refs are per-user-unique, so the
  tenant filter is now an explicit invariant, not a bet on uuid uniqueness.
- outgoing_files stat_ref/delete_ref fail closed on an ownerless sidecar
  (no `None != None` fall-through).
- The new sweep symbols are exported from cleanup_runner.__all__.

Backend stays at 100% coverage; parity verified on Postgres.
Addresses the BIO/DPIA review of PR #145:
- Encrypt parked attachment bytes at rest (new HKDF domain waggle-parkedfile-v1),
  mirroring the token pattern; quota now counts logical (plaintext) size.
- Persist the mm_file_id memo across scheduled-send retries so a transient MM
  failure no longer re-uploads and posts a duplicate attachment.
- Unlinking a Mattermost account now purges its outgoing-posts' parked files,
  matching full-account deletion (was left to the 7-day orphan sweep).
Addresses the WCAG findings on PR #145:
- Keep the attachment-status and error live regions in the DOM (empty -> 0
  height) so a screen reader announces changes reliably instead of missing a
  freshly-inserted node (WCAG 4.1.3).
- Announce the general error region via role=alert.
- Fold the per-file upload error reason into the live announcement.
- Return focus to the editor after removing/retrying an attachment chip.
- Make the schedule chevron aria-disabled (stays focusable) with its reason
  wired via aria-describedby, so the reason reaches assistive tech, not only the
  tooltip.
- Audit the two trust-boundary events for attachments: sending them to
  Mattermost (immediate reply + scheduled) and a per-user quota refusal. Draft
  attach/remove churn is deliberately not logged (self-service, not a boundary).
- Document that the parked-attachment quota is a soft, bounded ceiling; that
  attachment bytes are app-encrypted at rest (volume encryption as defense in
  depth); and the DPIA/verwerkingsregister duty and retention.
- OutgoingFileUpload docstring pointed at the retired /api/cards/{id}/outgoing-files
  route; it is /api/outgoing-files since the card-less split.
- Move the reply-attachment reload/upload-status note out of the released
  v2026.7.20 section (it is fase-2b behaviour) into the Unreleased feature entry.
- A dragged file is always attached, never inserted at a caret, so the
  ProseMirror drop-cursor (which promised placement) is turned off, and a
  dragover overlay ('Sleep bestanden hier...') makes the drop target explicit.
- Attachment chips show an image thumbnail (client-side object URL, revoked on
  removal/unmount) or a paperclip icon; there is no read-back route, so a
  rehydrated image falls back to the icon.
Fix Postgres parity: shorten the quota audit event_type
Some checks failed
CI / pre-commit (pull_request) Successful in 51s
CI / release-scripts (pull_request) Has been cancelled
security-scan / Python SCA (pip-audit) (pull_request) Has been cancelled
security-scan / Python SAST (bandit) (pull_request) Has been cancelled
security-scan / JS SCA (npm audit) (pull_request) Has been cancelled
security-scan / Filesystem scan (trivy fs) (pull_request) Has been cancelled
security-scan / SBOM (trivy) (pull_request) Has been cancelled
CI / frontend-test (pull_request) Has been cancelled
test-build / build (pull_request) Has been cancelled
test-build / build (backend) (pull_request) Has been cancelled
test-build / build (frontend) (pull_request) Has been cancelled
CI / backend-test-postgres (pull_request) Has been cancelled
CI / backend-test (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
9da2b529e8
audit_events.event_type is VARCHAR(32); 'outgoing_attachment_quota_exceeded'
is 34 chars. SQLite ignores the length so it passed there, but Postgres raised
StringDataRightTruncationError -> backend-test-postgres failed. Renamed to
'outgoing_attachment_quota' (25).
robbertbos force-pushed fase-2b-kabel-v2 from b2510e0755
Some checks failed
CI / pre-commit (pull_request) Has been cancelled
CI / backend-test (pull_request) Has been cancelled
CI / backend-test-postgres (pull_request) Has been cancelled
CI / frontend-test (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
CI / release-scripts (pull_request) Has been cancelled
security-scan / Python SCA (pip-audit) (pull_request) Has been cancelled
security-scan / Python SAST (bandit) (pull_request) Has been cancelled
security-scan / JS SCA (npm audit) (pull_request) Has been cancelled
security-scan / Filesystem scan (trivy fs) (pull_request) Has been cancelled
security-scan / SBOM (trivy) (pull_request) Has been cancelled
test-build / build (pull_request) Has been cancelled
test-build / build (backend) (pull_request) Has been cancelled
test-build / build (frontend) (pull_request) Has been cancelled
to bcd090a158
Some checks failed
CI / pre-commit (pull_request) Has been cancelled
CI / backend-test (pull_request) Has been cancelled
CI / backend-test-postgres (pull_request) Has been cancelled
CI / frontend-test (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
CI / release-scripts (pull_request) Has been cancelled
security-scan / Python SCA (pip-audit) (pull_request) Has been cancelled
security-scan / Python SAST (bandit) (pull_request) Has been cancelled
security-scan / JS SCA (npm audit) (pull_request) Has been cancelled
security-scan / Filesystem scan (trivy fs) (pull_request) Has been cancelled
security-scan / SBOM (trivy) (pull_request) Has been cancelled
test-build / build (pull_request) Has been cancelled
test-build / build (backend) (pull_request) Has been cancelled
test-build / build (frontend) (pull_request) Has been cancelled
2026-07-24 05:41:00 +00:00
Compare
robbertbos force-pushed fase-2b-kabel-v2 from bcd090a158
Some checks failed
CI / pre-commit (pull_request) Has been cancelled
CI / backend-test (pull_request) Has been cancelled
CI / backend-test-postgres (pull_request) Has been cancelled
CI / frontend-test (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
CI / release-scripts (pull_request) Has been cancelled
security-scan / Python SCA (pip-audit) (pull_request) Has been cancelled
security-scan / Python SAST (bandit) (pull_request) Has been cancelled
security-scan / JS SCA (npm audit) (pull_request) Has been cancelled
security-scan / Filesystem scan (trivy fs) (pull_request) Has been cancelled
security-scan / SBOM (trivy) (pull_request) Has been cancelled
test-build / build (pull_request) Has been cancelled
test-build / build (backend) (pull_request) Has been cancelled
test-build / build (frontend) (pull_request) Has been cancelled
to 7957c50149
Some checks failed
CI / pre-commit (pull_request) Has been cancelled
CI / backend-test (pull_request) Has been cancelled
CI / backend-test-postgres (pull_request) Has been cancelled
CI / frontend-test (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
CI / release-scripts (pull_request) Has been cancelled
security-scan / Python SCA (pip-audit) (pull_request) Has been cancelled
security-scan / Python SAST (bandit) (pull_request) Has been cancelled
security-scan / JS SCA (npm audit) (pull_request) Has been cancelled
security-scan / Filesystem scan (trivy fs) (pull_request) Has been cancelled
security-scan / SBOM (trivy) (pull_request) Has been cancelled
test-build / build (pull_request) Has been cancelled
test-build / build (backend) (pull_request) Has been cancelled
test-build / build (frontend) (pull_request) Has been cancelled
2026-07-24 05:51:37 +00:00
Compare
robbertbos force-pushed fase-2b-kabel-v2 from 7957c50149
Some checks failed
CI / pre-commit (pull_request) Has been cancelled
CI / backend-test (pull_request) Has been cancelled
CI / backend-test-postgres (pull_request) Has been cancelled
CI / frontend-test (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
CI / release-scripts (pull_request) Has been cancelled
security-scan / Python SCA (pip-audit) (pull_request) Has been cancelled
security-scan / Python SAST (bandit) (pull_request) Has been cancelled
security-scan / JS SCA (npm audit) (pull_request) Has been cancelled
security-scan / Filesystem scan (trivy fs) (pull_request) Has been cancelled
security-scan / SBOM (trivy) (pull_request) Has been cancelled
test-build / build (pull_request) Has been cancelled
test-build / build (backend) (pull_request) Has been cancelled
test-build / build (frontend) (pull_request) Has been cancelled
to d6644cdbe7
Some checks failed
CI / pre-commit (push) Successful in 1m45s
CI / pre-commit (pull_request) Has been cancelled
CI / backend-test (pull_request) Has been cancelled
CI / backend-test-postgres (pull_request) Has been cancelled
CI / frontend-test (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
CI / release-scripts (pull_request) Has been cancelled
security-scan / Python SCA (pip-audit) (pull_request) Has been cancelled
security-scan / Python SAST (bandit) (pull_request) Has been cancelled
security-scan / JS SCA (npm audit) (pull_request) Has been cancelled
security-scan / Filesystem scan (trivy fs) (pull_request) Has been cancelled
security-scan / SBOM (trivy) (pull_request) Has been cancelled
test-build / build (pull_request) Has been cancelled
test-build / build (backend) (pull_request) Has been cancelled
test-build / build (frontend) (pull_request) Has been cancelled
CI / e2e (push) Failing after 6m12s
CI / release-scripts (push) Successful in 10s
CI / frontend-test (push) Successful in 8m52s
publish-main / build (backend) (push) Successful in 2m29s
publish-main / build (frontend) (push) Successful in 2m41s
publish-main / build (push) Successful in 0s
security-scan / Python SCA (pip-audit) (push) Successful in 59s
security-scan / Python SAST (bandit) (push) Successful in 40s
security-scan / JS SCA (npm audit) (push) Successful in 43s
CI / backend-test (push) Successful in 12m39s
security-scan / Filesystem scan (trivy fs) (push) Successful in 23s
security-scan / SBOM (trivy) (push) Successful in 15s
CI / backend-test-postgres (push) Successful in 13m3s
2026-07-24 06:03:19 +00:00
Compare
robbertbos deleted branch fase-2b-kabel-v2 2026-07-24 07:00:49 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
robbertbos/waggle!164
No description provided.