Encrypt content at rest - phase 1 of the #189 data-minimization track #322

Merged

robbertbos merged 1 commit from phase-1-at-rest-encryption into main 2026-08-15 21:33:07 +00:00

Conversation 0 Commits 1 Files changed 14 +627 -50

robbertbos commented 2026-08-15 21:15:28 +00:00

Owner

Copy link

cards.source_data (the message snapshot), cards.notes and outgoing_posts.content become Fernet ciphertext under two new HKDF domains (waggle-cardcontent-v1, waggle-outgoing-v1), decrypted transparently at the ORM boundary (EncryptedJSON/EncryptedText over bytea in models/base.py). A database dump, backup or SQL-injection read no longer yields message content, notes or unsent drafts.

Why this is functionally free: nothing filters on these columns in SQL. The note filter and the retention sweep's note check already ran in Python after load; the one cheap SQL pre-filter on notes is removed. Import dedupe keeps working through column projections, which decrypt like any ORM read.

The revision (e5a9c31d7f02) converts existing rows in place and is NOT rolling-safe - the changelog carries a 'Note when rolling out': stop the old pod first and take a database backup; reading the converted columns requires the WAGGLE_SECRET_KEY the migration ran with.

waggle rotate-keys now also re-encrypts the three content columns, on raw table clauses rather than the ORM (the mapped types decrypt with process settings, not --from/--to). docs/security.md gains the two domains and a 'Content at rest' section; the key-rotation runbook in deployment.md names the covered columns.

Gates: 1981 backend tests at 100% coverage (including two new migration tests: in-place conversion of seeded plaintext, and bytea column types at head), 1177 vitest, vue-tsc + build, pinned pre-commit, full e2e suite green against a stack that boots through the new migration.

cards.source_data (the message snapshot), cards.notes and outgoing_posts.content become Fernet ciphertext under two new HKDF domains (waggle-cardcontent-v1, waggle-outgoing-v1), decrypted transparently at the ORM boundary (EncryptedJSON/EncryptedText over bytea in models/base.py). A database dump, backup or SQL-injection read no longer yields message content, notes or unsent drafts. Why this is functionally free: nothing filters on these columns in SQL. The note filter and the retention sweep's note check already ran in Python after load; the one cheap SQL pre-filter on notes is removed. Import dedupe keeps working through column projections, which decrypt like any ORM read. The revision (e5a9c31d7f02) converts existing rows in place and is NOT rolling-safe - the changelog carries a 'Note when rolling out': stop the old pod first and take a database backup; reading the converted columns requires the WAGGLE_SECRET_KEY the migration ran with. waggle rotate-keys now also re-encrypts the three content columns, on raw table clauses rather than the ORM (the mapped types decrypt with process settings, not --from/--to). docs/security.md gains the two domains and a 'Content at rest' section; the key-rotation runbook in deployment.md names the covered columns. Gates: 1981 backend tests at 100% coverage (including two new migration tests: in-place conversion of seeded plaintext, and bytea column types at head), 1177 vitest, vue-tsc + build, pinned pre-commit, full e2e suite green against a stack that boots through the new migration.

robbertbos added 1 commit 2026-08-15 21:15:28 +00:00

Encrypt card and outgoing content at rest - phase 1 of #189 ... f8c82565c9

cards.source_data (the message snapshot), cards.notes and
outgoing_posts.content become Fernet ciphertext under two new HKDF domains
(waggle-cardcontent-v1, waggle-outgoing-v1), decrypted transparently at the
ORM boundary (EncryptedJSON/EncryptedText over bytea). A database dump,
backup or SQL-injection read no longer yields message content. Feasible
without functional loss because nothing filters on these columns in SQL:
the note filter and retention checks already ran in Python, and the one
cheap SQL pre-filter on notes is gone.

The revision converts existing rows in place and is not rolling-safe (see
the changelog's rollout note). waggle rotate-keys covers the new columns on
raw bytes, since the mapped types decrypt with the process settings rather
than --from/--to.

robbertbos force-pushed phase-1-at-rest-encryption from f8c82565c9 to d93c086c3a 2026-08-15 21:25:48 +00:00 Compare

robbertbos merged commit d93c086c3a into main 2026-08-15 21:33:07 +00:00

robbertbos deleted branch phase-1-at-rest-encryption 2026-08-15 21:33:07 +00:00

robbertbos referenced this pull request 2026-08-16 12:47:37 +00:00

Explore: fetch Mattermost data directly from the frontend (backend data minimization) #189

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

help wanted

Need some help

  

invalid

Something is wrong

  

nldd ds

  

question

More information is needed

  

wontfix

This won't be fixed

No labels

bug

duplicate

enhancement

help wanted

invalid

nldd ds

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

robbertbos/waggle!322

No description provided.