Office-document previews through a Gotenberg converter #461

Open
robbertbos wants to merge 1 commit from office-document-preview into main
Owner

Word, ODF, Excel and PowerPoint attachments (.doc .docx .odt .rtf .xls .xlsx .ods .ppt .pptx .odp) open as a PDF preview instead of downloading. The backend converts them with a Gotenberg converter that runs as its own ZAD component; the browser's PDF viewer renders the result. Reasoning in WDR-0015, security in docs/security.md#office-document-previews, component setup in docs/deployment.md ("Service: converter").

What changes

  • GET /api/mm/file/{id}/pdf/{name}?account_id=: fetches through the existing token-scoped proxy (streamed, stops above 20 MB), converts, returns application/pdf inline and no-store. Converter answers map to 413/415/422/503/504; 20/min per user.
  • WAGGLE_DOCUMENT_CONVERTER_URL switches it on. /api/auth/me then carries document_preview: true; without it, file cards download as before.
  • Frontend: file cards get "Document", "Rekenblad" and "Presentatie" (icons file-text-on-file-text, table, display). OfficePreviewModal fetches the PDF, shows "Voorbeeld maken...", frames it as a blob: URL, and on failure shows the reason with a download. Converted PDFs stay in tab memory (50 MB cap, cleared on logout and session expiry), so reopening does not convert again.
  • CSP: frame-src 'self' blob: in nginx and in the backend middleware. productionCsp.test.ts now requires it.
  • The upload to the converter is named document.<ext>. The real name only travels as the PDF Title: for a blob URL that is the only thing Chrome's viewer titles itself after.
  • Deploy jobs: a step builds the component list and adds the converter when ZAD_CONVERTER_ENABLED is true.
  • compose.dev.yaml: converter service under the converter profile, pinned gotenberg/gotenberg:8.37.0@sha256:f29984bd... (manifest-list digest checked against Docker Hub). fake-mm serves minimal .docx, .odt and .xlsx fixtures, so the whole chain can be checked locally.

Before this reaches users

The merge releases (Unreleased entry), but the feature stays off until these three one-off steps (also in docs/deployment.md, "Service: converter"):

  1. Create component converter in ZAD: port 3000, no publish-on-web, requests 50m/256Mi, limits 1 CPU/1Gi, env vars from the table. It has to exist first: an upsert adds a missing component without these settings.
  2. One alias on the backend, valid for every deployment: zad alias add -c backend 'WAGGLE_DOCUMENT_CONVERTER_URL=http://${DEPLOYMENT_NAME}-converter:3000'. ZAD names a component's Service <deployment>-<component> and gives every pod DEPLOYMENT_NAME (verified in RIG-Cluster and zad-cli v0.12.0 source, not yet against the live platform: an unknown variable answers 422 on save).
  3. Forgejo variable ZAD_CONVERTER_ENABLED=true. From then on both deploy jobs (production and staging) send the converter along with backend and frontend.

The converter image is pinned in CONVERTER_IMAGE in both deploy jobs and in compose.dev.yaml; a Renovate group ("document converter") moves all three together, and the release after that merge rolls it out. The step that builds the component list was run locally with the variable unset, false and true.

Measured (local, Gotenberg 8.37.0, 1 CPU / 1 GiB)

  • Idle 6 MB, 40 MB after LibreOffice's idle shutdown; 150 to 450 MB while converting.
  • Cold start plus first .docx 2.1 s, then 0.3 to 1.2 s for all ten formats.
  • Spreadsheets about 1.6 s per 1000 rows of 20 columns; 20.000 rows (5 MB) 32 s, so the 20 s timeout turns those into "te groot".
  • One LibreOffice process: a spreadsheet that runs into the timeout still holds the next conversion about 11 s longer.
  • A corrupt .docx converts to a PDF of garbage (200), not an error.
  • Chrome for Testing 153 renders a blob: PDF in an iframe under object-src 'none'.

Test plan

  • Backend: new test_mm_file_pdf_preview.py, test_document_preview.py, /me flag tests; coverage of document_preview.py and mm_avatar.py 100%.
  • Full backend suite: 2614 passed. 3 failures in test_mm_unread_api.py ("mattermost not linked", an FK violation) under load 5 on mybee; the same file passes on its own. Not touched here, worth watching in CI.
  • Frontend: 1716 vitest tests, vue-tsc, eslint, build.
  • Real chain in the preview (fake-mm, backend, Gotenberg, Chromium): .docx preview in 0.8 to 1.9 s, reopening makes no second request, .odt/.xlsx/.docx return a PDF through the route, an image gets 415. Loading and error states checked in light and dark.
  • Staging: create the component, set the URL, open a real Word and ODF attachment.

Noticed, not changed

  • nginx img-src has no blob:, while the backend CSP allows it for the composer's image thumbnail before upload. If that thumbnail is meant to work in production, nginx blocks it.
  • Gotenberg has --libreoffice-allow-list for LibreOffice's outbound fetches. An allow-list that matches nothing might close egress at the application level while ZAD cannot; not tested, so not set.
Word, ODF, Excel and PowerPoint attachments (`.doc .docx .odt .rtf .xls .xlsx .ods .ppt .pptx .odp`) open as a PDF preview instead of downloading. The backend converts them with a Gotenberg converter that runs as its own ZAD component; the browser's PDF viewer renders the result. Reasoning in WDR-0015, security in `docs/security.md#office-document-previews`, component setup in `docs/deployment.md` ("Service: `converter`"). ## What changes - `GET /api/mm/file/{id}/pdf/{name}?account_id=`: fetches through the existing token-scoped proxy (streamed, stops above 20 MB), converts, returns `application/pdf` inline and `no-store`. Converter answers map to 413/415/422/503/504; 20/min per user. - `WAGGLE_DOCUMENT_CONVERTER_URL` switches it on. `/api/auth/me` then carries `document_preview: true`; without it, file cards download as before. - Frontend: file cards get "Document", "Rekenblad" and "Presentatie" (icons `file-text-on-file-text`, `table`, `display`). `OfficePreviewModal` fetches the PDF, shows "Voorbeeld maken...", frames it as a `blob:` URL, and on failure shows the reason with a download. Converted PDFs stay in tab memory (50 MB cap, cleared on logout and session expiry), so reopening does not convert again. - CSP: `frame-src 'self' blob:` in nginx and in the backend middleware. `productionCsp.test.ts` now requires it. - The upload to the converter is named `document.<ext>`. The real name only travels as the PDF Title: for a blob URL that is the only thing Chrome's viewer titles itself after. - Deploy jobs: a step builds the component list and adds the converter when `ZAD_CONVERTER_ENABLED` is `true`. - `compose.dev.yaml`: `converter` service under the `converter` profile, pinned `gotenberg/gotenberg:8.37.0@sha256:f29984bd...` (manifest-list digest checked against Docker Hub). fake-mm serves minimal `.docx`, `.odt` and `.xlsx` fixtures, so the whole chain can be checked locally. ## Before this reaches users The merge releases (Unreleased entry), but the feature stays off until these three one-off steps (also in `docs/deployment.md`, "Service: `converter`"): 1. Create component `converter` in ZAD: port 3000, **no `publish-on-web`**, requests 50m/256Mi, limits 1 CPU/1Gi, env vars from the table. It has to exist first: an upsert adds a missing component without these settings. 2. One alias on the backend, valid for every deployment: `zad alias add -c backend 'WAGGLE_DOCUMENT_CONVERTER_URL=http://${DEPLOYMENT_NAME}-converter:3000'`. ZAD names a component's Service `<deployment>-<component>` and gives every pod `DEPLOYMENT_NAME` (verified in RIG-Cluster and zad-cli v0.12.0 source, not yet against the live platform: an unknown variable answers 422 on save). 3. Forgejo variable `ZAD_CONVERTER_ENABLED=true`. From then on both deploy jobs (production and staging) send the converter along with backend and frontend. The converter image is pinned in `CONVERTER_IMAGE` in both deploy jobs and in `compose.dev.yaml`; a Renovate group ("document converter") moves all three together, and the release after that merge rolls it out. The step that builds the component list was run locally with the variable unset, `false` and `true`. ## Measured (local, Gotenberg 8.37.0, 1 CPU / 1 GiB) - Idle 6 MB, 40 MB after LibreOffice's idle shutdown; 150 to 450 MB while converting. - Cold start plus first `.docx` 2.1 s, then 0.3 to 1.2 s for all ten formats. - Spreadsheets about 1.6 s per 1000 rows of 20 columns; 20.000 rows (5 MB) 32 s, so the 20 s timeout turns those into "te groot". - One LibreOffice process: a spreadsheet that runs into the timeout still holds the next conversion about 11 s longer. - A corrupt `.docx` converts to a PDF of garbage (200), not an error. - Chrome for Testing 153 renders a `blob:` PDF in an iframe under `object-src 'none'`. ## Test plan - [x] Backend: new `test_mm_file_pdf_preview.py`, `test_document_preview.py`, `/me` flag tests; coverage of `document_preview.py` and `mm_avatar.py` 100%. - [x] Full backend suite: 2614 passed. 3 failures in `test_mm_unread_api.py` ("mattermost not linked", an FK violation) under load 5 on mybee; the same file passes on its own. Not touched here, worth watching in CI. - [x] Frontend: 1716 vitest tests, `vue-tsc`, eslint, build. - [x] Real chain in the preview (fake-mm, backend, Gotenberg, Chromium): `.docx` preview in 0.8 to 1.9 s, reopening makes no second request, `.odt`/`.xlsx`/`.docx` return a PDF through the route, an image gets 415. Loading and error states checked in light and dark. - [ ] Staging: create the component, set the URL, open a real Word and ODF attachment. ## Noticed, not changed - nginx `img-src` has no `blob:`, while the backend CSP allows it for the composer's image thumbnail before upload. If that thumbnail is meant to work in production, nginx blocks it. - Gotenberg has `--libreoffice-allow-list` for LibreOffice's outbound fetches. An allow-list that matches nothing might close egress at the application level while ZAD cannot; not tested, so not set.
Preview Word, ODF, Excel and PowerPoint attachments
All checks were successful
CI / pre-commit (pull_request) Successful in 2m5s
CI / release-scripts (pull_request) Successful in 16s
security-scan / Python SCA (pip-audit) (pull_request) Successful in 51s
security-scan / Python SAST (bandit) (pull_request) Successful in 32s
security-scan / JS SCA (npm audit) (pull_request) Successful in 33s
CI / frontend-test (pull_request) Successful in 4m44s
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 24s
security-scan / SBOM (trivy) (pull_request) Successful in 16s
test-build / build (backend) (pull_request) Successful in 2m14s
test-build / build (frontend) (pull_request) Successful in 2m13s
test-build / build (pull_request) Successful in 0s
CI / e2e (pull_request) Successful in 15m26s
CI / backend-test (pull_request) Successful in 16m38s
2f43a62acd
Office attachments (.doc .docx .odt .rtf .xls .xlsx .ods .ppt .pptx
.odp) open as a preview instead of downloading. The backend fetches
the file through the token-scoped proxy, converts it to PDF with a
Gotenberg converter and returns it inline and no-store; the browser's
PDF viewer renders it, as for PDF attachments.

- GET /api/mm/file/{id}/pdf/{name}: streams at most 20 MB from
  Mattermost, maps converter answers to 413/415/422/503/504, 20/min
  per user. The upload is named document.<ext>; the real name only
  travels as the PDF Title, which the viewer shows for a blob URL.
- The converter is optional (WAGGLE_DOCUMENT_CONVERTER_URL). /me
  announces it as document_preview; without it cards download.
- The modal fetches the PDF and frames it as a blob URL, so a slow or
  failed conversion ends in a message. That takes frame-src blob: in
  both CSPs. The blob stays in tab memory (50 MB cap, cleared on
  logout), so reopening does not convert again.
- compose.dev.yaml gets a converter service (profile), fake-mm serves
  minimal docx/odt/xlsx fixtures for a local end-to-end check.
- WDR-0015, security.md and deployment.md cover the component, its
  flags and what it may reach.
robbertbos force-pushed office-document-preview from 2f43a62acd
All checks were successful
CI / pre-commit (pull_request) Successful in 2m5s
CI / release-scripts (pull_request) Successful in 16s
security-scan / Python SCA (pip-audit) (pull_request) Successful in 51s
security-scan / Python SAST (bandit) (pull_request) Successful in 32s
security-scan / JS SCA (npm audit) (pull_request) Successful in 33s
CI / frontend-test (pull_request) Successful in 4m44s
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 24s
security-scan / SBOM (trivy) (pull_request) Successful in 16s
test-build / build (backend) (pull_request) Successful in 2m14s
test-build / build (frontend) (pull_request) Successful in 2m13s
test-build / build (pull_request) Successful in 0s
CI / e2e (pull_request) Successful in 15m26s
CI / backend-test (pull_request) Successful in 16m38s
to 63bb4b0ae9
All checks were successful
CI / release-scripts (pull_request) Successful in 13s
security-scan / SBOM (trivy) (pull_request) Successful in 12s
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 17s
security-scan / JS SCA (npm audit) (pull_request) Successful in 17s
security-scan / Python SAST (bandit) (pull_request) Successful in 19s
security-scan / Python SCA (pip-audit) (pull_request) Successful in 37s
test-build / build (frontend) (pull_request) Successful in 1m4s
test-build / build (backend) (pull_request) Successful in 1m9s
test-build / build (pull_request) Successful in 0s
CI / frontend-test (pull_request) Successful in 1m16s
CI / pre-commit (pull_request) Successful in 1m58s
CI / backend-test (pull_request) Successful in 3m8s
CI / e2e (pull_request) Successful in 4m37s
2026-09-27 08:08:42 +00:00
Compare
All checks were successful
CI / release-scripts (pull_request) Successful in 13s
Required
Details
security-scan / SBOM (trivy) (pull_request) Successful in 12s
Required
Details
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 17s
Required
Details
security-scan / JS SCA (npm audit) (pull_request) Successful in 17s
Required
Details
security-scan / Python SAST (bandit) (pull_request) Successful in 19s
Required
Details
security-scan / Python SCA (pip-audit) (pull_request) Successful in 37s
Required
Details
test-build / build (frontend) (pull_request) Successful in 1m4s
Required
Details
test-build / build (backend) (pull_request) Successful in 1m9s
Required
Details
test-build / build (pull_request) Successful in 0s
Required
Details
CI / frontend-test (pull_request) Successful in 1m16s
Required
Details
CI / pre-commit (pull_request) Successful in 1m58s
Required
Details
CI / backend-test (pull_request) Successful in 3m8s
Required
Details
CI / e2e (pull_request) Successful in 4m37s
Required
Details
This pull request has changes conflicting with the target branch.
  • CHANGELOG.md
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin office-document-preview:office-document-preview
git switch office-document-preview
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
robbertbos/waggle!461
No description provided.