robbertbos/waggle
robbertbos/waggle
1
0
Fork 0
Code Issues 6 Pull requests 5 Releases 8 Packages 2 Activity Actions

Test that Alembic migrations run and match the models #133

Closed
robbertbos wants to merge 1 commit from test-alembic-migrations into main
pull from: test-alembic-migrations
merge into: robbertbos:main
Conversation 1 Commits 1 Files changed 3 +179
robbertbos commented 2026-07-19 21:08:00 +00:00
Owner
Copy link

The backend suite builds its schema with Base.metadata.create_all (tests/conftest.py), so no test ever executed a migration: a revision that is broken, missing, or out of step with the models still showed a green run - and upgrade head runs at application startup, so that is exactly the kind of thing that only surfaces in production.

What this adds

backend/tests/integration/test_migrations.py with two integration tests:

  • upgrade -> downgrade -> upgrade is stable: the chain runs both ways and lands back on the same schema.
  • the migrated schema matches the models: uses Alembic's own autogenerate diff rather than a column-name comparison, so a revision that forgot a unique constraint, an index or a foreign key fails too.

The tests run against a file-backed throwaway database. Pointing WAGGLE_TEST_MIGRATION_URL at a Postgres database runs the same assertions there; a guard rail refuses any database whose name does not end in _migrations or _test, because the fixture wipes it with DROP SCHEMA public CASCADE.

CI runs the tests twice:

  • in the SQLite suite (part of the normal pytest run), and
  • as a separate step in the backend-test-postgres job, against a dedicated waggle_migrations database.

SQLite (batch-mode table rebuild) and Postgres (ALTER TABLE) are different Alembic code paths; a revision can pass on one and fail on the other, and production runs Postgres - this is what validates the production configuration.

Relation to #123

Split out of #123 so the migration safety net is not tied to the composite-FK work. #123 keeps its FK-specific migration tests (constraint survival, accept/reject behaviour, data backfill) and will be rebased on top once this merges.

The backend suite builds its schema with `Base.metadata.create_all` (tests/conftest.py), so no test ever executed a migration: a revision that is broken, missing, or out of step with the models still showed a green run - and `upgrade head` runs at application startup, so that is exactly the kind of thing that only surfaces in production. ## What this adds **`backend/tests/integration/test_migrations.py`** with two integration tests: - **upgrade -> downgrade -> upgrade is stable**: the chain runs both ways and lands back on the same schema. - **the migrated schema matches the models**: uses Alembic's own autogenerate diff rather than a column-name comparison, so a revision that forgot a unique constraint, an index or a foreign key fails too. The tests run against a file-backed throwaway database. Pointing `WAGGLE_TEST_MIGRATION_URL` at a Postgres database runs the same assertions there; a guard rail refuses any database whose name does not end in `_migrations` or `_test`, because the fixture wipes it with `DROP SCHEMA public CASCADE`. **CI** runs the tests twice: - in the SQLite suite (part of the normal pytest run), and - as a separate step in the `backend-test-postgres` job, against a dedicated `waggle_migrations` database. SQLite (batch-mode table rebuild) and Postgres (`ALTER TABLE`) are different Alembic code paths; a revision can pass on one and fail on the other, and production runs Postgres - this is what validates the production configuration. ## Relation to #123 Split out of #123 so the migration safety net is not tied to the composite-FK work. #123 keeps its FK-specific migration tests (constraint survival, accept/reject behaviour, data backfill) and will be rebased on top once this merges.
Test that Alembic migrations run and match the models
Some checks failed
CI / pre-commit (pull_request) Successful in 54s
Details
CI / frontend-test (pull_request) Successful in 4m59s
Details
security-scan / Python SCA (pip-audit) (pull_request) Successful in 56s
Details
CI / backend-test (pull_request) Successful in 8m29s
Details
security-scan / Python SAST (bandit) (pull_request) Successful in 33s
Details
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 24s
Details
security-scan / JS SCA (npm audit) (pull_request) Successful in 40s
Details
security-scan / SBOM (trivy) (pull_request) Successful in 16s
Details
CI / backend-test-postgres (pull_request) Failing after 9m23s
Details
CI / e2e (pull_request) Successful in 8m17s
Details
test-build / build (backend) (pull_request) Successful in 2m2s
Details
test-build / build (frontend) (pull_request) Successful in 2m17s
Details
test-build / build (pull_request) Successful in 0s
Details
CI / release-scripts (pull_request) Has been cancelled
Details
8919ecf6b5
 
The suite builds its schema with Base.metadata.create_all
(tests/conftest.py), so no test ever executed a migration: a broken or
missing revision shipped green. Two integration tests close that gap:
upgrade/downgrade/upgrade lands on the same schema, and the migrated
schema is diffed against the models with Alembic's own autogenerate
compare, which also catches forgotten indexes, constraints and FKs.

CI runs them twice: within the SQLite suite, and in the Postgres job
against a dedicated throwaway waggle_migrations database. SQLite's batch
rebuild and Postgres's ALTER TABLE are different code paths, and
production runs Postgres.
robbertbos commented 2026-07-20 05:37:48 +00:00
Author
Owner
Copy link

Overbodig geworden: PR #123 is gemerged en bracht de volledige migratie-testharnas al naar main - dezelfde twee tests hier plus vier extra (composite FKs, partiële index, insert/reject, backfill), dezelfde CI-stap 'Alembic migrations against Postgres' en dezelfde CHANGELOG-regel. Het add/add-conflict op test_migrations.py is precies die overlap. Deze PR sluit daarom zonder merge; er gaat niets verloren.

Overbodig geworden: PR #123 is gemerged en bracht de volledige migratie-testharnas al naar main - dezelfde twee tests hier plus vier extra (composite FKs, partiële index, insert/reject, backfill), dezelfde CI-stap 'Alembic migrations against Postgres' en dezelfde CHANGELOG-regel. Het add/add-conflict op test_migrations.py is precies die overlap. Deze PR sluit daarom zonder merge; er gaat niets verloren.
robbertbos closed this pull request 2026-07-20 05:37:48 +00:00
Some checks failed
CI / pre-commit (pull_request) Successful in 54s
Required
Details
CI / frontend-test (pull_request) Successful in 4m59s
Required
Details
security-scan / Python SCA (pip-audit) (pull_request) Successful in 56s
Required
Details
CI / backend-test (pull_request) Successful in 8m29s
Required
Details
security-scan / Python SAST (bandit) (pull_request) Successful in 33s
Required
Details
security-scan / Filesystem scan (trivy fs) (pull_request) Successful in 24s
Required
Details
security-scan / JS SCA (npm audit) (pull_request) Successful in 40s
Required
Details
security-scan / SBOM (trivy) (pull_request) Successful in 16s
Required
Details
CI / backend-test-postgres (pull_request) Failing after 9m23s
Required
Details
CI / e2e (pull_request) Successful in 8m17s
Required
Details
test-build / build (backend) (pull_request) Successful in 2m2s
Details
test-build / build (frontend) (pull_request) Successful in 2m17s
Details
test-build / build (pull_request) Successful in 0s
Details
CI / release-scripts (pull_request) Has been cancelled
Required
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
robbertbos/waggle!133
No description provided.