frontend-test fails intermittently in CI and passes on a retry #404

New issue

Closed

opened 2026-09-20 22:10:27 +00:00 by robbertbos · 0 comments

robbertbos commented

2026-09-20 22:10:27 +00:00

Owner

frontend-test goes red on unrelated pushes and is green when the run is restarted, so the failure is in the run, not in the diff.

Measured on #403, a CHANGELOG-only change: the job failed after 220s, while the same suite locally is 1634 passed in 10.5s. Its neighbours on that push were just as slow (backend-test 254s, e2e 254s), which points at runner contention rather than a bad test - we push 13 jobs at one runner. Related: the e2e flakiness under the same contention, and #402.

Worth finding out before adding more retries: which tests fail, whether it is always the same ones, and whether it is timing out rather than asserting. The job's own log is the starting point - the Forgejo API does not serve it, so this needs the web interface.

Note a restart is not free: it hides how often this happens, and the number is what decides whether this is worth fixing at the runner or in the suite.

`frontend-test` goes red on unrelated pushes and is green when the run is restarted, so the failure is in the run, not in the diff. Measured on #403, a CHANGELOG-only change: the job failed after **220s**, while the same suite locally is **1634 passed in 10.5s**. Its neighbours on that push were just as slow (`backend-test` 254s, `e2e` 254s), which points at runner contention rather than a bad test - we push 13 jobs at one runner. Related: the e2e flakiness under the same contention, and #402. Worth finding out before adding more retries: which tests fail, whether it is always the same ones, and whether it is timing out rather than asserting. The job's own log is the starting point - the Forgejo API does not serve it, so this needs the web interface. Note a restart is not free: it hides how often this happens, and the number is what decides whether this is worth fixing at the runner or in the suite.

robbertbos referenced this issue from a pull request that will close it,

2026-09-23 21:15:48 +00:00

Make the frontend tests independent of runner load #425

robbertbos closed this issue

2026-09-23 23:21:43 +00:00