You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem — The 2026-08-30 hub upgrade crash-looped 29 times against the production database because a new server binary was validated by booting it AT production: the binary carried a pending migration whose ALTER collided with drifted-but-legal database state (duplicate column name: directories). Nothing in the upgrade path stands between "built" and "serving production" — a migration that dies on the real DB is discovered by crash-looping the hub.
Approach — Make the upgrade verb's contract: build → boot-smoke against a copy of the production DB (consistent sqlite .backup snapshot, isolated XDG data dir, expect serving + migrations reconciled) → stage → atomic swap → restart → verify. The smoke harness already exists as an ad-hoc script from the incident; promote it into the verb. The hub-side stage/swap/restart/verify primitives land separately via the hub-restart ops script and are adopted as-is.
Scope — in: DB-copy smoke step in the upgrade verb, isolated-data-dir harness, honest refusal when the smoke fails · out: migration content fixes (landed separately in the fork), the client-side restart command.
Assumptions — the upgrade verb is currently gated on the pending server-binary PR; the smoke reuses the snapshot technique proven in the incident (.backup API, ~1.4 s on a 2.1 GB DB).
Acceptance Criteria
Upgrading a binary whose migrations would die against the current DB fails BEFORE any swap, with the migration id and the error in the refusal.
A clean binary passes the smoke (serves on a scratch port against the copy; journal row count >= pre-copy count) and the upgrade proceeds.
The smoke never touches the production DB (read-only snapshot via the backup API; the copy lives in a temp dir with an isolated data-dir env).
The refusal message names the recovery: fix the migration, don't retry the swap.
Important
Problem — The 2026-08-30 hub upgrade crash-looped 29 times against the production database because a new server binary was validated by booting it AT production: the binary carried a pending migration whose
ALTERcollided with drifted-but-legal database state (duplicate column name: directories). Nothing in the upgrade path stands between "built" and "serving production" — a migration that dies on the real DB is discovered by crash-looping the hub.Approach — Make the upgrade verb's contract: build → boot-smoke against a copy of the production DB (consistent sqlite
.backupsnapshot, isolated XDG data dir, expect serving + migrations reconciled) → stage → atomic swap → restart → verify. The smoke harness already exists as an ad-hoc script from the incident; promote it into the verb. The hub-side stage/swap/restart/verify primitives land separately via the hub-restart ops script and are adopted as-is.Scope — in: DB-copy smoke step in the upgrade verb, isolated-data-dir harness, honest refusal when the smoke fails · out: migration content fixes (landed separately in the fork), the client-side restart command.
Assumptions — the upgrade verb is currently gated on the pending server-binary PR; the smoke reuses the snapshot technique proven in the incident (
.backupAPI, ~1.4 s on a 2.1 GB DB).Acceptance Criteria