From bc5e8840a9f33aa4ea337fd79bc6f327946ef78b Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 13:57:43 -0600 Subject: [PATCH 01/15] docs(plan): split Step 3b-2b into the REST sampler and verify; settle verify's edges (D36) Step 3b-2b becomes 3b-2b-1 (the REST sampler and its recorder mode) and 3b-2b-2 (verify, its timeline lookup, and the live acceptance). 5.4 states verify's lock check, repository set under --repo, report on each exit, unverified repositories, link-total differences, and cost fields (D36, gap 94). 5.8 passes the timeline lookup's since one second before the boundary, as measured (gap 95). 3b-2b-1 gains a paging rule and recorder-mode rules (gap 96). Section 7 records the timeline measurements and Step 3b-2a-2c's acceptance file; section 8 drops the items that file resolved. --- CLAUDE.md | 6 +- docs/redesign-plan.md | 183 +++++++++++++++++++++++++++++------------- 2 files changed, 130 insertions(+), 59 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1d99cc6..5b7602b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -62,10 +62,10 @@ uv run pytest # CI also runs Python 3.12, `alembic check`, - `ghactivity sync` (Step 3b-2a-2c, `cli/sync.py`): the engine's run with the options of plan 5.7; a run that never starts exits 1, and a run that starts exits with the engine's 0, 2, or 1 -**Not implemented yet:** `verify` (Step 3b-2b). The tracked `github_activity.db` stays -frozen at the first schema (D26). +**Not implemented yet:** the REST sampler (Step 3b-2b-1) and `verify` (Step 3b-2b-2). The +tracked `github_activity.db` stays frozen at the first schema (D26). -**Planned in the redesign plan:** verify (Step 3b-2b); GitHub Issues and +**Planned in the redesign plan:** the REST sampler (Step 3b-2b-1) and verify (Step 3b-2b-2); GitHub Issues and closing-issue links (Step 4); scheduled sync and release publication (Step 6). **Out of Scope (Future):** diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index effefdc..3f2618c 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -10,7 +10,9 @@ GitHub, GitHub's documentation, the installed libraries, and `main`, and correct Step 3 was planned against `main` at `c1f474b`, and the library behaviours it relies on were probed with the installed versions (section 7); Step 3a's review probes ran on the PR #11 branch (merged as `6fedb05`). On 2026-10-18 the plan and `main` at `6fedb05` were audited against D29, and the -collection D29 calls for was measured against live GitHub (section 7). Counts that drift (PR and +collection D29 calls for was measured against live GitHub (section 7). On 2026-10-21 Step 3b-2b +was planned against `main` at `2a85eb9`, and the timeline lookup's `since` and `filteredCount` were +measured against live GitHub (section 7). Counts that drift (PR and issue totals) are dated; regenerate them with `ghactivity verify` once Step 3b-2b lands. ## Contents @@ -31,7 +33,7 @@ issue totals) are dated; regenerate them with `ghactivity verify` once Step 3b-2 |------|-------|--------| | 1 | Guardrails | done (PR #5) | | 2 | GitHub source layer | done (PR #7) | -| 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, and 3b-2b) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | +| 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2b-1, and 3b-2b-2) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | | 4 | Issues, closing-issue links, and default-branch activity (pull requests 4a and 4b) | not started | | 5 | Full bootstrap and verification | not started | | 6 | Scheduled sync and publication | not started | @@ -86,6 +88,7 @@ Within scope the database collects and distinguishes data and never interprets i | D33 | `prebid-agent-skills` is never queried by an agent. Commands an agent runs (sync, verify, the REST sampler, the difference report, recordings, and `ghactivity github test`) name their repositories with `--repo` and leave it out; its rows enter the database through the scheduled App's workflow (Step 6) or a run the maintainer starts. Recordings come from public repositories only (Step 2). | | D34 | Decision of 2026-10-20 (the maintainer): wherever stored data must tell runs apart, a run is its `sync_runs` row id, stored with the data, never a clock reading and never state the writer or the engine keeps in memory, which neither a later run nor a reader of the file can see. `sync_failures.last_run_id` names the latest run in which the item failed, and `attempts` adds one only for a failure in another run, whatever the two runs' clocks read (5.1, gap 91). | | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | +| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and takes no lock: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit, and one that cannot be written makes the exit 1, because Step 6a reads it; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch makes a repository unverified, and verify's report records its own points, REST calls, and time (5.4, gap 94). | ## 3. Coverage rule @@ -196,7 +199,7 @@ Every gap found in the audit, and where it is resolved. | 71 | A batch alter of a table that a view reads fails ("error in view ...: no such table") | Step 3a: migrations drop and recreate the dependent views (5.1) | | 72 | Batch recreate drops unnamed CHECK constraints; SQLAlchemy emits no CHECK for `Boolean` or `Enum` on SQLite; `alembic check` compares neither | Step 3a: every CHECK named, through a naming convention; the 5.1 comparison test compares CHECK names | | 73 | An explicit `DATABASE_URL` naming the tracked file (the `.env` line `docs/development.md` shows) is migrated without complaint: the tag-row guard passes on its 0 tag rows | Step 3a: `alembic/env.py` refuses the repository-root `github_activity.db` (5.1, D26) | -| 74 | Nothing stops `sync` or `verify` from running against an older revision or an empty file (the new default URL creates one on first connect), and the wheel ships no `alembic/` directory (`pyproject.toml` packages only `src/github_activity_db`), so the installed code cannot read the revision chain | Step 3a: a packaged tuple of revision ids, kept equal to the chain by a test; Step 3b-2a-2: `sync` (the head check 3b-2a-2a's, called by the engine), and Step 3b-2b: online `verify`, exit 1 unless the database is at head (5.3, 5.4) | +| 74 | Nothing stops `sync` or `verify` from running against an older revision or an empty file (the new default URL creates one on first connect), and the wheel ships no `alembic/` directory (`pyproject.toml` packages only `src/github_activity_db`), so the installed code cannot read the revision chain | Step 3a: a packaged tuple of revision ids, kept equal to the chain by a test; Step 3b-2a-2: `sync` (the head check 3b-2a-2a's, called by the engine), and Step 3b-2b-2: online `verify`, exit 1 unless the database is at head (5.3, 5.4) | | 75 | typer prints local variable values in tracebacks by default and `cli/app.py` does not turn it off; every loguru handler `setup_logging` adds shows them too (both stderr handlers set `diagnose=True`, and the optional file handler takes loguru's default, also true). Step 6 runs the CLI in a public repository's Actions logs, which mask only registered secrets | Step 3a: typer's `pretty_exceptions_show_locals` and `diagnose` on all three handlers off (5.7) | | 76 | `source/` imports httpx directly (`client.py`, `transport.py`), but `pyproject.toml` does not declare it; it is installed only as a githubkit dependency | Resolved before Step 3a: `pyproject.toml` declares `httpx>=0.28`, the line `uv.lock` resolves (0.28.1) and the tests run on; githubkit allows `>=0.23.0,<1.0.0`. `tests/test_declared_dependencies.py` fails when an `import` or `from` statement in a `src/**/*.py` file (`TYPE_CHECKING` blocks and function bodies included) names a module that no declared dependency provides; imports by string (`importlib.import_module`, `__import__`) and `.pyi` stubs are not read. Installed metadata (`importlib.metadata.packages_distributions`) maps only top-level names, so an import is accepted only when every distribution installing its top-level name is declared: a namespace package that several distributions share is reported with each of them, because a declared sibling must not stand in for the provider. A provider whose metadata names no importable module (an editable install recorded only as a `.pth` file) is not seen | | 77 | In rollback-journal mode a read left half-consumed on another connection blocks a commit until the busy timeout, which then raises "database is locked"; the comment on `SQLITE_BUSY_TIMEOUT_MS` in `db/engine.py` says a lock fails the item. Through the aiosqlite dialect a query fetches every row and closes its cursor inside `execute`, so only a server-side cursor leaves a result half-read; a reader in another process (an open `sqlite3` shell) can still hold the lock | Step 3b-2a-1: no code in `src/` reads through a server-side cursor (`stream`, `stream_scalars`, `stream_results=True`, `yield_per`), the snapshot reader first; Step 3b-2a-2b-2: a database error in the writer is fatal (5.3 step 4); Step 3b-2a-2a: the comment is corrected | @@ -216,6 +219,9 @@ Every gap found in the audit, and where it is resolved. | 91 | `sync_failures.attempts` told runs apart by comparing the stored `last_failed_at` with the run's start time, so a failure in a run that started, by its clock, at or before the previous failure did not count, and no column named the run that failed | Step 3b-2a-2a: `sync_failures.last_run_id` (migration `2d2ba76b177c`; D34, 5.1) | | 92 | `plan()` raises `ValueError` when two configured entries resolve to one `github_id` (an old name and the new one both listed), and nothing catches it, so the run would end as an unexpected error | Step 3b-2a-2b-2: the later entry fails for the run, naming both, and makes the exit 2 (5.3 step 1) | | 93 | A cancel that lands inside a transaction on the writer's connection can leave the abandoned connection holding the write lock; the run's close then fails after the busy timeout and the row stays `running` (measured: a cancel during a slow `SELECT` 3 of 3, during a slow `INSERT` 0 of 3, among alternating short `SELECT`s and `INSERT`s 3 of 24; section 7, "Cancelled writer transaction") | Step 3b-2a-2b-2: accepted; the row is left as a crash leaves it, for the stale-row rule or `--break-lock` (5.3) | +| 94 | 5.4 left verify's edges unstated: its lock check (the claim's rule runs only inside the claim, which writes); whether `--repo` limits the untracked repositories it reports; whether `--report` is written when verify stops before comparing or when the path cannot be written; the class of a resolve or index failure other than the three it listed (not found, no default branch, a number listed twice or out of creation order, a response the typed models reject) and of a repository error during a lookup by number; the issue a link difference matches when only the totals differ; and where its cost is recorded, since it writes no `sync_runs` row | Step 3b-2b-2 (5.4, D36) | +| 95 | `timelineItems(since:)` keeps only items created after `since`, so a lookup passing the boundary would miss an event at it; GitHub's schema describes `filteredCount` as applying only `before` and `after` | Step 3b-2b-2: `since` one second before the boundary, and `filteredCount` used as measured (5.8; section 7, "Timeline `since`") | +| 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode and `--only documents` drop unit kinds they do not record, and its public-only check runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 (section 6) | ## 5. Design @@ -820,11 +826,19 @@ an interrupt, which is re-raised. `ghactivity verify [--repo owner/name]... [--exact] [--max-minutes N] [--report PATH]` (online) checks the database against a fresh index, as of the sync it follows: for each repository, "that run" is `repo_sync_state.last_run_id` and the boundary is its `last_index_started_at`. verify uses -the 5.3 step 4 failure classes and retries. It covers the configured repositories (after `--repo`) -and every stored untracked repository, which it reports as not synced without a request. Like +the 5.3 step 4 failure classes and retries. It covers the configured repositories and, without +`--repo`, every stored untracked repository: one that is no configured entry's stored row (the +`is_tracked` row whose `full_name` equals the configured name, ignoring case, 5.3 step 1) and that +no configured entry resolves to, which it reports as not synced without a request. `--repo` limits +verify to the repositories it names, each a configured repository, ignoring case, as it limits a +sync (D27). Like `sync`, it exits 1 before any request unless the database is at the code's migration head and `user_version` equals `schema_version` (gap 74), and it uses no server-side cursor (gap 77); it -also exits 1 while a sync run holds the lock (5.3). It re-indexes and applies the coverage rule, then +also exits 1, naming the run, while a sync run holds the lock: before any request it reads +`sync_runs` and stops when a row is `running` with a null `deadline_at` or one less than 10 minutes +in the past, the claim's rule (5.3). verify writes nothing to the database and takes no lock, so a +stale row stays for the next claim to reclaim, and a crashed run with a null `deadline_at` stops +verify until `sync --break-lock` closes it (D36). It re-indexes and applies the coverage rule, then compares by `github_id` the stored set with the result (missing and extra items) and, for items in both, the stored `github_updated_at`, `state`, `is_draft`, `author_association`, and closing links with the fresh values @@ -846,10 +860,12 @@ the first class that matches, in this order: connected or disconnected event matches a link difference when its `subject` is the differing issue (its repository and number); a link made by a closing keyword produces no such event, so its difference is drift only through `createdAt` or `updatedAt` (section 7, "Closing-link - timeline events"); + timeline events"). When the links differ only in their totals (a PR with more links than the + index carries, every indexed link stored), no issue is named, and any connected or disconnected + event at or after the boundary matches; - removed since sync: a stored item missing from the fresh index whose lookup by number returns `NOT_FOUND`, listed by number (the lookup is a hydration batch: `Gone` is removed since sync, a - hydrated PR an error); + hydrated PR an error, and a repository error in the batch makes the repository unverified); - unchecked: a differing item whose timeline lookup or lookup by number still fails after the retries, listed by number; - error: anything else. @@ -857,15 +873,28 @@ the first class that matches, in this order: Only an error fails verify. `verify --exact` also fails on not synced, pending, halted deletion, removed since sync, unchecked, and known failures not listed in `[sync].accepted_failures`; it is the mode the Step 3-5 acceptance uses, and drift is reported in both modes. A repository that verify cannot -re-index (a fetch that still fails after its retries, a second count mismatch, or a resolve to a -`github_id` other than its stored row's) is reported as -unverified: it fails `--exact` and is reported, without failing, otherwise. A verify that reaches +resolve, re-index, or look up is reported as unverified: every failure 5.3 steps 1 and 2 give a +repository (not found, no default branch, a resolve to a `github_id` other than its stored row's, a +fetch that still fails after its retries, a response the typed models reject, a number listed twice +or out of creation order, a second count mismatch), and a repository error during a lookup by number +(5.3 step 4). An unverified repository +fails `--exact` and is reported, without failing, otherwise. A verify that reaches `--max-minutes` stops with a timeout instead of waiting past it. Exit 0 when no failing class -occurred, 2 when one did, 1 on a fatal error or a timeout. `--report PATH` writes a versioned JSON -report on every exit, 1 included (marked incomplete): the exit code and whether `--exact` was given; +occurred, 2 when one did, 1 on a fatal error or a timeout, and 130 on an interrupt, the CLI's code +(5.3). A verify that never starts exits 1 before any request and writes no report: a usage error +(`--repo` naming no configured repository, a single-valued option given twice), an invalid +`tracking.toml`, missing credentials, the head check, a held lock, or a report path that cannot be +opened for writing, which verify opens last before its first request. Once it starts, +`--report PATH` writes a versioned JSON report on every exit, 1 and 130 included, marked incomplete +with the reason when verify stops before its comparisons finish; a report it cannot write at exit +makes the exit 1, because Step 6a reads it (5.6, D36). The report holds the exit code and whether +`--exact` was given; per class, the count of items and their numbers (an item whose differences take two classes counts -in each); per repository, its `last_run_id`, boundary, per-class counts, and per-state counts; and -the unverified repositories with their reasons. +in each); per repository, its `last_run_id`, boundary, per-class counts, and per-state counts; +the unverified repositories with their reasons; and verify's GraphQL points, REST calls, and +elapsed seconds, counted as a run's are (5.3 step 4), since verify writes no `sync_runs` row. +What verify prints follows sync's rule: output that cannot be written leaves the exit code +verify's (5.3 step 8). The enumerated count must equal the last page's unfiltered `totalCount` (rescanned once, as in 5.3); per-state counts are @@ -874,9 +903,10 @@ moves an item between states. Stored child rows must equal stored totals: `pr_fi `files_total`, `pr_commits` rows `commits_total`, `pr_reviews` rows `reviews_total`, `pr_labels` rows `labels_total`, `pr_closing_issues` rows `closing_issues_total` and those with `user_linked` `user_linked_closing_issues_total`, `pr_comments` rows `comments_total`, `pr_review_requests` rows -`review_requests_total`, `pr_assignees` rows `assignees_total`, and -`issue_labels` rows equal `issues.labels_total`, and `repo_labels` rows `repositories.labels_total`; -from Step 4b, `repository_activity_compare_commits` rows equal `total_commits` for each compare +`review_requests_total`, `pr_assignees` rows `assignees_total`, and `repo_labels` rows +`repositories.labels_total`; from Step 4a, `issue_labels` rows equal `issues.labels_total`, and the +issue comment and assignee rows their totals (5.1); from Step 4b, +`repository_activity_compare_commits` rows equal `total_commits` for each compare with a 2xx `http_status`, and `default_branch_commits` rows equal `repo_sync_state.default_branch_commits_total`. A child-count mismatch is an error. Each check is one SQL statement, so a verify that reads beside a sync's item commits compares one consistent @@ -1057,7 +1087,7 @@ second test fails when `source/` imports SQLAlchemy or `db/`, so fetchers cannot |-------------|---------| | Keep | `config.py` (without `RateLimitConfig`, `PacingConfig`, `SyncConfig`, and the `rate_limit`, `pacing`, and `sync` settings; `env_nested_delimiter="__"`, gap 64; `github_token` an optional `SecretStr`, which one CLI helper requires for the commands that call GitHub), `tracking.py` and `tracking.toml`, `logging.py` (loguru `diagnose` off, gap 75), `db/engine.py` (without `create_tables` and `drop_tables`; databases are built by `alembic upgrade head`, `tests/db/test_application_engine.py` builds its database that way and keeps both gap 46 checks, and `tests/test_db_engine.py` drops `test_create_tables`), `cli/app.py` (without the `search` and `tags` stubs, D17; tracebacks without local variables, gap 75; a run that never starts exits 1, gap 69), `cli/common.py` (gains `parse_repo_string`; `OutputFormat` goes with the removed subcommands, since no new command takes `--format`), the package and `cli/` `__init__.py` files, `py.typed` | | New in Step 2 | `source/`: the client factory, failure classification, budget per pool, query documents with their per-kind `hydration_version`, private typed response models, typed invariant errors, resolve, index, hydration with overflow and compare, user lookup, and the typed records that cross into `sync/` (a PR record with its children; per-item outcomes hydrated, gone, or failed) | -| New in Step 3 | `sync/`: plan, writer, engine, verify, run records. The writer's SQL lives in `sync/` (`INSERT ... ON CONFLICT ... DO UPDATE`, with `COALESCE` keeping a reference D28 keeps when GitHub returns null), so one module owns each item's transaction. `db/` gains the timestamp type, `SCHEMA_VERSION`, and the packaged tuple of migration revision ids (gap 74). `source/` gains the rate-limit reading for `ghactivity github rate-limit` (Step 3a), the fields and connections 5.8 adds in Step 3b-1, and verify's timeline lookup (Step 3b-2) | +| New in Step 3 | `sync/`: plan, writer, engine, verify, run records. The writer's SQL lives in `sync/` (`INSERT ... ON CONFLICT ... DO UPDATE`, with `COALESCE` keeping a reference D28 keeps when GitHub returns null), so one module owns each item's transaction. `db/` gains the timestamp type, `SCHEMA_VERSION`, and the packaged tuple of migration revision ids (gap 74). `source/` gains the rate-limit reading for `ghactivity github rate-limit` (Step 3a), the fields and connections 5.8 adds in Step 3b-1, and verify's timeline lookup (Step 3b-2b-2) | | New in Step 4b | `source/`: the repository activity reader (REST `GET /repos/{owner}/{repo}/activity?ref=refs/heads/`), the compare of each event, and the default-branch history reader (5.8); `sync/`: their planner and writer steps | | Replace in Step 3 | `db/models.py`, `db/__init__.py` (its re-exports follow the new models), and the shared test fixtures that import them (`tests/conftest.py`; Step 3a removed `tests/factories.py`, and `tests/db/seed.py` writes rows through the models' tables); `cli/sync.py` becomes one `ghactivity sync [--repo owner/name]... [--max-minutes N] [--dry-run] [--allow-deletions owner/name=N] [--prune-untracked] [--break-lock]` command (the `pr`, `repo`, `all`, and `retry` subcommands go), plus `ghactivity verify [--repo owner/name]... [--exact] [--max-minutes N] [--report PATH]` and, in Step 6a, `--db-only` and `ghactivity snapshot`; `cli/github.py` keeps `rate-limit` over the `source/` factory (GraphQL `rateLimit` and REST `/rate_limit`; `--all` stays and `--verbose` goes), and `test [--repo owner/name] --pr N` resolves one repository (by default the first configured) and hydrates that PR | | Remove in Step 3 | all of `github/` (`client.py`, `exceptions.py`, `pacing/*`, `rate_limit/*`, `sync/*`), all of `schemas/`, `db/repositories/*`, and their tests and the fixtures only they use | @@ -1214,14 +1244,20 @@ pool's limit and use): query RateLimit { rateLimit { cost remaining resetAt } } ``` -Verify's timeline lookup (Step 3b-2b) reads, for each differing PR, its ready-for-review, +Verify's timeline lookup (Step 3b-2b-2) reads, for each differing PR, its ready-for-review, convert-to-draft, connected, and disconnected events at or after the repository's boundary. It +passes `since` one second before the boundary: `since` keeps only items created after it, and the +event times and the boundary are whole seconds (section 7, "Timeline `since`"). It pages by `pageInfo.hasNextPage` and checks the entries it read against the first page's -`filteredCount`, which applies `itemTypes` and `since`; a later page's `filteredCount` counts only +`filteredCount`, which applies `itemTypes` and `since` (measured; GitHub's schema describes it as +applying only `before` and `after`); a later page's `filteredCount` counts only the items after its cursor, and `totalCount` is neither filtered nor the number of entries GitHub -returns (section 7, "Closing-link timeline events"). Its document, formula cost, and recordings +returns (section 7, "Closing-link timeline events"). A lookup whose entries differ from that count +fails like an item error (5.3 step 4). Its document, formula cost, and recordings are added here when it is written; the recorder adds them without changing the existing -recordings. +recordings. The document takes no `tests/query_versions.json` entry: verify stores nothing, so no +`hydration_version` depends on it, and once it changes the replay fails with "re-record" on its +recordings, whose stamp is the old query hash (Step 2). Default-branch history (Step 4b; 100 commits per page cost 1, dry run on Prebid.js, 2026-10-18): @@ -1250,7 +1286,8 @@ nested node. ## 6. Execution steps Each step is one pull request from a `feature/` branch, except Step 3 (pull requests 3a, 3b-1, -3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, and 3b-2b), Step 4 (pull requests 4a and 4b), +3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2b-1, and 3b-2b-2), Step 4 (pull requests +4a and 4b), and Step 6 (pull requests 6a and 6c, with the 6b manual runs between them). A step, or a pull request within one, starts after the previous one merges. @@ -1334,18 +1371,20 @@ repository) and runs on the exact PR head; CI passes on the exact PR head. ### Step 3: New schema, PR sync engine, cutover -Eight pull requests. 3a lands the schema and removes the old pipeline, which imports the old models +Nine pull requests. 3a lands the schema and removes the old pipeline, which imports the old models and so cannot outlive them; 3b-1 adds what the sync collects under D29 (the query documents, typed records, schema, and row-level views of D3-D6, D8, D14, and D28); 3b-2a-1 adds the planner and the configuration it reads; 3b-2a-2a adds the writer and the run records on that schema, so the writer is built against the final tables; 3b-2a-2b-1 adds the scenario model the engine's tests run against; 3b-2a-2b-2 adds the sync engine over the planner and the writer, `--dry-run` -included; 3b-2a-2c adds `ghactivity sync`, the seeded runs, and the live acceptance; and 3b-2b adds -verify and the REST sampler that checks what the sync wrote. "Step 3b-2" elsewhere in this plan -means 3b-2a-1, the parts of 3b-2a-2, and 3b-2b together; "Step 3b-2a" means 3b-2a-1 and 3b-2a-2; -"Step 3b-2a-2" means 3b-2a-2a, the two parts of 3b-2a-2b, and 3b-2a-2c; and "Step 3b-2a-2b" -means 3b-2a-2b-1 and 3b-2a-2b-2. Verify (5.4), its timeline lookup, and the REST sampler -are 3b-2b's; the planner, its snapshot reader, step 7's candidate, guard, and `--allow-deletions N` +included; 3b-2a-2c adds `ghactivity sync`, the seeded runs, and the live acceptance; 3b-2b-1 adds +the REST sampler that checks what the sync wrote; and 3b-2b-2 adds verify, whose live acceptance +runs the sampler. "Step 3b-2" elsewhere in this plan means 3b-2a-1, the parts of 3b-2a-2, and the +two parts of 3b-2b together; "Step 3b-2a" means 3b-2a-1 and 3b-2a-2; "Step 3b-2a-2" means +3b-2a-2a, the two parts of 3b-2a-2b, and 3b-2a-2c; "Step 3b-2a-2b" means 3b-2a-2b-1 and +3b-2a-2b-2; and "Step 3b-2b" means 3b-2b-1 and 3b-2b-2. The REST sampler and its recorder mode are +3b-2b-1's; verify (5.4), its timeline lookup, and the scenario model's timeline responses are +3b-2b-2's; the planner, its snapshot reader, step 7's candidate, guard, and `--allow-deletions N` functions, and the `tracking.toml` keys the planner reads (`[coverage]` and `[sync]`'s `unsettled_minutes`, `delete_guard_min_rows`, and `delete_guard_percent`) with their loader and the file's two comment changes are 3b-2a-1's; `[automation]` and `[sync].accepted_failures` with their @@ -1594,7 +1633,7 @@ arrives, and that a test changes between renderings: the changes 3b-2a-2b-2's ca repository and a `NOT_FOUND` item or account included. It renders only GraphQL documents it was checked against, so the replay's stamp check still applies. The engine's tests share the replay's pytest fixtures and a test clock they can advance; `docs/testing.md` describes the model. 3b-2a-2c -and 3b-2b extend it as their tests need (expected states over several runs, timeline responses). +and 3b-2b-2 extend it as their tests need (expected states over several runs, timeline responses). Tests: every recorded pull-request, index, user-lookup, overflow, resolve, and repository-label unit parses into the model and re-renders to the recording's data and errors exactly, compared as @@ -1714,12 +1753,12 @@ syncs, the pull request records that PR's `updatedAt` and `mergeStateStatus` fro 8). An adversarial review of the change, by reviewers who did not write it, finds no invariant check without a test that targets it; CI passes on the exact PR head. -#### Step 3b-2b: Verify +#### Step 3b-2b-1: REST sampler Prerequisite: Step 3b-2a-2 has merged. -Scope: `ghactivity verify` (5.4; `--db-only` is Step 6a) and its head check (gap 74). Verify's -timeline lookup in `source/` with its recordings (5.8). The REST sampler the acceptance uses, laid +Scope: tests, `scripts/`, and `docs/testing.md`; `Transport.rest_get` already takes any path and +query, so nothing under `src/` changes. The REST sampler 3b-2b-2's acceptance uses, laid out like the recorder: comparison logic in a module under `tests/source/`, tested offline on recorded or synthetic REST bodies with one planted mismatch per compared field, REST's `ghost` matching a null reference or a stored reference to an @@ -1731,38 +1770,70 @@ returns among them (`closed_at`, the head and base commits, the merge commit, th times and commits, comment times, review requests, assignees) and those REST does not return named, the stack fields among them; REST's `mergeable`, `rebaseable`, and `mergeable_state` are returned but not compared, since the stored merge state is as of `hydrated_at` (5.1); -REST bodies recorded as their own unit kind by an additive recorder mode; run -from `scripts/`, reaching GitHub through `GitHubSource.transport`; Steps 4 and 5 reuse it. +each REST list read 100 entries per page until a page holds fewer, since `rest_get` returns no +`Link` header (gap 96); REST bodies recorded as their own unit kind by an additive recorder mode, +which resolves each repository through the recorder's public-only source before its first REST +request (D33), and which the recorder's default mode and `--only documents` record or keep, so a +later re-recording does not drop them (gap 96); run from `scripts/`, reaching GitHub through +`GitHubSource.transport`; Steps 3b-2b-2, 4, and 5 reuse it. + +Tests: the offline tests the scope lists; a list read across pages, one of them ending on a full +page followed by an empty one; the recorder mode refusing a private repository before any REST +request; and a re-recording in the default mode and with `--only documents` keeping the REST units. + +Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule +of the sampler without a test that targets it; CI passes on the exact PR head. + +#### Step 3b-2b-2: Verify + +Prerequisite: Step 3b-2b-1 has merged. + +Scope: `ghactivity verify` (5.4; `--db-only` is Step 6a) and its head check (gap 74), with D36's +lock check, repository set, report, and unverified repositories. Verify's timeline lookup in +`source/` with its recordings (5.8). Tests through 3b-2a-2b-1's scenario model, extended to render the timeline lookup's responses from -each PR's draft and link history (the timeline recordings join the model's round-trip test), and +each PR's draft and link event history, which the model gains here (the timeline recordings join +the model's round-trip test), and fake GitHub: each verify class, each overlapping pair of classes taking the first in the 5.4 -order, a stale item, verify's repository set, its child-count checks (each by a planted row -deletion), an unverified repository, a `CLOSED` PR whose `closedAt` falls on each side of the +order, a stale item, verify's repository set with and without `--repo` (a stored untracked +repository, a stored row no configured entry names, and a renamed repository), its child-count +checks (each by a planted row deletion), an unverified repository for each failure 5.4 lists, a +`CLOSED` PR whose `closedAt` falls on each side of the floor, verify's head check refusing an older revision, an empty file, and a missing path before -any request and without creating a file, verify exiting 1 while a sync run holds the lock (5.4), a +any request and without creating a file, verify exiting 1 while a sync run holds the lock (5.4), +with a `running` row whose `deadline_at` is null, one less than 10 minutes past, and one 10 minutes +past, which does not stop it, and the database file unchanged by verify, a verify started before a sync's claim reading beside the sync's item commits without blocking them -(gap 77), a verify reaching `--max-minutes` exiting 1, and its report written on exit 1. The +(gap 77), a verify reaching `--max-minutes` exiting 1, and its report written on exits 0, 2, 1, and +130, marked incomplete where 5.4 says, with the source's points and REST calls; no report from a +verify that never starts, a report path that cannot be opened stopping verify before any request, +a report that cannot be written at exit making the exit 1, and a closed stdout leaving the exit +code verify's. The timeline lookup through the replay: a connected event naming the differing issue, one naming -another issue, a link difference made by a closing keyword with no event, a synthetic -disconnected event, a filtered page whose `totalCount` differs from its entries, and a lookup +another issue, a link difference made by a closing keyword with no event, a difference in the +link totals alone matched by an event naming any issue, a synthetic +disconnected event, an event at the boundary counted and one a second before it not, a filtered +page whose `totalCount` differs from its entries, a lookup whose entries differ from its +`filteredCount` leaving its item unchecked, and a lookup spanning two pages whose second page's `filteredCount` counts only the entries after its cursor -(5.8). The REST sampler's offline tests as its scope lists them. +(5.8). Acceptance: `sync --repo` and `verify --exact --repo` on Prebid.js, salesagent, and prebid-server (without `--repo`, verify would report the other configured repositories as not synced); a REST sample of PRs that includes a null author, a null merger, a closed-unmerged PR, more than 250 -commits, overflow, a duplicate path, and a dismissed review matches field by field under the -sampler's rules above, with REST's `ghost` account (id 10137) matching a null reference or a +commits, overflow, a duplicate path, and a dismissed review matches field by field under +3b-2b-1's sampler rules, with REST's `ghost` account (id 10137) matching a null reference or a reference to an account a lookup now returns `NOT_FOUND` for, per-file line counts compared only where REST's per-file sums equal the PR totals, and comment ids and authors compared as D24 stores them (section 7, "Source vs REST"). The live run starts from a fresh database at an explicit path, with `GITHUB_TOKEN="$(gh auth token)"` and the token never printed; reads `ghactivity github rate-limit` before and after, and does not start with fewer than 1,500 GraphQL points remaining; runs a second, incremental sync -before verify; and records the points, REST calls, and time it spent. The sample's cases are chosen -by SQL queries recorded in the pull request; if none of the three repositories holds an in-coverage -dismissed review, prebid-server-java joins the sync and `verify --exact`. A live item failure that +before verify; and records the points, REST calls, and time it spent, verify's from its report. The +sample's cases are chosen by SQL queries recorded in the pull request; all three repositories hold +in-coverage dismissed reviews (section 7, "Step 3b-2a-2c acceptance"). The pull request lists each +closing-link error verify reports, with its PR and link (section 8). A live item failure that cannot pass is listed in `[sync].accepted_failures` with REST evidence of why, as in Step 5. An adversarial review of the change, by reviewers who did not write it, finds no invariant check without a test that targets it; CI passes on the exact PR head. @@ -1955,6 +2026,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Dismissed verdicts | `prebid-server-java#4637`: Lightwood13 submitted CHANGES_REQUESTED (5276426920), then a review now DISMISSED (5289072517); `latestOpinionatedReviews` lists only the two APPROVED reviewers. A rule taking the latest APPROVED or CHANGES_REQUESTED review would report CHANGES_REQUESTED | live query | | Late closing links | `prebid-server-java#4380`, `#4350`, `#4397`: a `ConnectedEvent` on 2026-08-14 linked an issue months after merge, and the PR's `updatedAt` did not change. Adding `fullDatabaseId`, `authorAssociation`, and `closingIssuesReferences(first: 10)` with `nameWithOwner` keeps an index page at 1 point | timeline scan; live index page and `rateLimit(dryRun: true)` | | Closing-link timeline events | Over the 100 most recent PRs each of Prebid.js, prebid-server-java, and prebid-server (2026-10-19), the `ConnectedEvent` subjects of each of the 48 PRs with closing links equal its user-linked issues (`userLinkedOnly: true`); each event's `source` is the PR and its `subject` the issue. None of the 30 links made only by a closing keyword has an event, nor does `Prebid.js#12975`'s ("Closes #12914"), read separately; no `DisconnectedEvent` occurs. `filteredCount` equals the filtered entries on all 300, each a single page. On a later page `filteredCount` counts only the items after its cursor (`Prebid.js#15694`, `ISSUE_COMMENT`, two entries per page: 11 on the first page, 9 on the second). `totalCount` is the same with and without `itemTypes` and is not the number of entries returned (56 on `prebid-server-java#4380`, whose unfiltered timeline returns 66 entries and one connected event) | `timelineItems(itemTypes: [CONNECTED_EVENT, DISCONNECTED_EVENT])` beside both closing lists, and `ISSUE_COMMENT` pages, live | +| Timeline `since` | On eight events, five ready-for-review or convert-to-draft events (`Prebid.js#15581`, `#15806`, `#15807`, `#15821`, and `#15823`) and three connected events (`prebid-server#4805`, `#4840`, and `#5007`), `timelineItems(since:)` returned the event for a `since` 1 ms before its `createdAt` (on the five, also 0.5 s and 1 s before) and not for a `since` equal to it (on the five, nor 1 ms, 0.5 s, 0.999 s, or 1 s after), so `since` keeps the items created after it; every `createdAt` read was a whole second. `filteredCount` applies `since`: on `Prebid.js#15807` it was 5 without `since`, 3 with `since` one second before the third event, and 2 at it, each equal to the entries returned, although GitHub's schema describes it as the count "after applying `before` and `after` filters". Paged one entry at a time with `since` set, `filteredCount` read 3, 2, and 1. Each read cost 1 point | live GraphQL `timelineItems(itemTypes: [READY_FOR_REVIEW_EVENT, CONVERT_TO_DRAFT_EVENT, CONNECTED_EVENT, DISCONNECTED_EVENT], since:)` and schema introspection, 2026-10-21 | | Copilot review requests | In 3 of the last 60 Prebid.js PRs, and none of the last 60 of prebid-server or prebid-server-java (2026-10-19), each `ReviewRequestedEvent` for Copilot names the bot `copilot-pull-request-reviewer` (`BOT_kgDOCnlnWA`, `databaseId` 175728472), the account that writes its review; no other Copilot account appears. A pending request was not caught | `reviews`, `reviewRequests`, and `timelineItems(itemTypes: [REVIEW_REQUESTED_EVENT, REVIEW_REQUEST_REMOVED_EVENT])`, live | | Label renames | The prebid-mobile-ios label "4.0 In-App Platfrom" became "4.0 In-App Platform"; issue #1030, which carries it, kept its `updatedAt` | REST events; live query | | Author association | GraphQL and REST agree on 100 of 100 PRs; on Prebid.js, the earliest merged PR of each of 60 recent first-time authors reads CONTRIBUTOR (59) or MEMBER (1), not FIRST_TIME_CONTRIBUTOR, so the value is computed when read, not stored at creation. `Prebid.js#15715`, merged, whose commits link to no account, still reads FIRST_TIME_CONTRIBUTOR (GitHub: "Author has not previously committed to the repository.") | live queries; introspection | @@ -2028,6 +2100,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Repository activity | `GET /repos/{owner}/{repo}/activity?ref=refs/heads/` returns events with `id`, `node_id`, `activity_type`, `ref`, `before`, `after`, `timestamp`, and `actor`; githubkit 0.14.2's `Activity` model types `activity_type` as `push`, `force_push`, `branch_deletion`, `branch_creation`, `pr_merge`, or `merge_queue_merge`, and `actor` as nullable. `prebid/github-activity-db` `main`: 40 events since the branch was created on 2026-01-15 (11 `pr_merge`, PRs #1-#11; 27 `push`; 1 `force_push`; 1 `branch_creation`). The oldest event on the default branches of `prebid/Prebid.js` and `prebid/prebid-server` is from 2023-03-07; Prebid.js's newest 100 `push` events span 2025-04-03 to 2026-10-16, and prebid-server has none. `Commit.pushedDate` is deprecated ("`pushedDate` is no longer supported. Removal on 2023-07-01 UTC.") and returned null on the three newest Prebid.js `master` commits. Each of the 11 `pr_merge` events' `after` equals its PR's `mergeCommit.oid`: PRs #1-#6 merged with a merge commit (2 parents), #7-#11 squashed (1 parent); no rebase merge exists to measure | live REST and GraphQL, introspection; installed githubkit `versions/v2022_11_28/models/group_0267.py`, 2026-10-18 | | Merge state and stacks | Adding `mergeable`, `mergeStateStatus`, `canBeRebased`, `potentialMergeCommit`, `stack`, and `stackEntry` keeps the 25-PR batch at 2 points (dry run, with the auto-merge and merge-queue fields also added; formula). Of the 17 public tracked repositories, none allowed auto-merge (`autoMergeAllowed`) or had a merge queue on its default branch, and no open PR had an auto-merge request or a queue entry; none of the 28 repositories read after the additions (D13) allows auto-merge. A real read of the 3 most recently updated open Prebid.js PRs returned them without error (each `MERGEABLE`, `BLOCKED`, in no stack). GitHub's descriptions: `stack` is "The stack this Pull Request belongs to, or null if it is not part of a stack"; a stack entry's `position` is "This entry's position in the stack, where 1 is the closest to the base branch"; `potentialMergeCommit` "will not return a value if the pull request is merged, or if the test merge commit is still being generated". The 17 public tracked repositories held 440 open PRs (prebid-server 137, salesagent 86, prebid.github.io 77, Prebid.js 35, 36 at a later read), 25 batches when batched per repository as the PR batch requires; the 28 tracked repositories other than `prebid-agent-skills` held 463 in 30 per-repository batches after the additions. `prebid` has 11 project boards; GitHub refused their `title` without the `read:project` scope | dry run and live GraphQL, introspection, 2026-10-18 | | Merge state after a base merge | Three PRs were merged into Prebid.js `master` (#15678 at 00:58:52, #15814 at 01:13:09, and #15825 at 01:19:19 UTC on 2026-10-21) between Step 3b-2a-2c's second and third acceptance runs. Of the 41 PRs open on `master` after the third run, 38 were stored before the merges and 37 of those kept their `updatedAt`, so the third run (01:36 to 01:38 UTC) did not hydrate them; it hydrated four: #15581, stored before the merges with a new `updatedAt` (01:22:15), and #15823, #15824, and #15826, opened between 01:05 and 01:09. Read live after the third run, all 41 had the stored `updatedAt`, and 8 of them a different `mergeStateStatus` than stored (6 `UNKNOWN` to `BLOCKED`, 2 `BLOCKED` to `UNKNOWN`); 38 of the 41 stored values were `UNKNOWN`. GitHub changed the merge state without changing `updatedAt`, so the stored merge state is as of `hydrated_at` (5.1, D32); the reading cannot tell whether the merges or GitHub's own recomputation changed it | the acceptance file and live GraphQL, 2026-10-21 | +| Step 3b-2a-2c acceptance | The acceptance file of Prebid.js, salesagent, and prebid-server (three runs, 2026-10-20 23:45 to 2026-10-21 01:38 UTC): the first sync spent 1,687 GraphQL points, 563 REST calls, and 3,872 s, and the two incremental syncs 179 and 180 points, no REST call, and 115 and 131 s. No run recorded an item failure (`sync_failures` is empty), so no lookup met a node id that REST compare and GraphQL gave in different forms; 59 stored PRs have more than 250 commits, read through compare. None of the 2,580 review-request rows names Copilot; its 224 reviews are stored under `BOT_kgDOCnlnWA`. Every stored PR is in coverage, and `DISMISSED` reviews are stored on Prebid.js (25), prebid-server (622, on 383 PRs), and salesagent (3) | the runs' `sync_runs` rows and `sqlite3` on the file, 2026-10-21 | | Account profiles | The lookup of 100 ids with every D31 field, and with `organizations(first: 100)` added, costs 1 point by `rateLimit(dryRun: true)`. A real read of `patmmccann`'s 15 `User` profile fields and `prebid`'s 6 `Organization` fields returned with no error at 1 point. A real read of a `User`'s `email` failed: "The 'email' field requires one of the following scopes: ['user:email', 'read:user']" (the token here holds `gist`, `read:org`, `repo`, `workflow`); the dry run did not report it. GraphQL `organizations` listed 3 for `ChrisHuie` and `prebid` for `patmmccann`, while REST `/users/{username}/orgs` (public memberships) listed none for either. An organization's 8 public fields D31 names read on `prebid` and on `github` (not a member) at 1 point; on `github`, `webCommitSignoffRequired`, `membersCanForkPrivateRepositories`, `ipAllowListEnabledSetting`, `ipAllowListForInstalledAppsEnabledSetting`, and `notificationDeliveryRestrictionEnabledSetting` each returned FORBIDDEN ("does not have the right permission") and nulled the organization in its alias, `announcementBanner` and `samlIdentityProvider` (shown only to enterprise members and to owners) returned null, and `requiresTwoFactorAuthentication` and `organizationBillingEmail` need `admin:org`. On 20 organizations (`prebid`, `github`, and 18 others, 12 with a sponsors listing, `Homebrew` with 643 sponsors), `estimatedNextSponsorsPayoutInCents` and `monthlyEstimatedSponsorsIncomeInCents` were 0, `anyPinnableItems` true, `interactionAbility` null, and `pinnedItemsRemaining` from 0 to 6; `totalSponsorshipAmountAsSponsorInCents` was set only on `prebid` | live GraphQL and REST, 2026-10-18 | | Empty fields | The plan's 5.8 documents run on 934 PRs (up to 100 numbers drawn per public tracked repository, all of them where it has fewer; seed 20261018), 400 node ids from those PRs (the first 400 by id: 328 users, 11 bots, and 61 review-request ids; no organization, mannequin, or enterprise account), and all 17 public tracked repositories, 148 points; then, after the additions (D13), on 206 PRs of the 11 added repositories (the same draw) and on the 28 tracked repositories other than `prebid-agent-skills`, 68 points. These fields were empty or identical everywhere and are not collected (D32): a PR's `createdViaEmail`, `locked`, and `activeLockReason`; a review's `createdViaEmail`, `isMinimized`, and `minimizedReason`; a comment's `isMinimized`, `minimizedReason`, `isPinned`, `pinnedAt`, and `pinnedBy`; a repository's `isFork`, `parent`, `isMirror`, `mirrorUrl`, `isTemplate`, `templateRepository`, `isUserConfigurationRepository`, `isEmpty`, `isDisabled`, `isLocked`, `lockReason`, `hasSponsorshipsEnabled`, `autoMergeAllowed`, `hasIssuesEnabled`, `hasPullRequestsEnabled`, `isBlankIssuesEnabled`, `isInOrganization`, `issueCreationPolicy`, `pullRequestCreationPolicy`, `codeOfConduct`, `fundingLinks`, `contactLinks`, `interactionAbility` (`NO_LIMIT` from `REPOSITORY` with no expiry where GitHub returned it), and `planFeatures`' `draftPullRequests` and `maximumAssignees`; and an organization's `anyPinnableItems` and `interactionAbility` on the 20 organizations read ("Account profiles"). Collected because they vary: a comment's `createdViaEmail` (true on 12 comments of the first sample), and once the added repositories were read, a repository's `forkingAllowed` and `hasVulnerabilityAlertsEnabled` (each constant across the 17), `usesCustomOpenGraphImage`, and `planFeatures`' `codeowners`, `teamReviewRequests`, and `maximumManualReviewRequests`. Kept though empty or constant: a repository's `isArchived` and `archivedAt` (the maintainer's exception for archive and visibility status; `isPrivate` and `visibility` vary across the 28); `owner`; the merge-method settings, `squashMergeAllowed`, `mergeCommitTitle`, and `mergeCommitMessage` among them; and `stack` and `stackEntry`, which the maintainer wants | the 5.8 documents over live GraphQL, 2026-10-18 | | Added repositories | The 11 repositories D13 adds: `prebid-cache` (147 PRs; 119 default-branch commits, 27 since the floor), `user-sync` (3; 16, 16), `shared-id` (34; 90, 5), `Shared-id-v2` (7; 144, 26), `prebid-mobile-ios-adapters` (0; 11, 11), `prebid-js-build-generator` (26; 42, 6), `code-scanner` (0; 29, 29), `currency-file` (0; 3,002, 1,424), `currency-file-generator` (45; 97, 24), and the private `prebidjs-releaser` (21; 401, 291) and `automation-testing` (6; 73, 73); 289 PRs in all; each pushed to since 2025-04-18 | live GraphQL, 2026-10-18 | @@ -2050,9 +2123,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Compare fallback on a PR whose head fork was deleted (only a live fork tested) | A count mismatch fails the item; Step 5 reports any | | Full bootstrap time, cost, and size are projections from two repositories | Step 5 measures them | | Overflow continuations per request (25), the server retry delays (2, 4, 8 s), the reset margin (1 s), retry jitter (25%), the index boundary margin (60 s), the budget reserves (100 points, 100 calls), the longest accepted rate-limit wait (65 min), and the HTTP timeouts (read 30 s, connect 10 s) are chosen, not measured | Step 5 records secondary-limit responses, budget readings, and request times at these values | -| Whether Prebid.js, salesagent, or prebid-server holds an in-coverage dismissed review (the plan's example, `prebid-server-java#4637`, is in none of them) | Step 3b-2b's acceptance queries its database; without one, prebid-server-java joins the acceptance run | -| Whether REST compare and GraphQL give one account's node id in the same form (the recordings use the `MDQ6...` form in both); a lookup checks the returned id against the requested one | A mismatch fails the waiting item, which exit 2 makes visible; Step 3b-2a-2c's acceptance reports any | -| The cost of one commit per item in rollback-journal mode, and of a live Step 3b-2a-2c acceptance run (not measured) | Step 3b-2a-2c's acceptance records its points, REST calls, and time; Step 5 measures the full run | +| The cost of one commit per item in rollback-journal mode at full scale (measured on three repositories only: section 7, "Step 3b-2a-2c acceptance") | Step 5 measures the full run | | Whether the activity log's 2023-03-07 start is fixed or a rolling window (seen on two repositories) | Step 4b records each repository's first logged event; it reads the whole log every run, so an event that ages out leaves the database as GitHub stops listing it (D7) | | Whether `history(since:)` filters on the committed or the authored date (only its `totalCount` was read) | Step 4b's recordings settle it before the coverage rule's wording for commits is final | | Whether REST compare caps the `files` it returns for a large push (not measured) | Step 4b measures it on its largest recorded push and stores a count only where GitHub reports one | @@ -2060,9 +2131,9 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | How a rebase merge appears in the activity log: whether its `pr_merge` event's `after` is the PR's `mergeCommit.oid` and its other commits come through the compare (the measured repositories hold merge-commit and squash merges only) | Step 4b records one on a tracked repository that allows rebase merges, before D30's join wording is final | | Whether GitHub returns `EnterpriseUserAccount.id` to a token without the `read:enterprise` scope (the dry run validated the document; no tracked repository has such an actor yet) | Step 3b-1 tests the synthetic actor. A live occurrence decides between two outcomes (a prediction): an error at the alias's path is an item failure (exit 2), and an error without a path is fatal (5.3 step 4, exit 1), which Step 5's bootstrap would surface | | Whether the scheduled App token can read every resolve field the token used here read (`diskUsage`, `webCommitSignoffRequired`, `hasVulnerabilityAlertsEnabled`, and `planFeatures` included) | Step 6b's first manual run; a field it cannot read is an error 5.3 step 4 classifies | -| A `DisconnectedEvent`'s `source` and `subject` (none in the 300 PRs read for section 7, "Closing-link timeline events"; GitHub's schema gives it `ConnectedEvent`'s fields) | Step 3b-2b tests a synthetic one; a live one is recorded when seen | -| Whether a closing-keyword link can change without the PR's `updatedAt` changing (an issue transfer or deletion, a body edit, a base-ref or default-branch change); 5.4 then reports the difference as an error | Step 3b-2b's acceptance lists each such error with its PR and link; Step 5 reports any | -| Whether a pending `reviewRequests` entry for Copilot carries `BOT_kgDOCnlnWA` (the review-request events and the reviews do; no pending request was seen, section 7, "Copilot review requests") | Step 3b-2a-2c's acceptance reports any Copilot request row; for a `databaseId` reported under a second node id, the writer keeps the stored node id and the run reports the second (5.2) | +| A `DisconnectedEvent`'s `source` and `subject` (none in the 300 PRs read for section 7, "Closing-link timeline events"; GitHub's schema gives it `ConnectedEvent`'s fields) | Step 3b-2b-2 tests a synthetic one; a live one is recorded when seen | +| Whether a closing-keyword link can change without the PR's `updatedAt` changing (an issue transfer or deletion, a body edit, a base-ref or default-branch change); 5.4 then reports the difference as an error | Step 3b-2b-2's acceptance lists each such error with its PR and link; Step 5 reports any | +| Whether a pending `reviewRequests` entry for Copilot carries `BOT_kgDOCnlnWA` (the review-request events and the reviews do; no pending request was seen, section 7, "Copilot review requests" and "Step 3b-2a-2c acceptance") | Step 5 reports any Copilot request row; for a `databaseId` reported under a second node id, the writer keeps the stored node id and the run reports the second (5.2) | | Whether to collect a `Mannequin`'s `email` ("The mannequin's email on the source instance"; the maintainer's profile decision named users and organizations only, and no mannequin was available to read, so whether GitHub requires a scope for it is unmeasured) | The maintainer decides; a live read on a mannequin comes first | | Whether GitHub returns an `EnterpriseUserAccount`'s `enterprise` to the tokens used (an `Enterprise` object; no tracked repository has such an actor) | Not selected until a live occurrence is read | | Whether GitHub returns an `EnterpriseTeam` requested reviewer's `id`, `fullDatabaseId`, and `slug` to the tokens used (the batch with them dry-runs at 2 points; no tracked repository has such a request) | Step 3b-1 tests a synthetic one; a live occurrence decides, as for `EnterpriseUserAccount.id`: an error at the alias's path fails the item (exit 2) | From 580d18d3d58fc046734592f1d25b45db05905ead Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 14:24:52 -0600 Subject: [PATCH 02/15] docs(plan): tighten Step 3b-2b's rules after review 5.4: verify's untracked set follows 5.3 step 1's every-entry-resolved rule; the lock rule names a future deadline; stored rows are read per repository before its index; the unverified list names every resolve and index failure and gap 92; a verify with no repository verified is fatal; a totals-only link difference needs as many connected or disconnected events as the change; an interrupt keeps 130 when the report cannot be written; elapsed seconds are defined. 5.6: snapshot refuses a missing or stale verify report. 5.8: the lookup keeps entries at or after the boundary, and a count mismatch leaves its item unchecked. 3b-2b-1: the sampler's REST units live in their own directory, the sampler takes --repo, and the offline tests pair REST and GraphQL units from one recorder run. 3b-2b-2: matching tests, a timeline re-recording rule, a 2,500-point floor, and the exit the acceptance requires. Step 4a: issue child-count tests. Section 7: per-run failure counts, code versions, and the compare-linked accounts. --- CLAUDE.md | 4 +- docs/redesign-plan.md | 193 ++++++++++++++++++++++++++---------------- 2 files changed, 120 insertions(+), 77 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 5b7602b..117c8fe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -65,8 +65,8 @@ uv run pytest # CI also runs Python 3.12, `alembic check`, **Not implemented yet:** the REST sampler (Step 3b-2b-1) and `verify` (Step 3b-2b-2). The tracked `github_activity.db` stays frozen at the first schema (D26). -**Planned in the redesign plan:** the REST sampler (Step 3b-2b-1) and verify (Step 3b-2b-2); GitHub Issues and -closing-issue links (Step 4); scheduled sync and release publication (Step 6). +**Planned in the redesign plan:** the REST sampler (Step 3b-2b-1) and verify (Step 3b-2b-2); +GitHub Issues and closing-issue links (Step 4); scheduled sync and release publication (Step 6). **Out of Scope (Future):** - Webhooks / real-time sync diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 3f2618c..50c9f9f 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -88,7 +88,7 @@ Within scope the database collects and distinguishes data and never interprets i | D33 | `prebid-agent-skills` is never queried by an agent. Commands an agent runs (sync, verify, the REST sampler, the difference report, recordings, and `ghactivity github test`) name their repositories with `--repo` and leave it out; its rows enter the database through the scheduled App's workflow (Step 6) or a run the maintainer starts. Recordings come from public repositories only (Step 2). | | D34 | Decision of 2026-10-20 (the maintainer): wherever stored data must tell runs apart, a run is its `sync_runs` row id, stored with the data, never a clock reading and never state the writer or the engine keeps in memory, which neither a later run nor a reader of the file can see. `sync_failures.last_run_id` names the latest run in which the item failed, and `attempts` adds one only for a failure in another run, whatever the two runs' clocks read (5.1, gap 91). | | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | -| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and takes no lock: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit, and one that cannot be written makes the exit 1, because Step 6a reads it; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch makes a repository unverified, and verify's report records its own points, REST calls, and time (5.4, gap 94). | +| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and takes no lock: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit, and one that cannot be written makes the exit 1, because Step 6a reads it; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch makes a repository unverified, and a verify that verifies no repository is fatal. verify reads each repository's stored rows before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | ## 3. Coverage rule @@ -219,9 +219,9 @@ Every gap found in the audit, and where it is resolved. | 91 | `sync_failures.attempts` told runs apart by comparing the stored `last_failed_at` with the run's start time, so a failure in a run that started, by its clock, at or before the previous failure did not count, and no column named the run that failed | Step 3b-2a-2a: `sync_failures.last_run_id` (migration `2d2ba76b177c`; D34, 5.1) | | 92 | `plan()` raises `ValueError` when two configured entries resolve to one `github_id` (an old name and the new one both listed), and nothing catches it, so the run would end as an unexpected error | Step 3b-2a-2b-2: the later entry fails for the run, naming both, and makes the exit 2 (5.3 step 1) | | 93 | A cancel that lands inside a transaction on the writer's connection can leave the abandoned connection holding the write lock; the run's close then fails after the busy timeout and the row stays `running` (measured: a cancel during a slow `SELECT` 3 of 3, during a slow `INSERT` 0 of 3, among alternating short `SELECT`s and `INSERT`s 3 of 24; section 7, "Cancelled writer transaction") | Step 3b-2a-2b-2: accepted; the row is left as a crash leaves it, for the stale-row rule or `--break-lock` (5.3) | -| 94 | 5.4 left verify's edges unstated: its lock check (the claim's rule runs only inside the claim, which writes); whether `--repo` limits the untracked repositories it reports; whether `--report` is written when verify stops before comparing or when the path cannot be written; the class of a resolve or index failure other than the three it listed (not found, no default branch, a number listed twice or out of creation order, a response the typed models reject) and of a repository error during a lookup by number; the issue a link difference matches when only the totals differ; and where its cost is recorded, since it writes no `sync_runs` row | Step 3b-2b-2 (5.4, D36) | -| 95 | `timelineItems(since:)` keeps only items created after `since`, so a lookup passing the boundary would miss an event at it; GitHub's schema describes `filteredCount` as applying only `before` and `after` | Step 3b-2b-2: `since` one second before the boundary, and `filteredCount` used as measured (5.8; section 7, "Timeline `since`") | -| 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode and `--only documents` drop unit kinds they do not record, and its public-only check runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 (section 6) | +| 94 | 5.4 left verify's edges unstated: its lock check (the claim's rule runs only inside the claim, which writes); whether `--repo` limits the untracked repositories it reports; whether `--report` is written when verify stops before comparing or when the path cannot be written; the class of the resolve and index failures it did not list (not found, no default branch, a label list that fails its count, a number listed twice or out of creation order, a response the resolve or index rejects), beyond the three it did (a fetch that still fails, a second count mismatch, a resolve to another `github_id`), and of a repository error during a lookup; a verify that verifies no repository; when it reads the stored rows, which a sync that claims after its lock check can change; which events match a link difference GitHub reports only in the totals; and where its cost is recorded, since it writes no `sync_runs` row | Step 3b-2b-2 (5.4, D36) | +| 95 | `timelineItems(since:)` keeps only items created after `since`, so a lookup passing the boundary would miss an event at it; GitHub's schema describes `filteredCount` as the count after applying `before` and `after`, and names neither `itemTypes` nor `since` | Step 3b-2b-2: `since` one second before the boundary, and `filteredCount` used as measured (5.8; section 7, "Timeline `since`") | +| 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode and `--only documents` rebuild `recorded/` and drop the unit kinds they do not record; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | ## 5. Design @@ -827,18 +827,22 @@ an interrupt, which is re-raised. checks the database against a fresh index, as of the sync it follows: for each repository, "that run" is `repo_sync_state.last_run_id` and the boundary is its `last_index_started_at`. verify uses the 5.3 step 4 failure classes and retries. It covers the configured repositories and, without -`--repo`, every stored untracked repository: one that is no configured entry's stored row (the -`is_tracked` row whose `full_name` equals the configured name, ignoring case, 5.3 step 1) and that -no configured entry resolves to, which it reports as not synced without a request. `--repo` limits -verify to the repositories it names, each a configured repository, ignoring case, as it limits a -sync (D27). Like +`--repo`, every stored untracked repository, which it reports as not synced without a request: a +stored repository that no configured entry resolves to without failing 5.3 step 1 is untracked when +its `is_tracked` is false, and, when every configured entry resolved, also when it is true, as a +sync marks it (5.3 step 1). `--repo` limits verify to the repositories it names, each a configured +repository, ignoring case, as it limits a sync (D27). Like `sync`, it exits 1 before any request unless the database is at the code's migration head and `user_version` equals `schema_version` (gap 74), and it uses no server-side cursor (gap 77); it also exits 1, naming the run, while a sync run holds the lock: before any request it reads `sync_runs` and stops when a row is `running` with a null `deadline_at` or one less than 10 minutes -in the past, the claim's rule (5.3). verify writes nothing to the database and takes no lock, so a -stale row stays for the next claim to reclaim, and a crashed run with a null `deadline_at` stops -verify until `sync --break-lock` closes it (D36). It re-indexes and applies the coverage rule, then +in the past, a deadline not yet reached included, the claim's rule (5.3). verify writes nothing to +the database and takes no lock, so a stale row stays for the next claim to reclaim, and a crashed +run with a null `deadline_at` stops verify until a `sync --break-lock` run marks it `aborted` (D36). +A sync can still claim after that check (gap 77), so for each repository verify reads its +`repo_sync_state` row, then its stored items with their `sync_failures` and `sync_pending` rows, +before the first request of its index, and the stored side is never newer than the fresh index. +It re-indexes and applies the coverage rule, then compares by `github_id` the stored set with the result (missing and extra items) and, for items in both, the stored `github_updated_at`, `state`, `is_draft`, `author_association`, and closing links with the fresh values @@ -860,41 +864,50 @@ the first class that matches, in this order: connected or disconnected event matches a link difference when its `subject` is the differing issue (its repository and number); a link made by a closing keyword produces no such event, so its difference is drift only through `createdAt` or `updatedAt` (section 7, "Closing-link - timeline events"). When the links differ only in their totals (a PR with more links than the - index carries, every indexed link stored), no issue is named, and any connected or disconnected - event at or after the boundary matches; + timeline events"). The index names at most 10 links, so a PR with more can differ in its totals + alone; that difference is drift when the connected events at or after the boundary number at + least the total's increase, or the disconnected events at least its decrease. A PR's link + difference is drift only when each part of it is: each differing issue the index names by its + own event, and a difference in the totals by that count; - removed since sync: a stored item missing from the fresh index whose lookup by number returns `NOT_FOUND`, listed by number (the lookup is a hydration batch: `Gone` is removed since sync, a - hydrated PR an error, and a repository error in the batch makes the repository unverified); -- unchecked: a differing item whose timeline lookup or lookup by number still fails after the + hydrated PR an error; a repository error in the batch makes the repository unverified, which keeps + the classes its comparisons already gave, and the missing items it did not look up are unchecked); +- unchecked: a differing item whose timeline lookup (5.8) or lookup by number still fails after the retries, listed by number; - error: anything else. Only an error fails verify. `verify --exact` also fails on not synced, pending, halted deletion, removed since sync, unchecked, and known failures not listed in `[sync].accepted_failures`; it is the mode the Step 3-5 acceptance uses, and drift is reported in both modes. A repository that verify cannot -resolve, re-index, or look up is reported as unverified: every failure 5.3 steps 1 and 2 give a -repository (not found, no default branch, a resolve to a `github_id` other than its stored row's, a -fetch that still fails after its retries, a response the typed models reject, a number listed twice -or out of creation order, a second count mismatch), and a repository error during a lookup by number -(5.3 step 4). An unverified repository -fails `--exact` and is reported, without failing, otherwise. A verify that reaches +resolve, re-index, or look up is reported as unverified: every repository error the source raises at +resolve or index (not found, no default branch, a fetch that still fails after its retries, a +response or GitHub error the resolve or index rejects, a label list that fails its count or names a +label twice, paging without progress, a number listed twice or out of creation order, a second +count mismatch), step 1's own checks (a resolve to a `github_id` other than its stored row's, a later +entry resolving to an earlier entry's `github_id`, gap 92), and a repository error during a lookup by +number or a timeline lookup (5.3 step 4). An unverified repository fails `--exact` and is reported, +without failing, otherwise; a verify in which every repository it attempts is unverified, and the +deadline stopped none, is fatal (exit 1), as a sync is when no configured repository resolves (5.3 +step 1). A verify that reaches `--max-minutes` stops with a timeout instead of waiting past it. Exit 0 when no failing class occurred, 2 when one did, 1 on a fatal error or a timeout, and 130 on an interrupt, the CLI's code -(5.3). A verify that never starts exits 1 before any request and writes no report: a usage error -(`--repo` naming no configured repository, a single-valued option given twice), an invalid -`tracking.toml`, missing credentials, the head check, a held lock, or a report path that cannot be -opened for writing, which verify opens last before its first request. Once it starts, -`--report PATH` writes a versioned JSON report on every exit, 1 and 130 included, marked incomplete -with the reason when verify stops before its comparisons finish; a report it cannot write at exit -makes the exit 1, because Step 6a reads it (5.6, D36). The report holds the exit code and whether -`--exact` was given; +(5.3). A verify that never starts exits 1 before any request, writes no report, and leaves PATH as +it was: a usage error (`--repo` naming no configured repository, a single-valued option given +twice), an invalid `tracking.toml`, missing credentials, the head check, a held lock, a database +error before its first request (the busy timeout included), or a report path that cannot be opened +for writing, which verify opens last before its first request. Once it starts, `--report PATH` +writes a versioned JSON report on every exit its own code handles, 1 and 130 included (not SIGKILL, +SIGTERM, or a second interrupt, as for sync, 5.3), marked incomplete with the reason when verify +stops before its comparisons finish. A report it cannot write at exit turns an exit 0 or 2 into 1, +because Step 6a reads it (5.6, D36); an interrupt exits 130 either way. The report holds the exit +code and whether `--exact` was given; per class, the count of items and their numbers (an item whose differences take two classes counts in each); per repository, its `last_run_id`, boundary, per-class counts, and per-state counts; -the unverified repositories with their reasons; and verify's GraphQL points, REST calls, and -elapsed seconds, counted as a run's are (5.3 step 4), since verify writes no `sync_runs` row. -What verify prints follows sync's rule: output that cannot be written leaves the exit code -verify's (5.3 step 8). +the unverified repositories with their reasons; and verify's GraphQL points and REST calls, counted +by its source as a run's are (5.3 step 4), and the seconds from its start to the report's write, +since verify writes no `sync_runs` row. A report verify prints that stdout cannot take leaves the +exit code verify's, and stderr follows 5.3 step 8, 120 included. The enumerated count must equal the last page's unfiltered `totalCount` (rescanned once, as in 5.3); per-state counts are @@ -1004,7 +1017,7 @@ Workflow `.github/workflows/sync.yml`: and `ghactivity verify --max-minutes 40 --report verify.json`, runs `ghactivity snapshot --verify-report verify.json`, and hands the snapshot and manifest to `publish` as a workflow artifact, with the sync and verify exit codes as job outputs. The job fails when - `verify --db-only` halts or sync exits 1. + `verify --db-only` halts, sync exits 1, or `snapshot` refuses the verify report. - `publish` (`needs: sync`; `permissions: contents: write`; no App secret, no project code, no PyPI dependencies) runs when D25's condition holds. It accepts only artifact files whose names match `^github_activity-[0-9]{8}T[0-9]{6}Z\.(db\.gz|manifest\.json)$` and reads only the @@ -1038,7 +1051,10 @@ Workflow `.github/workflows/sync.yml`: (`sync_runs.config_sha256`), migration head, `schema_version`, coverage floor, `bootstrap`, per-repository PR and issue counts, per kind the number of rows whose `hydration_version` differs from the code's, the sync exit code, verify's - exit code and counts per class (from `--verify-report`; null without it), and the SHA-256. + exit code, counts per class, and unverified repositories (from `--verify-report`; null without + it), and the SHA-256. `snapshot` exits 1 when the report `--verify-report` names is missing or + unreadable, or names a `last_run_id` other than the file's `repo_sync_state`, so a verify that + never started, which leaves no report or an earlier one (5.4), fails the job. - Uploads never use `--clobber`. Before each upload, `publish` deletes a same-name asset in any state other than `uploaded`. A same-name `uploaded` asset whose `digest` is `sha256:` followed by the expected SHA-256 (or, when `digest` is null, whose downloaded SHA-256 matches) is skipped, so a @@ -1246,18 +1262,21 @@ query RateLimit { rateLimit { cost remaining resetAt } } Verify's timeline lookup (Step 3b-2b-2) reads, for each differing PR, its ready-for-review, convert-to-draft, connected, and disconnected events at or after the repository's boundary. It -passes `since` one second before the boundary: `since` keeps only items created after it, and the -event times and the boundary are whole seconds (section 7, "Timeline `since`"). It +passes `since` one second before the boundary and counts an entry only when its `createdAt` is at or +after the boundary: `since` keeps only items created after it, the event times are whole seconds +(section 7, "Timeline `since`"), and so is the boundary (5.3 step 2). It pages by `pageInfo.hasNextPage` and checks the entries it read against the first page's `filteredCount`, which applies `itemTypes` and `since` (measured; GitHub's schema describes it as -applying only `before` and `after`); a later page's `filteredCount` counts only +the count after applying `before` and `after`, and names neither); a later page's `filteredCount` +counts only the items after its cursor, and `totalCount` is neither filtered nor the number of entries GitHub returns (section 7, "Closing-link timeline events"). A lookup whose entries differ from that count -fails like an item error (5.3 step 4). Its document, formula cost, and recordings -are added here when it is written; the recorder adds them without changing the existing -recordings. The document takes no `tests/query_versions.json` entry: verify stores nothing, so no -`hydration_version` depends on it, and once it changes the replay fails with "re-record" on its -recordings, whose stamp is the old query hash (Step 2). +is retried once alone, as an item error is (5.3 step 4), and then leaves its item unchecked (5.4). +Its document, formula cost, and recordings are added here when it is written; the recorder adds +them without changing the existing recordings, and its default mode and `--only documents` record +them too, so a re-recording keeps them (gap 96). The document takes no `tests/query_versions.json` +entry: verify stores nothing, so no `hydration_version` depends on it, and once it changes the +replay fails with "re-record" on its recordings, whose stamp is the old query hash (Step 2). Default-branch history (Step 4b; 100 commits per page cost 1, dry run on Prebid.js, 2026-10-18): @@ -1771,15 +1790,22 @@ times and commits, comment times, review requests, assignees) and those REST doe the stack fields among them; REST's `mergeable`, `rebaseable`, and `mergeable_state` are returned but not compared, since the stored merge state is as of `hydrated_at` (5.1); each REST list read 100 entries per page until a page holds fewer, since `rest_get` returns no -`Link` header (gap 96); REST bodies recorded as their own unit kind by an additive recorder mode, -which resolves each repository through the recorder's public-only source before its first REST -request (D33), and which the recorder's default mode and `--only documents` record or keep, so a -later re-recording does not drop them (gap 96); run from `scripts/`, reaching GitHub through +`Link` header (gap 96); the offline tests compare REST bodies with rows written from GraphQL units +recorded in the same recorder run as the REST units of the same PRs, or with rows seeded from a +synthetic body; REST bodies recorded as their own unit kind by an additive recorder mode, each +stamped as the other kinds are (Step 2), into their own directory beside `recorded/`, as the +first-sync recordings are, so the default mode and `--only documents`, which rebuild `recorded/`, +leave them as they are (gap 96); the mode refuses a repository the recorder's `PRIVATE` list names +before any request and resolves each other repository through the recorder's public-only source +before its first REST request (D33); the sampler takes its repositories with `--repo` and refuses +`prebid-agent-skills` before any request (D33); run from `scripts/`, reaching GitHub through `GitHubSource.transport`; Steps 3b-2b-2, 4, and 5 reuse it. Tests: the offline tests the scope lists; a list read across pages, one of them ending on a full -page followed by an empty one; the recorder mode refusing a private repository before any REST -request; and a re-recording in the default mode and with `--only documents` keeping the REST units. +page followed by an empty one; the recorder mode sending no request for a repository `PRIVATE` +lists and refusing a private repository before any REST request; a re-recording in the default mode +and with `--only documents` leaving the REST units as they are; and the sampler reading only the +repositories `--repo` names and refusing `prebid-agent-skills` before any request. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule of the sampler without a test that targets it; CI passes on the exact PR head. @@ -1790,34 +1816,44 @@ Prerequisite: Step 3b-2b-1 has merged. Scope: `ghactivity verify` (5.4; `--db-only` is Step 6a) and its head check (gap 74), with D36's lock check, repository set, report, and unverified repositories. Verify's timeline lookup in -`source/` with its recordings (5.8). +`source/` with its recordings, which the recorder's default mode and `--only documents` also record +(5.8, gap 96). Any change 3b-2b-1's sampler needs on live REST bodies. Tests through 3b-2a-2b-1's scenario model, extended to render the timeline lookup's responses from -each PR's draft and link event history, which the model gains here (the timeline recordings join -the model's round-trip test), and +each PR's draft and link event history, which the model gains here, rendering `since` and +`filteredCount` as section 7 measured them (entries created after `since`; each page's +`filteredCount` counting the matching entries after its cursor), with the replay matching a timeline +unit by its `since` (the timeline recordings join the model's round-trip test), and fake GitHub: each verify class, each overlapping pair of classes taking the first in the 5.4 order, a stale item, verify's repository set with and without `--repo` (a stored untracked -repository, a stored row no configured entry names, and a renamed repository), its child-count -checks (each by a planted row deletion), an unverified repository for each failure 5.4 lists, a +repository, a stored row no configured entry names, a renamed repository, and a renamed repository +whose configured entry fails to resolve, reported unverified and not untracked), its child-count +checks (each by a planted row deletion), an unverified repository for each failure 5.4 lists, one +made unverified by a lookup by number keeping the classes it already had, a verify with every +repository unverified exiting 1, a `CLOSED` PR whose `closedAt` falls on each side of the floor, verify's head check refusing an older revision, an empty file, and a missing path before -any request and without creating a file, verify exiting 1 while a sync run holds the lock (5.4), -with a `running` row whose `deadline_at` is null, one less than 10 minutes past, and one 10 minutes -past, which does not stop it, and the database file unchanged by verify, a +any request and without creating a file, verify exiting 1 and naming the run while a sync run +holds the lock (5.4), with a `running` row whose `deadline_at` is null, one in the future, one less +than 10 minutes past, and one 10 minutes past, which does not stop it, and the database file +unchanged by verify, a verify started before a sync's claim reading beside the sync's item commits without blocking them -(gap 77), a verify reaching `--max-minutes` exiting 1, and its report written on exits 0, 2, 1, and -130, marked incomplete where 5.4 says, with the source's points and REST calls; no report from a -verify that never starts, a report path that cannot be opened stopping verify before any request, -a report that cannot be written at exit making the exit 1, and a closed stdout leaving the exit -code verify's. The +(gap 77), with the classes it reports when that sync writes a repository's items while verify +indexes it, a verify reaching `--max-minutes` exiting 1, and its report written on exits 0, 2, 1, +and 130 (the 130 from a real SIGINT to a child process, as for sync), marked incomplete where 5.4 +says, with the source's points and REST calls and the clock's seconds; each cause 5.4 lists for a +verify that never starts exiting 1 before any request, with no report and PATH unchanged; a report +that cannot be written at exit turning 0 or 2 into 1 and leaving an interrupt's 130; and a closed +stdout leaving the exit code verify's. The timeline lookup through the replay: a connected event naming the differing issue, one naming another issue, a link difference made by a closing keyword with no event, a difference in the -link totals alone matched by an event naming any issue, a synthetic +link totals alone with enough connected events, with only a disconnected event, and with too few, a +synthetic disconnected event, an event at the boundary counted and one a second before it not, a filtered page whose `totalCount` differs from its entries, a lookup whose entries differ from its `filteredCount` leaving its item unchecked, and a lookup spanning two pages whose second page's `filteredCount` counts only the entries after its cursor -(5.8). +(5.8). A re-recording in the default mode and with `--only documents` keeping the timeline units. Acceptance: `sync --repo` and `verify --exact --repo` on Prebid.js, salesagent, and prebid-server (without `--repo`, verify would report the other configured repositories as not synced); a REST @@ -1829,11 +1865,15 @@ per-file line counts compared only where REST's per-file sums equal the PR total and authors compared as D24 stores them (section 7, "Source vs REST"). The live run starts from a fresh database at an explicit path, with `GITHUB_TOKEN="$(gh auth token)"` and the token never printed; reads `ghactivity github rate-limit` before and after, and does not start with fewer than -1,500 GraphQL points remaining; runs a second, incremental sync -before verify; and records the points, REST calls, and time it spent, verify's from its report. The -sample's cases are chosen by SQL queries recorded in the pull request; all three repositories hold -in-coverage dismissed reviews (section 7, "Step 3b-2a-2c acceptance"). The pull request lists each -closing-link error verify reports, with its PR and link (section 8). A live item failure that +2,500 GraphQL points remaining (the first sync of these repositories spent 1,687, section 7, +"Step 3b-2a-2c acceptance"), recording any rate-limit wait apart from the run's time; runs a second, +incremental sync before verify; and records the points, REST calls, and time it spent, verify's +from its report. The sample's cases are chosen by SQL queries recorded in the pull request; all +three repositories hold in-coverage dismissed reviews (section 7, "Step 3b-2a-2c acceptance"). The +pull request lists each +closing-link error verify reports, with its PR and link (section 8). `verify --exact` exits 0, or +exits 2 with no failing difference but such closing-link errors, each with REST evidence that +GitHub changed the link without changing the PR's `updatedAt`. A live item failure that cannot pass is listed in `[sync].accepted_failures` with REST evidence of why, as in Step 5. An adversarial review of the change, by reviewers who did not write it, finds no invariant check without a test that targets it; CI passes on the exact PR head. @@ -1858,7 +1898,9 @@ included; the data dictionary for issues. Tests: issue first sync, update, close, and reopen; unsettled re-hydration and version bump for issues; an issue's stored author kept when GitHub returns it as null (D28); issue comments and assignees written and replaced; a transfer arriving with a new id; an old number returning -`NOT_FOUND`, treated as gone; a snapshot one migration behind head upgrades and syncs. +`NOT_FOUND`, treated as gone; a snapshot one migration behind head upgrades and syncs; verify's +child-count checks for `issue_labels`, `issue_comments`, and `issue_assignees`, each by a planted +row deletion. Acceptance: `verify --exact` for issues on the same repositories; a REST sample of issues matches field by field. @@ -1941,7 +1983,8 @@ Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snap `ghactivity verify --db-only` (5.4), and adds the `githubkit[auth-app]` extra to `pyproject.toml` and `uv.lock`. Its tests: `verify --db-only` accepting the gz and manifest `ghactivity snapshot` wrote, and halting with exit 1 on a tampered -copy; through +copy; `snapshot --verify-report` exiting 1 on a missing report and on one whose `last_run_id` +differs from the file's (5.6); through `httpx.MockTransport`, a mocked installation-token response whose `expires_at` lies just over a minute ahead, a clock moved past it mid-run, and a second token request before the next API request (githubkit keeps a token until one minute before `expires_at`); and a 401 on an API request still @@ -2100,7 +2143,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Repository activity | `GET /repos/{owner}/{repo}/activity?ref=refs/heads/` returns events with `id`, `node_id`, `activity_type`, `ref`, `before`, `after`, `timestamp`, and `actor`; githubkit 0.14.2's `Activity` model types `activity_type` as `push`, `force_push`, `branch_deletion`, `branch_creation`, `pr_merge`, or `merge_queue_merge`, and `actor` as nullable. `prebid/github-activity-db` `main`: 40 events since the branch was created on 2026-01-15 (11 `pr_merge`, PRs #1-#11; 27 `push`; 1 `force_push`; 1 `branch_creation`). The oldest event on the default branches of `prebid/Prebid.js` and `prebid/prebid-server` is from 2023-03-07; Prebid.js's newest 100 `push` events span 2025-04-03 to 2026-10-16, and prebid-server has none. `Commit.pushedDate` is deprecated ("`pushedDate` is no longer supported. Removal on 2023-07-01 UTC.") and returned null on the three newest Prebid.js `master` commits. Each of the 11 `pr_merge` events' `after` equals its PR's `mergeCommit.oid`: PRs #1-#6 merged with a merge commit (2 parents), #7-#11 squashed (1 parent); no rebase merge exists to measure | live REST and GraphQL, introspection; installed githubkit `versions/v2022_11_28/models/group_0267.py`, 2026-10-18 | | Merge state and stacks | Adding `mergeable`, `mergeStateStatus`, `canBeRebased`, `potentialMergeCommit`, `stack`, and `stackEntry` keeps the 25-PR batch at 2 points (dry run, with the auto-merge and merge-queue fields also added; formula). Of the 17 public tracked repositories, none allowed auto-merge (`autoMergeAllowed`) or had a merge queue on its default branch, and no open PR had an auto-merge request or a queue entry; none of the 28 repositories read after the additions (D13) allows auto-merge. A real read of the 3 most recently updated open Prebid.js PRs returned them without error (each `MERGEABLE`, `BLOCKED`, in no stack). GitHub's descriptions: `stack` is "The stack this Pull Request belongs to, or null if it is not part of a stack"; a stack entry's `position` is "This entry's position in the stack, where 1 is the closest to the base branch"; `potentialMergeCommit` "will not return a value if the pull request is merged, or if the test merge commit is still being generated". The 17 public tracked repositories held 440 open PRs (prebid-server 137, salesagent 86, prebid.github.io 77, Prebid.js 35, 36 at a later read), 25 batches when batched per repository as the PR batch requires; the 28 tracked repositories other than `prebid-agent-skills` held 463 in 30 per-repository batches after the additions. `prebid` has 11 project boards; GitHub refused their `title` without the `read:project` scope | dry run and live GraphQL, introspection, 2026-10-18 | | Merge state after a base merge | Three PRs were merged into Prebid.js `master` (#15678 at 00:58:52, #15814 at 01:13:09, and #15825 at 01:19:19 UTC on 2026-10-21) between Step 3b-2a-2c's second and third acceptance runs. Of the 41 PRs open on `master` after the third run, 38 were stored before the merges and 37 of those kept their `updatedAt`, so the third run (01:36 to 01:38 UTC) did not hydrate them; it hydrated four: #15581, stored before the merges with a new `updatedAt` (01:22:15), and #15823, #15824, and #15826, opened between 01:05 and 01:09. Read live after the third run, all 41 had the stored `updatedAt`, and 8 of them a different `mergeStateStatus` than stored (6 `UNKNOWN` to `BLOCKED`, 2 `BLOCKED` to `UNKNOWN`); 38 of the 41 stored values were `UNKNOWN`. GitHub changed the merge state without changing `updatedAt`, so the stored merge state is as of `hydrated_at` (5.1, D32); the reading cannot tell whether the merges or GitHub's own recomputation changed it | the acceptance file and live GraphQL, 2026-10-21 | -| Step 3b-2a-2c acceptance | The acceptance file of Prebid.js, salesagent, and prebid-server (three runs, 2026-10-20 23:45 to 2026-10-21 01:38 UTC): the first sync spent 1,687 GraphQL points, 563 REST calls, and 3,872 s, and the two incremental syncs 179 and 180 points, no REST call, and 115 and 131 s. No run recorded an item failure (`sync_failures` is empty), so no lookup met a node id that REST compare and GraphQL gave in different forms; 59 stored PRs have more than 250 commits, read through compare. None of the 2,580 review-request rows names Copilot; its 224 reviews are stored under `BOT_kgDOCnlnWA`. Every stored PR is in coverage, and `DISMISSED` reviews are stored on Prebid.js (25), prebid-server (622, on 383 PRs), and salesagent (3) | the runs' `sync_runs` rows and `sqlite3` on the file, 2026-10-21 | +| Step 3b-2a-2c acceptance | The acceptance file of Prebid.js, salesagent, and prebid-server (three runs, 2026-10-20 23:45 to 2026-10-21 01:38 UTC): the first sync spent 1,687 GraphQL points, 563 REST calls, and 3,872 s, and the two incremental syncs 179 and 180 points, no REST call, and 115 and 131 s. The runs ran `9270c64`, `c1e70d4` with uncommitted changes, and `d08d98d`. No run counted an item failure (`sync_runs.failures` is 0 on each, and `sync_failures` is empty). 59 stored PRs have more than 250 commits, read through compare, and 247 accounts only their commits link (225 `MDQ6...` and 20 `U_kgDO...` users, 2 `MDEy...` organizations) were stored by lookups that answered with the requested node id. The one compare link a lookup answered `NOT_FOUND` (`github_id` 1708813, 21 commit rows on 7 Prebid.js PRs) has no account in GraphQL either (its commit's `author.user` is null), and is stored as 5.2 stores such a link, which no run reports, so exit 2 would not show a node id form GraphQL answers `NOT_FOUND`. None of the 2,580 review-request rows names Copilot; its 224 reviews are stored under `BOT_kgDOCnlnWA`. Every stored PR is in coverage, and `DISMISSED` reviews are stored on Prebid.js (25), prebid-server (622, on 383 PRs), and salesagent (3) | the runs' `sync_runs` rows and `sqlite3` on the file, 2026-10-21 | | Account profiles | The lookup of 100 ids with every D31 field, and with `organizations(first: 100)` added, costs 1 point by `rateLimit(dryRun: true)`. A real read of `patmmccann`'s 15 `User` profile fields and `prebid`'s 6 `Organization` fields returned with no error at 1 point. A real read of a `User`'s `email` failed: "The 'email' field requires one of the following scopes: ['user:email', 'read:user']" (the token here holds `gist`, `read:org`, `repo`, `workflow`); the dry run did not report it. GraphQL `organizations` listed 3 for `ChrisHuie` and `prebid` for `patmmccann`, while REST `/users/{username}/orgs` (public memberships) listed none for either. An organization's 8 public fields D31 names read on `prebid` and on `github` (not a member) at 1 point; on `github`, `webCommitSignoffRequired`, `membersCanForkPrivateRepositories`, `ipAllowListEnabledSetting`, `ipAllowListForInstalledAppsEnabledSetting`, and `notificationDeliveryRestrictionEnabledSetting` each returned FORBIDDEN ("does not have the right permission") and nulled the organization in its alias, `announcementBanner` and `samlIdentityProvider` (shown only to enterprise members and to owners) returned null, and `requiresTwoFactorAuthentication` and `organizationBillingEmail` need `admin:org`. On 20 organizations (`prebid`, `github`, and 18 others, 12 with a sponsors listing, `Homebrew` with 643 sponsors), `estimatedNextSponsorsPayoutInCents` and `monthlyEstimatedSponsorsIncomeInCents` were 0, `anyPinnableItems` true, `interactionAbility` null, and `pinnedItemsRemaining` from 0 to 6; `totalSponsorshipAmountAsSponsorInCents` was set only on `prebid` | live GraphQL and REST, 2026-10-18 | | Empty fields | The plan's 5.8 documents run on 934 PRs (up to 100 numbers drawn per public tracked repository, all of them where it has fewer; seed 20261018), 400 node ids from those PRs (the first 400 by id: 328 users, 11 bots, and 61 review-request ids; no organization, mannequin, or enterprise account), and all 17 public tracked repositories, 148 points; then, after the additions (D13), on 206 PRs of the 11 added repositories (the same draw) and on the 28 tracked repositories other than `prebid-agent-skills`, 68 points. These fields were empty or identical everywhere and are not collected (D32): a PR's `createdViaEmail`, `locked`, and `activeLockReason`; a review's `createdViaEmail`, `isMinimized`, and `minimizedReason`; a comment's `isMinimized`, `minimizedReason`, `isPinned`, `pinnedAt`, and `pinnedBy`; a repository's `isFork`, `parent`, `isMirror`, `mirrorUrl`, `isTemplate`, `templateRepository`, `isUserConfigurationRepository`, `isEmpty`, `isDisabled`, `isLocked`, `lockReason`, `hasSponsorshipsEnabled`, `autoMergeAllowed`, `hasIssuesEnabled`, `hasPullRequestsEnabled`, `isBlankIssuesEnabled`, `isInOrganization`, `issueCreationPolicy`, `pullRequestCreationPolicy`, `codeOfConduct`, `fundingLinks`, `contactLinks`, `interactionAbility` (`NO_LIMIT` from `REPOSITORY` with no expiry where GitHub returned it), and `planFeatures`' `draftPullRequests` and `maximumAssignees`; and an organization's `anyPinnableItems` and `interactionAbility` on the 20 organizations read ("Account profiles"). Collected because they vary: a comment's `createdViaEmail` (true on 12 comments of the first sample), and once the added repositories were read, a repository's `forkingAllowed` and `hasVulnerabilityAlertsEnabled` (each constant across the 17), `usesCustomOpenGraphImage`, and `planFeatures`' `codeowners`, `teamReviewRequests`, and `maximumManualReviewRequests`. Kept though empty or constant: a repository's `isArchived` and `archivedAt` (the maintainer's exception for archive and visibility status; `isPrivate` and `visibility` vary across the 28); `owner`; the merge-method settings, `squashMergeAllowed`, `mergeCommitTitle`, and `mergeCommitMessage` among them; and `stack` and `stackEntry`, which the maintainer wants | the 5.8 documents over live GraphQL, 2026-10-18 | | Added repositories | The 11 repositories D13 adds: `prebid-cache` (147 PRs; 119 default-branch commits, 27 since the floor), `user-sync` (3; 16, 16), `shared-id` (34; 90, 5), `Shared-id-v2` (7; 144, 26), `prebid-mobile-ios-adapters` (0; 11, 11), `prebid-js-build-generator` (26; 42, 6), `code-scanner` (0; 29, 29), `currency-file` (0; 3,002, 1,424), `currency-file-generator` (45; 97, 24), and the private `prebidjs-releaser` (21; 401, 291) and `automation-testing` (6; 73, 73); 289 PRs in all; each pushed to since 2025-04-18 | live GraphQL, 2026-10-18 | @@ -2123,7 +2166,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Compare fallback on a PR whose head fork was deleted (only a live fork tested) | A count mismatch fails the item; Step 5 reports any | | Full bootstrap time, cost, and size are projections from two repositories | Step 5 measures them | | Overflow continuations per request (25), the server retry delays (2, 4, 8 s), the reset margin (1 s), retry jitter (25%), the index boundary margin (60 s), the budget reserves (100 points, 100 calls), the longest accepted rate-limit wait (65 min), and the HTTP timeouts (read 30 s, connect 10 s) are chosen, not measured | Step 5 records secondary-limit responses, budget readings, and request times at these values | -| The cost of one commit per item in rollback-journal mode at full scale (measured on three repositories only: section 7, "Step 3b-2a-2c acceptance") | Step 5 measures the full run | +| The cost of one commit per item in rollback-journal mode (not separated; whole-run times on three repositories are in section 7, "Step 3b-2a-2c acceptance") | Step 5 measures the full run | | Whether the activity log's 2023-03-07 start is fixed or a rolling window (seen on two repositories) | Step 4b records each repository's first logged event; it reads the whole log every run, so an event that ages out leaves the database as GitHub stops listing it (D7) | | Whether `history(since:)` filters on the committed or the authored date (only its `totalCount` was read) | Step 4b's recordings settle it before the coverage rule's wording for commits is final | | Whether REST compare caps the `files` it returns for a large push (not measured) | Step 4b measures it on its largest recorded push and stores a count only where GitHub reports one | From cfc040e3181e159959e93f560a4390015bebb322 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 14:48:15 -0600 Subject: [PATCH 03/15] docs(plan): close the gaps the convergence review found in Step 3b-2b 5.4: verify reads each repository's state and pending rows in one statement, then its items with their links and failures in one statement; the untracked set requires every entry to resolve without failing 5.3 step 1; the unverified list adds the full_name check; a repository error in either lookup leaves the items no lookup answered unchecked; the per-issue link rule applies only to an earlier updatedAt, and the report names each link difference's issues and totals. 5.8: the timeline lookup reads through repository(owner:, name:). 3b-2b-1: the sampler's recorder mode writes every unit of its run into its own directory. D36, gap 96, 5.6, section 8, and the 3b-2b-2 and 6a test lists follow. --- docs/redesign-plan.md | 118 +++++++++++++++++++++++++----------------- 1 file changed, 70 insertions(+), 48 deletions(-) diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 50c9f9f..c975927 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -88,7 +88,7 @@ Within scope the database collects and distinguishes data and never interprets i | D33 | `prebid-agent-skills` is never queried by an agent. Commands an agent runs (sync, verify, the REST sampler, the difference report, recordings, and `ghactivity github test`) name their repositories with `--repo` and leave it out; its rows enter the database through the scheduled App's workflow (Step 6) or a run the maintainer starts. Recordings come from public repositories only (Step 2). | | D34 | Decision of 2026-10-20 (the maintainer): wherever stored data must tell runs apart, a run is its `sync_runs` row id, stored with the data, never a clock reading and never state the writer or the engine keeps in memory, which neither a later run nor a reader of the file can see. `sync_failures.last_run_id` names the latest run in which the item failed, and `attempts` adds one only for a failure in another run, whatever the two runs' clocks read (5.1, gap 91). | | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | -| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and takes no lock: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit, and one that cannot be written makes the exit 1, because Step 6a reads it; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch makes a repository unverified, and a verify that verifies no repository is fatal. verify reads each repository's stored rows before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | +| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, and a repository error during a timeline lookup, makes a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | ## 3. Coverage rule @@ -221,7 +221,7 @@ Every gap found in the audit, and where it is resolved. | 93 | A cancel that lands inside a transaction on the writer's connection can leave the abandoned connection holding the write lock; the run's close then fails after the busy timeout and the row stays `running` (measured: a cancel during a slow `SELECT` 3 of 3, during a slow `INSERT` 0 of 3, among alternating short `SELECT`s and `INSERT`s 3 of 24; section 7, "Cancelled writer transaction") | Step 3b-2a-2b-2: accepted; the row is left as a crash leaves it, for the stale-row rule or `--break-lock` (5.3) | | 94 | 5.4 left verify's edges unstated: its lock check (the claim's rule runs only inside the claim, which writes); whether `--repo` limits the untracked repositories it reports; whether `--report` is written when verify stops before comparing or when the path cannot be written; the class of the resolve and index failures it did not list (not found, no default branch, a label list that fails its count, a number listed twice or out of creation order, a response the resolve or index rejects), beyond the three it did (a fetch that still fails, a second count mismatch, a resolve to another `github_id`), and of a repository error during a lookup; a verify that verifies no repository; when it reads the stored rows, which a sync that claims after its lock check can change; which events match a link difference GitHub reports only in the totals; and where its cost is recorded, since it writes no `sync_runs` row | Step 3b-2b-2 (5.4, D36) | | 95 | `timelineItems(since:)` keeps only items created after `since`, so a lookup passing the boundary would miss an event at it; GitHub's schema describes `filteredCount` as the count after applying `before` and `after`, and names neither `itemTypes` nor `since` | Step 3b-2b-2: `since` one second before the boundary, and `filteredCount` used as measured (5.8; section 7, "Timeline `since`") | -| 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode and `--only documents` rebuild `recorded/` and drop the unit kinds they do not record; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | +| 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode rebuilds `recorded/` from what it records, and `--only documents` drops every unit kind but the compare and rate-limit kinds it keeps; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | ## 5. Design @@ -829,19 +829,22 @@ run" is `repo_sync_state.last_run_id` and the boundary is its `last_index_starte the 5.3 step 4 failure classes and retries. It covers the configured repositories and, without `--repo`, every stored untracked repository, which it reports as not synced without a request: a stored repository that no configured entry resolves to without failing 5.3 step 1 is untracked when -its `is_tracked` is false, and, when every configured entry resolved, also when it is true, as a -sync marks it (5.3 step 1). `--repo` limits verify to the repositories it names, each a configured +its `is_tracked` is false, and, when every configured entry resolved without failing 5.3 step 1 +(to its stored `github_id` or to a new repository), also when it is true, as a sync marks it (5.3 +step 1). `--repo` limits verify to the repositories it names, each a configured repository, ignoring case, as it limits a sync (D27). Like `sync`, it exits 1 before any request unless the database is at the code's migration head and `user_version` equals `schema_version` (gap 74), and it uses no server-side cursor (gap 77); it also exits 1, naming the run, while a sync run holds the lock: before any request it reads `sync_runs` and stops when a row is `running` with a null `deadline_at` or one less than 10 minutes in the past, a deadline not yet reached included, the claim's rule (5.3). verify writes nothing to -the database and takes no lock, so a stale row stays for the next claim to reclaim, and a crashed +the database and claims no run, so a stale row stays for the next claim to reclaim, and a crashed run with a null `deadline_at` stops verify until a `sync --break-lock` run marks it `aborted` (D36). -A sync can still claim after that check (gap 77), so for each repository verify reads its -`repo_sync_state` row, then its stored items with their `sync_failures` and `sync_pending` rows, -before the first request of its index, and the stored side is never newer than the fresh index. +A sync can still claim after that check (gap 94), so for each repository, before the first request +of its index, verify reads its `repo_sync_state` row with its `sync_pending` rows in one statement, +then its stored items with their closing links and `sync_failures` rows in one statement. Each +statement reads one consistent state (gap 77), and the stored side is never newer than the fresh +index. It re-indexes and applies the coverage rule, then compares by `github_id` the stored set with the result (missing and extra items) and, for items in both, the stored `github_updated_at`, `state`, `is_draft`, `author_association`, and closing links @@ -866,15 +869,17 @@ the first class that matches, in this order: its difference is drift only through `createdAt` or `updatedAt` (section 7, "Closing-link timeline events"). The index names at most 10 links, so a PR with more can differ in its totals alone; that difference is drift when the connected events at or after the boundary number at - least the total's increase, or the disconnected events at least its decrease. A PR's link - difference is drift only when each part of it is: each differing issue the index names by its - own event, and a difference in the totals by that count; + least the total's increase, or the disconnected events at least its decrease, every such event + counted, those that match a differing issue included. For a link difference with an earlier + `updatedAt`, the difference is drift only when each part of it is: each differing issue (indexed + and not stored, or stored and not indexed when the index lists every link) by an event whose + `subject` is that issue, and a difference in the totals by that count; - removed since sync: a stored item missing from the fresh index whose lookup by number returns `NOT_FOUND`, listed by number (the lookup is a hydration batch: `Gone` is removed since sync, a - hydrated PR an error; a repository error in the batch makes the repository unverified, which keeps - the classes its comparisons already gave, and the missing items it did not look up are unchecked); + hydrated PR an error); - unchecked: a differing item whose timeline lookup (5.8) or lookup by number still fails after the - retries, listed by number; + retries, or that a lookup had not answered when a repository error stopped it, those of the batch + the error ended included, listed by number; - error: anything else. Only an error fails verify. `verify --exact` also fails on not synced, pending, halted deletion, @@ -885,8 +890,11 @@ resolve or index (not found, no default branch, a fetch that still fails after i response or GitHub error the resolve or index rejects, a label list that fails its count or names a label twice, paging without progress, a number listed twice or out of creation order, a second count mismatch), step 1's own checks (a resolve to a `github_id` other than its stored row's, a later -entry resolving to an earlier entry's `github_id`, gap 92), and a repository error during a lookup by -number or a timeline lookup (5.3 step 4). An unverified repository fails `--exact` and is reported, +entry resolving to an earlier entry's `github_id`, gap 92, and another `is_tracked` stored repository +holding the resolved `full_name`, ignoring case, checked as a dry run checks it, against the stored +names and the names earlier entries resolved to, 5.1), and a repository error during a lookup by +number or a timeline lookup (5.3 step 4), which keeps the classes the repository's comparisons +already gave. An unverified repository fails `--exact` and is reported, without failing, otherwise; a verify in which every repository it attempts is unverified, and the deadline stopped none, is fatal (exit 1), as a sync is when no configured repository resolves (5.3 step 1). A verify that reaches @@ -904,7 +912,8 @@ because Step 6a reads it (5.6, D36); an interrupt exits 130 either way. The repo code and whether `--exact` was given; per class, the count of items and their numbers (an item whose differences take two classes counts in each); per repository, its `last_run_id`, boundary, per-class counts, and per-state counts; -the unverified repositories with their reasons; and verify's GraphQL points and REST calls, counted +for each closing-link difference, the issues it names and the stored and fresh totals; the +unverified repositories with their reasons; and verify's GraphQL points and REST calls, counted by its source as a run's are (5.3 step 4), and the seconds from its start to the report's write, since verify writes no `sync_runs` row. A report verify prints that stdout cannot take leaves the exit code verify's, and stderr follows 5.3 step 8, 120 included. @@ -1054,7 +1063,7 @@ Workflow `.github/workflows/sync.yml`: exit code, counts per class, and unverified repositories (from `--verify-report`; null without it), and the SHA-256. `snapshot` exits 1 when the report `--verify-report` names is missing or unreadable, or names a `last_run_id` other than the file's `repo_sync_state`, so a verify that - never started, which leaves no report or an earlier one (5.4), fails the job. + never started, which leaves no report or one from an earlier sync (5.4), fails the job. - Uploads never use `--clobber`. Before each upload, `publish` deletes a same-name asset in any state other than `uploaded`. A same-name `uploaded` asset whose `digest` is `sha256:` followed by the expected SHA-256 (or, when `digest` is null, whose downloaded SHA-256 matches) is skipped, so a @@ -1264,19 +1273,20 @@ Verify's timeline lookup (Step 3b-2b-2) reads, for each differing PR, its ready- convert-to-draft, connected, and disconnected events at or after the repository's boundary. It passes `since` one second before the boundary and counts an entry only when its `createdAt` is at or after the boundary: `since` keeps only items created after it, the event times are whole seconds -(section 7, "Timeline `since`"), and so is the boundary (5.3 step 2). It -pages by `pageInfo.hasNextPage` and checks the entries it read against the first page's -`filteredCount`, which applies `itemTypes` and `since` (measured; GitHub's schema describes it as -the count after applying `before` and `after`, and names neither); a later page's `filteredCount` -counts only -the items after its cursor, and `totalCount` is neither filtered nor the number of entries GitHub -returns (section 7, "Closing-link timeline events"). A lookup whose entries differ from that count -is retried once alone, as an item error is (5.3 step 4), and then leaves its item unchecked (5.4). -Its document, formula cost, and recordings are added here when it is written; the recorder adds -them without changing the existing recordings, and its default mode and `--only documents` record -them too, so a re-recording keeps them (gap 96). The document takes no `tests/query_versions.json` -entry: verify stores nothing, so no `hydration_version` depends on it, and once it changes the -replay fails with "re-record" on its recordings, whose stamp is the old query hash (Step 2). +(section 7, "Timeline `since`"), and so is the boundary (5.3 step 2). It reads each PR through +`repository(owner:, name:)` and `pullRequest(number:)`, so a null `repository` is the repository +error 5.4 names. It pages by `pageInfo.hasNextPage` and checks the entries it read against the +first page's `filteredCount`, which applies `itemTypes` and `since` (measured; GitHub's schema +describes it as the count after applying `before` and `after`, and names neither); a later page's +`filteredCount` counts only the items after its cursor, and `totalCount` is neither filtered nor +the number of entries GitHub returns (section 7, "Closing-link timeline events"). A lookup whose +entries differ from that count is retried once alone, as an item error is (5.3 step 4), and then +leaves its item unchecked (5.4). Its document, formula cost, and recordings are added here when it +is written; the recorder adds them without changing the existing recordings, and its default mode +and `--only documents` record them too, so a re-recording keeps them (gap 96). The document takes +no `tests/query_versions.json` entry: verify stores nothing, so no `hydration_version` depends on +it, and once it changes the replay fails with "re-record" on its recordings, whose stamp is the old +query hash (Step 2). Default-branch history (Step 4b; 100 commits per page cost 1, dry run on Prebid.js, 2026-10-18): @@ -1792,10 +1802,12 @@ but not compared, since the stored merge state is as of `hydrated_at` (5.1); each REST list read 100 entries per page until a page holds fewer, since `rest_get` returns no `Link` header (gap 96); the offline tests compare REST bodies with rows written from GraphQL units recorded in the same recorder run as the REST units of the same PRs, or with rows seeded from a -synthetic body; REST bodies recorded as their own unit kind by an additive recorder mode, each -stamped as the other kinds are (Step 2), into their own directory beside `recorded/`, as the -first-sync recordings are, so the default mode and `--only documents`, which rebuild `recorded/`, -leave them as they are (gap 96); the mode refuses a repository the recorder's `PRIVATE` list names +synthetic body; REST bodies recorded as their own unit kind, each stamped as the other kinds are +(Step 2), by a recorder mode of their own that writes every unit of its run, the sampled PRs' +GraphQL units included, into its own directory beside `recorded/`, replacing that directory alone, +as the first-sync mode does, and that a document change re-runs; the offline tests replay that +directory alone, and the default mode and `--only documents`, which rebuild `recorded/`, leave it +as it is (gap 96); the mode refuses a repository the recorder's `PRIVATE` list names before any request and resolves each other repository through the recorder's public-only source before its first REST request (D33); the sampler takes its repositories with `--repo` and refuses `prebid-agent-skills` before any request (D33); run from `scripts/`, reaching GitHub through @@ -1804,7 +1816,8 @@ before its first REST request (D33); the sampler takes its repositories with `-- Tests: the offline tests the scope lists; a list read across pages, one of them ending on a full page followed by an empty one; the recorder mode sending no request for a repository `PRIVATE` lists and refusing a private repository before any REST request; a re-recording in the default mode -and with `--only documents` leaving the REST units as they are; and the sampler reading only the +and with `--only documents` leaving the mode's directory, REST and GraphQL units, as it is; and the +sampler reading only the repositories `--repo` names and refusing `prebid-agent-skills` before any request. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule @@ -1827,10 +1840,14 @@ unit by its `since` (the timeline recordings join the model's round-trip test), fake GitHub: each verify class, each overlapping pair of classes taking the first in the 5.4 order, a stale item, verify's repository set with and without `--repo` (a stored untracked repository, a stored row no configured entry names, a renamed repository, and a renamed repository -whose configured entry fails to resolve, reported unverified and not untracked), its child-count -checks (each by a planted row deletion), an unverified repository for each failure 5.4 lists, one -made unverified by a lookup by number keeping the classes it already had, a verify with every -repository unverified exiting 1, a +whose configured entry fails to resolve, reported unverified and not untracked, and an entry that +resolves to a `github_id` other than its stored row's, with no stored `is_tracked` row reported +untracked), its child-count checks (each by a planted row deletion), an unverified repository for +each failure 5.4 lists, one made unverified by a lookup by number and one by a timeline lookup, each +keeping the classes it already had and leaving unchecked the items no lookup had answered, those of +the batch the error ended included, a verify with every repository unverified exiting 1, a +repository's `sync_pending` rows replaced by a sync's close between verify's two reads and the item +still reported pending, a `CLOSED` PR whose `closedAt` falls on each side of the floor, verify's head check refusing an older revision, an empty file, and a missing path before any request and without creating a file, verify exiting 1 and naming the run while a sync run @@ -1849,9 +1866,12 @@ timeline lookup through the replay: a connected event naming the differing issue another issue, a link difference made by a closing keyword with no event, a difference in the link totals alone with enough connected events, with only a disconnected event, and with too few, a synthetic -disconnected event, an event at the boundary counted and one a second before it not, a filtered -page whose `totalCount` differs from its entries, a lookup whose entries differ from its -`filteredCount` leaving its item unchecked, and a lookup +disconnected event, an event at the boundary counted and one a second before it not, entries +planted in the lookup's response at a second and at half a second before the boundary and not +counted, a stored link no longer indexed on a PR whose index lists every link matched only by an +event naming it, a filtered page whose `totalCount` differs from its entries, a lookup whose entries +differ from its `filteredCount` leaving its item unchecked, one that differs once and matches on +its retry, classed by its events, and a lookup spanning two pages whose second page's `filteredCount` counts only the entries after its cursor (5.8). A re-recording in the default mode and with `--only documents` keeping the timeline units. @@ -1866,12 +1886,13 @@ and authors compared as D24 stores them (section 7, "Source vs REST"). The live fresh database at an explicit path, with `GITHUB_TOKEN="$(gh auth token)"` and the token never printed; reads `ghactivity github rate-limit` before and after, and does not start with fewer than 2,500 GraphQL points remaining (the first sync of these repositories spent 1,687, section 7, -"Step 3b-2a-2c acceptance"), recording any rate-limit wait apart from the run's time; runs a second, +"Step 3b-2a-2c acceptance"); runs a second, incremental sync before verify; and records the points, REST calls, and time it spent, verify's from its report. The sample's cases are chosen by SQL queries recorded in the pull request; all three repositories hold in-coverage dismissed reviews (section 7, "Step 3b-2a-2c acceptance"). The pull request lists each -closing-link error verify reports, with its PR and link (section 8). `verify --exact` exits 0, or +closing-link error verify reports, with its PR and link (for a difference in the totals alone, +found by reading the PR's full closing list; section 8). `verify --exact` exits 0, or exits 2 with no failing difference but such closing-link errors, each with REST evidence that GitHub changed the link without changing the PR's `updatedAt`. A live item failure that cannot pass is listed in `[sync].accepted_failures` with REST evidence of why, as in Step 5. An @@ -1983,8 +2004,9 @@ Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snap `ghactivity verify --db-only` (5.4), and adds the `githubkit[auth-app]` extra to `pyproject.toml` and `uv.lock`. Its tests: `verify --db-only` accepting the gz and manifest `ghactivity snapshot` wrote, and halting with exit 1 on a tampered -copy; `snapshot --verify-report` exiting 1 on a missing report and on one whose `last_run_id` -differs from the file's (5.6); through +copy; `snapshot --verify-report` exiting 1 on a missing report, a truncated one, one of an unknown +version, and one whose `last_run_id` differs from the file's, and the manifest listing the report's +unverified repositories (5.6); through `httpx.MockTransport`, a mocked installation-token response whose `expires_at` lies just over a minute ahead, a clock moved past it mid-run, and a second token request before the next API request (githubkit keeps a token until one minute before `expires_at`); and a 401 on an API request still @@ -2175,7 +2197,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Whether GitHub returns `EnterpriseUserAccount.id` to a token without the `read:enterprise` scope (the dry run validated the document; no tracked repository has such an actor yet) | Step 3b-1 tests the synthetic actor. A live occurrence decides between two outcomes (a prediction): an error at the alias's path is an item failure (exit 2), and an error without a path is fatal (5.3 step 4, exit 1), which Step 5's bootstrap would surface | | Whether the scheduled App token can read every resolve field the token used here read (`diskUsage`, `webCommitSignoffRequired`, `hasVulnerabilityAlertsEnabled`, and `planFeatures` included) | Step 6b's first manual run; a field it cannot read is an error 5.3 step 4 classifies | | A `DisconnectedEvent`'s `source` and `subject` (none in the 300 PRs read for section 7, "Closing-link timeline events"; GitHub's schema gives it `ConnectedEvent`'s fields) | Step 3b-2b-2 tests a synthetic one; a live one is recorded when seen | -| Whether a closing-keyword link can change without the PR's `updatedAt` changing (an issue transfer or deletion, a body edit, a base-ref or default-branch change); 5.4 then reports the difference as an error | Step 3b-2b-2's acceptance lists each such error with its PR and link; Step 5 reports any | +| Whether a closing-keyword link can change without the PR's `updatedAt` changing (an issue transfer or deletion, a body edit, a base-ref or default-branch change); 5.4 then reports the difference as an error, unless it is a difference in the totals alone that the connected or disconnected events since the boundary cover (5.4) | Step 3b-2b-2's acceptance lists each such error with its PR and link; Step 5 reports any | | Whether a pending `reviewRequests` entry for Copilot carries `BOT_kgDOCnlnWA` (the review-request events and the reviews do; no pending request was seen, section 7, "Copilot review requests" and "Step 3b-2a-2c acceptance") | Step 5 reports any Copilot request row; for a `databaseId` reported under a second node id, the writer keeps the stored node id and the run reports the second (5.2) | | Whether to collect a `Mannequin`'s `email` ("The mannequin's email on the source instance"; the maintainer's profile decision named users and organizations only, and no mannequin was available to read, so whether GitHub requires a scope for it is unmeasured) | The maintainer decides; a live read on a mannequin comes first | | Whether GitHub returns an `EnterpriseUserAccount`'s `enterprise` to the tokens used (an `Enterprise` object; no tracked repository has such an actor) | Not selected until a live occurrence is read | From 9c2b80326a783f7c87f1786ae1464c5491e65d93 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 14:55:55 -0600 Subject: [PATCH 04/15] docs(plan): fail verify on anything it could not check (D37) An unchecked item and an unverified repository now fail verify in both modes, not only under --exact, so the scheduled verify cannot exit 0 on a database it did not check and D25 does not publish it. A timeline lookup whose entries still differ from filteredCount after its retry, or whose response the typed models reject, is an error. D37 records the rule: an error over a silent failure or silently wrong data. Gap 97, D25's wording, 5.8, and the 3b-2b-2 tests follow. --- docs/redesign-plan.md | 42 +++++++++++++++++++++++++----------------- 1 file changed, 25 insertions(+), 17 deletions(-) diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index c975927..23b9710 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -77,7 +77,7 @@ Within scope the database collects and distinguishes data and never interprets i | D22 | The CI test gate allows no skipped tests (`scripts/check_junit.py --max-skipped 0`). A test that must skip in CI needs the allowance raised in the workflow, in the same change. pytest reports `xfail` as skipped, so the same applies to an `xfail`. | | D23 | Closing-issue links are read by the index as well as by hydration, and the stored links are compared with the index on every run, so a link added after merge, which does not change the PR's `updatedAt`, is captured: a PR whose indexed links differ from its stored links is planned for hydration (5.3 step 3). Hydration also reads which links GitHub lists as manually linked (`userLinkedOnly`), at no added cost (section 7). | | D24 | Conversation comments are stored one row per comment, its `github_id`, author, and times and never its text, so a comment by an account the database never stored is a row with a null author, never dropped (D28). Review bodies, inline review comment text, and commit messages are not stored either. | -| D25 | A run, scheduled or manual, publishes when its starting database is valid (the downloaded snapshot passed `verify --db-only`, or `bootstrap` started from an empty database), sync exited 0 or 2, and verify exited 0 (no error, 5.4). The manifest records the sync exit code and verify's counts per class. A final job that runs whatever the others did then marks the workflow failed if sync exited 2, verify failed, or nothing was published, so a failure is visible after the data is saved. | +| D25 | A run, scheduled or manual, publishes when its starting database is valid (the downloaded snapshot passed `verify --db-only`, or `bootstrap` started from an empty database), sync exited 0 or 2, and verify exited 0 (no failing class, 5.4). The manifest records the sync exit code and verify's counts per class. A final job that runs whatever the others did then marks the workflow failed if sync exited 2, verify failed, or nothing was published, so a failure is visible after the data is saved. | | D26 | The Step 5 database is not committed; it becomes the first snapshot of release `db` (Step 6b). From Step 3 until 6c removes it, the tracked `github_activity.db` keeps its pre-Step-3 content: the default `DATABASE_URL` moves to a git-ignored file, and `scripts/check_frozen_db.py` fails when the file's SHA-256 differs from the digest it pins, as a pre-commit hook on the file (so CI's `--all-files` run checks it) and as a test; the tests also fail when git stores the file other than as one regular file, and the `tracked-db-no-sidecar` hook refuses a committed `-journal` or `-wal` file beside it, which SQLite would apply on the next open, and the `-shm` index that goes with a `-wal`. | | D27 | A stored repository no longer listed in `tracking.toml` is reported, with exit 2, on every run without `--repo` (5.3 steps 1 and 8), and its rows stay until `ghactivity sync --prune-untracked` deletes them; the prune, on the operator's explicit flag, removes rows section 1 puts out of scope (repositories not in the tracked list) and is recorded in `sync_runs.deletions_approved` (5.1). | | D28 | The database keeps a person attached to everything it attributed to them. A stored account stays stored after a lookup returns `NOT_FOUND` for it, with its last known login and type, and every reference to it stays: when GitHub later returns that actor as null, as a re-hydration does once GitHub stops returning the account, the writer keeps the stored reference of the same object (the PR's author and merger, the issue's author, the review's reviewer or the comment's author with the same `github_id`, this PR's commit with the same SHA, and the default-branch commit with the same repository and SHA, Step 4b), and the row's `*_reported` column records that GitHub returned no actor in that hydration (5.1). Other actor references (editors, requested reviewers, event actors) take GitHub's current value, null included, and the assignees are GitHub's current `assignedActors` list. `not_found_at` (named `deleted_at` until Step 3b-1) records when a lookup returned `NOT_FOUND`; it changes no reference, the 24-hour refresh skips the account, and a later sighting of the account clears it. GitHub does not say why it returns a null actor (`Prebid.js#13769` is merged, with a null `mergedBy` and an existing author), so a null reference means only that GitHub returned no actor and the object had none stored, and an account never stored has no `github_users` row. A non-null actor GitHub returns replaces the stored one. On a commit, `linked_github_id` holds the account id GitHub linked in the latest hydration whether or not a lookup found the account, so a link whose lookup returned null is told apart from an email linked to no account. The attribution lives in the stored rows and cascades with them, so anything that re-creates a row from GitHub drops it for accounts GitHub no longer returns: a bootstrap (it starts from an empty file), a transferred issue (new `github_id`), and a repository pruned and re-added (D27). | @@ -89,6 +89,7 @@ Within scope the database collects and distinguishes data and never interprets i | D34 | Decision of 2026-10-20 (the maintainer): wherever stored data must tell runs apart, a run is its `sync_runs` row id, stored with the data, never a clock reading and never state the writer or the engine keeps in memory, which neither a later run nor a reader of the file can see. `sync_failures.last_run_id` names the latest run in which the item failed, and `attempts` adds one only for a failure in another run, whatever the two runs' clocks read (5.1, gap 91). | | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, and a repository error during a timeline lookup, makes a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | +| D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | ## 3. Coverage rule @@ -222,6 +223,7 @@ Every gap found in the audit, and where it is resolved. | 94 | 5.4 left verify's edges unstated: its lock check (the claim's rule runs only inside the claim, which writes); whether `--repo` limits the untracked repositories it reports; whether `--report` is written when verify stops before comparing or when the path cannot be written; the class of the resolve and index failures it did not list (not found, no default branch, a label list that fails its count, a number listed twice or out of creation order, a response the resolve or index rejects), beyond the three it did (a fetch that still fails, a second count mismatch, a resolve to another `github_id`), and of a repository error during a lookup; a verify that verifies no repository; when it reads the stored rows, which a sync that claims after its lock check can change; which events match a link difference GitHub reports only in the totals; and where its cost is recorded, since it writes no `sync_runs` row | Step 3b-2b-2 (5.4, D36) | | 95 | `timelineItems(since:)` keeps only items created after `since`, so a lookup passing the boundary would miss an event at it; GitHub's schema describes `filteredCount` as the count after applying `before` and `after`, and names neither `itemTypes` nor `since` | Step 3b-2b-2: `since` one second before the boundary, and `filteredCount` used as measured (5.8; section 7, "Timeline `since`") | | 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode rebuilds `recorded/` from what it records, and `--only documents` drops every unit kind but the compare and rate-limit kinds it keeps; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | +| 97 | Without `--exact`, an unchecked item and an unverified repository did not fail verify, and a timeline lookup whose entries differed from `filteredCount` left its item unchecked, so the scheduled verify (5.6, no `--exact`) could exit 0 and D25 publish a database it had not checked | D37: both fail verify in every mode, and an answer verify cannot accept is an error (5.4, 5.8) | ## 5. Design @@ -878,13 +880,17 @@ the first class that matches, in this order: `NOT_FOUND`, listed by number (the lookup is a hydration batch: `Gone` is removed since sync, a hydrated PR an error); - unchecked: a differing item whose timeline lookup (5.8) or lookup by number still fails after the - retries, or that a lookup had not answered when a repository error stopped it, those of the batch - the error ended included, listed by number; -- error: anything else. - -Only an error fails verify. `verify --exact` also fails on not synced, pending, halted deletion, -removed since sync, unchecked, and known failures not listed in `[sync].accepted_failures`; it is -the mode the Step 3-5 acceptance uses, and drift is reported in both modes. A repository that verify cannot + retries on a request GitHub did not answer (a transport, server, timeout, or rate-limit failure), + or that a lookup had not answered when a repository error stopped it, those of the batch the error + ended included, listed by number; +- error: anything else, a lookup whose answer verify cannot accept included (a timeline lookup + whose entries still differ from `filteredCount` after its retry, a value the typed models reject). + +An error, an unchecked item, and an unverified repository fail verify in both modes: what verify +could not check is a failure, never a pass (D37). `verify --exact` also fails on not synced, +pending, halted deletion, removed since sync, and known failures not listed in +`[sync].accepted_failures`; it is the mode the Step 3-5 acceptance uses, and drift is reported in +both modes. A repository that verify cannot resolve, re-index, or look up is reported as unverified: every repository error the source raises at resolve or index (not found, no default branch, a fetch that still fails after its retries, a response or GitHub error the resolve or index rejects, a label list that fails its count or names a @@ -894,10 +900,9 @@ entry resolving to an earlier entry's `github_id`, gap 92, and another `is_track holding the resolved `full_name`, ignoring case, checked as a dry run checks it, against the stored names and the names earlier entries resolved to, 5.1), and a repository error during a lookup by number or a timeline lookup (5.3 step 4), which keeps the classes the repository's comparisons -already gave. An unverified repository fails `--exact` and is reported, -without failing, otherwise; a verify in which every repository it attempts is unverified, and the -deadline stopped none, is fatal (exit 1), as a sync is when no configured repository resolves (5.3 -step 1). A verify that reaches +already gave. An unverified repository fails verify and is reported with its reason; a verify in +which every repository it attempts is unverified, and the deadline stopped none, is fatal (exit 1), +as a sync is when no configured repository resolves (5.3 step 1). A verify that reaches `--max-minutes` stops with a timeout instead of waiting past it. Exit 0 when no failing class occurred, 2 when one did, 1 on a fatal error or a timeout, and 130 on an interrupt, the CLI's code (5.3). A verify that never starts exits 1 before any request, writes no report, and leaves PATH as @@ -1280,8 +1285,9 @@ first page's `filteredCount`, which applies `itemTypes` and `since` (measured; G describes it as the count after applying `before` and `after`, and names neither); a later page's `filteredCount` counts only the items after its cursor, and `totalCount` is neither filtered nor the number of entries GitHub returns (section 7, "Closing-link timeline events"). A lookup whose -entries differ from that count is retried once alone, as an item error is (5.3 step 4), and then -leaves its item unchecked (5.4). Its document, formula cost, and recordings are added here when it +entries differ from that count is retried once alone, as an item error is (5.3 step 4), and a +difference that remains makes its item an error, as a value the typed models reject does (5.4, +D37). Its document, formula cost, and recordings are added here when it is written; the recorder adds them without changing the existing recordings, and its default mode and `--only documents` record them too, so a re-recording keeps them (gap 96). The document takes no `tests/query_versions.json` entry: verify stores nothing, so no `hydration_version` depends on @@ -1845,7 +1851,8 @@ resolves to a `github_id` other than its stored row's, with no stored `is_tracke untracked), its child-count checks (each by a planted row deletion), an unverified repository for each failure 5.4 lists, one made unverified by a lookup by number and one by a timeline lookup, each keeping the classes it already had and leaving unchecked the items no lookup had answered, those of -the batch the error ended included, a verify with every repository unverified exiting 1, a +the batch the error ended included, an unchecked item and an unverified repository each making a +verify without `--exact` exit 2, a verify with every repository unverified exiting 1, a repository's `sync_pending` rows replaced by a sync's close between verify's two reads and the item still reported pending, a `CLOSED` PR whose `closedAt` falls on each side of the @@ -1870,8 +1877,9 @@ disconnected event, an event at the boundary counted and one a second before it planted in the lookup's response at a second and at half a second before the boundary and not counted, a stored link no longer indexed on a PR whose index lists every link matched only by an event naming it, a filtered page whose `totalCount` differs from its entries, a lookup whose entries -differ from its `filteredCount` leaving its item unchecked, one that differs once and matches on -its retry, classed by its events, and a lookup +differ from its `filteredCount` after its retry and one whose response the typed models reject, +each an error, one that differs once and matches on its retry, classed by its events, and a +lookup spanning two pages whose second page's `filteredCount` counts only the entries after its cursor (5.8). A re-recording in the default mode and with `--only documents` keeping the timeline units. From 19d4f3f0648898fae8c46e713f6209271ab7d37b Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 15:30:23 -0600 Subject: [PATCH 05/15] docs(plan): apply D37 to the sync and to Steps 4b-6 (D38) Records the decisions on the silent-failure audit as D38 and gaps 98-110: Step 3b-2a-3 makes a failed account refresh, a time-limited run with unreached items, and a hydration that contradicts the run's own index each exit 2, scopes accepted failures to an error class, re-hydrates PRs with more links than the index carries, stores every node id GitHub reports, and closes the source and run-record holes the audit found. Steps 4b, 5, and 6 and 5.6 gain the completeness, permission, staleness, and workflow rules. --- docs/redesign-plan.md | 286 +++++++++++++++++++++++++++++------------- 1 file changed, 200 insertions(+), 86 deletions(-) diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 23b9710..bf51efd 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -33,7 +33,7 @@ issue totals) are dated; regenerate them with `ghactivity verify` once Step 3b-2 |------|-------|--------| | 1 | Guardrails | done (PR #5) | | 2 | GitHub source layer | done (PR #7) | -| 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2b-1, and 3b-2b-2) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | +| 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | | 4 | Issues, closing-issue links, and default-branch activity (pull requests 4a and 4b) | not started | | 5 | Full bootstrap and verification | not started | | 6 | Scheduled sync and publication | not started | @@ -90,6 +90,7 @@ Within scope the database collects and distinguishes data and never interprets i | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, and a repository error during a timeline lookup, makes a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | +| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when a time-limited run leaves planned items unreached, and when a hydration contradicts the run's own index (Step 3b-2a-3). An `accepted_failures` entry accepts one error class. A PR with more closing links than the index carries is hydrated every run. Every node id GitHub reports for a stored account is stored. Step 4b marks a compare file list that reaches GitHub's 300-file cap, completes an activity read only at the log's end or the floor, and fails a history read whose count still differs after a rescan. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before the schedule starts, pins the database file the job checked, compares exit-code outputs as non-empty strings, ties a snapshot to the job's own sync, and gives consumers a staleness check that does not depend on the schedule. Left as they are, because the stored data already says what happened: a compare-linked account a lookup answers `NOT_FOUND` (D28), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (the retained snapshots hold the identities), and the merge state as of `hydrated_at` (D32). | ## 3. Coverage rule @@ -224,6 +225,19 @@ Every gap found in the audit, and where it is resolved. | 95 | `timelineItems(since:)` keeps only items created after `since`, so a lookup passing the boundary would miss an event at it; GitHub's schema describes `filteredCount` as the count after applying `before` and `after`, and names neither `itemTypes` nor `since` | Step 3b-2b-2: `since` one second before the boundary, and `filteredCount` used as measured (5.8; section 7, "Timeline `since`") | | 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode rebuilds `recorded/` from what it records, and `--only documents` drops every unit kind but the compare and rate-limit kinds it keeps; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | | 97 | Without `--exact`, an unchecked item and an unverified repository did not fail verify, and a timeline lookup whose entries differed from `filteredCount` left its item unchecked, so the scheduled verify (5.6, no `--exact`) could exit 0 and D25 publish a database it had not checked | D37: both fail verify in every mode, and an answer verify cannot accept is an error (5.4, 5.8) | +| 98 | A 24-hour account refresh that still fails, or whose answer the writer refuses (another `databaseId` for a stored node id), left the run `complete` with exit 0; the only trace was a printed line, and a test pinned exit 0 | Step 3b-2a-3: counted in `sync_runs.refresh_failures`, listed in the report, and exit 2 (5.3 step 4, D38) | +| 99 | A run that reached `--max-minutes` with planned items unreached exited 0, and a deadline inside a batch turned the item failures it had computed into pending items | Step 3b-2a-3: such a run exits 2; D25 still publishes it, and the workflow is red (5.3 step 8, D38) | +| 100 | A PR the run's own index listed in coverage that hydration then answered `NOT_FOUND`, or returned out of coverage, was neither written, failed, nor pending, and nothing reported it; a hydration older than the index entry was written without a check | Step 3b-2a-3: each is an item error (5.3 steps 4 and 5, D38) | +| 101 | An `accepted_failures` entry accepted any failure of its item, so a new cause stayed out of exit 2 | Step 3b-2a-3: an entry names the error class it accepts (5.5, D38) | +| 102 | A PR with more than the 10 closing links the index carries was not re-hydrated when a link beyond the 10th was swapped with equal totals, and verify compares those links the same way | Step 3b-2a-3: such a PR is hydrated every run (5.3 step 3, D38) | +| 103 | A `databaseId` reported under a second node id kept the stored node id and dropped the reported one; only a printed line recorded it | Step 3b-2a-3: `account_node_ids` stores every node id GitHub reports for an account (5.1, 5.2, D38) | +| 104 | Source checks that accepted what they should refuse: a REST response without readable rate-limit headers, a GraphQL `errors` present but not a list (`{}`, `""`), a compare above 250 commits checked by count but not membership, and an index page from a repository other than the one resolved | Step 3b-2a-3 (5.3 steps 2, 4, and 5, D38) | +| 105 | Run records and setup that hid a failure: `code_version` without `-dirty` when `git status` failed, a close that ignored outcomes it could not write, a missing `[automation]` table emptying the automation lists, `foreign_keys` never read back, `alembic stamp` accepted, a wheel check comparing repository names only, and a log file no record reaches | Step 3b-2a-3 (5.1, 5.3 step 8, 5.5, D38) | +| 106 | Step 4b: compare `files` stop at GitHub's documented 300 per comparison with no total, so a capped list read as complete; the activity read had no paging or completeness rule; a history count mismatch had no outcome; a refused compare was never retried; and the log's first event was not stored | Step 4b (5.1, D38) | +| 107 | Step 6: an App token may return null where the maintainer's token read a value; a stopped schedule turns nothing red; a missing `DATABASE_URL` builds from empty; an empty exit-code output compares equal to 0 in `if:`; and `snapshot` did not check that the job's own sync ran | Step 6 and 5.6 (D38) | +| 108 | The data dictionary presented head-repository, milestone, and team names as GitHub's current values, though they are as of `hydrated_at` | Step 3b-2a-3: described as of `hydrated_at` (D38) | +| 109 | Edits that may leave `updatedAt` unchanged (body, base ref, review dismissal) were accepted as missed | Step 5 measures them; one that does gets an index-side check or a periodic re-hydration (section 8, D38) | +| 110 | A GitHub-derived column a later migration adds to a table without `hydration_version` (accounts, compares) is never filled for existing rows | 5.1: such a migration adds a per-row read version or refuses while the table holds rows (D38) | ## 5. Design @@ -319,15 +333,16 @@ and `color`, `pr_user_tags.created_at`). | `issue_labels` | `issue_id`, `name` | PK (`issue_id`, `name`) | | (Step 4a) | | The two issue rows above are the Step 3 plan. Step 4a completes them under D29 the way Step 3b-1 completes the PR rows: `issues.author_reported` (D28), the label id on `issue_labels` (D8), the issue's other free fields (`author_association`, `last_edited_at`, `published_at`, milestone, issue type, the parent and the `duplicateOf` issue by repository id and number), `issue_comments` and `issue_assignees` tables shaped like `pr_comments` and `pr_assignees` with their totals, and their 5.4 child-count checks | | `repository_activities` | `repository_id`, `github_id`, `node_id`, `activity_type`, `ref`, `before_sha`, `after_sha`, `occurred_at`, `actor_github_id`, `actor_id` | Step 4b (D30); `github_id` unique (GitHub's event `id`); `activity_type` is GitHub's value as text (`push`, `force_push`, `branch_creation`, `branch_deletion`, `pr_merge`, `merge_queue_merge` in githubkit 0.14.2's model); `occurred_at` is GitHub's `timestamp`; `actor_github_id` is the REST actor's `id` and `actor_id` its `github_users` row once a lookup resolves it (5.2), both null when GitHub returns no actor | -| `repository_activity_compares` | `activity_id`, `http_status`, `status`, `ahead_by`, `behind_by`, `total_commits`, `merge_base_sha` | Step 4b; PK `activity_id`; GitHub's REST compare of `before_sha...after_sha` as it returns them, so the files and counts run from the merge base, not from `before_sha`, when the push diverged (section 7, "Compare"); no row when either SHA is all zeros (a branch creation or deletion); `http_status` records the response, so a refused compare is a row with the other columns null (a compare across a force push, `5110f3d...fccb1f5`, returned 200, section 7) | +| `repository_activity_compares` | `activity_id`, `http_status`, `status`, `ahead_by`, `behind_by`, `total_commits`, `merge_base_sha`, `files_at_cap` | Step 4b; PK `activity_id`; `files_at_cap` is true when compare returned 300 files, the cap GitHub documents for one comparison, so its file rows may be partial (gap 106); GitHub's REST compare of `before_sha...after_sha` as it returns them, so the files and counts run from the merge base, not from `before_sha`, when the push diverged (section 7, "Compare"); no row when either SHA is all zeros (a branch creation or deletion); `http_status` records the response, so a refused compare is a row with the other columns null (a compare across a force push, `5110f3d...fccb1f5`, returned 200, section 7) | | `repository_activity_compare_files` | `activity_id`, `ordinal`, `filename`, `previous_filename`, `status`, `additions`, `deletions` | Step 4b; PK (`activity_id`, `ordinal`); compare's `files` entries as returned | | `repository_activity_compare_commits` | `activity_id`, `ordinal`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at` | Step 4b; PK (`activity_id`, `ordinal`); compare's `commits` list in full, each commit stored like a `pr_commits` row (D19), so a commit an event brought joins to its event; written once with its compare and never re-read, so its author link is the one GitHub returned then | | `default_branch_commits` | `repository_id`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at`, `committed_at`, `parent_count`, `additions`, `deletions`, `changed_files` | Step 4b; PK (`repository_id`, `sha`); the default branch's history since the floor as GitHub lists it (D30); the author link as in `pr_commits`, a replace keeping the stored link of the row with the same SHA when GitHub returns none (D28); `parent_count` is GitHub's `parents.totalCount`; `changed_files` is GitHub's `changedFilesIfAvailable`, null when GitHub returns none; no message (D19) | | `automation_accounts` | `github_id`, `login`, `reason`, `run_id` | integer primary key; `github_id` unique; replaced from `tracking.toml` at the start of every run, `run_id` naming that run (a reference to `sync_runs`) | | `account_profiles` | `account_id`, `company`, `location`, `website_url`, and an organization's `description`, `email`, `twitter_username`, `is_verified`, `archived_at`, `has_sponsors_listing`, and `pinned_items_remaining`, `refreshed_at` | PK `account_id`, a reference to `github_users` (`ON DELETE CASCADE`); rows only for a `User` or an `Organization` (D31); each lookup that returns the account replaces the whole row with the profile it returned, through the writer and in the transaction that writes `github_users`' login and type, so no earlier value survives a field GitHub now returns as null or a change of type; a lookup that returns `NOT_FOUND` leaves the row unchanged, marked by `github_users.not_found_at` (D28); every field nullable as GitHub's is, the organization fields null for a user and `company` null for an organization; `refreshed_at` as on `github_users` | +| `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting or a lookup, the one `github_users.node_id` holds included, with the first and latest runs that reported it, so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); consumer | | `automation_emails` | `email`, `reason`, `run_id` | integer primary key; `email` unique, stored lowercase ASCII (SQLite's `lower()`, which the views apply to commit emails, folds only ASCII letters); replaced the same way | -| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `deletions_approved` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, else `manual`; `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when its tracked files have changes; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted; `failures` counts this run's item failures, accepted ones included; `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | -| `repo_sync_state` | `repository_id`, `last_index_started_at`, `index_pr_total`, `index_issue_total`, `last_success_at`, `last_run_id`, `last_outcome`, `halt_reason`, and from Step 4b `activity_read_at`, `activity_last_outcome`, `activity_halt_reason`, `history_read_at`, and `default_branch_commits_total` | PK `repository_id`; `activity_read_at` is the GitHub `Date` header of the first page of the latest activity-log read that completed and was written; `history_read_at` and `default_branch_commits_total` are the first page's `Date` header and the `totalCount` of the latest history read that replaced the rows, written only with that replace; `activity_last_outcome` in (`complete`, `time_limited`, `failed`, `halted`); `activity_halt_reason` holds a Step 4b deletion halt's reason and candidate counts and is cleared by a run whose Step 4b did not halt (Step 4b); `last_run_id` is the latest run that attempted the repository, or found it untracked, and `last_outcome` its result, in (`complete`, `time_limited`, `failed`, `incomplete_index`, `halted`, `untracked`), both written by step 8, or by the close of a fatal or interrupted run (5.3 step 8); `last_index_started_at` is written only when that index completed and the repository did not fail later in the run, a time-limited repository included (its unreached items are in `sync_pending`), and never by the close of a fatal or interrupted run; `last_success_at` is the run's `finished_at` when `last_outcome` is `complete` and no item of the repository failed outside `[sync].accepted_failures`; `halt_reason` holds step 7's halt and is cleared by a run whose step 7 ran for the repository and did not halt, a run whose step 7 did not run for it leaving it | +| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `deletions_approved` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, else `manual`; `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when its tracked files have changes, and `-unknown` when a git call failed or timed out, gap 105; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted; `failures` counts this run's item failures, accepted ones included, and `refresh_failures` its failed refresh lookups (5.3 step 4); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | +| `repo_sync_state` | `repository_id`, `last_index_started_at`, `index_pr_total`, `index_issue_total`, `last_success_at`, `last_run_id`, `last_outcome`, `halt_reason`, and from Step 4b `activity_read_at`, `activity_first_event_at`, `activity_last_outcome`, `activity_halt_reason`, `history_read_at`, and `default_branch_commits_total` | PK `repository_id`; `activity_first_event_at` is the `timestamp` of the oldest event the latest complete activity read listed, so a consumer can tell a quiet period from a log that does not reach back (gap 106); `activity_read_at` is the GitHub `Date` header of the first page of the latest activity-log read that completed and was written; `history_read_at` and `default_branch_commits_total` are the first page's `Date` header and the `totalCount` of the latest history read that replaced the rows, written only with that replace; `activity_last_outcome` in (`complete`, `time_limited`, `failed`, `halted`); `activity_halt_reason` holds a Step 4b deletion halt's reason and candidate counts and is cleared by a run whose Step 4b did not halt (Step 4b); `last_run_id` is the latest run that attempted the repository, or found it untracked, and `last_outcome` its result, in (`complete`, `time_limited`, `failed`, `incomplete_index`, `halted`, `untracked`), both written by step 8, or by the close of a fatal or interrupted run (5.3 step 8); `last_index_started_at` is written only when that index completed and the repository did not fail later in the run, a time-limited repository included (its unreached items are in `sync_pending`), and never by the close of a fatal or interrupted run; `last_success_at` is the run's `finished_at` when `last_outcome` is `complete` and no item of the repository failed outside `[sync].accepted_failures`; `halt_reason` holds step 7's halt and is cleared by a run whose step 7 ran for the repository and did not halt, a run whose step 7 did not run for it leaving it | | `sync_pending` | `repository_id`, `item_kind`, `number`, `run_id` | PK (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); items a time-limited run planned but did not reach; step 8 replaces the rows of each completely indexed repository that did not fail later in the run (a repository gone during hydration keeps its rows, 5.3 step 4) with this run's unreached items, so a run that reached them all leaves none; consumer, so an in-coverage item not yet hydrated is told apart from one outside coverage | | `sync_failures` | `repository_id`, `item_kind`, `number`, `attempts`, `last_error`, `first_failed_at`, `last_failed_at`, `last_run_id` | unique (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); `attempts` counts the runs in which the item failed: `last_run_id` is the latest such run, and a failure adds one only when its run is another, whatever the runs' clocks read; `last_error` is the error's class and message; row deleted on success, or by step 7 when its repository's in-coverage set no longer lists the item | | `user_tags`, `pr_user_tags` | unchanged except `created_at`, which uses the 5.1 timestamp type with a `datetime.now(UTC)` default | FK to new `pull_requests` | @@ -372,6 +387,10 @@ Column rules: automation tables), as Step 3a's refuses tag rows, and its added columns need no value for existing rows. A column a later migration adds to tables that hold rows is null on rows hydrated at an older version until they re-hydrate, and `hydration_version` is a consumer column, so a reader can tell. + A table without `hydration_version` (`github_users`, `account_profiles`, the Step 4b compare + tables) to which a later migration adds a GitHub-derived column gains a per-row read version + that the planner compares the same way, or the migration refuses while the table holds rows, so + a column is never left unread on old rows (gap 110). - `state_reason` is GitHub's value including `DUPLICATE` (5.8). Descriptive enums, every GitHub enum column except a PR's, issue's, and review's `state` (`github_users.type` keeps its CHECK), such as `change_type`, `author_association`, `state_reason`, `merge_state_status`, and @@ -510,10 +529,11 @@ same number. author and `dependabot[bot]` as a commit's `GitActor.user`, whose declared type is `User`; REST uses `dependabot[bot]`. `copilot-pull-request-reviewer[bot]` and `Copilot` share id 175728472. Identity never joins on login. A sighting or lookup whose `databaseId` is stored under another node id - refers to the stored row, which keeps its node id, and the run reports the second one; one whose + refers to the stored row, which keeps its node id; the run reports the second one and stores it in + `account_node_ids`, which holds every node id GitHub reported for the account (gap 103). One whose node id is stored under another `databaseId` fails the item, or the lookup, as does an item whose - sightings and lookups report one node under two `databaseId` values, a second node id never - stored included. A commit link resolves by its `databaseId` alone. + sightings and lookups report one node under two `databaseId` values, a node id in + `account_node_ids` included. A commit link resolves by its `databaseId` alone. - Type and login come only from GraphQL: actor sightings (PR author, merger, review author, comment author, editor, requested reviewer, assignee), which carry `__typename`, and `nodes(ids:)` lookups. A commit-linked account (GraphQL `GitActor.user` or the REST compare @@ -545,7 +565,8 @@ same number. returned the account, the same clock as `hydrated_at`; a `NOT_FOUND` lookup sets `not_found_at` from its own `Date` header and leaves `refreshed_at`. The refresh runs after the last hydration write and before step 7, oldest `refreshed_at` first, and stops at the deadline; in it, a - `databaseId` that differs from the stored `github_id` is a failed refresh. Accounts that no stored + `databaseId` that differs from the stored `github_id` is a failed refresh, which makes the exit 2 + (5.3 step 4). Accounts that no stored row references any more stay stored and refreshed: within one file, `github_users` holds every account it ever stored (a bootstrap starts without them, section 9). - Automation lists: `[automation].accounts` and `[automation].emails` name accounts and commit @@ -620,7 +641,8 @@ an interrupt, which is re-raised. (the PR connection's when both are indexed) minus 60 s, because the header has one-second resolution and is set after the page is read. Creation order is stable, so an update during the scan cannot make pagination skip an item; a number listed twice or out of creation order - fails the repository. The enumerated count must equal the `totalCount` + fails the repository, as does a page whose repository `databaseId` differs from the resolved + `github_id` (gap 104). The enumerated count must equal the `totalCount` returned with the last page. A transfer or deletion during the scan breaks that equality, so on a mismatch the connection is rescanned once; a second mismatch leaves the repository with no item writes, in-place updates, or deletions this run (step 1's repository row and labels, read by the @@ -634,16 +656,17 @@ an interrupt, which is re-raised. longer has (step 1), its indexed links differ from its stored links (its link `totalCount` differs from `closing_issues_total`, an indexed link is not stored, or, when the index lists all of them, a stored link is not indexed; D23), the index lists fewer distinct links - than the smaller of its `totalCount` and 10 (a link listed twice), or it has a `sync_failures` - row, so a failed item is retried even when nothing else changed. A link change goes through - hydration, which reads the fields the index does not (`user_linked`, the issue's id). Update in - place without hydration, for items not planned for hydration in this run: `is_draft` and - `author_association` from the index (ready-for-review is not proven to change `updatedAt`, 0 of - 151 counterexamples; association changes do not change it); `repo_sync_state.last_index_started_at` - is their as-of time. The writer applies in-place updates before the first hydrated write; a hydrated - item takes every field from its hydration response, except that a null actor never replaces a - reference D28 keeps. Order the hydration queue: items in - `sync_pending` first, then open items, then by `updatedAt` descending, so a time-limited run + than the smaller of its `totalCount` and 10 (a link listed twice), its link `totalCount` exceeds + the 10 links the index carries, so a change beyond the 10th is never missed (gap 102), or it has + a `sync_failures` row, so a failed item is retried even when nothing else changed. A link change + goes through hydration, which reads the fields the index does not (`user_linked`, the issue's + id). Update in place without hydration, for items not planned for hydration in this run: + `is_draft` and `author_association` from the index (ready-for-review is not proven to change + `updatedAt`, 0 of 151 counterexamples; association changes do not change it); + `repo_sync_state.last_index_started_at` is their as-of time. The writer applies in-place updates + before the first hydrated write; a hydrated item takes every field from its hydration response, + except that a null actor never replaces a reference D28 keeps. Order the hydration queue: items + in `sync_pending` first, then open items, then by `updatedAt` descending, so a time-limited run refreshes the newest data first and the next run finishes what it left. The queue spans the attempted repositories: steps 1-3 run for all of them before any hydration, and each batch takes the first unbatched item and further unbatched items of the same repository and kind, in queue @@ -679,12 +702,12 @@ an interrupt, which is re-raised. |-------|-------------|--------| | Rate limited | Any HTTP 429 (GraphQL or REST); HTTP 403 with `retry-after`, `x-ratelimit-remaining: 0`, a secondary-limit message, or `errors[].type == "RATE_LIMITED"`; GraphQL HTTP 200 with `RATE_LIMITED` or an error message naming a secondary rate limit (GitHub documents 200 and 403 for GraphQL, 403 and 429 for REST) | Wait for `retry-after`; else, when `x-ratelimit-remaining` is 0 or the error is `RATE_LIMITED`, until `x-ratelimit-reset` or the last `resetAt`; else 60 s. Each further limit before a success waits at least 60 s doubled per repeat, whatever the headers say (GitHub: "If your request continues to fail due to a secondary rate limit, wait for an exponentially increasing amount of time between retries, and throw an error after a specific number of retries"). Never split; not an attempt. A wait that would end after the `--max-minutes` deadline stops queueing instead | | Timeout or resource limit | HTTP 502/504 (GitHub documents both for a request over 10 s); a GraphQL error at any path reporting a timeout or exceeded resource limits (GitHub returns "partial results along with an error indicating that resource limits were exceeded") | Split the batch in half at once; a same-size retry repeats the cost | - | Server or transport error | other 5xx; a GraphQL server fault at any path; connection error; client timeout; a connection the server broke off or a body it encoded wrongly; a response without data or errors, without a readable `Date`, or with an error path that is not strings and list indices (never trimmed, since an exact path is what makes an item gone) | Retry up to 3 more times (2, 4, 8 s, with jitter), then split | - | Item gone | alias `null` with `NOT_FOUND` | No write and no failure; the next index removes it | + | Server or transport error | other 5xx; a GraphQL server fault at any path; connection error; client timeout; a connection the server broke off or a body it encoded wrongly; a GraphQL `errors` that is present and not a list (gap 104); a response without data or errors, without a readable `Date`, or with an error path that is not strings and list indices (never trimmed, since an exact path is what makes an item gone) | Retry up to 3 more times (2, 4, 8 s, with jitter), then split | + | Item gone | alias `null` with `NOT_FOUND` | In a sync, an item error: the run's own index listed the number in coverage, so GitHub's two answers disagree (gap 100); an index that no longer lists the number removes the item and its failure (step 7). In verify's lookup by number, removed since sync (5.4) | | Item error | another alias error; partial alias data; a value the typed models reject; a failed check in step 5; HTTP 404/422 on a compare page | Retry the item once alone, then record it in `sync_failures` and write nothing for it | | Repository gone during hydration | a batch's `repository` is null (`RepositoryNotFound`, `RepositoryShapeError`; gap 79) | The repository fails for the rest of the run: its written items stay, step 7 skips it, and its `sync_pending` rows are unchanged | | Local database error | any error the database raises while the writer writes (a lock held past the busy timeout, an unexpected constraint failure; gap 77) | Fatal: roll back the item, close the run, exit 1 | - | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | + | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; a REST response without readable `x-ratelimit-remaining` and `x-ratelimit-reset` (gap 104); another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | | Unauthorized or forbidden | HTTP 401; a failed credential request (an App token mint answering 4xx without a rate-limit signal); HTTP 403 with no rate-limit signal (a missing permission); an HTTP client failure outside the network (a redirect loop, a proxy or URL problem, a missing auth dependency) | Fatal: close the run, exit 1 | Typed response models check types strictly: GitHub's JSON has exact types, and lax coercion @@ -696,9 +719,11 @@ an interrupt, which is re-raised. request; every other item error gets its solo retry. A resolve or index page that still fails fails its repository for this run. A compare page that still fails is an item error. A user lookup failure is scoped to what it covers, one node or one request of 100 ids, and is an - item error for each item waiting on those accounts (5.2); a failed 24-hour - refresh keeps the stored values, is logged, and is retried next run without changing the exit - code. Fields that drive logic (`state`, review `state`) are closed enums in the typed models; + item error for each item waiting on those accounts (5.2). A failed 24-hour refresh (a lookup that + still fails after its retries, an answer the typed models or the id check reject, or a + `databaseId` other than the stored `github_id`) keeps the stored values (D28), is listed in the + report, counted in `sync_runs.refresh_failures`, retried next run, and makes the exit 2 (D37, + gap 98). Fields that drive logic (`state`, review `state`) are closed enums in the typed models; descriptive enums are text (5.1). Any other exception after the claim is fatal too. A fatal run closes as `failed` (exit 1), with @@ -721,7 +746,8 @@ an interrupt, which is re-raised. issues, user-linked closing issues, review requests, and assignees; every user-linked closing issue is also in the full closing list; every overflow page reports the first page's `totalCount`; a null connection or null node is a failure, never a count of zero; the compare `total_commits` equals the PR's commit - `totalCount`, and the fetched compare commits equal `total_commits`; the returned number equals + `totalCount`, the fetched compare commits equal `total_commits`, and every commit on the commit + connection's first page is among them (gap 104); the returned number equals the request (the batch names the resolved repository); `state` is `MERGED` exactly when `mergedAt` is set, and a `CLOSED` or `MERGED` PR has `closedAt`, which the coverage rule reads; the entries the database keys (labels, closing issues, user-linked closing issues, reviews, @@ -731,8 +757,9 @@ an interrupt, which is re-raised. solo retry, since no retry of the transport preceded it. The hydrated PR's `github_id` equals the one the index listed for its number, and a review or comment whose `github_id` is stored under another PR fails the item. A PR that closed without merge between index and hydration is written - with state `CLOSED` (D3); an item that left coverage between index and hydration is skipped - without a failure, before any account lookup for it, and the next index removes it. + with state `CLOSED` (D3). A hydration that contradicts the run's own index is an item error, + before any account lookup for it: an item outside coverage, or an `updatedAt` earlier than the + index entry's (gap 100). 6. Write: one batch at a time, on the writer's one connection. The run hydrates a batch, checks it (step 5), looks up its accounts (5.2), and writes each of its items before it requests the next batch; at the deadline it writes the fetched items whose accounts it has and records the rest in @@ -785,16 +812,19 @@ an interrupt, which is re-raised. 8. Record: update `repo_sync_state` and close the `sync_runs` row. Exit 0 when every repository indexed completely and nothing failed; 2 when an item failed in this run (other than one listed in `[sync].accepted_failures`), a repository failed, had an incomplete index, or halted, a stored - repository is untracked in a run without `--repo`, or a prune was refused (step 7); 1 on a fatal - error. The exit code is decided after the prune. An accepted failure is still recorded and - reported; it names the repository, number, and reason in `tracking.toml`, so a permanent failure - does not turn every run red. An entry is reported as not having failed when its repository's - outcome in this run is `complete`, `time_limited`, or `halted` and its item neither failed in - this run nor is pending (5.5). `--max-minutes` stops queueing new hydration, records in - `sync_pending` the planned items it did not reach (a repository that failed during hydration - keeps its rows, step 4), still runs step 7 for complete repositories and step 8, and sets status - `time_limited`; unreached items are not failures. Step 8 writes `last_run_id` and `last_outcome` - for every repository the run attempted. `--dry-run` takes no lock, inserts no `sync_runs` row, + repository is untracked in a run without `--repo`, a prune was refused (step 7), a refresh + failed (step 4), or the deadline left planned items unreached (gap 99); 1 on a fatal error. The + exit code is decided after the prune. An accepted failure is still recorded and reported; it + names the repository, number, error class, and reason in `tracking.toml`, so a permanent failure + does not turn every run red, and a failure of another class is not accepted (gap 101). An entry + is reported as not having failed when its repository's outcome in this run is `complete`, + `time_limited`, or `halted` and its item neither failed in this run nor is pending (5.5). + `--max-minutes` stops queueing new hydration, records in `sync_pending` the planned items it did + not reach (a repository that failed during hydration keeps its rows, step 4), still runs step 7 + for complete repositories and step 8, and sets status `time_limited`; unreached items are not + item failures, but they make the exit 2 (D37). Step 8 writes `last_run_id` and `last_outcome` for + every repository the run attempted. A close that is given an outcome for a repository it finds no + stored row for is fatal (exit 1, gap 105). `--dry-run` takes no lock, inserts no `sync_runs` row, and writes nothing: it runs steps 1-3 as the run would and reports step 1's updates, the planned hydrations with their reasons, the in-place updates, and the delete candidates. It exits as the run would for every step-8 condition steps 1-3 can show (a failed or incompletely indexed @@ -967,12 +997,15 @@ strictly typed throughout: an account `github_id` is at least 1, and `login`, `e Python `timedelta` holds, so the planner can build the window), `delete_guard_min_rows` an integer of at least 0, and `delete_guard_percent` a number above 0 and at most 100; an `accepted_failures` entry has a -`number` of at least 1, is unique by repository and number, must name a configured repository, and -matches by that entry and the number, so a rename does not unhook it; an entry whose item did not -fail is reported (5.3 step 8). The file holds domain policy only. Step 1 added `[[repositories]]`; -Step 3b-2a-1 adds `[coverage]` and `[sync]`'s `unsettled_minutes`, `delete_guard_min_rows`, and -`delete_guard_percent`, which the planner reads; Step 3b-2a-2a adds `[automation]` and -`[sync].accepted_failures`, which the run claim and the exit code read. +`number` of at least 1, an `error` naming the error class it accepts (the class +`sync_failures.last_error` starts with), is unique by repository and number, must name a configured +repository, and matches by that entry, the number, and the class, so a rename does not unhook it and +a new cause is not accepted (gap 101); an entry whose item did not fail is reported (5.3 step 8). +`[automation]` is required, its lists may be empty, so a file without it is refused instead of +emptying the automation tables (gap 105). The file holds domain policy only. Step 1 added +`[[repositories]]`; Step 3b-2a-1 adds `[coverage]` and `[sync]`'s `unsettled_minutes`, +`delete_guard_min_rows`, and `delete_guard_percent`, which the planner reads; Step 3b-2a-2a adds +`[automation]` and `[sync].accepted_failures`, which the run claim and the exit code read. ```toml [coverage] @@ -1000,7 +1033,7 @@ delete_guard_min_rows = 25 delete_guard_percent = 2.0 # Items whose failure is understood and does not count toward exit 2 (5.3 step 8). accepted_failures = [ - # { repository = "prebid/Prebid.js", number = 12345, reason = "..." }, + # { repository = "prebid/Prebid.js", number = 12345, error = "ItemConflict", reason = "..." }, ] ``` @@ -1026,7 +1059,9 @@ Workflow `.github/workflows/sync.yml`: - `sync` (`permissions: contents: read`; `timeout-minutes: 355`, under GitHub's 6-hour job limit) holds the App credentials. Without `bootstrap`, it downloads the latest snapshot and runs `verify --db-only`; a missing or failing snapshot halts the run. With `bootstrap`, it downloads - nothing and starts from an empty file, and the manifest records `bootstrap: true`. It then runs + nothing and starts from an empty file, and the manifest records `bootstrap: true`. One job-level + `DATABASE_URL` names the file every step uses; without `bootstrap` the job fails unless that file + exists and its SHA-256 equals the one `verify --db-only` checked (gap 107). It then runs `alembic upgrade head`, asserts the revision is head, runs `ghactivity sync --max-minutes 300` and `ghactivity verify --max-minutes 40 --report verify.json`, runs `ghactivity snapshot --verify-report verify.json`, and hands the snapshot and manifest to `publish` as a workflow @@ -1044,7 +1079,9 @@ Workflow `.github/workflows/sync.yml`: Values that `sync` produces (job outputs, artifact names, manifest fields) are untrusted in the other jobs: they are used only in `if:` expressions, or through `env:` after validation as an - integer or a 64-character hex digest, and never interpolated into a `run:` script. + integer or a 64-character hex digest, and never interpolated into a `run:` script. An `if:` + compares an exit-code output as a quoted string (`== '0'`) and requires it to be non-empty, since + GitHub's expressions read an empty string as 0 (gap 107). - Authentication (D10): `ghactivity sync` and `ghactivity verify` authenticate in-process with githubkit's `AppInstallationAuthStrategy` (app ID, private key, and installation ID from secrets; contents, pull requests, issues, and metadata read), which mints a new installation token before @@ -1054,8 +1091,8 @@ Workflow `.github/workflows/sync.yml`: dependencies (Step 6a). The App key is readable by the sync job's dependencies, so the App stays read-only and its key rotatable. The workflow's own `GITHUB_TOKEN` (`contents: write`) is used only in `publish`. -- Snapshots: `ghactivity snapshot --out DIR [--run-id ID] [--verify-report PATH]` writes the copy - with `VACUUM INTO`, checks its header for rollback-journal mode (the check +- Snapshots: `ghactivity snapshot --out DIR [--run-id ID] [--sync-run ID] [--verify-report PATH]` + writes the copy with `VACUUM INTO`, checks its header for rollback-journal mode (the check `scripts/check_db_journal.py` makes), gzips it, and writes the manifest, in the workflow and for 6b's first snapshot (`run_id` null). Names are `github_activity-.db.gz` and `github_activity-.manifest.json`, with `T` as `YYYYMMDDTHHMMSSZ`, because GitHub renames asset @@ -1068,7 +1105,12 @@ Workflow `.github/workflows/sync.yml`: exit code, counts per class, and unverified repositories (from `--verify-report`; null without it), and the SHA-256. `snapshot` exits 1 when the report `--verify-report` names is missing or unreadable, or names a `last_run_id` other than the file's `repo_sync_state`, so a verify that - never started, which leaves no report or one from an earlier sync (5.4), fails the job. + never started, which leaves no report or one from an earlier sync (5.4), fails the job. In the + workflow, `--sync-run` is the `sync_runs` id of the sync this job ran; `snapshot` exits 1 unless + that row is newer than every id in the downloaded file, is closed with the exit code the job + recorded, and is every attempted repository's `last_run_id`, so a job whose sync did not run + cannot republish the old file (gap 107). The manifest records that id and the run's + `finished_at`. - Uploads never use `--clobber`. Before each upload, `publish` deletes a same-name asset in any state other than `uploaded`. A same-name `uploaded` asset whose `digest` is `sha256:` followed by the expected SHA-256 (or, when `digest` is null, whose downloaded SHA-256 matches) is skipped, so a @@ -1094,8 +1136,11 @@ Workflow `.github/workflows/sync.yml`: downloads the `.db.gz`, checks its SHA-256 against the manifest, decompresses it once, and replaces its local copy by writing a temporary file and renaming it over the old one, so no query reads a half-written file. It does this once per new snapshot (about every 6 hours, D11), not per - read, and queries its copy with SQLite. The download runs outside a browser, on a server or in a - build step: the asset host (`release-assets.githubusercontent.com`) answered without + read, and queries its copy with SQLite. It treats a pair whose manifest `finished_at` is older + than twice D11's interval (12 hours) as stale and says so wherever it shows the data, with the + manifest's sync exit code and verify counts beside it: a stopped schedule runs nothing, so only a + check outside the workflow can see it (gap 107). The download runs outside a browser, on a server + or in a build step: the asset host (`release-assets.githubusercontent.com`) answered without `access-control-allow-origin`, so a page on another origin is not expected to read the file (inferred from the response headers, no browser was run; section 7, "Release downloads"). How a page reads its copy is the frontend's design, outside this plan. @@ -1171,11 +1216,14 @@ query RepositoryLabels($id: ID!, $after: String!) { ``` PR index (cost 1 point per 100 items, measured with and without the closing-link selection, and -with `closedAt` by `rateLimit(dryRun: true)`, 2026-10-18; `closedAt` is new in Step 3b-1): +with `closedAt` by `rateLimit(dryRun: true)`, 2026-10-18; `closedAt` is new in Step 3b-1; the +repository's `databaseId`, which Step 3b-2a-3 adds for 5.3 step 2's check, keeps it at 1 point, +dry run on Prebid.js, 2026-10-21): ```graphql query($owner: String!, $name: String!, $after: String) { repository(owner: $owner, name: $name) { + databaseId pullRequests(first: 100, after: $after, orderBy: {field: CREATED_AT, direction: ASC}) { totalCount pageInfo { hasNextPage endCursor } @@ -1321,8 +1369,8 @@ nested node. ## 6. Execution steps Each step is one pull request from a `feature/` branch, except Step 3 (pull requests 3a, 3b-1, -3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2b-1, and 3b-2b-2), Step 4 (pull requests -4a and 4b), +3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2), Step 4 (pull +requests 4a and 4b), and Step 6 (pull requests 6a and 6c, with the 6b manual runs between them). A step, or a pull request within one, starts after the previous one merges. @@ -1406,16 +1454,17 @@ repository) and runs on the exact PR head; CI passes on the exact PR head. ### Step 3: New schema, PR sync engine, cutover -Nine pull requests. 3a lands the schema and removes the old pipeline, which imports the old models +Ten pull requests. 3a lands the schema and removes the old pipeline, which imports the old models and so cannot outlive them; 3b-1 adds what the sync collects under D29 (the query documents, typed records, schema, and row-level views of D3-D6, D8, D14, and D28); 3b-2a-1 adds the planner and the configuration it reads; 3b-2a-2a adds the writer and the run records on that schema, so the writer is built against the final tables; 3b-2a-2b-1 adds the scenario model the engine's tests run against; 3b-2a-2b-2 adds the sync engine over the planner and the writer, `--dry-run` -included; 3b-2a-2c adds `ghactivity sync`, the seeded runs, and the live acceptance; 3b-2b-1 adds +included; 3b-2a-2c adds `ghactivity sync`, the seeded runs, and the live acceptance; 3b-2a-3 +applies D37 to the sync (D38); 3b-2b-1 adds the REST sampler that checks what the sync wrote; and 3b-2b-2 adds verify, whose live acceptance runs the sampler. "Step 3b-2" elsewhere in this plan means 3b-2a-1, the parts of 3b-2a-2, and the -two parts of 3b-2b together; "Step 3b-2a" means 3b-2a-1 and 3b-2a-2; "Step 3b-2a-2" means +two parts of 3b-2b together; "Step 3b-2a" means 3b-2a-1, 3b-2a-2, and 3b-2a-3; "Step 3b-2a-2" means 3b-2a-2a, the two parts of 3b-2a-2b, and 3b-2a-2c; "Step 3b-2a-2b" means 3b-2a-2b-1 and 3b-2a-2b-2; and "Step 3b-2b" means 3b-2b-1 and 3b-2b-2. The REST sampler and its recorder mode are 3b-2b-1's; verify (5.4), its timeline lookup, and the scenario model's timeline responses are @@ -1788,10 +1837,54 @@ syncs, the pull request records that PR's `updatedAt` and `mergeStateStatus` fro 8). An adversarial review of the change, by reviewers who did not write it, finds no invariant check without a test that targets it; CI passes on the exact PR head. -#### Step 3b-2b-1: REST sampler +#### Step 3b-2a-3: D37 in the sync Prerequisite: Step 3b-2a-2 has merged. +Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh_failures` and the +`account_node_ids` table (5.1): +- a failed 24-hour refresh is counted, listed in the report, and makes the exit 2 (5.3 step 4, + gap 98); +- a run whose deadline leaves planned items unreached exits 2 (5.3 step 8, gap 99); +- a hydration that contradicts the run's own index (an alias `NOT_FOUND`, an item outside coverage, + an `updatedAt` earlier than the index entry's) is an item error (5.3 steps 4 and 5, gap 100); +- an `accepted_failures` entry names the error class it accepts (5.5, gap 101); +- a PR whose link `totalCount` exceeds the 10 links the index carries is hydrated every run (5.3 + step 3, gap 102); +- every node id GitHub reports for a stored account is stored in `account_node_ids` (5.2, gap 103); +- the source checks of gap 104: a REST response without readable rate-limit headers is not + understood, a GraphQL `errors` present and not a list is a server error, a compare above 250 + commits holds every commit on the commit connection's first page, and every index page selects + the repository's `databaseId` and is checked against the resolved `github_id` (the index units are + re-recorded; no `hydration_version` changes); +- the run-record and setup checks of gap 105: `code_version` ends in `-unknown` when a git call + failed or timed out, a close given an outcome it cannot write is fatal, `[automation]` is + required, the connect listener reads `foreign_keys` back and refuses a connection where it is + off, `alembic/env.py` refuses `stamp`, the wheel check compares the packaged `tracking.toml` + bytes with the source file's, and the log-file handler receives every record at its level; +- the data dictionary describes the head-repository, milestone, and team names as of + `hydrated_at` (gap 108). + +Tests: each rule with planted inputs on both sides: a refresh request that fails and one that +succeeds, and a refresh answer with another `databaseId`; a deadline before and after the last +planned item, and inside a batch whose items failed; a `NOT_FOUND` alias for a number the index +listed, then a later run whose index omits the number and whose step 7 removes the item and its +failure; a hydration outside coverage; a hydrated `updatedAt` earlier than, equal to, and later +than the index entry's; an accepted entry whose class matches and one whose class differs; PRs with +10 and with 11 links; a second node id stored and sighted again, and a node id in +`account_node_ids` reported under another `databaseId` failing the item; each source check accepting +the valid shape and refusing the planted one; each run-record and setup check. Each is a regression +test that fails against the parent's code, and the tests that pinned the old behaviour (a failed +refresh and a time-limited run exiting 0, a gone item writing nothing, a PR with 11 links not +planned) change with the rule they pinned. + +Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule of +this step without a test that targets it; CI passes on the exact PR head. + +#### Step 3b-2b-1: REST sampler + +Prerequisite: Step 3b-2a-3 has merged. + Scope: tests, `scripts/`, and `docs/testing.md`; `Transport.rest_get` already takes any path and query, so nothing under `src/` changes. The REST sampler 3b-2b-2's acceptance uses, laid out like the recorder: comparison logic in a @@ -1945,15 +2038,22 @@ Scope (D30): the repository activity reader, its compares, and the default-branc (`activity_last_outcome`: `complete`, `time_limited`, `failed`, or `halted`; `activity_halt_reason`; 5.1); the planner and writer steps, which run after 5.3 step 6's writes and before step 7. Each run reads the -activity log since the floor in full and upserts its events by `github_id`, keeping each stored +activity log since the floor in full, following GitHub's `Link` cursors, which the source then +returns with a REST page; a read is complete only when a page has no next link or its oldest event +is before the floor, and any other end fails the repository's activity for the run (exit 2, +gap 106). It upserts the events by `github_id`, records the oldest one's `timestamp` in +`activity_first_event_at`, keeping each stored event's compare, then deletes the stored events the log no longer lists (D7) unless they exceed max(25, 2%) of the repository's stored events, which halts that deletion as step 7 halts deletions. It compares each event not yet compared whose `before` and `after` are both non-zero, -storing compare's commits in full (counted against `total_commits`) and its files; a compare GitHub -refuses (HTTP 404 or 422) is stored with its `http_status` and not retried. It reads the default -branch's history since the floor in full and replaces that repository's `default_branch_commits` -rows only when the read count equals `totalCount` and the commits it would remove are within the -same guard, so a commit a force push removed leaves as GitHub no longer lists it. A log or history +storing compare's commits in full (counted against `total_commits`) and its files, with +`files_at_cap` set when compare returned 300 files, GitHub's documented cap for one comparison +(gap 106); a compare GitHub refuses (HTTP 404 or 422) is stored with its `http_status` and requested +again on each later run until GitHub answers it. It reads the default branch's history since the +floor in full and replaces that repository's `default_branch_commits` rows only when the read count +equals `totalCount` and the commits it would remove are within the same guard, so a commit a force +push removed leaves as GitHub no longer lists it; a count that differs is rescanned once, and a +second mismatch fails the repository's activity for the run (exit 2, gap 106). A log or history page, or a compare, that still fails under 5.3 step 4's transient classes makes the repository's activity `failed` for the run: it writes none of that read, and the run exits 2. The deadline stops the readers as it stops hydration, a stopped read writes nothing (`time_limited`), and a halt is @@ -1969,16 +2069,19 @@ fresh read no longer lists, and error otherwise. The data dictionary for the new recordings from public repositories. Tests: an event of each `activity_type` the recordings hold; an event with an all-zero SHA and no -compare row; a refused compare stored with its status and not retried; a second run sending no -compare for an event already compared; an event deletion past the guard halting; a diverged push whose compare runs from the -merge base; a push of several commits whose compare commits join each commit to its event; a force -push removing commits from `default_branch_commits`; a short history read leaving the stored rows -unchanged; a history read past the guard halting; a null actor; an event GitHub stops listing -removed on the next run; an event the log stops listing after the stored read classified as -removed since sync; `activity_halt_reason` set by a halt and cleared by the next run that does not -halt, `halt_reason` unchanged; `--allow-deletions` applying N to events and history commits -separately; a history replace keeping a commit's stored author link when GitHub returns none (D28); -each verify check and each stored-row check, by a planted row deletion. +compare row; a refused compare stored with its status and requested again on the next run; a +compare of exactly 300 files marked `files_at_cap` and one of 299 not; an activity read that ends +without reaching the floor or the log's end failing the activity; a history count mismatch rescanned +once and, on a second mismatch, failing the activity; a second run sending no compare for an event +already compared; an event deletion past the guard halting; a diverged push whose compare runs from +the merge base; a push of several commits whose compare commits join each commit to its event; a +force push removing commits from `default_branch_commits`; a history read past the guard halting; a +null actor; an event GitHub stops listing removed on the next run; an event the log stops listing +after the stored read classified as removed since sync; `activity_halt_reason` set by a halt and +cleared by the next run that does not halt, `halt_reason` unchanged; `--allow-deletions` applying N +to events and history commits separately; a history replace keeping a commit's stored author link +when GitHub returns none (D28); each verify check and each stored-row check, by a planted row +deletion. Acceptance: on the Step 3b-2 acceptance repositories and on `prebid/github-activity-db`, the stored events equal the log and the stored commits equal the history; each repository's first logged event @@ -1999,7 +2102,10 @@ failures, each listed in the PR with REST evidence of why it cannot pass; a 50-i matches REST; a difference report against the tracked file classifies every differing item (closed-unmerged PR, file-status rebuild, commit list beyond 250, login rename, changed on GitHub after the old file's sync, in-rule item absent from the old file, schema-only) with per-category counts, zero -unclassified items, and five REST-checked examples per category; the run's time, cost, file size, and +unclassified items, and five REST-checked examples per category; the REST sample measures whether a +body edit, a base-ref change, and a review dismissal change `updatedAt`, and one that does not gets +an index-side check or a periodic re-hydration before Step 6 (gap 109); the run's time, cost, file +size, and secondary-limit responses at one request in flight are recorded (and at any wider setting proposed). ### Step 6: Scheduled sync and publication @@ -2018,19 +2124,27 @@ unverified repositories (5.6); through `httpx.MockTransport`, a mocked installation-token response whose `expires_at` lies just over a minute ahead, a clock moved past it mid-run, and a second token request before the next API request (githubkit keeps a token until one minute before `expires_at`); and a 401 on an API request still -exiting 1 with the run closed (5.3 step 4). 6b creates release `db` and uploads the Step 5 database +exiting 1 with the run closed (5.3 step 4); `snapshot --sync-run` refusing an id that is not newer +than the downloaded file's, and the job failing when its database file is missing or differs from +the one `verify --db-only` checked (5.6). 6b creates release `db` and uploads the Step 5 database with its manifest (`ghactivity snapshot --verify-report` from Step 5's `verify --exact --report`) as the first snapshot. It creates release `db-test` the same way, holding snapshots prepared offline: the corrupted cases below, one with a `pr_files` row deleted (a verify error), one built at the revision before the last migration, and one holding an extra repository absent from `tracking.toml` -(a sync exit 2). It then runs one normal manual run and one bootstrap run. 6c enables the +(a sync exit 2). It then runs one normal manual run and one bootstrap run, and one run whose +verify exit-code output is unset, which publishes nothing and marks the workflow failed. Before 6c, +6b hydrates with the App the recorded sample the maintainer's token read (team review requests, +organization assignees, every resolve field, and each repository's index totals) and compares the +two column by column; a null, false, or zero the App returns where the token read a value blocks 6c +until the permission is granted or the field is dropped (gap 107). 6c enables the schedule and stops tracking `github_activity.db`: it removes the file from the index, the `!/github_activity.db` exception from `.gitignore`, the `check-added-large-files` exclude, the `tracked-db-rollback-journal` and `tracked-db-no-sidecar` hooks, the D26 freeze check, and `alembic/env.py`'s refusal of the repository-root `github_activity.db` and of any file of that name (gap 73); replaces the tracked-file journal test with one on the snapshot `VACUUM INTO` writes; records the D20 WAL decision for the -working file; and documents how to download the latest snapshot. +working file; and documents how to download the latest snapshot and the staleness check 5.6 +gives consumers. Acceptance: two consecutive scheduled runs publish, and their manifests list drift, pending, and failure counts; the bootstrap run publishes, and the next scheduled run starts from its snapshot; a @@ -2187,19 +2301,19 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in |------|------------------| | Intermittent GraphQL failure rate at full scale (prototype needed 0 retries) | Step 5 measures it | | Secondary-limit responses with more than one request in flight | Step 5 measures before any wider setting | -| Whether a heartbeat commit counts as repository activity for schedule disabling (undocumented); a disabled schedule fails no run | Not provable inside Step 6; the heartbeat is best effort; a consumer-side check of manifest age is later work (section 9) | +| Whether a heartbeat commit counts as repository activity for schedule disabling (undocumented); a disabled schedule fails no run | Not provable inside Step 6; the heartbeat is best effort, and the consumer's staleness check (5.6, gap 107) shows a schedule that stopped | | Issues read permission for the App | Step 6b's first manual run exercises it with a real token | | GraphQL `rateLimit` readings alternating between two windows (seen with a personal token; GitHub documents that values vary by the region handling a request, but not differing `resetAt` values) | Step 2's guard handles it; Step 6 records App-token readings | -| Whether body edits, base-ref changes, review dismissals, and issue label events always change `updatedAt` (not measured; a state change is caught by the index) | A change that does not is missed until the item's next update or a `hydration_version` bump; accepted | +| Whether body edits, base-ref changes, review dismissals, and issue label events always change `updatedAt` (not measured; a state change is caught by the index) | Step 5's REST sample measures it; a change that does not change `updatedAt` gets an index-side check or a periodic re-hydration before Step 6 (gap 109) | | The error body of a GraphQL secondary limit returned as HTTP 200 (not observed; documented only as "an error message") | Step 2 classifies by type and message; a message it does not recognize is "not understood" (5.3 step 4) and stops the run with exit 1, naming the message; Step 5 records secondary-limit responses | | A PR file list above 3,000 entries (no tracked PR found) | Validation fails the item if `files` stops short of `totalCount`; Step 5 reports any | | Compare fallback on a PR whose head fork was deleted (only a live fork tested) | A count mismatch fails the item; Step 5 reports any | | Full bootstrap time, cost, and size are projections from two repositories | Step 5 measures them | | Overflow continuations per request (25), the server retry delays (2, 4, 8 s), the reset margin (1 s), retry jitter (25%), the index boundary margin (60 s), the budget reserves (100 points, 100 calls), the longest accepted rate-limit wait (65 min), and the HTTP timeouts (read 30 s, connect 10 s) are chosen, not measured | Step 5 records secondary-limit responses, budget readings, and request times at these values | | The cost of one commit per item in rollback-journal mode (not separated; whole-run times on three repositories are in section 7, "Step 3b-2a-2c acceptance") | Step 5 measures the full run | -| Whether the activity log's 2023-03-07 start is fixed or a rolling window (seen on two repositories) | Step 4b records each repository's first logged event; it reads the whole log every run, so an event that ages out leaves the database as GitHub stops listing it (D7) | +| Whether the activity log's 2023-03-07 start is fixed or a rolling window (seen on two repositories) | Step 4b stores each repository's first logged event in `activity_first_event_at`; it reads the whole log every run, so an event that ages out leaves the database as GitHub stops listing it (D7) | | Whether `history(since:)` filters on the committed or the authored date (only its `totalCount` was read) | Step 4b's recordings settle it before the coverage rule's wording for commits is final | -| Whether REST compare caps the `files` it returns for a large push (not measured) | Step 4b measures it on its largest recorded push and stores a count only where GitHub reports one | +| Whether REST compare caps the `files` it returns for a large push (GitHub documents a cap of 300 files per comparison; not measured live) | Step 4b marks a list of 300 `files_at_cap` (gap 106) and records its largest push's file count | | The REST cost of comparing every activity event on a first sync, and of reading every log in full each run (about one call per event compared, and one per 100 events read; Prebid.js's default branch has 4,560 commits since the floor) | Step 4b records it; Step 5 measures the full run | | How a rebase merge appears in the activity log: whether its `pr_merge` event's `after` is the PR's `mergeCommit.oid` and its other commits come through the compare (the measured repositories hold merge-commit and squash merges only) | Step 4b records one on a tracked repository that allows rebase merges, before D30's join wording is final | | Whether GitHub returns `EnterpriseUserAccount.id` to a token without the `read:enterprise` scope (the dry run validated the document; no tracked repository has such an actor yet) | Step 3b-1 tests the synthetic actor. A live occurrence decides between two outcomes (a prediction): an error at the alias's path is an item failure (exit 2), and an error without a path is fatal (5.3 step 4, exit 1), which Step 5's bootstrap would surface | @@ -2210,15 +2324,15 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Whether to collect a `Mannequin`'s `email` ("The mannequin's email on the source instance"; the maintainer's profile decision named users and organizations only, and no mannequin was available to read, so whether GitHub requires a scope for it is unmeasured) | The maintainer decides; a live read on a mannequin comes first | | Whether GitHub returns an `EnterpriseUserAccount`'s `enterprise` to the tokens used (an `Enterprise` object; no tracked repository has such an actor) | Not selected until a live occurrence is read | | Whether GitHub returns an `EnterpriseTeam` requested reviewer's `id`, `fullDatabaseId`, and `slug` to the tokens used (the batch with them dry-runs at 2 points; no tracked repository has such a request) | Step 3b-1 tests a synthetic one; a live occurrence decides, as for `EnterpriseUserAccount.id`: an error at the alias's path fails the item (exit 2) | -| Whether the scheduled App token reads a `Team` requested reviewer and an `Organization` assignee as the token used here does (null or FORBIDDEN not measured) | Step 6b's first manual run; a FORBIDDEN fails the item, which 5.3 step 4 classifies | +| Whether the scheduled App token reads a `Team` requested reviewer and an `Organization` assignee as the token used here does (null or FORBIDDEN not measured) | Step 6b's first manual run; a FORBIDDEN fails the item, which 5.3 step 4 classifies, and a null where the token read a value blocks 6c (Step 6, gap 107) | | `--help` anywhere on a command line, or `--version` before the subcommand, exits 0 without doing any work (click's eager options), the code D25 publishes on, so a workflow input that expands to either would publish the unchanged database | Step 6's workflow passes inputs as validated option values, never as raw arguments | ## 9. Later work Dashboard integration and replacement of its data layer (it reads the snapshot as 5.6 "Consumer access" describes), a query API, AI enrichment (D12), -company affiliation, CI check status, releases, additional repositories, and a consumer-side -check that the latest manifest is recent (a disabled schedule fails no run). File classification +company affiliation, CI check status, releases, and additional repositories (the consumer's +staleness check moved to 5.6 and Step 6, gap 107). File classification and dependency tagging are not planned; excluding paths from line counts is the consumer's: `pr_files` keeps every path with its counts (D29). From d77f91969a7a77e1c21b060fe492af9880abc7ce Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 15:59:38 -0600 Subject: [PATCH 06/15] docs(plan): close the gaps the review of D38 found A hydration that contradicts the run's index gets one engine retry, then fails. Any deadline makes the exit 2, and the failures a batch made final before it are recorded. The migration refuses a populated file. A close that cannot write an outcome raises before writing. code_version names which git answer it trusts. Commit-link node ids are stored; accepted failures match the class exactly and can name an account. verify --db-only writes and digests the working file, snapshot checks the job's own sync through --after-run, and 6b compares App and token reads before any App run publishes. The stopped-schedule case is stated as visible only to a consumer. --- CLAUDE.md | 10 +- docs/redesign-plan.md | 366 +++++++++++++++++++++++++----------------- 2 files changed, 221 insertions(+), 155 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 117c8fe..f9649fb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -62,11 +62,13 @@ uv run pytest # CI also runs Python 3.12, `alembic check`, - `ghactivity sync` (Step 3b-2a-2c, `cli/sync.py`): the engine's run with the options of plan 5.7; a run that never starts exits 1, and a run that starts exits with the engine's 0, 2, or 1 -**Not implemented yet:** the REST sampler (Step 3b-2b-1) and `verify` (Step 3b-2b-2). The -tracked `github_activity.db` stays frozen at the first schema (D26). +**Not implemented yet:** D37 in the sync (Step 3b-2a-3: no silent failures, D38), the REST +sampler (Step 3b-2b-1), and `verify` (Step 3b-2b-2). The tracked `github_activity.db` stays +frozen at the first schema (D26). -**Planned in the redesign plan:** the REST sampler (Step 3b-2b-1) and verify (Step 3b-2b-2); -GitHub Issues and closing-issue links (Step 4); scheduled sync and release publication (Step 6). +**Planned in the redesign plan:** D37 in the sync (Step 3b-2a-3), the REST sampler (Step +3b-2b-1), and verify (Step 3b-2b-2); GitHub Issues and closing-issue links (Step 4); scheduled +sync and release publication (Step 6). **Out of Scope (Future):** - Webhooks / real-time sync diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index bf51efd..faa5c1a 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -12,8 +12,10 @@ with the installed versions (section 7); Step 3a's review probes ran on the PR # `6fedb05`). On 2026-10-18 the plan and `main` at `6fedb05` were audited against D29, and the collection D29 calls for was measured against live GitHub (section 7). On 2026-10-21 Step 3b-2b was planned against `main` at `2a85eb9`, and the timeline lookup's `since` and `filteredCount` were -measured against live GitHub (section 7). Counts that drift (PR and -issue totals) are dated; regenerate them with `ghactivity verify` once Step 3b-2b lands. +measured against live GitHub (section 7). The same day the plan and `main` at `2a85eb9` were +audited for silent failures and silently wrong data under D37, and D38 records the decisions +(gaps 98-110). Counts that drift (PR and issue totals) are dated; regenerate them with +`ghactivity verify` once Step 3b-2b lands. ## Contents @@ -90,7 +92,7 @@ Within scope the database collects and distinguishes data and never interprets i | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, and a repository error during a timeline lookup, makes a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | -| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when a time-limited run leaves planned items unreached, and when a hydration contradicts the run's own index (Step 3b-2a-3). An `accepted_failures` entry accepts one error class. A PR with more closing links than the index carries is hydrated every run. Every node id GitHub reports for a stored account is stored. Step 4b marks a compare file list that reaches GitHub's 300-file cap, completes an activity read only at the log's end or the floor, and fails a history read whose count still differs after a rescan. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before the schedule starts, pins the database file the job checked, compares exit-code outputs as non-empty strings, ties a snapshot to the job's own sync, and gives consumers a staleness check that does not depend on the schedule. Left as they are, because the stored data already says what happened: a compare-linked account a lookup answers `NOT_FOUND` (D28), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (the retained snapshots hold the identities), and the merge state as of `hydrated_at` (D32). | +| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class, of an item or of an account's refresh, so a permanent failure can be accepted without turning every run red. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write or determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the log's end or the floor, and fails a history read whose count still differs after a rescan. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a compare-linked account a lookup answers `NOT_FOUND` (D28; now also counted per run), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | ## 3. Coverage rule @@ -226,18 +228,18 @@ Every gap found in the audit, and where it is resolved. | 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode rebuilds `recorded/` from what it records, and `--only documents` drops every unit kind but the compare and rate-limit kinds it keeps; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | | 97 | Without `--exact`, an unchecked item and an unverified repository did not fail verify, and a timeline lookup whose entries differed from `filteredCount` left its item unchecked, so the scheduled verify (5.6, no `--exact`) could exit 0 and D25 publish a database it had not checked | D37: both fail verify in every mode, and an answer verify cannot accept is an error (5.4, 5.8) | | 98 | A 24-hour account refresh that still fails, or whose answer the writer refuses (another `databaseId` for a stored node id), left the run `complete` with exit 0; the only trace was a printed line, and a test pinned exit 0 | Step 3b-2a-3: counted in `sync_runs.refresh_failures`, listed in the report, and exit 2 (5.3 step 4, D38) | -| 99 | A run that reached `--max-minutes` with planned items unreached exited 0, and a deadline inside a batch turned the item failures it had computed into pending items | Step 3b-2a-3: such a run exits 2; D25 still publishes it, and the workflow is red (5.3 step 8, D38) | +| 99 | A run that reached `--max-minutes` exited 0, and a deadline inside a batch turned the item failures it had computed into pending items | Step 3b-2a-3: a run that reaches its deadline exits 2, which D25 still publishes with the workflow red, and the failures a batch made final before the deadline are recorded (5.3 steps 6 and 8, D38) | | 100 | A PR the run's own index listed in coverage that hydration then answered `NOT_FOUND`, or returned out of coverage, was neither written, failed, nor pending, and nothing reported it; a hydration older than the index entry was written without a check | Step 3b-2a-3: each is an item error (5.3 steps 4 and 5, D38) | | 101 | An `accepted_failures` entry accepted any failure of its item, so a new cause stayed out of exit 2 | Step 3b-2a-3: an entry names the error class it accepts (5.5, D38) | -| 102 | A PR with more than the 10 closing links the index carries was not re-hydrated when a link beyond the 10th was swapped with equal totals, and verify compares those links the same way | Step 3b-2a-3: such a PR is hydrated every run (5.3 step 3, D38) | -| 103 | A `databaseId` reported under a second node id kept the stored node id and dropped the reported one; only a printed line recorded it | Step 3b-2a-3: `account_node_ids` stores every node id GitHub reports for an account (5.1, 5.2, D38) | +| 102 | A PR with more than the 10 closing links the index carries was not re-hydrated when a link beyond the 10th was swapped with equal totals | Step 3b-2a-3: such a PR is hydrated every run (5.3 step 3, D38). Verify keeps comparing the totals and the indexed links: the sync re-reads every link of such a PR each run, and verify's child-count check holds the stored rows to the hydrated total | +| 103 | A `databaseId` reported under a second node id kept the stored node id and dropped the reported one; only a printed line recorded it; a commit link's node id was never read | Step 3b-2a-3: `account_node_ids` stores every node id a sighting, a lookup, or a commit link reports for an account (5.1, 5.2, D38) | | 104 | Source checks that accepted what they should refuse: a REST response without readable rate-limit headers, a GraphQL `errors` present but not a list (`{}`, `""`), a compare above 250 commits checked by count but not membership, and an index page from a repository other than the one resolved | Step 3b-2a-3 (5.3 steps 2, 4, and 5, D38) | -| 105 | Run records and setup that hid a failure: `code_version` without `-dirty` when `git status` failed, a close that ignored outcomes it could not write, a missing `[automation]` table emptying the automation lists, `foreign_keys` never read back, `alembic stamp` accepted, a wheel check comparing repository names only, and a log file no record reaches | Step 3b-2a-3 (5.1, 5.3 step 8, 5.5, D38) | +| 105 | Run records and setup that hid a failure: `code_version` without `-dirty` when `git status` failed, a close that ignored outcomes it could not write, a missing `[automation]` table emptying the automation lists, `foreign_keys` never read back, `alembic stamp` accepted, a wheel check comparing repository names only, a log file no record reaches, and no record of a run's `--repo` filter | Step 3b-2a-3 (5.1, 5.3 step 8, 5.5, D38) | | 106 | Step 4b: compare `files` stop at GitHub's documented 300 per comparison with no total, so a capped list read as complete; the activity read had no paging or completeness rule; a history count mismatch had no outcome; a refused compare was never retried; and the log's first event was not stored | Step 4b (5.1, D38) | -| 107 | Step 6: an App token may return null where the maintainer's token read a value; a stopped schedule turns nothing red; a missing `DATABASE_URL` builds from empty; an empty exit-code output compares equal to 0 in `if:`; and `snapshot` did not check that the job's own sync ran | Step 6 and 5.6 (D38) | +| 107 | Step 6: an App token may return null where the maintainer's token read a value; a stopped schedule turns nothing red; a missing `DATABASE_URL` builds from empty; an empty exit-code output compares equal to 0 in `if:`; and `snapshot` did not check that the job's own sync ran | Step 6 and 5.6 (D38), except the stopped schedule: no run starts, so this repository cannot see it; 5.6 states the consumer's staleness rule, which the dashboard's integration builds (section 9) | | 108 | The data dictionary presented head-repository, milestone, and team names as GitHub's current values, though they are as of `hydrated_at` | Step 3b-2a-3: described as of `hydrated_at` (D38) | | 109 | Edits that may leave `updatedAt` unchanged (body, base ref, review dismissal) were accepted as missed | Step 5 measures them; one that does gets an index-side check or a periodic re-hydration (section 8, D38) | -| 110 | A GitHub-derived column a later migration adds to a table without `hydration_version` (accounts, compares) is never filled for existing rows | 5.1: such a migration adds a per-row read version or refuses while the table holds rows (D38) | +| 110 | A GitHub-derived column a later migration adds to a table without `hydration_version` (accounts, compares), or a new table of such data, is never filled for existing rows | 5.1: such a migration adds a per-row read version or refuses while the file holds the rows it would leave unread (D38) | ## 5. Design @@ -267,8 +269,9 @@ IMMEDIATE`, and revision the upgrade started from, with no revision of that upgrade applied. `PRAGMA foreign_key_check` runs inside that transaction after the last revision and raises on any row, before the commit. It runs before every command commits, one that runs no revision (`current`, -`check`, `stamp`) included: alembic loads each revision module inside the transaction, so a -module's top-level code can change rows or the schema without a revision running (section 7, +`check`, and a `stamp` that changes nothing) included; a `stamp` that would change the revision is +refused before any statement (gap 105): alembic loads each revision module inside the transaction, +so a module's top-level code can change rows or the schema without a revision running (section 7, "Revision module code"), and a file that already holds a violation fails every command that connects. A migration that batch-alters a table a view reads drops those views first and recreates them afterwards from its own frozen SQL; `legacy_alter_table` stays off, so a view left @@ -339,9 +342,9 @@ and `color`, `pr_user_tags.created_at`). | `default_branch_commits` | `repository_id`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at`, `committed_at`, `parent_count`, `additions`, `deletions`, `changed_files` | Step 4b; PK (`repository_id`, `sha`); the default branch's history since the floor as GitHub lists it (D30); the author link as in `pr_commits`, a replace keeping the stored link of the row with the same SHA when GitHub returns none (D28); `parent_count` is GitHub's `parents.totalCount`; `changed_files` is GitHub's `changedFilesIfAvailable`, null when GitHub returns none; no message (D19) | | `automation_accounts` | `github_id`, `login`, `reason`, `run_id` | integer primary key; `github_id` unique; replaced from `tracking.toml` at the start of every run, `run_id` naming that run (a reference to `sync_runs`) | | `account_profiles` | `account_id`, `company`, `location`, `website_url`, and an organization's `description`, `email`, `twitter_username`, `is_verified`, `archived_at`, `has_sponsors_listing`, and `pinned_items_remaining`, `refreshed_at` | PK `account_id`, a reference to `github_users` (`ON DELETE CASCADE`); rows only for a `User` or an `Organization` (D31); each lookup that returns the account replaces the whole row with the profile it returned, through the writer and in the transaction that writes `github_users`' login and type, so no earlier value survives a field GitHub now returns as null or a change of type; a lookup that returns `NOT_FOUND` leaves the row unchanged, marked by `github_users.not_found_at` (D28); every field nullable as GitHub's is, the organization fields null for a user and `company` null for an organization; `refreshed_at` as on `github_users` | -| `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting or a lookup, the one `github_users.node_id` holds included, with the first and latest runs that reported it, so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); consumer | +| `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting, a lookup, or a commit link, the one `github_users.node_id` holds included, with the first and latest runs that reported it, so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); its migration refuses a file whose `github_users` holds rows (gap 110); consumer | | `automation_emails` | `email`, `reason`, `run_id` | integer primary key; `email` unique, stored lowercase ASCII (SQLite's `lower()`, which the views apply to commit emails, folds only ASCII letters); replaced the same way | -| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `deletions_approved` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, else `manual`; `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when its tracked files have changes, and `-unknown` when a git call failed or timed out, gap 105; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted; `failures` counts this run's item failures, accepted ones included, and `refresh_failures` its failed refresh lookups (5.3 step 4); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | +| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `links_not_found`, `repo_filter`, `deletions_approved` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, else `manual`; `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes and `-unknown` when it fails or times out, gap 105; `git ls-files --error-unmatch` exiting 1 means the package is not tracked, and any other failure of it or of `git rev-parse`, a missing `git` included, records the package version followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted; `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `links_not_found` the commit-link accounts a lookup answered `NOT_FOUND` this run (5.2), and `repo_filter` the repositories `--repo` named, null for a run without it; the migration that adds these three refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | | `repo_sync_state` | `repository_id`, `last_index_started_at`, `index_pr_total`, `index_issue_total`, `last_success_at`, `last_run_id`, `last_outcome`, `halt_reason`, and from Step 4b `activity_read_at`, `activity_first_event_at`, `activity_last_outcome`, `activity_halt_reason`, `history_read_at`, and `default_branch_commits_total` | PK `repository_id`; `activity_first_event_at` is the `timestamp` of the oldest event the latest complete activity read listed, so a consumer can tell a quiet period from a log that does not reach back (gap 106); `activity_read_at` is the GitHub `Date` header of the first page of the latest activity-log read that completed and was written; `history_read_at` and `default_branch_commits_total` are the first page's `Date` header and the `totalCount` of the latest history read that replaced the rows, written only with that replace; `activity_last_outcome` in (`complete`, `time_limited`, `failed`, `halted`); `activity_halt_reason` holds a Step 4b deletion halt's reason and candidate counts and is cleared by a run whose Step 4b did not halt (Step 4b); `last_run_id` is the latest run that attempted the repository, or found it untracked, and `last_outcome` its result, in (`complete`, `time_limited`, `failed`, `incomplete_index`, `halted`, `untracked`), both written by step 8, or by the close of a fatal or interrupted run (5.3 step 8); `last_index_started_at` is written only when that index completed and the repository did not fail later in the run, a time-limited repository included (its unreached items are in `sync_pending`), and never by the close of a fatal or interrupted run; `last_success_at` is the run's `finished_at` when `last_outcome` is `complete` and no item of the repository failed outside `[sync].accepted_failures`; `halt_reason` holds step 7's halt and is cleared by a run whose step 7 ran for the repository and did not halt, a run whose step 7 did not run for it leaving it | | `sync_pending` | `repository_id`, `item_kind`, `number`, `run_id` | PK (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); items a time-limited run planned but did not reach; step 8 replaces the rows of each completely indexed repository that did not fail later in the run (a repository gone during hydration keeps its rows, 5.3 step 4) with this run's unreached items, so a run that reached them all leaves none; consumer, so an in-coverage item not yet hydrated is told apart from one outside coverage | | `sync_failures` | `repository_id`, `item_kind`, `number`, `attempts`, `last_error`, `first_failed_at`, `last_failed_at`, `last_run_id` | unique (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); `attempts` counts the runs in which the item failed: `last_run_id` is the latest such run, and a failure adds one only when its run is another, whatever the runs' clocks read; `last_error` is the error's class and message; row deleted on success, or by step 7 when its repository's in-coverage set no longer lists the item | @@ -390,7 +393,9 @@ Column rules: A table without `hydration_version` (`github_users`, `account_profiles`, the Step 4b compare tables) to which a later migration adds a GitHub-derived column gains a per-row read version that the planner compares the same way, or the migration refuses while the table holds rows, so - a column is never left unread on old rows (gap 110). + a column is never left unread on old rows; a migration that adds a table of GitHub-derived data, + or a run counter, refuses while the file holds the rows the new table or column would describe, + so no row reads as "none" where nothing was recorded (gap 110). - `state_reason` is GitHub's value including `DUPLICATE` (5.8). Descriptive enums, every GitHub enum column except a PR's, issue's, and review's `state` (`github_users.type` keeps its CHECK), such as `change_type`, `author_association`, `state_reason`, `merge_state_status`, and @@ -488,12 +493,12 @@ GitHub is consumer. The columns that say how current a row is are consumer time of everything the index wrote), `sync_failures`' repository, kind, number, error, and failure times, `sync_pending` (an in-coverage item a time-limited run did not reach), and every `sync_runs` column except the run counters (`graphql_points`, `rest_calls`, `items_*`, -`failures`), which with the tag tables are internal. Integer row ids change on a rebuild, but a pull request's labels, -reviews, comments, commits, files, and closing links, and its author and merger, are reachable -only through them, so the foreign-key columns of consumer tables and the row ids they reference -are consumer, valid within one file; other row ids are internal. Consumers never store a row id -or compare one across files, where a row is identified by its `github_id` (an account with a null -`github_id` by its `node_id`), and a child row +`failures`, `refresh_failures`, `links_not_found`), which with the tag tables are internal. Integer +row ids change on a rebuild, but a pull request's labels, reviews, comments, commits, files, and +closing links, and its author and merger, are reachable only through them, so the foreign-key +columns of consumer tables and the row ids they reference are consumer, valid within one file; other +row ids are internal. Consumers never store a row id or compare one across files, where a row is +identified by its `github_id` (an account with a null `github_id` by its `node_id`), and a child row without one (a label, commit, file, or closing link) by its pull request's `github_id` and its own key; (`full_name`, `number`) identifies a pull request among tracked repositories within one file and changes with a rename. Views carry no row ids. `PRAGMA user_version` holds `schema_version`, set by @@ -533,7 +538,10 @@ same number. `account_node_ids`, which holds every node id GitHub reported for the account (gap 103). One whose node id is stored under another `databaseId` fails the item, or the lookup, as does an item whose sightings and lookups report one node under two `databaseId` values, a node id in - `account_node_ids` included. A commit link resolves by its `databaseId` alone. + `account_node_ids` included. A commit link resolves by its `databaseId`; the node id it carries + (GraphQL `GitActor.user.id`, REST compare `author.node_id`) is stored in `account_node_ids` and + checked as a sighting's is (gap 103). A commit-link lookup that answers `NOT_FOUND` is counted in + `sync_runs.links_not_found` and listed in the report (D28 stores the commit as before). - Type and login come only from GraphQL: actor sightings (PR author, merger, review author, comment author, editor, requested reviewer, assignee), which carry `__typename`, and `nodes(ids:)` lookups. A commit-linked account (GraphQL `GitActor.user` or the REST compare @@ -682,9 +690,10 @@ an interrupt, which is re-raised. A failure the engine detects itself (these two, a hydrated `github_id` other than the index's, a review or comment id stored under another PR, a per-batch lookup's `databaseId` other than the commit link's) is recorded at once, without the solo retry, which would repeat a deterministic - result. The planner is a pure function over snapshots of the stored rows, which `--dry-run` - reuses. The run reads its snapshot once, after the claim and before step 1 (a dry run, after the - head check). + result. A hydration that contradicts the run's own index (step 5) is the exception: a lagging + read can clear, so the engine requests the item once more alone before it records the failure. + The planner is a pure function over snapshots of the stored rows, which `--dry-run` reuses. The + run reads its snapshot once, after the claim and before step 1 (a dry run, after the head check). 4. Hydrate: GraphQL batches of 25 PRs or 100 issues of one repository via aliased `pullRequest(number:)` and `issue(number:)` fields (5.8). Read each alias separately. Overflow beyond 100 entries on any connection (reviews, comments, commits, files, labels, closing issues, @@ -703,11 +712,11 @@ an interrupt, which is re-raised. | Rate limited | Any HTTP 429 (GraphQL or REST); HTTP 403 with `retry-after`, `x-ratelimit-remaining: 0`, a secondary-limit message, or `errors[].type == "RATE_LIMITED"`; GraphQL HTTP 200 with `RATE_LIMITED` or an error message naming a secondary rate limit (GitHub documents 200 and 403 for GraphQL, 403 and 429 for REST) | Wait for `retry-after`; else, when `x-ratelimit-remaining` is 0 or the error is `RATE_LIMITED`, until `x-ratelimit-reset` or the last `resetAt`; else 60 s. Each further limit before a success waits at least 60 s doubled per repeat, whatever the headers say (GitHub: "If your request continues to fail due to a secondary rate limit, wait for an exponentially increasing amount of time between retries, and throw an error after a specific number of retries"). Never split; not an attempt. A wait that would end after the `--max-minutes` deadline stops queueing instead | | Timeout or resource limit | HTTP 502/504 (GitHub documents both for a request over 10 s); a GraphQL error at any path reporting a timeout or exceeded resource limits (GitHub returns "partial results along with an error indicating that resource limits were exceeded") | Split the batch in half at once; a same-size retry repeats the cost | | Server or transport error | other 5xx; a GraphQL server fault at any path; connection error; client timeout; a connection the server broke off or a body it encoded wrongly; a GraphQL `errors` that is present and not a list (gap 104); a response without data or errors, without a readable `Date`, or with an error path that is not strings and list indices (never trimmed, since an exact path is what makes an item gone) | Retry up to 3 more times (2, 4, 8 s, with jitter), then split | - | Item gone | alias `null` with `NOT_FOUND` | In a sync, an item error: the run's own index listed the number in coverage, so GitHub's two answers disagree (gap 100); an index that no longer lists the number removes the item and its failure (step 7). In verify's lookup by number, removed since sync (5.4) | + | Item gone | alias `null` with `NOT_FOUND` | The hydrator reports it gone and does not retry it. In a sync, the run's own index listed the number in coverage, so GitHub's two answers disagree: the engine requests the item once more alone, and a second `NOT_FOUND` is an item failure (step 5, gap 100); an index that no longer lists the number removes the item and its failure (step 7). In verify's lookup by number, removed since sync (5.4) | | Item error | another alias error; partial alias data; a value the typed models reject; a failed check in step 5; HTTP 404/422 on a compare page | Retry the item once alone, then record it in `sync_failures` and write nothing for it | | Repository gone during hydration | a batch's `repository` is null (`RepositoryNotFound`, `RepositoryShapeError`; gap 79) | The repository fails for the rest of the run: its written items stay, step 7 skips it, and its `sync_pending` rows are unchanged | | Local database error | any error the database raises while the writer writes (a lock held past the busy timeout, an unexpected constraint failure; gap 77) | Fatal: roll back the item, close the run, exit 1 | - | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; a REST response without readable `x-ratelimit-remaining` and `x-ratelimit-reset` (gap 104); another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | + | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; a 2xx REST response without readable `x-ratelimit-remaining` and `x-ratelimit-reset` (gap 104; other statuses take the rows above whatever their headers); another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | | Unauthorized or forbidden | HTTP 401; a failed credential request (an App token mint answering 4xx without a rate-limit signal); HTTP 403 with no rate-limit signal (a missing permission); an HTTP client failure outside the network (a redirect loop, a proxy or URL problem, a missing auth dependency) | Fatal: close the run, exit 1 | Typed response models check types strictly: GitHub's JSON has exact types, and lax coercion @@ -757,19 +766,24 @@ an interrupt, which is re-raised. solo retry, since no retry of the transport preceded it. The hydrated PR's `github_id` equals the one the index listed for its number, and a review or comment whose `github_id` is stored under another PR fails the item. A PR that closed without merge between index and hydration is written - with state `CLOSED` (D3). A hydration that contradicts the run's own index is an item error, - before any account lookup for it: an item outside coverage, or an `updatedAt` earlier than the - index entry's (gap 100). + with state `CLOSED` (D3). A hydration that contradicts the run's own index (an alias `NOT_FOUND`, + an item outside coverage, an `updatedAt` earlier than the index entry's) is requested once more + alone by the engine, which holds the index, and a second contradiction is an item failure, + recorded before any account lookup for it (gap 100). Above 250 commits, only the commit + connection's first page is checked against compare's list: it holds the oldest commits, which a + wrong base adds or drops, and reading the rest would cost continuations a count check already + covers. 6. Write: one batch at a time, on the writer's one connection. The run hydrates a batch, checks it (step 5), looks up its accounts (5.2), and writes each of its items before it requests the next batch; at the deadline it writes the fetched items whose accounts it has and records the rest in `sync_pending`. The hydrator delivers a batch's outcomes together (Step 2), so a deadline inside the batch's own requests (a split half, a solo retry, an overflow or compare page) writes none - of its items, and records none of the failures the batch computed: all of them wait in - `sync_pending`, which puts them first in the queue of the next run that indexes the repository - completely and selects them (step 3). Each item is one transaction: upsert by `github_id` with - `INSERT ... ON CONFLICT (github_id) DO UPDATE`, which keeps the row id (never - `INSERT OR REPLACE`, which deletes the row and cascades); replace the hydrated child rows + of its items. The hydrator hands over, with the deadline, the outcomes it had made final (a + failure after its solo retry, or one that gets none), and those failures are recorded (gap 99); + the rest of the batch waits in `sync_pending`, which puts it first in the queue of the next run + that indexes the repository completely and selects it (step 3). Each item is one transaction: + upsert by `github_id` with `INSERT ... ON CONFLICT (github_id) DO UPDATE`, which keeps the row id + (never `INSERT OR REPLACE`, which deletes the row and cascades); replace the hydrated child rows (`pr_labels`, `pr_reviews`, `pr_comments`, `pr_commits`, `pr_files`, `pr_closing_issues`, `pr_review_requests`, `pr_assignees`, `issue_labels`), where a new review or comment with a null author takes the stored author of the row with the same `github_id`, a commit with a null link @@ -813,18 +827,22 @@ an interrupt, which is re-raised. indexed completely and nothing failed; 2 when an item failed in this run (other than one listed in `[sync].accepted_failures`), a repository failed, had an incomplete index, or halted, a stored repository is untracked in a run without `--repo`, a prune was refused (step 7), a refresh - failed (step 4), or the deadline left planned items unreached (gap 99); 1 on a fatal error. The + failed (step 4), or the run reached its deadline (status `time_limited`, whatever it stopped: + hydration, the refresh, or a Step 4b read; gap 99); 1 on a fatal error. The exit code is decided after the prune. An accepted failure is still recorded and reported; it - names the repository, number, error class, and reason in `tracking.toml`, so a permanent failure - does not turn every run red, and a failure of another class is not accepted (gap 101). An entry + names the repository and number, or the account's `github_id`, with the error class and reason + in `tracking.toml`, so a permanent failure does not turn every run red, and a failure of another + class is not accepted (gap 101). An entry is reported as not having failed when its repository's outcome in this run is `complete`, `time_limited`, or `halted` and its item neither failed in this run nor is pending (5.5). `--max-minutes` stops queueing new hydration, records in `sync_pending` the planned items it did not reach (a repository that failed during hydration keeps its rows, step 4), still runs step 7 for complete repositories and step 8, and sets status `time_limited`; unreached items are not - item failures, but they make the exit 2 (D37). Step 8 writes `last_run_id` and `last_outcome` for - every repository the run attempted. A close that is given an outcome for a repository it finds no - stored row for is fatal (exit 1, gap 105). `--dry-run` takes no lock, inserts no `sync_runs` row, + item failures, but the deadline makes the exit 2 (D37). Step 8 writes `last_run_id` and + `last_outcome` for every repository the run attempted. A close given an outcome for a repository + it finds no stored row for raises inside its transaction before it writes, so nothing is written + and the run exits 1 with its row `running` for the stale-row rule or `--break-lock`, as a close + that fails does (gap 105). `--dry-run` takes no lock, inserts no `sync_runs` row, and writes nothing: it runs steps 1-3 as the run would and reports step 1's updates, the planned hydrations with their reasons, the in-place updates, and the delete candidates. It exits as the run would for every step-8 condition steps 1-3 can show (a failed or incompletely indexed @@ -918,9 +936,9 @@ the first class that matches, in this order: An error, an unchecked item, and an unverified repository fail verify in both modes: what verify could not check is a failure, never a pass (D37). `verify --exact` also fails on not synced, -pending, halted deletion, removed since sync, and known failures not listed in -`[sync].accepted_failures`; it is the mode the Step 3-5 acceptance uses, and drift is reported in -both modes. A repository that verify cannot +pending, halted deletion, removed since sync, and known failures no `[sync].accepted_failures` +entry matches by repository, number, and error class; it is the mode the Step 3-5 acceptance uses, +and drift is reported in both modes. A repository that verify cannot resolve, re-index, or look up is reported as unverified: every repository error the source raises at resolve or index (not found, no default branch, a fetch that still fails after its retries, a response or GitHub error the resolve or index rejects, a label list that fails its count or names a @@ -947,11 +965,13 @@ because Step 6a reads it (5.6, D36); an interrupt exits 130 either way. The repo code and whether `--exact` was given; per class, the count of items and their numbers (an item whose differences take two classes counts in each); per repository, its `last_run_id`, boundary, per-class counts, and per-state counts; -for each closing-link difference, the issues it names and the stored and fresh totals; the -unverified repositories with their reasons; and verify's GraphQL points and REST calls, counted -by its source as a run's are (5.3 step 4), and the seconds from its start to the report's write, -since verify writes no `sync_runs` row. A report verify prints that stdout cannot take leaves the -exit code verify's, and stderr follows 5.3 step 8, 120 included. +for each closing-link difference, the issues it names and the stored and fresh totals; from +Step 4b, per repository the compares stored with `files_at_cap` and those GitHub refused, which do +not fail verify (GitHub's limit and GitHub's refusal, which no retry lifts) but are never reported +as complete; the unverified repositories with their reasons; and verify's GraphQL points and REST +calls, counted by its source as a run's are (5.3 step 4), and the seconds from its start to the +report's write, since verify writes no `sync_runs` row. A report verify prints that stdout cannot +take leaves the exit code verify's, and stderr follows 5.3 step 8, 120 included. The enumerated count must equal the last page's unfiltered `totalCount` (rescanned once, as in 5.3); per-state counts are @@ -970,8 +990,9 @@ one SQL statement, so a verify that reads beside a sync's item commits compares state per check. `ghactivity verify --db-only` (Step 6a, beside the `snapshot` command that writes its inputs; it takes -the snapshot gz, its manifest, and the asset digest as explicit arguments, and a halt exits 1) -checks a downloaded snapshot before anything touches it: the gz SHA-256 +the snapshot gz, its manifest, the asset digest, and `--out PATH` as explicit arguments, and a halt +exits 1) decompresses to PATH, prints that file's SHA-256 on its last line once every check passes, +and checks a downloaded snapshot before anything touches it: the gz SHA-256 equals the manifest's (and the release asset's `digest` when GitHub reports one; a null digest leaves the manifest as the only check); decompression succeeds; `PRAGMA integrity_check` returns `ok` and `PRAGMA foreign_key_check` returns no rows; per-repository PR and issue counts equal the manifest's; @@ -996,16 +1017,21 @@ strictly typed throughout: an account `github_id` is at least 1, and `login`, `e `unsettled_minutes` is an integer of at least 1 and at most 1,439,999,999,999 (the most minutes a Python `timedelta` holds, so the planner can build the window), `delete_guard_min_rows` an integer of at least 0, -and `delete_guard_percent` a number above 0 and at most 100; an `accepted_failures` entry has a -`number` of at least 1, an `error` naming the error class it accepts (the class -`sync_failures.last_error` starts with), is unique by repository and number, must name a configured -repository, and matches by that entry, the number, and the class, so a rename does not unhook it and -a new cause is not accepted (gap 101); an entry whose item did not fail is reported (5.3 step 8). +and `delete_guard_percent` a number above 0 and at most 100; an `accepted_failures` entry names +an item (a configured `repository` and a `number` of at least 1) or an account (`github_id`, for a +refresh failure), and an `error`: the error class it accepts, compared for equality with the text +of `sync_failures.last_error` (or the refresh failure's text) before its first `": "`, which the +loader refuses unless it is one of the failure classes the sync records; for a class whose message +carries the cause (`AliasError`, `UserLookupFailed`), the entry also gives `message`, compared for +equality with the rest of that text. Entries are unique by item or account, an item entry matches +by its configured repository, so a rename does not unhook it, and a new cause is not accepted +(gap 101); an entry whose item or account did not fail is reported (5.3 step 8). `[automation]` is required, its lists may be empty, so a file without it is refused instead of emptying the automation tables (gap 105). The file holds domain policy only. Step 1 added `[[repositories]]`; Step 3b-2a-1 adds `[coverage]` and `[sync]`'s `unsettled_minutes`, `delete_guard_min_rows`, and `delete_guard_percent`, which the planner reads; Step 3b-2a-2a adds -`[automation]` and `[sync].accepted_failures`, which the run claim and the exit code read. +`[automation]` and `[sync].accepted_failures`, which the run claim and the exit code read; Step +3b-2a-3 adds an entry's `error`, `message`, and account form, and makes `[automation]` required. ```toml [coverage] @@ -1060,8 +1086,9 @@ Workflow `.github/workflows/sync.yml`: holds the App credentials. Without `bootstrap`, it downloads the latest snapshot and runs `verify --db-only`; a missing or failing snapshot halts the run. With `bootstrap`, it downloads nothing and starts from an empty file, and the manifest records `bootstrap: true`. One job-level - `DATABASE_URL` names the file every step uses; without `bootstrap` the job fails unless that file - exists and its SHA-256 equals the one `verify --db-only` checked (gap 107). It then runs + `DATABASE_URL` names the file every step uses, the `--out` path of `verify --db-only`; without + `bootstrap`, before `alembic upgrade head` the job records the file's highest `sync_runs` id and + fails unless the file's SHA-256 equals the one `verify --db-only` printed (gap 107). It then runs `alembic upgrade head`, asserts the revision is head, runs `ghactivity sync --max-minutes 300` and `ghactivity verify --max-minutes 40 --report verify.json`, runs `ghactivity snapshot --verify-report verify.json`, and hands the snapshot and manifest to `publish` as a workflow @@ -1102,15 +1129,18 @@ Workflow `.github/workflows/sync.yml`: (`sync_runs.config_sha256`), migration head, `schema_version`, coverage floor, `bootstrap`, per-repository PR and issue counts, per kind the number of rows whose `hydration_version` differs from the code's, the sync exit code, verify's - exit code, counts per class, and unverified repositories (from `--verify-report`; null without + exit code, counts per class, unverified repositories, and from Step 4b the compares at the file + cap and those refused (from `--verify-report`; null without it), and the SHA-256. `snapshot` exits 1 when the report `--verify-report` names is missing or unreadable, or names a `last_run_id` other than the file's `repo_sync_state`, so a verify that never started, which leaves no report or one from an earlier sync (5.4), fails the job. In the - workflow, `--sync-run` is the `sync_runs` id of the sync this job ran; `snapshot` exits 1 unless - that row is newer than every id in the downloaded file, is closed with the exit code the job - recorded, and is every attempted repository's `last_run_id`, so a job whose sync did not run - cannot republish the old file (gap 107). The manifest records that id and the run's - `finished_at`. + workflow, `snapshot --after-run N --sync-exit C` takes the highest `sync_runs` id the job recorded + before the sync (0 for a bootstrap) and the sync's exit code, and exits 1 unless exactly one + `sync_runs` row is newer than N, that row is closed with exit code C, and it is the + `last_run_id` of every tracked repository's `repo_sync_state` row (the workflow's sync has no + `--repo`), so a job whose sync did not run cannot republish the old file (gap 107). For 6b's + first snapshot, `--sync-run ID` names Step 5's last run instead. The manifest records that run's + id and `finished_at`. - Uploads never use `--clobber`. Before each upload, `publish` deletes a same-name asset in any state other than `uploaded`. A same-name `uploaded` asset whose `digest` is `sha256:` followed by the expected SHA-256 (or, when `digest` is null, whose downloaded SHA-256 matches) is skipped, so a @@ -1137,13 +1167,15 @@ Workflow `.github/workflows/sync.yml`: replaces its local copy by writing a temporary file and renaming it over the old one, so no query reads a half-written file. It does this once per new snapshot (about every 6 hours, D11), not per read, and queries its copy with SQLite. It treats a pair whose manifest `finished_at` is older - than twice D11's interval (12 hours) as stale and says so wherever it shows the data, with the - manifest's sync exit code and verify counts beside it: a stopped schedule runs nothing, so only a - check outside the workflow can see it (gap 107). The download runs outside a browser, on a server - or in a build step: the asset host (`release-assets.githubusercontent.com`) answered without - `access-control-allow-origin`, so a page on another origin is not expected to read the file - (inferred from the response headers, no browser was run; section 7, "Release downloads"). How a - page reads its copy is the frontend's design, outside this plan. + than 13 hours (D11's 6-hour interval, a job of up to 355 minutes, and an hour of schedule delay) + as stale and says so wherever it shows the data, with the manifest's sync exit code and verify + counts beside it: a stopped schedule runs nothing, so only a check outside this repository can + see it, and the dashboard's integration builds this one (section 9, gap 107). The download runs + outside a browser, on a server or in a build step: the asset host + (`release-assets.githubusercontent.com`) answered without `access-control-allow-origin`, so a page + on another origin is not expected to read the file (inferred from the response headers, no browser + was run; section 7, "Release downloads"). How a page reads its copy is the frontend's design, + outside this plan. A `workflow_dispatch` trigger runs only when the workflow file exists on the default branch, so the workflow merges first with the manual trigger only, is proven by manual runs, and the schedule is @@ -1352,9 +1384,10 @@ repository(owner: $owner, name: $name) { defaultBranchRef { name target { ... on ``` Repository activity (Step 4b; REST `GET /repos/{owner}/{repo}/activity?ref=refs/heads/&per_page=100`, newest first) and, for each event whose `before` and `after` are both -non-zero, REST `GET /repos/{owner}/{repo}/compare/{before}...{after}`, paged like the commit-list -compare (5.3 step 4). Their REST cost is measured in Step 4b. +branch>&per_page=100`, newest first, paged by the cursor in each response's `Link` header) and, +for each event whose `before` and `after` are both non-zero, REST +`GET /repos/{owner}/{repo}/compare/{before}...{after}`, paged like the commit-list compare (5.3 +step 4). Their REST cost is measured in Step 4b. `Actor` selects `__typename login` plus `id databaseId` on `User`, `Bot`, `Organization`, and `Mannequin`, and from Step 3b-1 `id user { databaseId }` on `EnterpriseUserAccount`, which has no @@ -1463,11 +1496,11 @@ against; 3b-2a-2b-2 adds the sync engine over the planner and the writer, `--dry included; 3b-2a-2c adds `ghactivity sync`, the seeded runs, and the live acceptance; 3b-2a-3 applies D37 to the sync (D38); 3b-2b-1 adds the REST sampler that checks what the sync wrote; and 3b-2b-2 adds verify, whose live acceptance -runs the sampler. "Step 3b-2" elsewhere in this plan means 3b-2a-1, the parts of 3b-2a-2, and the -two parts of 3b-2b together; "Step 3b-2a" means 3b-2a-1, 3b-2a-2, and 3b-2a-3; "Step 3b-2a-2" means -3b-2a-2a, the two parts of 3b-2a-2b, and 3b-2a-2c; "Step 3b-2a-2b" means 3b-2a-2b-1 and -3b-2a-2b-2; and "Step 3b-2b" means 3b-2b-1 and 3b-2b-2. The REST sampler and its recorder mode are -3b-2b-1's; verify (5.4), its timeline lookup, and the scenario model's timeline responses are +runs the sampler. "Step 3b-2" elsewhere in this plan means 3b-2a-1, the parts of 3b-2a-2, 3b-2a-3, +and the two parts of 3b-2b together; "Step 3b-2a" means 3b-2a-1, 3b-2a-2, and 3b-2a-3; "Step +3b-2a-2" means 3b-2a-2a, the two parts of 3b-2a-2b, and 3b-2a-2c; "Step 3b-2a-2b" means 3b-2a-2b-1 +and 3b-2a-2b-2; and "Step 3b-2b" means 3b-2b-1 and 3b-2b-2. The REST sampler and its recorder mode +are 3b-2b-1's; verify (5.4), its timeline lookup, and the scenario model's timeline responses are 3b-2b-2's; the planner, its snapshot reader, step 7's candidate, guard, and `--allow-deletions N` functions, and the `tracking.toml` keys the planner reads (`[coverage]` and `[sync]`'s `unsettled_minutes`, `delete_guard_min_rows`, and `delete_guard_percent`) with their loader and the @@ -1475,8 +1508,9 @@ file's two comment changes are 3b-2a-1's; `[automation]` and `[sync].accepted_fa loading, every write the sync makes, run records, the run lock, the head check, and the busy-timeout comment are 3b-2a-2a's; the scenario model and its round-trip test are 3b-2a-2b-1's; the command with its usage errors and the runs that never start, the seeded runs, the go-gdpr recordings, and -the live acceptance are 3b-2a-2c's; and the rest, `--dry-run` and the exit code of a run that starts -included, is 3b-2a-2b-2's. Until 3b-2a-2c `main` has no `sync` command and the tracked database +the live acceptance are 3b-2a-2c's; the changes D38 makes to the sync, its exit code included, +are 3b-2a-3's; and the rest, `--dry-run` and the exit code of a run that starts included, is +3b-2a-2b-2's. Until 3b-2a-2c `main` has no `sync` command and the tracked database stays frozen (D26). No sync has written the new schema before 3b-2a-2, so 3b-1's changes touch no stored data: the tracked `github_activity.db` holds the first schema's data, which D18 rebuilds from GitHub rather than migrating (Step 5). Prerequisites for 3a: the gap 68 counter follow-up and the @@ -1841,42 +1875,60 @@ check without a test that targets it; CI passes on the exact PR head. Prerequisite: Step 3b-2a-2 has merged. -Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh_failures` and the -`account_node_ids` table (5.1): -- a failed 24-hour refresh is counted, listed in the report, and makes the exit 2 (5.3 step 4, - gap 98); -- a run whose deadline leaves planned items unreached exits 2 (5.3 step 8, gap 99); +Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh_failures`, +`links_not_found`, and `repo_filter` and the `account_node_ids` table, and refuses a file whose +`sync_runs` or `github_users` holds rows (5.1, gap 110; a development file is rebuilt by a sync): +- a failed 24-hour refresh is counted (accounts), listed in the report, and makes the exit 2 (5.3 + step 4, gap 98); +- a run that reaches its deadline exits 2, and the failures a batch made final before the deadline + are recorded, the hydrator handing them over with the deadline (5.3 steps 6 and 8, gap 99); - a hydration that contradicts the run's own index (an alias `NOT_FOUND`, an item outside coverage, - an `updatedAt` earlier than the index entry's) is an item error (5.3 steps 4 and 5, gap 100); -- an `accepted_failures` entry names the error class it accepts (5.5, gap 101); + an `updatedAt` earlier than the index entry's) is requested once more alone by the engine and + then is an item failure (5.3 steps 3, 4, and 5, gap 100); +- an `accepted_failures` entry names the error class it accepts, and may name an account for a + refresh failure (5.5, gap 101); - a PR whose link `totalCount` exceeds the 10 links the index carries is hydrated every run (5.3 step 3, gap 102); -- every node id GitHub reports for a stored account is stored in `account_node_ids` (5.2, gap 103); -- the source checks of gap 104: a REST response without readable rate-limit headers is not +- every node id a sighting, a lookup, or a commit link reports for a stored account is stored in + `account_node_ids`, and a commit-link lookup answered `NOT_FOUND` is counted (5.2, gap 103); +- the source checks of gap 104: a 2xx REST response without readable rate-limit headers is not understood, a GraphQL `errors` present and not a list is a server error, a compare above 250 commits holds every commit on the commit connection's first page, and every index page selects the repository's `databaseId` and is checked against the resolved `github_id` (the index units are re-recorded; no `hydration_version` changes); -- the run-record and setup checks of gap 105: `code_version` ends in `-unknown` when a git call - failed or timed out, a close given an outcome it cannot write is fatal, `[automation]` is - required, the connect listener reads `foreign_keys` back and refuses a connection where it is - off, `alembic/env.py` refuses `stamp`, the wheel check compares the packaged `tracking.toml` - bytes with the source file's, and the log-file handler receives every record at its level; +- the run-record and setup checks of gap 105: `code_version` as 5.1 defines it, with `_git` + returning the exit status; a close given an outcome it cannot write raises before writing; + `[automation]` is required; the connect listener reads `foreign_keys` back and refuses a + connection where it is off; `alembic/env.py` refuses a `stamp` that would change the revision; + the wheel check compares the packaged `tracking.toml` bytes with the source file's; the log-file + handler receives every record at its level; and `sync_runs.repo_filter` records `--repo`; - the data dictionary describes the head-repository, milestone, and team names as of - `hydrated_at` (gap 108). + `hydrated_at` (gap 108); +- the fixtures these rules break: the scenario model renders `x-ratelimit-*` headers on REST units + and the repository `databaseId` on index pages, and every test configuration gains + `[automation]`. Tests: each rule with planted inputs on both sides: a refresh request that fails and one that -succeeds, and a refresh answer with another `databaseId`; a deadline before and after the last -planned item, and inside a batch whose items failed; a `NOT_FOUND` alias for a number the index -listed, then a later run whose index omits the number and whose step 7 removes the item and its -failure; a hydration outside coverage; a hydrated `updatedAt` earlier than, equal to, and later -than the index entry's; an accepted entry whose class matches and one whose class differs; PRs with -10 and with 11 links; a second node id stored and sighted again, and a node id in -`account_node_ids` reported under another `databaseId` failing the item; each source check accepting -the valid shape and refusing the planted one; each run-record and setup check. Each is a regression -test that fails against the parent's code, and the tests that pinned the old behaviour (a failed -refresh and a time-limited run exiting 0, a gone item writing nothing, a PR with 11 links not -planned) change with the rule they pinned. +succeeds, a refresh answer with another `databaseId`, and an accepted account entry; a run that +finishes before its deadline (exit 0) and one whose deadline stops hydration, the refresh, or both +(exit 2); a deadline inside a batch after one item's failure became final, which is recorded while +the rest wait in `sync_pending`; a `NOT_FOUND` alias for a number the index listed that the retry +clears and one it does not, then a later run whose index omits the number and whose step 7 removes +the item and its failure; a hydration outside coverage; a hydrated `updatedAt` earlier than, equal +to, and later than the index entry's, the earlier one cleared by the retry and not; an accepted +entry whose class matches, one whose class differs, one whose `message` differs, and a loader +refusing an unknown class; PRs with 10 and with 11 links; a second node id from a sighting, a +lookup, and a REST compare link stored and seen again, and a node id in `account_node_ids` reported +under another `databaseId` failing the item; a commit-link lookup answered `NOT_FOUND` counted; each +source check accepting the valid shape and refusing the planted one, and a 502 without rate-limit +headers still retried; `code_version` outside a checkout, in a clean and a dirty checkout, and with +`git status`, `ls-files`, and `rev-parse` failing or timing out; a close given an outcome with no +stored row leaving the row `running`; a `stamp` to another revision refused and one at head +accepted; the migration refusing a populated file. Each is a regression test that fails against the +parent's code. The tests that pinned the old behaviour change with the rule they pinned: in +`tests/sync/test_engine.py` the failed-refresh, deadline (hydration and refresh), gone-item, +leaving-coverage, and 11-link tests; in `tests/sync/test_runs.py` the skipped-close and +`code_version` tests; and in `tests/test_tracking_config.py` the defaults without `[automation]`. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule of this step without a test that targets it; CI passes on the exact PR head. @@ -2019,8 +2071,9 @@ included; the data dictionary for issues. Tests: issue first sync, update, close, and reopen; unsettled re-hydration and version bump for issues; an issue's stored author kept when GitHub returns it as null (D28); issue comments and -assignees written and replaced; a transfer arriving with a new id; an old number returning -`NOT_FOUND`, treated as gone; a snapshot one migration behind head upgrades and syncs; verify's +assignees written and replaced; a transfer arriving with a new id; an old number the run's index +listed that hydration answers `NOT_FOUND`, an item failure until an index that omits it removes it +(5.3 step 4); a snapshot one migration behind head upgrades and syncs; verify's child-count checks for `issue_labels`, `issue_comments`, and `issue_assignees`, each by a planted row deletion. @@ -2040,16 +2093,18 @@ Scope (D30): the repository activity reader, its compares, and the default-branc planner and writer steps, which run after 5.3 step 6's writes and before step 7. Each run reads the activity log since the floor in full, following GitHub's `Link` cursors, which the source then returns with a REST page; a read is complete only when a page has no next link or its oldest event -is before the floor, and any other end fails the repository's activity for the run (exit 2, -gap 106). It upserts the events by `github_id`, records the oldest one's `timestamp` in -`activity_first_event_at`, keeping each stored +is before the floor, a deadline gives `time_limited` (exit 2, 5.3 step 8), and any other end fails +the repository's activity for the run (exit 2, gap 106). A complete read upserts the events from +the floor on by `github_id` and records in `activity_first_event_at` the `timestamp` of the oldest +event it listed, one before the floor included, keeping each stored event's compare, then deletes the stored events the log no longer lists (D7) unless they exceed max(25, 2%) of the repository's stored events, which halts that deletion as step 7 halts deletions. It compares each event not yet compared whose `before` and `after` are both non-zero, storing compare's commits in full (counted against `total_commits`) and its files, with `files_at_cap` set when compare returned 300 files, GitHub's documented cap for one comparison (gap 106); a compare GitHub refuses (HTTP 404 or 422) is stored with its `http_status` and requested -again on each later run until GitHub answers it. It reads the default branch's history since the +again on each later run until it returns 2xx, and verify's report and the manifest count both kinds +(5.4, 5.6). It reads the default branch's history since the floor in full and replaces that repository's `default_branch_commits` rows only when the read count equals `totalCount` and the commits it would remove are within the same guard, so a commit a force push removed leaves as GitHub no longer lists it; a count that differs is rescanned once, and a @@ -2070,22 +2125,25 @@ recordings from public repositories. Tests: an event of each `activity_type` the recordings hold; an event with an all-zero SHA and no compare row; a refused compare stored with its status and requested again on the next run; a -compare of exactly 300 files marked `files_at_cap` and one of 299 not; an activity read that ends -without reaching the floor or the log's end failing the activity; a history count mismatch rescanned -once and, on a second mismatch, failing the activity; a second run sending no compare for an event -already compared; an event deletion past the guard halting; a diverged push whose compare runs from -the merge base; a push of several commits whose compare commits join each commit to its event; a -force push removing commits from `default_branch_commits`; a history read past the guard halting; a -null actor; an event GitHub stops listing removed on the next run; an event the log stops listing -after the stored read classified as removed since sync; `activity_halt_reason` set by a halt and -cleared by the next run that does not halt, `halt_reason` unchanged; `--allow-deletions` applying N -to events and history commits separately; a history replace keeping a commit's stored author link -when GitHub returns none (D28); each verify check and each stored-row check, by a planted row -deletion. +compare of exactly 300 files marked `files_at_cap` and one of 299 not; a two-page activity read +through a `Link` cursor that completes and sets `activity_first_event_at`, one that stops at the +floor, one the deadline stops (`time_limited`, `activity_first_event_at` unchanged), and one that +ends without reaching the floor or the log's end failing the activity; a history count mismatch +rescanned once and, on a second mismatch, failing the activity; a second run sending no compare for +an event already compared; an event deletion past the guard halting; a diverged push whose compare +runs from the merge base; a push of several commits whose compare commits join each commit to its +event; a force push removing commits from `default_branch_commits`; a history read past the guard +halting; a null actor; an event GitHub stops listing removed on the next run; an event the log stops +listing after the stored read classified as removed since sync; `activity_halt_reason` set by a halt +and cleared by the next run that does not halt, `halt_reason` unchanged; `--allow-deletions` +applying N to events and history commits separately; a history replace keeping a commit's stored +author link when GitHub returns none (D28); each verify check and each stored-row check, by a +planted row deletion. Acceptance: on the Step 3b-2 acceptance repositories and on `prebid/github-activity-db`, the stored events equal the log and the stored commits equal the history; each repository's first logged event -is recorded (section 8); the run records its REST calls, points, and time. +is recorded (section 8), with the file count of its largest push's compare; the run records its +REST calls, points, and time. ### Step 5: Full bootstrap and verification @@ -2103,8 +2161,11 @@ matches REST; a difference report against the tracked file classifies every diff (closed-unmerged PR, file-status rebuild, commit list beyond 250, login rename, changed on GitHub after the old file's sync, in-rule item absent from the old file, schema-only) with per-category counts, zero unclassified items, and five REST-checked examples per category; the REST sample measures whether a -body edit, a base-ref change, and a review dismissal change `updatedAt`, and one that does not gets -an index-side check or a periodic re-hydration before Step 6 (gap 109); the run's time, cost, file +body edit, a base-ref change, a review dismissal, and (from Step 4a) an issue label event change +`updatedAt`, at least 20 cases of each where the repositories hold them; a kind with a case that +leaves `updatedAt` unchanged, or with fewer than 20 cases, gets an index-side check or a periodic +re-hydration in a pull request of its own before Step 6a, which adds the 5.3 step 3 rule and its +tests (gap 109); the run's time, cost, file size, and secondary-limit responses at one request in flight are recorded (and at any wider setting proposed). @@ -2124,27 +2185,29 @@ unverified repositories (5.6); through `httpx.MockTransport`, a mocked installation-token response whose `expires_at` lies just over a minute ahead, a clock moved past it mid-run, and a second token request before the next API request (githubkit keeps a token until one minute before `expires_at`); and a 401 on an API request still -exiting 1 with the run closed (5.3 step 4); `snapshot --sync-run` refusing an id that is not newer -than the downloaded file's, and the job failing when its database file is missing or differs from -the one `verify --db-only` checked (5.6). 6b creates release `db` and uploads the Step 5 database -with its manifest (`ghactivity snapshot --verify-report` from Step 5's `verify --exact --report`) as -the first snapshot. It creates release `db-test` the same way, holding snapshots prepared offline: -the corrupted cases below, one with a `pr_files` row deleted (a verify error), one built at the -revision before the last migration, and one holding an extra repository absent from `tracking.toml` -(a sync exit 2). It then runs one normal manual run and one bootstrap run, and one run whose -verify exit-code output is unset, which publishes nothing and marks the workflow failed. Before 6c, -6b hydrates with the App the recorded sample the maintainer's token read (team review requests, -organization assignees, every resolve field, and each repository's index totals) and compares the -two column by column; a null, false, or zero the App returns where the token read a value blocks 6c -until the permission is granted or the field is dropped (gap 107). 6c enables the -schedule and stops tracking `github_activity.db`: it removes the file +exiting 1 with the run closed (5.3 step 4); `verify --db-only --out` writing the file and printing +its SHA-256; `snapshot --after-run --sync-exit` refusing no newer row, two newer rows, a row closed +with another exit code, and a tracked repository whose `last_run_id` is older (5.6); and a test +that reads `sync.yml` and fails unless every `if:` that reads an exit-code output compares a quoted +string and requires it to be non-empty. 6b creates release `db` and uploads the Step 5 database +with its manifest (`ghactivity snapshot --sync-run` naming Step 5's last run, and +`--verify-report` from Step 5's `verify --exact --report`) as the first snapshot. Before any App +run writes to release `db`, 6b reads one sample with the App and with the maintainer's token back +to back (team review requests, organization assignees, every resolve field, and each repository's +index totals) and compares them, every column and child row; any difference blocks App runs that +publish to `db`, and 6c, until it is explained or the permission is granted, and until then App +runs publish only to `db-test` (gap 107). It creates release `db-test` the same way, holding +snapshots prepared offline: the corrupted cases below, one with a `pr_files` row deleted (a verify +error), one built at the revision before the last migration, and one holding an extra repository +absent from `tracking.toml` (a sync exit 2). It then runs one normal manual run and one bootstrap +run. 6c enables the schedule and stops tracking `github_activity.db`: it removes the file from the index, the `!/github_activity.db` exception from `.gitignore`, the `check-added-large-files` exclude, the `tracked-db-rollback-journal` and `tracked-db-no-sidecar` hooks, the D26 freeze check, and `alembic/env.py`'s refusal of the repository-root `github_activity.db` and of any file of that name (gap 73); replaces the tracked-file journal test with one on the snapshot `VACUUM INTO` writes; records the D20 WAL decision for the -working file; and documents how to download the latest snapshot and the staleness check 5.6 -gives consumers. +working file; and documents how to download the latest snapshot and the staleness rule 5.6 gives +consumers. Acceptance: two consecutive scheduled runs publish, and their manifests list drift, pending, and failure counts; the bootstrap run publishes, and the next scheduled run starts from its snapshot; a @@ -2301,38 +2364,39 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in |------|------------------| | Intermittent GraphQL failure rate at full scale (prototype needed 0 retries) | Step 5 measures it | | Secondary-limit responses with more than one request in flight | Step 5 measures before any wider setting | -| Whether a heartbeat commit counts as repository activity for schedule disabling (undocumented); a disabled schedule fails no run | Not provable inside Step 6; the heartbeat is best effort, and the consumer's staleness check (5.6, gap 107) shows a schedule that stopped | +| Whether a heartbeat commit counts as repository activity for schedule disabling (undocumented); a disabled schedule fails no run | Not provable inside Step 6; the heartbeat is best effort. A schedule that stops starts no run, so this repository cannot see it; 5.6 states the consumer's staleness rule, and the dashboard's integration builds it (section 9, gap 107) | | Issues read permission for the App | Step 6b's first manual run exercises it with a real token | | GraphQL `rateLimit` readings alternating between two windows (seen with a personal token; GitHub documents that values vary by the region handling a request, but not differing `resetAt` values) | Step 2's guard handles it; Step 6 records App-token readings | -| Whether body edits, base-ref changes, review dismissals, and issue label events always change `updatedAt` (not measured; a state change is caught by the index) | Step 5's REST sample measures it; a change that does not change `updatedAt` gets an index-side check or a periodic re-hydration before Step 6 (gap 109) | +| Whether body edits, base-ref changes, review dismissals, and issue label events always change `updatedAt` (not measured; a state change is caught by the index) | Step 5's REST sample measures each kind, issue label events from Step 4a; a kind not shown to change `updatedAt` gets an index-side check or a periodic re-hydration before Step 6a (gap 109) | | The error body of a GraphQL secondary limit returned as HTTP 200 (not observed; documented only as "an error message") | Step 2 classifies by type and message; a message it does not recognize is "not understood" (5.3 step 4) and stops the run with exit 1, naming the message; Step 5 records secondary-limit responses | | A PR file list above 3,000 entries (no tracked PR found) | Validation fails the item if `files` stops short of `totalCount`; Step 5 reports any | | Compare fallback on a PR whose head fork was deleted (only a live fork tested) | A count mismatch fails the item; Step 5 reports any | | Full bootstrap time, cost, and size are projections from two repositories | Step 5 measures them | | Overflow continuations per request (25), the server retry delays (2, 4, 8 s), the reset margin (1 s), retry jitter (25%), the index boundary margin (60 s), the budget reserves (100 points, 100 calls), the longest accepted rate-limit wait (65 min), and the HTTP timeouts (read 30 s, connect 10 s) are chosen, not measured | Step 5 records secondary-limit responses, budget readings, and request times at these values | | The cost of one commit per item in rollback-journal mode (not separated; whole-run times on three repositories are in section 7, "Step 3b-2a-2c acceptance") | Step 5 measures the full run | -| Whether the activity log's 2023-03-07 start is fixed or a rolling window (seen on two repositories) | Step 4b stores each repository's first logged event in `activity_first_event_at`; it reads the whole log every run, so an event that ages out leaves the database as GitHub stops listing it (D7) | +| Whether the activity log's 2023-03-07 start is fixed or a rolling window (seen on two repositories) | Step 4b stores in `activity_first_event_at` the oldest event each complete read lists, one before the floor included; it reads the whole log every run, so an event that ages out leaves the database as GitHub stops listing it (D7) | | Whether `history(since:)` filters on the committed or the authored date (only its `totalCount` was read) | Step 4b's recordings settle it before the coverage rule's wording for commits is final | | Whether REST compare caps the `files` it returns for a large push (GitHub documents a cap of 300 files per comparison; not measured live) | Step 4b marks a list of 300 `files_at_cap` (gap 106) and records its largest push's file count | | The REST cost of comparing every activity event on a first sync, and of reading every log in full each run (about one call per event compared, and one per 100 events read; Prebid.js's default branch has 4,560 commits since the floor) | Step 4b records it; Step 5 measures the full run | | How a rebase merge appears in the activity log: whether its `pr_merge` event's `after` is the PR's `mergeCommit.oid` and its other commits come through the compare (the measured repositories hold merge-commit and squash merges only) | Step 4b records one on a tracked repository that allows rebase merges, before D30's join wording is final | | Whether GitHub returns `EnterpriseUserAccount.id` to a token without the `read:enterprise` scope (the dry run validated the document; no tracked repository has such an actor yet) | Step 3b-1 tests the synthetic actor. A live occurrence decides between two outcomes (a prediction): an error at the alias's path is an item failure (exit 2), and an error without a path is fatal (5.3 step 4, exit 1), which Step 5's bootstrap would surface | -| Whether the scheduled App token can read every resolve field the token used here read (`diskUsage`, `webCommitSignoffRequired`, `hasVulnerabilityAlertsEnabled`, and `planFeatures` included) | Step 6b's first manual run; a field it cannot read is an error 5.3 step 4 classifies | +| Whether the scheduled App token can read every resolve field the token used here read (`diskUsage`, `webCommitSignoffRequired`, `hasVulnerabilityAlertsEnabled`, and `planFeatures` included) | Step 6b's App-and-token comparison, before any App run publishes to `db`; a field it cannot read is an error 5.3 step 4 classifies, and a difference blocks App publication (Step 6, gap 107) | | A `DisconnectedEvent`'s `source` and `subject` (none in the 300 PRs read for section 7, "Closing-link timeline events"; GitHub's schema gives it `ConnectedEvent`'s fields) | Step 3b-2b-2 tests a synthetic one; a live one is recorded when seen | | Whether a closing-keyword link can change without the PR's `updatedAt` changing (an issue transfer or deletion, a body edit, a base-ref or default-branch change); 5.4 then reports the difference as an error, unless it is a difference in the totals alone that the connected or disconnected events since the boundary cover (5.4) | Step 3b-2b-2's acceptance lists each such error with its PR and link; Step 5 reports any | | Whether a pending `reviewRequests` entry for Copilot carries `BOT_kgDOCnlnWA` (the review-request events and the reviews do; no pending request was seen, section 7, "Copilot review requests" and "Step 3b-2a-2c acceptance") | Step 5 reports any Copilot request row; for a `databaseId` reported under a second node id, the writer keeps the stored node id and the run reports the second (5.2) | | Whether to collect a `Mannequin`'s `email` ("The mannequin's email on the source instance"; the maintainer's profile decision named users and organizations only, and no mannequin was available to read, so whether GitHub requires a scope for it is unmeasured) | The maintainer decides; a live read on a mannequin comes first | | Whether GitHub returns an `EnterpriseUserAccount`'s `enterprise` to the tokens used (an `Enterprise` object; no tracked repository has such an actor) | Not selected until a live occurrence is read | | Whether GitHub returns an `EnterpriseTeam` requested reviewer's `id`, `fullDatabaseId`, and `slug` to the tokens used (the batch with them dry-runs at 2 points; no tracked repository has such a request) | Step 3b-1 tests a synthetic one; a live occurrence decides, as for `EnterpriseUserAccount.id`: an error at the alias's path fails the item (exit 2) | -| Whether the scheduled App token reads a `Team` requested reviewer and an `Organization` assignee as the token used here does (null or FORBIDDEN not measured) | Step 6b's first manual run; a FORBIDDEN fails the item, which 5.3 step 4 classifies, and a null where the token read a value blocks 6c (Step 6, gap 107) | +| Whether the scheduled App token reads a `Team` requested reviewer and an `Organization` assignee as the token used here does (null or FORBIDDEN not measured) | Step 6b's App-and-token comparison, before any App run publishes to `db`; a FORBIDDEN fails the item, which 5.3 step 4 classifies, and any difference blocks App publication (Step 6, gap 107) | | `--help` anywhere on a command line, or `--version` before the subcommand, exits 0 without doing any work (click's eager options), the code D25 publishes on, so a workflow input that expands to either would publish the unchanged database | Step 6's workflow passes inputs as validated option values, never as raw arguments | ## 9. Later work Dashboard integration and replacement of its data layer (it reads the snapshot as 5.6 "Consumer access" describes), a query API, AI enrichment (D12), -company affiliation, CI check status, releases, and additional repositories (the consumer's -staleness check moved to 5.6 and Step 6, gap 107). File classification +company affiliation, CI check status, releases, and additional repositories. The dashboard's +integration includes 5.6's staleness rule, the only check that can see a schedule that stopped +(gap 107). File classification and dependency tagging are not planned; excluding paths from line counts is the consumer's: `pr_files` keeps every path with its counts (D29). From 423881db8b24895dedfbbefc5c2992555d748fe4 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 16:28:25 -0600 Subject: [PATCH 07/15] docs(plan): settle the remaining gaps in D38's rules Accepted failures match class and message exactly and can name an account by github_id or node_id; a refresh failure an account entry accepts leaves the exit code alone, and verify --exact matches entries as 5.5 does. code_version decides a checkout by a .git entry before reading git's exit codes. A stamp is refused by on_version_apply inside the transaction. Gap 110 refuses only for rows no later run reads. snapshot takes --after-run/--sync-exit or --sync-run, checks the stored row of every configured entry, and names its manifest keys. 6b's App-and-token comparison names its runs and script. Tests follow each rule. --- CLAUDE.md | 4 +- docs/redesign-plan.md | 264 ++++++++++++++++++++++++------------------ 2 files changed, 154 insertions(+), 114 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index f9649fb..1841613 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -67,8 +67,8 @@ sampler (Step 3b-2b-1), and `verify` (Step 3b-2b-2). The tracked `github_activit frozen at the first schema (D26). **Planned in the redesign plan:** D37 in the sync (Step 3b-2a-3), the REST sampler (Step -3b-2b-1), and verify (Step 3b-2b-2); GitHub Issues and closing-issue links (Step 4); scheduled -sync and release publication (Step 6). +3b-2b-1), and verify (Step 3b-2b-2); GitHub Issues, closing-issue links, and default-branch +activity (Step 4); the full bootstrap (Step 5); scheduled sync and release publication (Step 6). **Out of Scope (Future):** - Webhooks / real-time sync diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index faa5c1a..aa09e71 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -92,7 +92,7 @@ Within scope the database collects and distinguishes data and never interprets i | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, and a repository error during a timeline lookup, makes a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | -| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class, of an item or of an account's refresh, so a permanent failure can be accepted without turning every run red. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write or determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the log's end or the floor, and fails a history read whose count still differs after a rescan. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a compare-linked account a lookup answers `NOT_FOUND` (D28; now also counted per run), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | +| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class, of an item or of an account's refresh, so a permanent failure can be accepted without turning every run red. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the log's end or the floor, and fails a history read whose count still differs after a rescan. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now also counted per run), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | ## 3. Coverage rule @@ -227,18 +227,18 @@ Every gap found in the audit, and where it is resolved. | 95 | `timelineItems(since:)` keeps only items created after `since`, so a lookup passing the boundary would miss an event at it; GitHub's schema describes `filteredCount` as the count after applying `before` and `after`, and names neither `itemTypes` nor `since` | Step 3b-2b-2: `since` one second before the boundary, and `filteredCount` used as measured (5.8; section 7, "Timeline `since`") | | 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode rebuilds `recorded/` from what it records, and `--only documents` drops every unit kind but the compare and rate-limit kinds it keeps; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | | 97 | Without `--exact`, an unchecked item and an unverified repository did not fail verify, and a timeline lookup whose entries differed from `filteredCount` left its item unchecked, so the scheduled verify (5.6, no `--exact`) could exit 0 and D25 publish a database it had not checked | D37: both fail verify in every mode, and an answer verify cannot accept is an error (5.4, 5.8) | -| 98 | A 24-hour account refresh that still fails, or whose answer the writer refuses (another `databaseId` for a stored node id), left the run `complete` with exit 0; the only trace was a printed line, and a test pinned exit 0 | Step 3b-2a-3: counted in `sync_runs.refresh_failures`, listed in the report, and exit 2 (5.3 step 4, D38) | +| 98 | A 24-hour account refresh that still fails, or whose answer the writer refuses (another `databaseId` for a stored node id), left the run `complete` with exit 0; the only trace was a printed line, and a test pinned exit 0 | Step 3b-2a-3: counted in `sync_runs.refresh_failures`, listed in the report, and exit 2 unless an account entry in `[sync].accepted_failures` accepts it (5.3 step 4, 5.5, D38) | | 99 | A run that reached `--max-minutes` exited 0, and a deadline inside a batch turned the item failures it had computed into pending items | Step 3b-2a-3: a run that reaches its deadline exits 2, which D25 still publishes with the workflow red, and the failures a batch made final before the deadline are recorded (5.3 steps 6 and 8, D38) | | 100 | A PR the run's own index listed in coverage that hydration then answered `NOT_FOUND`, or returned out of coverage, was neither written, failed, nor pending, and nothing reported it; a hydration older than the index entry was written without a check | Step 3b-2a-3: each is an item error (5.3 steps 4 and 5, D38) | -| 101 | An `accepted_failures` entry accepted any failure of its item, so a new cause stayed out of exit 2 | Step 3b-2a-3: an entry names the error class it accepts (5.5, D38) | +| 101 | An `accepted_failures` entry accepted any failure of its item, so a new cause stayed out of exit 2 | Step 3b-2a-3: an entry names the error class and the message it accepts, and may name an account (5.5, D38) | | 102 | A PR with more than the 10 closing links the index carries was not re-hydrated when a link beyond the 10th was swapped with equal totals | Step 3b-2a-3: such a PR is hydrated every run (5.3 step 3, D38). Verify keeps comparing the totals and the indexed links: the sync re-reads every link of such a PR each run, and verify's child-count check holds the stored rows to the hydrated total | -| 103 | A `databaseId` reported under a second node id kept the stored node id and dropped the reported one; only a printed line recorded it; a commit link's node id was never read | Step 3b-2a-3: `account_node_ids` stores every node id a sighting, a lookup, or a commit link reports for an account (5.1, 5.2, D38) | +| 103 | A `databaseId` reported under a second node id kept the stored node id and dropped the reported one; only a printed line recorded it; the sync used a commit link's node id, which the source reads, only as a lookup key, never storing or checking it | Step 3b-2a-3: `account_node_ids` stores every node id a sighting, a lookup, or a commit link reports for an account (5.1, 5.2, D38) | | 104 | Source checks that accepted what they should refuse: a REST response without readable rate-limit headers, a GraphQL `errors` present but not a list (`{}`, `""`), a compare above 250 commits checked by count but not membership, and an index page from a repository other than the one resolved | Step 3b-2a-3 (5.3 steps 2, 4, and 5, D38) | | 105 | Run records and setup that hid a failure: `code_version` without `-dirty` when `git status` failed, a close that ignored outcomes it could not write, a missing `[automation]` table emptying the automation lists, `foreign_keys` never read back, `alembic stamp` accepted, a wheel check comparing repository names only, a log file no record reaches, and no record of a run's `--repo` filter | Step 3b-2a-3 (5.1, 5.3 step 8, 5.5, D38) | | 106 | Step 4b: compare `files` stop at GitHub's documented 300 per comparison with no total, so a capped list read as complete; the activity read had no paging or completeness rule; a history count mismatch had no outcome; a refused compare was never retried; and the log's first event was not stored | Step 4b (5.1, D38) | | 107 | Step 6: an App token may return null where the maintainer's token read a value; a stopped schedule turns nothing red; a missing `DATABASE_URL` builds from empty; an empty exit-code output compares equal to 0 in `if:`; and `snapshot` did not check that the job's own sync ran | Step 6 and 5.6 (D38), except the stopped schedule: no run starts, so this repository cannot see it; 5.6 states the consumer's staleness rule, which the dashboard's integration builds (section 9) | | 108 | The data dictionary presented head-repository, milestone, and team names as GitHub's current values, though they are as of `hydrated_at` | Step 3b-2a-3: described as of `hydrated_at` (D38) | -| 109 | Edits that may leave `updatedAt` unchanged (body, base ref, review dismissal) were accepted as missed | Step 5 measures them; one that does gets an index-side check or a periodic re-hydration (section 8, D38) | +| 109 | Edits that may leave `updatedAt` unchanged (body, base ref, review dismissal, issue label events) were accepted as missed | Step 5 measures at least 20 cases of each; a kind not shown to change `updatedAt` gets an index-side check or a periodic re-hydration before Step 6a (section 8, D38) | | 110 | A GitHub-derived column a later migration adds to a table without `hydration_version` (accounts, compares), or a new table of such data, is never filled for existing rows | 5.1: such a migration adds a per-row read version or refuses while the file holds the rows it would leave unread (D38) | ## 5. Design @@ -269,15 +269,16 @@ IMMEDIATE`, and revision the upgrade started from, with no revision of that upgrade applied. `PRAGMA foreign_key_check` runs inside that transaction after the last revision and raises on any row, before the commit. It runs before every command commits, one that runs no revision (`current`, -`check`, and a `stamp` that changes nothing) included; a `stamp` that would change the revision is -refused before any statement (gap 105): alembic loads each revision module inside the transaction, -so a module's top-level code can change rows or the schema without a revision running (section 7, -"Revision module code"), and a file that already holds a violation fails every command that -connects. A migration that batch-alters a table a view reads drops those views first and -recreates them afterwards from its own frozen SQL; `legacy_alter_table` stays off, so a view left -in place fails the migration (gap 71). Migrations import only the standard library, alembic, and -SQLAlchemy, never application code, and name every index, CHECK constraint, and foreign key they create through -`op.f()`: env.py passes +`check`, and a `stamp` that changes nothing) included; a `stamp` that changes the revision is +refused by an `on_version_apply` callback, which raises on alembic's `MigrationInfo.is_stamp` +inside the command's transaction, so nothing commits, and a `stamp` at head has no step (gap 105): +alembic loads each revision module inside the transaction, so a module's top-level code can change +rows or the schema without a revision running (section 7, "Revision module code"), and a file that +already holds a violation fails every command that connects. A migration that batch-alters a table a +view reads drops those views first and recreates them afterwards from its own frozen SQL; +`legacy_alter_table` stays off, so a view left in place fails the migration (gap 71). Migrations +import only the standard library, alembic, and SQLAlchemy, never application code, and name every +index, CHECK constraint, and foreign key they create through `op.f()`: env.py passes `Base.metadata` as `target_metadata`, and alembic applies its naming convention to every migration's operations, old revisions included (on a fresh build, the unnamed foreign keys of `01421d8dfaeb` and `a2b3c4d5e6f7` take the head convention's names, which is harmless only because Step 3 drops @@ -342,9 +343,9 @@ and `color`, `pr_user_tags.created_at`). | `default_branch_commits` | `repository_id`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at`, `committed_at`, `parent_count`, `additions`, `deletions`, `changed_files` | Step 4b; PK (`repository_id`, `sha`); the default branch's history since the floor as GitHub lists it (D30); the author link as in `pr_commits`, a replace keeping the stored link of the row with the same SHA when GitHub returns none (D28); `parent_count` is GitHub's `parents.totalCount`; `changed_files` is GitHub's `changedFilesIfAvailable`, null when GitHub returns none; no message (D19) | | `automation_accounts` | `github_id`, `login`, `reason`, `run_id` | integer primary key; `github_id` unique; replaced from `tracking.toml` at the start of every run, `run_id` naming that run (a reference to `sync_runs`) | | `account_profiles` | `account_id`, `company`, `location`, `website_url`, and an organization's `description`, `email`, `twitter_username`, `is_verified`, `archived_at`, `has_sponsors_listing`, and `pinned_items_remaining`, `refreshed_at` | PK `account_id`, a reference to `github_users` (`ON DELETE CASCADE`); rows only for a `User` or an `Organization` (D31); each lookup that returns the account replaces the whole row with the profile it returned, through the writer and in the transaction that writes `github_users`' login and type, so no earlier value survives a field GitHub now returns as null or a change of type; a lookup that returns `NOT_FOUND` leaves the row unchanged, marked by `github_users.not_found_at` (D28); every field nullable as GitHub's is, the organization fields null for a user and `company` null for an organization; `refreshed_at` as on `github_users` | -| `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting, a lookup, or a commit link, the one `github_users.node_id` holds included, with the first and latest runs that reported it, so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); its migration refuses a file whose `github_users` holds rows (gap 110); consumer | +| `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting, a lookup, or a commit link, the one `github_users.node_id` holds included, with the first and latest runs that reported it, so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); its migration refuses a file whose `github_users` holds rows, since no file was published before Step 6b (gap 110); consumer | | `automation_emails` | `email`, `reason`, `run_id` | integer primary key; `email` unique, stored lowercase ASCII (SQLite's `lower()`, which the views apply to commit emails, folds only ASCII letters); replaced the same way | -| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `links_not_found`, `repo_filter`, `deletions_approved` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, else `manual`; `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes and `-unknown` when it fails or times out, gap 105; `git ls-files --error-unmatch` exiting 1 means the package is not tracked, and any other failure of it or of `git rev-parse`, a missing `git` included, records the package version followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted; `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `links_not_found` the commit-link accounts a lookup answered `NOT_FOUND` this run (5.2), and `repo_filter` the repositories `--repo` named, null for a run without it; the migration that adds these three refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | +| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `links_not_found`, `repo_filter`, `deletions_approved` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, else `manual`; `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes and `-unknown` when it fails or times out, gap 105; the package runs from a checkout only when its directory or a parent holds a `.git` entry, and then `git ls-files --error-unmatch` exiting 1 means the package is not tracked, while any other failure of a git call, exit 128 and a missing `git` included, records the package version, or the commit when `rev-parse` answered, followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted; `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `links_not_found` the commit-link accounts a lookup answered `NOT_FOUND` this run, gap 90's lookups included (5.2), and `repo_filter` a JSON array of the `--repo` values as given, sorted, null for a run without `--repo`; `refresh_failures`, like `failures`, counts accepted ones; the migration that adds these three refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | | `repo_sync_state` | `repository_id`, `last_index_started_at`, `index_pr_total`, `index_issue_total`, `last_success_at`, `last_run_id`, `last_outcome`, `halt_reason`, and from Step 4b `activity_read_at`, `activity_first_event_at`, `activity_last_outcome`, `activity_halt_reason`, `history_read_at`, and `default_branch_commits_total` | PK `repository_id`; `activity_first_event_at` is the `timestamp` of the oldest event the latest complete activity read listed, so a consumer can tell a quiet period from a log that does not reach back (gap 106); `activity_read_at` is the GitHub `Date` header of the first page of the latest activity-log read that completed and was written; `history_read_at` and `default_branch_commits_total` are the first page's `Date` header and the `totalCount` of the latest history read that replaced the rows, written only with that replace; `activity_last_outcome` in (`complete`, `time_limited`, `failed`, `halted`); `activity_halt_reason` holds a Step 4b deletion halt's reason and candidate counts and is cleared by a run whose Step 4b did not halt (Step 4b); `last_run_id` is the latest run that attempted the repository, or found it untracked, and `last_outcome` its result, in (`complete`, `time_limited`, `failed`, `incomplete_index`, `halted`, `untracked`), both written by step 8, or by the close of a fatal or interrupted run (5.3 step 8); `last_index_started_at` is written only when that index completed and the repository did not fail later in the run, a time-limited repository included (its unreached items are in `sync_pending`), and never by the close of a fatal or interrupted run; `last_success_at` is the run's `finished_at` when `last_outcome` is `complete` and no item of the repository failed outside `[sync].accepted_failures`; `halt_reason` holds step 7's halt and is cleared by a run whose step 7 ran for the repository and did not halt, a run whose step 7 did not run for it leaving it | | `sync_pending` | `repository_id`, `item_kind`, `number`, `run_id` | PK (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); items a time-limited run planned but did not reach; step 8 replaces the rows of each completely indexed repository that did not fail later in the run (a repository gone during hydration keeps its rows, 5.3 step 4) with this run's unreached items, so a run that reached them all leaves none; consumer, so an in-coverage item not yet hydrated is told apart from one outside coverage | | `sync_failures` | `repository_id`, `item_kind`, `number`, `attempts`, `last_error`, `first_failed_at`, `last_failed_at`, `last_run_id` | unique (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); `attempts` counts the runs in which the item failed: `last_run_id` is the latest such run, and a failure adds one only when its run is another, whatever the runs' clocks read; `last_error` is the error's class and message; row deleted on success, or by step 7 when its repository's in-coverage set no longer lists the item | @@ -393,9 +394,13 @@ Column rules: A table without `hydration_version` (`github_users`, `account_profiles`, the Step 4b compare tables) to which a later migration adds a GitHub-derived column gains a per-row read version that the planner compares the same way, or the migration refuses while the table holds rows, so - a column is never left unread on old rows; a migration that adds a table of GitHub-derived data, - or a run counter, refuses while the file holds the rows the new table or column would describe, - so no row reads as "none" where nothing was recorded (gap 110). + a column is never left unread on old rows. A migration that adds a table of GitHub-derived data + refuses only while the file holds rows no later run reads for it, so no row reads as "none" where + nothing was recorded; a table or column every run fills for every stored parent, whose unread + state is already marked, does not refuse (Step 4a's issue tables, Step 4b's activity, compare, + and history tables and `repo_sync_state` columns, unread while `activity_last_outcome` or + `history_read_at` is null). A run counter added after Step 6b is nullable, null meaning not + recorded, since a refusal there would force a bootstrap (gap 110). - `state_reason` is GitHub's value including `DUPLICATE` (5.8). Descriptive enums, every GitHub enum column except a PR's, issue's, and review's `state` (`github_users.type` keeps its CHECK), such as `change_type`, `author_association`, `state_reason`, `merge_state_status`, and @@ -574,9 +579,9 @@ same number. from its own `Date` header and leaves `refreshed_at`. The refresh runs after the last hydration write and before step 7, oldest `refreshed_at` first, and stops at the deadline; in it, a `databaseId` that differs from the stored `github_id` is a failed refresh, which makes the exit 2 - (5.3 step 4). Accounts that no stored - row references any more stay stored and refreshed: within one file, `github_users` holds every - account it ever stored (a bootstrap starts without them, section 9). + unless an account entry in `[sync].accepted_failures` accepts it (5.3 step 4). Accounts that no + stored row references any more stay stored and refreshed: within one file, `github_users` holds + every account it ever stored (a bootstrap starts without them, section 9). - Automation lists: `[automation].accounts` and `[automation].emails` name accounts and commit emails the maintainer lists, each with a reason. Each run copies both lists into `automation_accounts` and `automation_emails` with its run id, and the views show the matching @@ -690,8 +695,9 @@ an interrupt, which is re-raised. A failure the engine detects itself (these two, a hydrated `github_id` other than the index's, a review or comment id stored under another PR, a per-batch lookup's `databaseId` other than the commit link's) is recorded at once, without the solo retry, which would repeat a deterministic - result. A hydration that contradicts the run's own index (step 5) is the exception: a lagging - read can clear, so the engine requests the item once more alone before it records the failure. + result. Step 5's three contradictions of the run's own index (an alias `NOT_FOUND`, an item + outside coverage, an earlier `updatedAt`) are the exception: a lagging read can clear, so the + engine requests the item once more alone before it records the failure. The planner is a pure function over snapshots of the stored rows, which `--dry-run` reuses. The run reads its snapshot once, after the claim and before step 1 (a dry run, after the head check). 4. Hydrate: GraphQL batches of 25 PRs or 100 issues of one repository via aliased @@ -731,9 +737,9 @@ an interrupt, which is re-raised. item error for each item waiting on those accounts (5.2). A failed 24-hour refresh (a lookup that still fails after its retries, an answer the typed models or the id check reject, or a `databaseId` other than the stored `github_id`) keeps the stored values (D28), is listed in the - report, counted in `sync_runs.refresh_failures`, retried next run, and makes the exit 2 (D37, - gap 98). Fields that drive logic (`state`, review `state`) are closed enums in the typed models; - descriptive enums are text (5.1). + report, counted in `sync_runs.refresh_failures`, retried next run, and makes the exit 2 unless + an account entry in `[sync].accepted_failures` accepts it (D37, gap 98). Fields that drive logic + (`state`, review `state`) are closed enums in the typed models; descriptive enums are text (5.1). Any other exception after the claim is fatal too. A fatal run closes as `failed` (exit 1), with step 7's halt reasons when step 7 ran. @@ -769,7 +775,9 @@ an interrupt, which is re-raised. with state `CLOSED` (D3). A hydration that contradicts the run's own index (an alias `NOT_FOUND`, an item outside coverage, an `updatedAt` earlier than the index entry's) is requested once more alone by the engine, which holds the index, and a second contradiction is an item failure, - recorded before any account lookup for it (gap 100). Above 250 commits, only the commit + recorded before any account lookup for it (gap 100); a deadline that refuses that request + leaves the item in `sync_pending`, and the batch's other checked items are written as at any + deadline. Above 250 commits, only the commit connection's first page is checked against compare's list: it holds the oldest commits, which a wrong base adds or drops, and reading the rest would cost continuations a count check already covers. @@ -827,14 +835,15 @@ an interrupt, which is re-raised. indexed completely and nothing failed; 2 when an item failed in this run (other than one listed in `[sync].accepted_failures`), a repository failed, had an incomplete index, or halted, a stored repository is untracked in a run without `--repo`, a prune was refused (step 7), a refresh - failed (step 4), or the run reached its deadline (status `time_limited`, whatever it stopped: - hydration, the refresh, or a Step 4b read; gap 99); 1 on a fatal error. The - exit code is decided after the prune. An accepted failure is still recorded and reported; it - names the repository and number, or the account's `github_id`, with the error class and reason - in `tracking.toml`, so a permanent failure does not turn every run red, and a failure of another - class is not accepted (gap 101). An entry + failed (step 4, other than one an account entry accepts), or the run reached its deadline (status + `time_limited`, whatever it stopped: hydration, the refresh, or a Step 4b read; gap 99); 1 on a + fatal error. The exit code is decided after the prune. An accepted failure is still recorded and + reported; it names the repository and number, or the account, with the error class, the message, + and a reason in `tracking.toml`, so a permanent failure does not turn every run red, and a + failure of another class or message is not accepted (5.5, gap 101). An item entry is reported as not having failed when its repository's outcome in this run is `complete`, - `time_limited`, or `halted` and its item neither failed in this run nor is pending (5.5). + `time_limited`, or `halted` and its item neither failed in this run nor is pending (5.5); an + account entry, only when this run's refresh looked the account up and the lookup succeeded. `--max-minutes` stops queueing new hydration, records in `sync_pending` the planned items it did not reach (a repository that failed during hydration keeps its rows, step 4), still runs step 7 for complete repositories and step 8, and sets status `time_limited`; unreached items are not @@ -937,8 +946,8 @@ the first class that matches, in this order: An error, an unchecked item, and an unverified repository fail verify in both modes: what verify could not check is a failure, never a pass (D37). `verify --exact` also fails on not synced, pending, halted deletion, removed since sync, and known failures no `[sync].accepted_failures` -entry matches by repository, number, and error class; it is the mode the Step 3-5 acceptance uses, -and drift is reported in both modes. A repository that verify cannot +entry accepts as 5.5 matches it (repository, number, error class, and message); it is the mode the +Step 3-5 acceptance uses, and drift is reported in both modes. A repository that verify cannot resolve, re-index, or look up is reported as unverified: every repository error the source raises at resolve or index (not found, no default branch, a fetch that still fails after its retries, a response or GitHub error the resolve or index rejects, a label list that fails its count or names a @@ -991,7 +1000,8 @@ state per check. `ghactivity verify --db-only` (Step 6a, beside the `snapshot` command that writes its inputs; it takes the snapshot gz, its manifest, the asset digest, and `--out PATH` as explicit arguments, and a halt -exits 1) decompresses to PATH, prints that file's SHA-256 on its last line once every check passes, +exits 1) decompresses to PATH, prints that file's SHA-256 and its highest `sync_runs` id (0 when +it has none) on its last line once every check passes, and checks a downloaded snapshot before anything touches it: the gz SHA-256 equals the manifest's (and the release asset's `digest` when GitHub reports one; a null digest leaves the manifest as the only check); decompression succeeds; `PRAGMA integrity_check` returns `ok` and @@ -1019,13 +1029,13 @@ Python `timedelta` holds, so the planner can build the window), `delete_guard_mi of at least 0, and `delete_guard_percent` a number above 0 and at most 100; an `accepted_failures` entry names an item (a configured `repository` and a `number` of at least 1) or an account (`github_id`, for a -refresh failure), and an `error`: the error class it accepts, compared for equality with the text -of `sync_failures.last_error` (or the refresh failure's text) before its first `": "`, which the -loader refuses unless it is one of the failure classes the sync records; for a class whose message -carries the cause (`AliasError`, `UserLookupFailed`), the entry also gives `message`, compared for -equality with the rest of that text. Entries are unique by item or account, an item entry matches -by its configured repository, so a rename does not unhook it, and a new cause is not accepted -(gap 101); an entry whose item or account did not fail is reported (5.3 step 8). +refresh failure, or `node_id` for an account stored with a null `github_id`), an `error`, and a +`message`: the class and the message of `sync_failures.last_error` (or of the refresh failure's +text), split at its first `": "` and each compared for equality, since several classes carry +more than one cause in their message; the loader refuses a class that is not one of the failure +classes the sync records, and an entry without `message`. Entries are unique by item or account, an +item entry matches by its configured repository, so a rename does not unhook it, and a new cause is +not accepted (gap 101); an entry whose item or account did not fail is reported (5.3 step 8). `[automation]` is required, its lists may be empty, so a file without it is refused instead of emptying the automation tables (gap 105). The file holds domain policy only. Step 1 added `[[repositories]]`; Step 3b-2a-1 adds `[coverage]` and `[sync]`'s `unsettled_minutes`, @@ -1057,9 +1067,11 @@ emails = [ unsettled_minutes = 10 delete_guard_min_rows = 25 delete_guard_percent = 2.0 -# Items whose failure is understood and does not count toward exit 2 (5.3 step 8). +# Items or accounts whose failure is understood and does not count toward exit 2 (5.3 step 8). accepted_failures = [ - # { repository = "prebid/Prebid.js", number = 12345, error = "ItemConflict", reason = "..." }, + # { repository = "prebid/Prebid.js", number = 12345, error = "ItemConflict", + # message = "#12345: ...", reason = "..." }, + # { github_id = 1234567, error = "UserLookupFailed", message = "...", reason = "..." }, ] ``` @@ -1087,13 +1099,14 @@ Workflow `.github/workflows/sync.yml`: `verify --db-only`; a missing or failing snapshot halts the run. With `bootstrap`, it downloads nothing and starts from an empty file, and the manifest records `bootstrap: true`. One job-level `DATABASE_URL` names the file every step uses, the `--out` path of `verify --db-only`; without - `bootstrap`, before `alembic upgrade head` the job records the file's highest `sync_runs` id and - fails unless the file's SHA-256 equals the one `verify --db-only` printed (gap 107). It then runs - `alembic upgrade head`, asserts the revision is head, runs `ghactivity sync --max-minutes 300` - and `ghactivity verify --max-minutes 40 --report verify.json`, runs `ghactivity snapshot - --verify-report verify.json`, and hands the snapshot and manifest to `publish` as a workflow + `bootstrap`, before `alembic upgrade head` the job records the highest `sync_runs` id that + `verify --db-only` printed (N; 0 for a bootstrap) and fails unless the file's SHA-256 equals the + one it printed (gap 107). It then runs `alembic upgrade head`, asserts the revision is head, runs + `ghactivity sync --max-minutes 300` and `ghactivity verify --max-minutes 40 --report + verify.json`, runs `ghactivity snapshot --after-run N --sync-exit C --verify-report verify.json` + with C the sync's exit code, and hands the snapshot and manifest to `publish` as a workflow artifact, with the sync and verify exit codes as job outputs. The job fails when - `verify --db-only` halts, sync exits 1, or `snapshot` refuses the verify report. + `verify --db-only` halts, the file's SHA-256 differs, sync exits 1, or `snapshot` refuses. - `publish` (`needs: sync`; `permissions: contents: write`; no App secret, no project code, no PyPI dependencies) runs when D25's condition holds. It accepts only artifact files whose names match `^github_activity-[0-9]{8}T[0-9]{6}Z\.(db\.gz|manifest\.json)$` and reads only the @@ -1118,14 +1131,16 @@ Workflow `.github/workflows/sync.yml`: dependencies (Step 6a). The App key is readable by the sync job's dependencies, so the App stays read-only and its key rotatable. The workflow's own `GITHUB_TOKEN` (`contents: write`) is used only in `publish`. -- Snapshots: `ghactivity snapshot --out DIR [--run-id ID] [--sync-run ID] [--verify-report PATH]` - writes the copy with `VACUUM INTO`, checks its header for rollback-journal mode (the check - `scripts/check_db_journal.py` makes), gzips it, and writes the manifest, in the workflow and for - 6b's first snapshot (`run_id` null). Names are `github_activity-.db.gz` and - `github_activity-.manifest.json`, with `T` as `YYYYMMDDTHHMMSSZ`, because GitHub renames asset - names that contain special characters. The latest snapshot is the newest `T` whose two assets are - both in state `uploaded`; `verify --db-only` on that pair decides, and a mismatch halts, with no - fallback to an older pair. The manifest records the run id, code SHA, the `tracking.toml` digest +- Snapshots: `ghactivity snapshot --out DIR [--run-id ID]` with either `--after-run N --sync-exit C` + or `--sync-run ID` (exactly one of the two; another combination is a usage error, exit 1), and + `[--verify-report PATH]`, writes the copy with `VACUUM INTO`, checks its header for + rollback-journal mode (the check `scripts/check_db_journal.py` makes), gzips it, and writes the + manifest, in the workflow and for 6b's first snapshot (`run_id` null). Names are + `github_activity-.db.gz` and `github_activity-.manifest.json`, with `T` as + `YYYYMMDDTHHMMSSZ`, because GitHub renames asset names that contain special characters. The latest + snapshot is the newest `T` whose two assets are both in state `uploaded`; `verify --db-only` on + that pair decides, and a mismatch halts, with no fallback to an older pair. The manifest records + `actions_run_id`, `sync_run_id`, `sync_finished_at`, the code SHA, the `tracking.toml` digest (`sync_runs.config_sha256`), migration head, `schema_version`, coverage floor, `bootstrap`, per-repository PR and issue counts, per kind the number of rows whose `hydration_version` differs from the code's, the sync exit code, verify's @@ -1137,10 +1152,12 @@ Workflow `.github/workflows/sync.yml`: workflow, `snapshot --after-run N --sync-exit C` takes the highest `sync_runs` id the job recorded before the sync (0 for a bootstrap) and the sync's exit code, and exits 1 unless exactly one `sync_runs` row is newer than N, that row is closed with exit code C, and it is the - `last_run_id` of every tracked repository's `repo_sync_state` row (the workflow's sync has no - `--repo`), so a job whose sync did not run cannot republish the old file (gap 107). For 6b's - first snapshot, `--sync-run ID` names Step 5's last run instead. The manifest records that run's - id and `finished_at`. + `last_run_id` of the stored row of every entry in the `tracking.toml` that run read (matched as + 5.3 step 1 matches; an entry that never resolved has no row), so a job whose sync did not run + cannot republish the old file (gap 107). For 6b's first snapshot, `--sync-run ID` names Step 5's + last run instead, which must exist and be closed with exit 0 or 2, and supplies the manifest's + sync exit code. The manifest records that run as `sync_run_id` and `sync_finished_at`, and + `--run-id` as `actions_run_id`. - Uploads never use `--clobber`. Before each upload, `publish` deletes a same-name asset in any state other than `uploaded`. A same-name `uploaded` asset whose `digest` is `sha256:` followed by the expected SHA-256 (or, when `digest` is null, whose downloaded SHA-256 matches) is skipped, so a @@ -1166,7 +1183,7 @@ Workflow `.github/workflows/sync.yml`: downloads the `.db.gz`, checks its SHA-256 against the manifest, decompresses it once, and replaces its local copy by writing a temporary file and renaming it over the old one, so no query reads a half-written file. It does this once per new snapshot (about every 6 hours, D11), not per - read, and queries its copy with SQLite. It treats a pair whose manifest `finished_at` is older + read, and queries its copy with SQLite. It treats a pair whose manifest `sync_finished_at` is older than 13 hours (D11's 6-hour interval, a job of up to 355 minutes, and an hour of schedule delay) as stale and says so wherever it shows the data, with the manifest's sync exit code and verify counts beside it: a stopped schedule runs nothing, so only a check outside this repository can @@ -1878,15 +1895,15 @@ Prerequisite: Step 3b-2a-2 has merged. Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh_failures`, `links_not_found`, and `repo_filter` and the `account_node_ids` table, and refuses a file whose `sync_runs` or `github_users` holds rows (5.1, gap 110; a development file is rebuilt by a sync): -- a failed 24-hour refresh is counted (accounts), listed in the report, and makes the exit 2 (5.3 - step 4, gap 98); +- a failed 24-hour refresh is counted (accounts), listed in the report, and makes the exit 2 + unless an account entry accepts it (5.3 step 4, gap 98); - a run that reaches its deadline exits 2, and the failures a batch made final before the deadline are recorded, the hydrator handing them over with the deadline (5.3 steps 6 and 8, gap 99); - a hydration that contradicts the run's own index (an alias `NOT_FOUND`, an item outside coverage, an `updatedAt` earlier than the index entry's) is requested once more alone by the engine and then is an item failure (5.3 steps 3, 4, and 5, gap 100); -- an `accepted_failures` entry names the error class it accepts, and may name an account for a - refresh failure (5.5, gap 101); +- an `accepted_failures` entry names the error class and the message it accepts, and may name an + account, by `github_id` or `node_id`, for a refresh failure (5.5, gap 101); - a PR whose link `totalCount` exceeds the 10 links the index carries is hydrated every run (5.3 step 3, gap 102); - every node id a sighting, a lookup, or a commit link reports for a stored account is stored in @@ -1905,30 +1922,43 @@ Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh - the data dictionary describes the head-repository, milestone, and team names as of `hydrated_at` (gap 108); - the fixtures these rules break: the scenario model renders `x-ratelimit-*` headers on REST units - and the repository `databaseId` on index pages, and every test configuration gains - `[automation]`. + and the repository `databaseId` on index pages; every test configuration gains `[automation]`; + every `accepted_failures` entry in the tests and in the packaged file's comment gains `error` and + `message`; `tests/test_cli_sync_report.py` follows the account and message forms. Tests: each rule with planted inputs on both sides: a refresh request that fails and one that -succeeds, a refresh answer with another `databaseId`, and an accepted account entry; a run that +succeeds, a refresh answer with another `databaseId`, an accepted account entry (exit 0) by +`github_id` and by `node_id`, and an account entry the refresh did not reach, not reported as not +having failed; a run that finishes before its deadline (exit 0) and one whose deadline stops hydration, the refresh, or both (exit 2); a deadline inside a batch after one item's failure became final, which is recorded while the rest wait in `sync_pending`; a `NOT_FOUND` alias for a number the index listed that the retry clears and one it does not, then a later run whose index omits the number and whose step 7 removes -the item and its failure; a hydration outside coverage; a hydrated `updatedAt` earlier than, equal +the item and its failure; a hydration outside coverage that the retry clears and one it does +not; a deadline that refuses the retry, leaving the item in `sync_pending`; a hydrated `updatedAt` +earlier than, equal to, and later than the index entry's, the earlier one cleared by the retry and not; an accepted -entry whose class matches, one whose class differs, one whose `message` differs, and a loader -refusing an unknown class; PRs with 10 and with 11 links; a second node id from a sighting, a -lookup, and a REST compare link stored and seen again, and a node id in `account_node_ids` reported -under another `databaseId` failing the item; a commit-link lookup answered `NOT_FOUND` counted; each -source check accepting the valid shape and refusing the planted one, and a 502 without rate-limit -headers still retried; `code_version` outside a checkout, in a clean and a dirty checkout, and with -`git status`, `ls-files`, and `rev-parse` failing or timing out; a close given an outcome with no -stored row leaving the row `running`; a `stamp` to another revision refused and one at head -accepted; the migration refusing a populated file. Each is a regression test that fails against the -parent's code. The tests that pinned the old behaviour change with the rule they pinned: in -`tests/sync/test_engine.py` the failed-refresh, deadline (hydration and refresh), gone-item, -leaving-coverage, and 11-link tests; in `tests/sync/test_runs.py` the skipped-close and -`code_version` tests; and in `tests/test_tracking_config.py` the defaults without `[automation]`. +entry whose class and message match, one whose class differs, one whose message differs, and a +loader refusing an unknown class and an entry without `message`; PRs with 10 and with 11 links; a +second node id from a sighting, a lookup, and a REST compare link stored and seen again, and a node +id in `account_node_ids` reported under another `databaseId` failing the item; a commit-link lookup +answered `NOT_FOUND` counted; each source check accepting the valid shape and refusing the planted +one, and a 502 without rate-limit headers still retried; `code_version` outside any `.git` (the +package version), in a checkout with the package untracked (the package version), in a clean and a +dirty checkout (the commit, and `-dirty`), and with `git status`, `ls-files`, and `rev-parse` +failing or timing out inside a checkout (`-unknown`); a close given an outcome with no stored row +leaving the row `running`; a `stamp` to another revision refused with nothing committed and one at +head accepted; the migration refusing a populated file; a connection whose `foreign_keys` reads back +0 refused; the wheel check failing when the packaged `tracking.toml` bytes differ while its +repository names match; a record at the log file's level reaching it and one below not; +`repo_filter` written for a `--repo` run and null without one; a file without `[automation]` +refused; and the report listing failed refreshes and commit links answered `NOT_FOUND`. Each is a +regression test that fails against the parent's code. The tests that pinned the old behaviour change +with the rule they pinned: in `tests/sync/test_engine.py` the failed-refresh, deadline (hydration +and refresh), gone-item, leaving-coverage, and 11-link tests; in `tests/sync/test_runs.py` the +skipped-close and `code_version` tests; in `tests/test_tracking_config.py` the defaults without +`[automation]` and the accepted-failure entries; and the deadline-inside-a-batch test, whose +failures are now recorded. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule of this step without a test that targets it; CI passes on the exact PR head. @@ -2023,6 +2053,7 @@ planted in the lookup's response at a second and at half a second before the bou counted, a stored link no longer indexed on a PR whose index lists every link matched only by an event naming it, a filtered page whose `totalCount` differs from its entries, a lookup whose entries differ from its `filteredCount` after its retry and one whose response the typed models reject, +a known failure whose class an entry accepts but whose message differs failing `--exact`, each an error, one that differs once and matches on its retry, classed by its events, and a lookup spanning two pages whose second page's `filteredCount` counts only the entries after its cursor @@ -2175,8 +2206,8 @@ Prerequisite: a GitHub App with read access to Contents, Pull requests, Issues, installed on the tracked repositories, with its app ID, private key, and installation ID stored as repository secrets. -Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snapshot` and -`ghactivity verify --db-only` (5.4), and adds the +Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snapshot`, +`ghactivity verify --db-only` (5.4), and `scripts/compare_reads.py` for 6b, and adds the `githubkit[auth-app]` extra to `pyproject.toml` and `uv.lock`. Its tests: `verify --db-only` accepting the gz and manifest `ghactivity snapshot` wrote, and halting with exit 1 on a tampered copy; `snapshot --verify-report` exiting 1 on a missing report, a truncated one, one of an unknown @@ -2186,28 +2217,35 @@ unverified repositories (5.6); through minute ahead, a clock moved past it mid-run, and a second token request before the next API request (githubkit keeps a token until one minute before `expires_at`); and a 401 on an API request still exiting 1 with the run closed (5.3 step 4); `verify --db-only --out` writing the file and printing -its SHA-256; `snapshot --after-run --sync-exit` refusing no newer row, two newer rows, a row closed -with another exit code, and a tracked repository whose `last_run_id` is older (5.6); and a test -that reads `sync.yml` and fails unless every `if:` that reads an exit-code output compares a quoted -string and requires it to be non-empty. 6b creates release `db` and uploads the Step 5 database -with its manifest (`ghactivity snapshot --sync-run` naming Step 5's last run, and -`--verify-report` from Step 5's `verify --exact --report`) as the first snapshot. Before any App -run writes to release `db`, 6b reads one sample with the App and with the maintainer's token back -to back (team review requests, organization assignees, every resolve field, and each repository's -index totals) and compares them, every column and child row; any difference blocks App runs that -publish to `db`, and 6c, until it is explained or the permission is granted, and until then App -runs publish only to `db-test` (gap 107). It creates release `db-test` the same way, holding -snapshots prepared offline: the corrupted cases below, one with a `pr_files` row deleted (a verify -error), one built at the revision before the last migration, and one holding an extra repository -absent from `tracking.toml` (a sync exit 2). It then runs one normal manual run and one bootstrap -run. 6c enables the schedule and stops tracking `github_activity.db`: it removes the file -from the index, the `!/github_activity.db` exception from `.gitignore`, the `check-added-large-files` -exclude, the `tracked-db-rollback-journal` and `tracked-db-no-sidecar` hooks, the D26 freeze -check, and `alembic/env.py`'s refusal of the repository-root `github_activity.db` and of any file -of that name (gap 73); replaces the tracked-file -journal test with one on the snapshot `VACUUM INTO` writes; records the D20 WAL decision for the -working file; and documents how to download the latest snapshot and the staleness rule 5.6 gives -consumers. +its SHA-256 and highest `sync_runs` id; `snapshot --after-run --sync-exit` refusing no newer row, +two newer rows, a row closed with another exit code, and a configured entry's stored row whose +`last_run_id` is older, and accepting a stored repository the run neither attempted nor marked +(5.6); `snapshot --sync-run` refusing a missing or unclosed row, and neither form or both a usage +error; the manifest carrying `sync_run_id`, `sync_finished_at`, `actions_run_id`, and the Step 4b +compare counts; and a test that reads `sync.yml` and fails unless every `if:` that reads an +exit-code output compares a quoted string and requires it to be non-empty, the job-level +`DATABASE_URL` equals `verify --db-only`'s `--out`, the SHA-256 comparison and the recording of N +come before `alembic upgrade head`, and `snapshot` is given `--after-run` and `--sync-exit`. 6b +creates release `db` and uploads the Step 5 database with its manifest +(`ghactivity snapshot --sync-run` naming Step 5's last run, and `--verify-report` from Step 5's +`verify --exact --report`) as the first snapshot. Before any App run writes to release `db`, 6b +compares the App's reads with the maintainer token's: a `workflow_dispatch` run with +`release: db-test` syncs a sample of the repositories with the App, and at once a local run syncs +the same repositories with the token into a fresh file; `scripts/compare_reads.py` compares the two +files over the hydrated tables, every column and child row except those stamped from `Date` headers +and the run ids. Any difference blocks App runs that publish to `db`, and 6c, until it is explained +or the permission is granted (gap 107). It creates release `db-test` the same way, holding snapshots +prepared offline: the corrupted cases below, one with a `pr_files` row deleted (a verify error), one +built at the revision before the last migration, and one holding an extra repository absent from +`tracking.toml` (a sync exit 2). Once the comparison shows no unexplained difference, it runs one +normal manual run and one bootstrap run with `release: db`. 6c enables the schedule and stops +tracking `github_activity.db`: it removes the file from the index, the `!/github_activity.db` +exception from `.gitignore`, the `check-added-large-files` exclude, the +`tracked-db-rollback-journal` and `tracked-db-no-sidecar` hooks, the D26 freeze check, and +`alembic/env.py`'s refusal of the repository-root `github_activity.db` and of any file of that name +(gap 73); replaces the tracked-file journal test with one on the snapshot `VACUUM INTO` writes; +records the D20 WAL decision for the working file; and documents how to download the latest snapshot +and the staleness rule 5.6 gives consumers. Acceptance: two consecutive scheduled runs publish, and their manifests list drift, pending, and failure counts; the bootstrap run publishes, and the next scheduled run starts from its snapshot; a @@ -2215,7 +2253,9 @@ run makes requests more than 60 minutes after its first installation-token mint, records at least two mints; against release `db-test`, the snapshot one migration behind head upgrades and publishes, the snapshot with an untracked repository publishes and the workflow is marked failed, the snapshot with a deleted `pr_files` row publishes nothing and the workflow is -marked failed, a re-run of `publish` after an interrupted upload completes, and each of these halts +marked failed, a re-run of `publish` after an interrupted upload completes, a run whose working +file differs from the one `verify --db-only` printed fails before `alembic upgrade head`, and each +of these halts without publishing: a truncated gz, a database changed after its manifest was written, a manifest whose counts disagree, and, each with a matching manifest, a gz that does not decompress, a database failing `integrity_check`, one with a foreign-key violation, From 506665e3cdd7b7d3281d32f2b57bf0dcd0b45a9f Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 17:40:28 -0600 Subject: [PATCH 08/15] docs: no planted, seeded, or tampered inputs, and no mutation testing (D39) D39 records the maintainer's rule: no test, review, or acceptance changes a copy of the code, a test, the plan, review material, a database, a snapshot, or a manifest to see whether something catches the change, and none measures how many such changes are found. Test inputs are recorded GitHub responses, stubs of what GitHub or the environment can send, calls and configuration the code must refuse, and files this project's own code wrote; a check only a defect could fail is tested on its passing side with the exact count it compared. - Step 3's evidence rule, the sampler, verify, Step 4a, Step 4b, and Step 6 no longer prescribe planted mismatches, planted row deletions, tampered copies, or a corrupted db-test release. - Section 7 drops the mutation result from the SQLite pragmas row. - Step 7's audit replaces merged tests that change a copy of a database or a file. - docs/testing.md, CONTRIBUTING.md, and CLAUDE.md state the same rule. --- CLAUDE.md | 9 ++++++ CONTRIBUTING.md | 6 ++-- docs/redesign-plan.md | 72 ++++++++++++++++++++++--------------------- docs/testing.md | 12 ++++++-- 4 files changed, 59 insertions(+), 40 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1841613..34dd57a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -40,6 +40,15 @@ uv run pytest # CI also runs Python 3.12, `alembic check`, | [Architecture](docs/architecture.md), [Data Model](docs/data-model.md) | History: the pipeline and schema Step 3a removed, kept until Step 7 | | [Roadmap](docs/roadmap.md) | History of earlier phases; superseded by the redesign plan | +## Test Evidence + +Nothing is planted, seeded, or tampered, and there is no mutation testing, under any name +(redesign plan D39): no test, review, or acceptance changes a copy of the code, a test, the plan, a +database, or a file to see whether something catches the change, or measures how many such +changes are found. Tests use recorded GitHub responses, stubs of what GitHub or the environment +can send, and databases and files the project's own code wrote, and assert exact results +([Testing](docs/testing.md)). + ## Current Scope **Implemented:** diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5c85c99..99b7109 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -246,8 +246,10 @@ def test_reviewers(migrated_database: Path) -> None: # a copy of the alembic-bu pr = seed.pull_request(seed.repository("prebid/Prebid.js"), 13481, seed.user("author")) ``` -**3. Prove a check with planted inputs on both sides of its boundary**, and a fix with a test -that fails against the parent commit's code ([Testing Guide](docs/testing.md#principles)). +**3. Test on recorded GitHub responses, stubs of what GitHub can send, and files the project's +own code wrote, never on planted or tampered inputs** (redesign plan D39): a check on both sides of +its boundary with the exact result asserted, and a fix with a test that fails against the parent +commit's code ([Testing Guide](docs/testing.md#principles)). ### Running Tests diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index aa09e71..c5b37d4 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -93,6 +93,7 @@ Within scope the database collects and distinguishes data and never interprets i | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, and a repository error during a timeline lookup, makes a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | | D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class, of an item or of an account's refresh, so a permanent failure can be accepted without turning every run red. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the log's end or the floor, and fails a history read whose count still differs after a rescan. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now also counted per run), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | +| D39 | Decision of 2026-10-21 (the maintainer): nothing is planted, seeded, or tampered, and there is no mutation testing, under any name. No test, review, or acceptance changes a copy of the code, a test, the plan, review material, a database, a snapshot, or a manifest to see whether something catches the change, and none measures how many such changes a test, a check, or a reviewer finds. A test's inputs are GitHub's recorded responses, stubs of what GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`), calls and configuration the code under test must refuse, valid rows a test writes through the models (`tests/db/seed.py`), and databases and files this project's own code wrote, at any revision; each test asserts the exact result. A check that only a defect of this code could fail is tested on its passing side, with the exact count of the rows or values it compared, so it cannot pass on nothing. A regression test that fails against the parent commit's code remains evidence. Tests merged before D39 that change a copy of a database or a file to see whether a check catches it are replaced in Step 7. | ## 3. Coverage rule @@ -1536,10 +1537,13 @@ httpx declaration (gap 76) have merged. Every Step 3 test of a GitHub interaction runs through the real `GitHubSource` over `httpx.MockTransport`. Databases come from one template per test session, built by `alembic upgrade head` and copied for each test; `metadata.create_all` is used only by the 5.1 -schema comparison. Evidence is direct: planted inputs on both sides of every boundary the design -states (the coverage floor, the 10-minute unsettled window, 10 minutes past `deadline_at` for the -lock, the 24-hour refresh, the delete guard's max(25, 2%)), tampered databases, and regression -tests that fail against the parent's code; never mutation testing. Each wait on a thread or +schema comparison. Evidence is direct and nothing is planted (D39): inputs at and on each side of +every boundary the design states (the coverage floor, the 10-minute unsettled window, 10 minutes +past `deadline_at` for the lock, the 24-hour refresh, the delete guard's max(25, 2%)), taken from +GitHub's recorded responses, stubs of what GitHub or the environment can send, and databases this +project's code wrote, with the exact result asserted, and regression tests that fail against the +parent's code. No test changes a copy of the code, a test, a database, or a file to see whether a +check catches it, and there is no mutation testing, under any name. Each wait on a thread or subprocess has an explicit timeout. #### Step 3a: New schema and cutover @@ -1926,7 +1930,7 @@ Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh every `accepted_failures` entry in the tests and in the packaged file's comment gains `error` and `message`; `tests/test_cli_sync_report.py` follows the account and message forms. -Tests: each rule with planted inputs on both sides: a refresh request that fails and one that +Tests, each rule on both sides with inputs D39 allows: a refresh request that fails and one that succeeds, a refresh answer with another `databaseId`, an accepted account entry (exit 0) by `github_id` and by `node_id`, and an account entry the refresh did not reach, not reported as not having failed; a run that @@ -1942,15 +1946,15 @@ entry whose class and message match, one whose class differs, one whose message loader refusing an unknown class and an entry without `message`; PRs with 10 and with 11 links; a second node id from a sighting, a lookup, and a REST compare link stored and seen again, and a node id in `account_node_ids` reported under another `databaseId` failing the item; a commit-link lookup -answered `NOT_FOUND` counted; each source check accepting the valid shape and refusing the planted -one, and a 502 without rate-limit headers still retried; `code_version` outside any `.git` (the +answered `NOT_FOUND` counted; each source check accepting the shapes GitHub's recordings hold and refusing +each shape its rule names, and a 502 without rate-limit headers still retried; `code_version` outside any `.git` (the package version), in a checkout with the package untracked (the package version), in a clean and a dirty checkout (the commit, and `-dirty`), and with `git status`, `ls-files`, and `rev-parse` failing or timing out inside a checkout (`-unknown`); a close given an outcome with no stored row leaving the row `running`; a `stamp` to another revision refused with nothing committed and one at head accepted; the migration refusing a populated file; a connection whose `foreign_keys` reads back -0 refused; the wheel check failing when the packaged `tracking.toml` bytes differ while its -repository names match; a record at the log file's level reaching it and one below not; +0 refused; the wheel check reporting the SHA-256 of the packaged `tracking.toml` it compared, +equal to the source file's; a record at the log file's level reaching it and one below not; `repo_filter` written for a `--repo` run and null without one; a file without `[automation]` refused; and the report listing failed refreshes and commit links answered `NOT_FOUND`. Each is a regression test that fails against the parent's code. The tests that pinned the old behaviour change @@ -1970,8 +1974,9 @@ Prerequisite: Step 3b-2a-3 has merged. Scope: tests, `scripts/`, and `docs/testing.md`; `Transport.rest_get` already takes any path and query, so nothing under `src/` changes. The REST sampler 3b-2b-2's acceptance uses, laid out like the recorder: comparison logic in a -module under `tests/source/`, tested offline on recorded or synthetic REST bodies with one planted -mismatch per compared field, REST's `ghost` matching a null reference or a stored reference to an +module under `tests/source/`, tested offline on REST and GraphQL units recorded from the same PRs, +each compared field matching, with the number of fields compared per PR asserted so a comparison +that reads nothing fails (D39), REST's `ghost` matching a null reference or a stored reference to an account a lookup now returns `NOT_FOUND` for (the sampler looks it up), REST's 250 commits of a PR above 250 compared as a subset of the stored commits with equal attribution, a PR whose REST `updated_at` differs from the stored `github_updated_at` reported as changed and not compared, and @@ -1982,8 +1987,7 @@ the stack fields among them; REST's `mergeable`, `rebaseable`, and `mergeable_st but not compared, since the stored merge state is as of `hydrated_at` (5.1); each REST list read 100 entries per page until a page holds fewer, since `rest_get` returns no `Link` header (gap 96); the offline tests compare REST bodies with rows written from GraphQL units -recorded in the same recorder run as the REST units of the same PRs, or with rows seeded from a -synthetic body; REST bodies recorded as their own unit kind, each stamped as the other kinds are +recorded in the same recorder run as the REST units of the same PRs; REST bodies recorded as their own unit kind, each stamped as the other kinds are (Step 2), by a recorder mode of their own that writes every unit of its run, the sampled PRs' GraphQL units included, into its own directory beside `recorded/`, replacing that directory alone, as the first-sync mode does, and that a document change re-runs; the offline tests replay that @@ -2023,7 +2027,8 @@ order, a stale item, verify's repository set with and without `--repo` (a stored repository, a stored row no configured entry names, a renamed repository, and a renamed repository whose configured entry fails to resolve, reported unverified and not untracked, and an entry that resolves to a `github_id` other than its stored row's, with no stored `is_tracked` row reported -untracked), its child-count checks (each by a planted row deletion), an unverified repository for +untracked), its child-count checks, each reporting the rows it compared, equal to the counts the +scenario model gives (D39), an unverified repository for each failure 5.4 lists, one made unverified by a lookup by number and one by a timeline lookup, each keeping the classes it already had and leaving unchecked the items no lookup had answered, those of the batch the error ended included, an unchecked item and an unverified repository each making a @@ -2049,8 +2054,7 @@ another issue, a link difference made by a closing keyword with no event, a diff link totals alone with enough connected events, with only a disconnected event, and with too few, a synthetic disconnected event, an event at the boundary counted and one a second before it not, entries -planted in the lookup's response at a second and at half a second before the boundary and not -counted, a stored link no longer indexed on a PR whose index lists every link matched only by an +the model renders at a second and at half a second before the boundary, not counted, a stored link no longer indexed on a PR whose index lists every link matched only by an event naming it, a filtered page whose `totalCount` differs from its entries, a lookup whose entries differ from its `filteredCount` after its retry and one whose response the typed models reject, a known failure whose class an entry accepts but whose message differs failing `--exact`, @@ -2105,8 +2109,8 @@ issues; an issue's stored author kept when GitHub returns it as null (D28); issu assignees written and replaced; a transfer arriving with a new id; an old number the run's index listed that hydration answers `NOT_FOUND`, an item failure until an index that omits it removes it (5.3 step 4); a snapshot one migration behind head upgrades and syncs; verify's -child-count checks for `issue_labels`, `issue_comments`, and `issue_assignees`, each by a planted -row deletion. +child-count checks for `issue_labels`, `issue_comments`, and `issue_assignees`, each reporting the +rows it compared, equal to the stored totals (D39). Acceptance: `verify --exact` for issues on the same repositories; a REST sample of issues matches field by field. @@ -2168,8 +2172,8 @@ halting; a null actor; an event GitHub stops listing removed on the next run; an listing after the stored read classified as removed since sync; `activity_halt_reason` set by a halt and cleared by the next run that does not halt, `halt_reason` unchanged; `--allow-deletions` applying N to events and history commits separately; a history replace keeping a commit's stored -author link when GitHub returns none (D28); each verify check and each stored-row check, by a -planted row deletion. +author link when GitHub returns none (D28); each verify check and each stored-row check on the recordings, each stored-row check reporting the +rows it compared (D39). Acceptance: on the Step 3b-2 acceptance repositories and on `prebid/github-activity-db`, the stored events equal the log and the stored commits equal the history; each repository's first logged event @@ -2209,9 +2213,11 @@ repository secrets. Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snapshot`, `ghactivity verify --db-only` (5.4), and `scripts/compare_reads.py` for 6b, and adds the `githubkit[auth-app]` extra to `pyproject.toml` and `uv.lock`. Its tests: `verify --db-only` -accepting the gz and manifest `ghactivity snapshot` wrote, and halting with exit 1 on a tampered -copy; `snapshot --verify-report` exiting 1 on a missing report, a truncated one, one of an unknown -version, and one whose `last_run_id` differs from the file's, and the manifest listing the report's +accepting the gz and manifest `ghactivity snapshot` wrote, and reporting the SHA-256 it computed, +the per-repository counts it compared, the revision it read, and the answers of `integrity_check` +and `foreign_key_check`, each equal to what the file holds (D39); a release with no complete +snapshot pair halting with exit 1; `snapshot --verify-report` exiting 1 on a missing report and on +one from a verify of an earlier sync, whose `last_run_id` differs from the file's, and the manifest listing the report's unverified repositories (5.6); through `httpx.MockTransport`, a mocked installation-token response whose `expires_at` lies just over a minute ahead, a clock moved past it mid-run, and a second token request before the next API request @@ -2235,9 +2241,8 @@ the same repositories with the token into a fresh file; `scripts/compare_reads.p files over the hydrated tables, every column and child row except those stamped from `Date` headers and the run ids. Any difference blocks App runs that publish to `db`, and 6c, until it is explained or the permission is granted (gap 107). It creates release `db-test` the same way, holding snapshots -prepared offline: the corrupted cases below, one with a `pr_files` row deleted (a verify error), one -built at the revision before the last migration, and one holding an extra repository absent from -`tracking.toml` (a sync exit 2). Once the comparison shows no unexplained difference, it runs one +this project's code wrote: one built at the revision before the last migration, and one synced with +a `tracking.toml` that listed a repository the current one does not (a sync exit 2). Once the comparison shows no unexplained difference, it runs one normal manual run and one bootstrap run with `release: db`. 6c enables the schedule and stops tracking `github_activity.db`: it removes the file from the index, the `!/github_activity.db` exception from `.gitignore`, the `check-added-large-files` exclude, the @@ -2252,14 +2257,8 @@ failure counts; the bootstrap run publishes, and the next scheduled run starts f run makes requests more than 60 minutes after its first installation-token mint, and its log records at least two mints; against release `db-test`, the snapshot one migration behind head upgrades and publishes, the snapshot with an untracked repository publishes and the workflow is -marked failed, the snapshot with a deleted `pr_files` row publishes nothing and the workflow is -marked failed, a re-run of `publish` after an interrupted upload completes, a run whose working -file differs from the one `verify --db-only` printed fails before `alembic upgrade head`, and each -of these halts -without publishing: a truncated gz, a database changed after -its manifest was written, a manifest whose counts disagree, and, each with a matching manifest, a gz -that does not decompress, a database failing `integrity_check`, one with a foreign-key violation, -and one at a revision unknown to the code. +marked failed, and a re-run of `publish` after an interrupted upload completes; `verify --db-only`'s +refusals are covered by the 6a tests of what it compared (D39). ### Step 7: Documentation rewrite and code audit @@ -2270,6 +2269,9 @@ counts. A final audit of the whole repository, made once Steps 1-6 have landed, - dependencies in `pyproject.toml` that nothing imports; - scripts, hooks, and CI steps for mechanisms a step removed; - tests and fixtures of removed paths, and fixtures no test reads; +- tests merged before D39 that change a copy of code, a database, or a file to see whether a check + catches it (`tests/db/test_schema.py`'s planted differences and `tests/db/test_migrations.py`'s + planted failure among them), each replaced under D39's rule; - every note that defers something until a step that has landed (in code, docs, `.gitignore`, and hook comments). @@ -2346,7 +2348,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Schedules | Public-repo schedules disable after 60 days without repository activity; start of hour is high load; queued jobs may be dropped; `workflow_dispatch` requires the workflow on the default branch | GitHub docs | | Concurrency | A new run replaces a pending run in the group; running runs are cancelled only with `cancel-in-progress: true` | GitHub docs | | App token action | `actions/create-github-app-token` v3.2.0 (2026-05-12) and v3.3.0 (2026-10-12); the sync no longer uses it (5.6) | GitHub API | -| SQLite pragmas | The `connect` listener on `engine.sync_engine` sets per connection through aiosqlite: `foreign_keys=1` and `busy_timeout=30000`. Without the listener a connection reports `foreign_keys=0` and `busy_timeout=5000` (Python `sqlite3` default `timeout=5.0`). A file once set to WAL stays WAL (header bytes 18-19 `0202`; rollback `0101`). A foreign-key violation is rejected and `ON DELETE CASCADE` removes children; the same values hold on every session from `get_session()`, and routing `get_engine()` around `build_engine()` fails 2 tests in `tests/db/test_application_engine.py` | `tests/test_db_engine.py` (`TestSqliteConnectionSettings`), `tests/db/test_application_engine.py`; mutation run | +| SQLite pragmas | The `connect` listener on `engine.sync_engine` sets per connection through aiosqlite: `foreign_keys=1` and `busy_timeout=30000`. Without the listener a connection reports `foreign_keys=0` and `busy_timeout=5000` (Python `sqlite3` default `timeout=5.0`). A file once set to WAL stays WAL (header bytes 18-19 `0202`; rollback `0101`). A foreign-key violation is rejected and `ON DELETE CASCADE` removes children; the same values hold on every session from `get_session()` | `tests/test_db_engine.py` (`TestSqliteConnectionSettings`), `tests/db/test_application_engine.py` | | Connect-listener failure | When a statement in the `connect` listener raises, SQLAlchemy 2.0.45 does not close the DBAPI connection and the aiosqlite 0.22.1 worker thread keeps the process alive. With `PRAGMA journal_mode=DELETE` in the listener, `ghactivity sync retry --dry-run` against a database held with `BEGIN EXCLUSIVE` printed "database is locked" and was still running at 60 s; without that statement, and separately with the connection closed on failure, it exited 1 after 31.6-31.8 s (the 30 s busy timeout). With the close in place and the statement restored, a WAL database held open by another process fails every connection with "database is locked" (exit 1 after 0.5 s), where without the statement the CLI uses the file | CLI runs against a locked file | | Build constraints | `uv build --build-constraints build-constraints.txt --require-hashes` builds the wheel; with both listed hatchling hashes altered it exits 2 (hash mismatch); with only the wheel's hash altered it succeeds, because uv rejects the wheel and builds hatchling from its sdist, whose hash still matches | local builds with uv 0.7.10, no cache | | Packaged config | The wheel built from the Step 1 branch (merged as `8c5b16b`) contains `github_activity_db/tracking.toml`; installed with the locked dependencies and run outside the checkout, `Settings().tracked_repos` returns the 18 repositories. A wheel built from `7555a6d` raises `TrackingConfigError` for `/lib/python3.13/config/tracking.toml`; a wheel excluding the file fails the same check | `uv build`, `scripts/wheel_smoke.py` in a new environment | diff --git a/docs/testing.md b/docs/testing.md index 2ed4d78..c9b06f6 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -10,9 +10,15 @@ How the test suite is built and what it counts as evidence. The plan of record i - **Real components, faked boundaries.** GitHub is the only fake: the real `GitHubSource` runs over `httpx.MockTransport`, answering from responses recorded from public repositories. Databases are real SQLite files built by `alembic upgrade head`. -- **Direct evidence.** A check is tested with planted inputs on both sides of the boundary it - states, tampered databases and artifacts, and a positive control showing it sees what it - checks; a fix comes with a regression test that fails against the parent commit's code. +- **Direct evidence, nothing planted** (redesign plan D39). A test's inputs are GitHub's recorded + responses, stubs of what GitHub or the environment can send, calls and configuration the code + must refuse, and databases and files the project's own code wrote; it asserts the exact result. + No test changes a copy of the code, a test, a database, or a file to see whether a check catches + it, and none measures how many such changes are caught: there is no mutation testing, under any + name. A check that only a defect of this code could fail is tested on its passing side, with the + exact count of what it compared. A fix comes with a regression test that fails against the parent + commit's code. Tests merged before D39 that change a copy of a database or a file are replaced in + Step 7. - **No skipped tests** (D22). CI runs `scripts/check_junit.py --max-skipped 0`, which fails a run where any test was skipped (pytest reports `xfail` as skipped) or none ran. - **Time.** Source tests pass a `FakeClock`, so waits and retries are asserted, never slept. From 12b245283c9cc8798f1b96bbbbbbd896db8ddda9 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 17:43:10 -0600 Subject: [PATCH 09/15] docs(plan): close the findings of the last review of D38 - 5.3 step 5: GraphQL's commit connection holds the newest 250, so above 250 commits its first page must equal compare's positions total-250..total-151 in order (Prebid.js #8858: positions 4-103); the oldest commits are checked by count alone. - 5.5: an accepted_failures entry matches the class and the cause, the first line of the failure's message, which carries no request id, count, or typed-model text; tracking.py keeps the recorded class list, held equal to the engine's classes by a test; index contradictions take the class IndexContradiction; Step 4b adds a compare entry form. - 5.6 and Step 6: a validated `repo` dispatch input (bootstrap only) for 6b's App-and-token comparison, which covers every table the sync writes from GitHub, names the cases its sample must hold, and lists App nulls; the manifest carries repo_filter, links_not_found, and Step 4b's counts; 6a tests compare_reads and lists each exit-code output for the sync.yml test. - Step 4b: completeness against an ascending read's first event, files from the first page, per-compare transactions, a stored error for a compare that still fails, deletion counters, verify's own read as unverified, halted vs not-synced classes, and merge-brought commits. - Step 5: the updatedAt measurement's instrument per kind and its counting rule, contradiction rates, the link NOT_FOUND share, and the bootstrap's projected time; the conditional pull request is 5b. - 5.1: a read version after Step 6b instead of a refusal, account_node_ids' keys and indexes, trigger `unknown`, untracked files outside -dirty, and `stamp --purge` at head. - Step 3b-2a-3: scope and tests follow the above, plus re-recording and docs/development.md. --- CLAUDE.md | 2 +- docs/redesign-plan.md | 481 +++++++++++++++++++++++++----------------- 2 files changed, 292 insertions(+), 191 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 34dd57a..ffee6fd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -71,7 +71,7 @@ can send, and databases and files the project's own code wrote, and assert exact - `ghactivity sync` (Step 3b-2a-2c, `cli/sync.py`): the engine's run with the options of plan 5.7; a run that never starts exits 1, and a run that starts exits with the engine's 0, 2, or 1 -**Not implemented yet:** D37 in the sync (Step 3b-2a-3: no silent failures, D38), the REST +**Not implemented yet:** D37 in the sync (Step 3b-2a-3, D38), the REST sampler (Step 3b-2b-1), and `verify` (Step 3b-2b-2). The tracked `github_activity.db` stays frozen at the first schema (D26). diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index c5b37d4..8f898f5 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -37,7 +37,7 @@ audited for silent failures and silently wrong data under D37, and D38 records t | 2 | GitHub source layer | done (PR #7) | | 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | | 4 | Issues, closing-issue links, and default-branch activity (pull requests 4a and 4b) | not started | -| 5 | Full bootstrap and verification | not started | +| 5 | Full bootstrap and verification (pull request 5, and 5b when Step 5 finds an edit kind that leaves `updatedAt` unchanged) | not started | | 6 | Scheduled sync and publication | not started | | 7 | Documentation rewrite and code audit | not started | @@ -90,9 +90,9 @@ Within scope the database collects and distinguishes data and never interprets i | D33 | `prebid-agent-skills` is never queried by an agent. Commands an agent runs (sync, verify, the REST sampler, the difference report, recordings, and `ghactivity github test`) name their repositories with `--repo` and leave it out; its rows enter the database through the scheduled App's workflow (Step 6) or a run the maintainer starts. Recordings come from public repositories only (Step 2). | | D34 | Decision of 2026-10-20 (the maintainer): wherever stored data must tell runs apart, a run is its `sync_runs` row id, stored with the data, never a clock reading and never state the writer or the engine keeps in memory, which neither a later run nor a reader of the file can see. `sync_failures.last_run_id` names the latest run in which the item failed, and `attempts` adds one only for a failure in another run, whatever the two runs' clocks read (5.1, gap 91). | | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | -| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, and a repository error during a timeline lookup, makes a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | +| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, a repository error during a timeline lookup, and from Step 4b a log or history read verify cannot complete make a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | -| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class, of an item or of an account's refresh, so a permanent failure can be accepted without turning every run red. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the log's end or the floor, and fails a history read whose count still differs after a rescan. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now also counted per run), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | +| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class and one cause, the first line of the failure's message, of an item, an account's refresh, or from Step 4b a compare, so a permanent failure can be accepted without turning every run red while a new cause is not. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the floor or at the log's first event, as a read in ascending order finds it, fails a history read whose count still differs after a rescan, and records a compare that still fails, which an entry can accept. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now counted per run and in the manifest, and from Step 6a a run whose share of link lookups answered `NOT_FOUND` exceeds the one Step 5 sets exits 2, since a lookup form GitHub stopped resolving would look the same per link), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (`items_deleted`, and from Step 4b `events_deleted` and `history_commits_deleted`; the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | | D39 | Decision of 2026-10-21 (the maintainer): nothing is planted, seeded, or tampered, and there is no mutation testing, under any name. No test, review, or acceptance changes a copy of the code, a test, the plan, review material, a database, a snapshot, or a manifest to see whether something catches the change, and none measures how many such changes a test, a check, or a reviewer finds. A test's inputs are GitHub's recorded responses, stubs of what GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`), calls and configuration the code under test must refuse, valid rows a test writes through the models (`tests/db/seed.py`), and databases and files this project's own code wrote, at any revision; each test asserts the exact result. A check that only a defect of this code could fail is tested on its passing side, with the exact count of the rows or values it compared, so it cannot pass on nothing. A regression test that fails against the parent commit's code remains evidence. Tests merged before D39 that change a copy of a database or a file to see whether a check catches it are replaced in Step 7. | ## 3. Coverage rule @@ -229,18 +229,18 @@ Every gap found in the audit, and where it is resolved. | 96 | The REST sampler's lists have no stop rule (`Transport.rest_get` returns no `Link` header); the recorder's default mode rebuilds `recorded/` from what it records, and `--only documents` drops every unit kind but the compare and rate-limit kinds it keeps; and its check that a repository is public, after the name list it refuses before any request, runs on a resolve, which a REST-only recording does not make | Step 3b-2b-1 for the sampler and its REST units, recorded in their own directory; Step 3b-2b-2 for the timeline units (5.8) | | 97 | Without `--exact`, an unchecked item and an unverified repository did not fail verify, and a timeline lookup whose entries differed from `filteredCount` left its item unchecked, so the scheduled verify (5.6, no `--exact`) could exit 0 and D25 publish a database it had not checked | D37: both fail verify in every mode, and an answer verify cannot accept is an error (5.4, 5.8) | | 98 | A 24-hour account refresh that still fails, or whose answer the writer refuses (another `databaseId` for a stored node id), left the run `complete` with exit 0; the only trace was a printed line, and a test pinned exit 0 | Step 3b-2a-3: counted in `sync_runs.refresh_failures`, listed in the report, and exit 2 unless an account entry in `[sync].accepted_failures` accepts it (5.3 step 4, 5.5, D38) | -| 99 | A run that reached `--max-minutes` exited 0, and a deadline inside a batch turned the item failures it had computed into pending items | Step 3b-2a-3: a run that reaches its deadline exits 2, which D25 still publishes with the workflow red, and the failures a batch made final before the deadline are recorded (5.3 steps 6 and 8, D38) | +| 99 | A run whose deadline came after every index completed (in hydration, its lookups, or the refresh) exited 0, and a deadline inside a batch turned the item failures it had computed into pending items | Step 3b-2a-3: a run that reaches its deadline exits 2, which D25 still publishes with the workflow red, and the failures a batch made final before the deadline are recorded (5.3 steps 6 and 8, D38) | | 100 | A PR the run's own index listed in coverage that hydration then answered `NOT_FOUND`, or returned out of coverage, was neither written, failed, nor pending, and nothing reported it; a hydration older than the index entry was written without a check | Step 3b-2a-3: each is an item error (5.3 steps 4 and 5, D38) | -| 101 | An `accepted_failures` entry accepted any failure of its item, so a new cause stayed out of exit 2 | Step 3b-2a-3: an entry names the error class and the message it accepts, and may name an account (5.5, D38) | +| 101 | An `accepted_failures` entry accepted any failure of its item, so a new cause stayed out of exit 2 | Step 3b-2a-3: an entry names the error class and the cause it accepts, the first line of the failure's message, which carries no value that changes between runs, and may name an account (5.5, D38) | | 102 | A PR with more than the 10 closing links the index carries was not re-hydrated when a link beyond the 10th was swapped with equal totals | Step 3b-2a-3: such a PR is hydrated every run (5.3 step 3, D38). Verify keeps comparing the totals and the indexed links: the sync re-reads every link of such a PR each run, and verify's child-count check holds the stored rows to the hydrated total | | 103 | A `databaseId` reported under a second node id kept the stored node id and dropped the reported one; only a printed line recorded it; the sync used a commit link's node id, which the source reads, only as a lookup key, never storing or checking it | Step 3b-2a-3: `account_node_ids` stores every node id a sighting, a lookup, or a commit link reports for an account (5.1, 5.2, D38) | -| 104 | Source checks that accepted what they should refuse: a REST response without readable rate-limit headers, a GraphQL `errors` present but not a list (`{}`, `""`), a compare above 250 commits checked by count but not membership, and an index page from a repository other than the one resolved | Step 3b-2a-3 (5.3 steps 2, 4, and 5, D38) | +| 104 | Source checks that accepted what they should refuse: a REST response without readable rate-limit headers, a GraphQL `errors` present but not a list (`{}`, `""`), a compare above 250 commits checked by count but not against the commits GraphQL returns, and an index page from a repository other than the one resolved | Step 3b-2a-3 (5.3 steps 2, 4, and 5, D38); the oldest `total_commits - 250` commits of such a compare stay checked by count alone, since no GraphQL read reaches them | | 105 | Run records and setup that hid a failure: `code_version` without `-dirty` when `git status` failed, a close that ignored outcomes it could not write, a missing `[automation]` table emptying the automation lists, `foreign_keys` never read back, `alembic stamp` accepted, a wheel check comparing repository names only, a log file no record reaches, and no record of a run's `--repo` filter | Step 3b-2a-3 (5.1, 5.3 step 8, 5.5, D38) | -| 106 | Step 4b: compare `files` stop at GitHub's documented 300 per comparison with no total, so a capped list read as complete; the activity read had no paging or completeness rule; a history count mismatch had no outcome; a refused compare was never retried; and the log's first event was not stored | Step 4b (5.1, D38) | +| 106 | Step 4b: compare `files` stop at GitHub's documented 300 per comparison with no total, so a capped list read as complete; the activity read had no paging or completeness rule; a history count mismatch had no outcome; a refused compare was never retried; the log's first event was not stored; a compare without `files` had no rule; a compare that always fails could not be accepted; and the events and history commits a run deleted were not counted | Step 4b (5.1, 5.5, D38) | | 107 | Step 6: an App token may return null where the maintainer's token read a value; a stopped schedule turns nothing red; a missing `DATABASE_URL` builds from empty; an empty exit-code output compares equal to 0 in `if:`; and `snapshot` did not check that the job's own sync ran | Step 6 and 5.6 (D38), except the stopped schedule: no run starts, so this repository cannot see it; 5.6 states the consumer's staleness rule, which the dashboard's integration builds (section 9) | | 108 | The data dictionary presented head-repository, milestone, and team names as GitHub's current values, though they are as of `hydrated_at` | Step 3b-2a-3: described as of `hydrated_at` (D38) | -| 109 | Edits that may leave `updatedAt` unchanged (body, base ref, review dismissal, issue label events) were accepted as missed | Step 5 measures at least 20 cases of each; a kind not shown to change `updatedAt` gets an index-side check or a periodic re-hydration before Step 6a (section 8, D38) | -| 110 | A GitHub-derived column a later migration adds to a table without `hydration_version` (accounts, compares), or a new table of such data, is never filled for existing rows | 5.1: such a migration adds a per-row read version or refuses while the file holds the rows it would leave unread (D38) | +| 109 | Edits that may leave `updatedAt` unchanged (body, base ref, review dismissal, issue label events) were accepted as missed | Step 5 measures at least 20 cases of each that no later change the index sees follows; a kind not shown to change `updatedAt` gets an index-side check or a periodic re-hydration in pull request 5b, before Step 6a (section 8, D38) | +| 110 | A GitHub-derived column a later migration adds to a table without `hydration_version` (accounts, compares), or a new table of such data, is never filled for existing rows | 5.1: such a migration adds a per-row read version, which the refresh or the compare reader reads, or, before Step 6b only, refuses while the file holds the rows it would leave unread (D38) | ## 5. Design @@ -251,7 +251,8 @@ Entity tables (`repositories`, `github_users`, `pull_requests`, `pr_reviews`, `p identifiers there are unique columns, never primary keys. Child tables use the composite keys listed. References to `repositories`, `pull_requests`, `issues`, and `repository_activities` use `ON DELETE CASCADE`; references to `github_users` use `ON DELETE SET NULL`, except those in a -primary key (`pr_assignees.account_id`, `account_profiles.account_id`), which use `ON DELETE CASCADE`; +primary key (`pr_assignees.account_id`, `account_profiles.account_id`, +`account_node_ids.account_id`), which use `ON DELETE CASCADE`; references to `sync_runs` take SQLite's default (no action). Application connections enforce foreign keys. Migrations run with `foreign_keys=OFF`, because alembic's SQLite batch mode drops and recreates tables and an enforced drop cascades into child rows. @@ -272,7 +273,8 @@ foreign_key_check` runs inside that transaction after the last revision and rais before the commit. It runs before every command commits, one that runs no revision (`current`, `check`, and a `stamp` that changes nothing) included; a `stamp` that changes the revision is refused by an `on_version_apply` callback, which raises on alembic's `MigrationInfo.is_stamp` -inside the command's transaction, so nothing commits, and a `stamp` at head has no step (gap 105): +inside the command's transaction, so nothing commits, and a `stamp` at head has no step, except +with `--purge`, which empties the version table first, so it has a step and is refused (gap 105): alembic loads each revision module inside the transaction, so a module's top-level code can change rows or the schema without a revision running (section 7, "Revision module code"), and a file that already holds a violation fails every command that connects. A migration that batch-alters a table a @@ -338,15 +340,15 @@ and `color`, `pr_user_tags.created_at`). | `issue_labels` | `issue_id`, `name` | PK (`issue_id`, `name`) | | (Step 4a) | | The two issue rows above are the Step 3 plan. Step 4a completes them under D29 the way Step 3b-1 completes the PR rows: `issues.author_reported` (D28), the label id on `issue_labels` (D8), the issue's other free fields (`author_association`, `last_edited_at`, `published_at`, milestone, issue type, the parent and the `duplicateOf` issue by repository id and number), `issue_comments` and `issue_assignees` tables shaped like `pr_comments` and `pr_assignees` with their totals, and their 5.4 child-count checks | | `repository_activities` | `repository_id`, `github_id`, `node_id`, `activity_type`, `ref`, `before_sha`, `after_sha`, `occurred_at`, `actor_github_id`, `actor_id` | Step 4b (D30); `github_id` unique (GitHub's event `id`); `activity_type` is GitHub's value as text (`push`, `force_push`, `branch_creation`, `branch_deletion`, `pr_merge`, `merge_queue_merge` in githubkit 0.14.2's model); `occurred_at` is GitHub's `timestamp`; `actor_github_id` is the REST actor's `id` and `actor_id` its `github_users` row once a lookup resolves it (5.2), both null when GitHub returns no actor | -| `repository_activity_compares` | `activity_id`, `http_status`, `status`, `ahead_by`, `behind_by`, `total_commits`, `merge_base_sha`, `files_at_cap` | Step 4b; PK `activity_id`; `files_at_cap` is true when compare returned 300 files, the cap GitHub documents for one comparison, so its file rows may be partial (gap 106); GitHub's REST compare of `before_sha...after_sha` as it returns them, so the files and counts run from the merge base, not from `before_sha`, when the push diverged (section 7, "Compare"); no row when either SHA is all zeros (a branch creation or deletion); `http_status` records the response, so a refused compare is a row with the other columns null (a compare across a force push, `5110f3d...fccb1f5`, returned 200, section 7) | +| `repository_activity_compares` | `activity_id`, `http_status`, `status`, `ahead_by`, `behind_by`, `total_commits`, `merge_base_sha`, `files_at_cap`, `error` | Step 4b; PK `activity_id`; `files_at_cap` is true when compare's first page, the only page GitHub lists files on, returned 300 files, the cap GitHub documents for one comparison, so its file rows may be partial (gap 106); `error` is the class and message of a compare that still failed after its retries or whose 2xx first page had no `files` list (Step 4b), with the other columns null, and null once GitHub answers it; GitHub's REST compare of `before_sha...after_sha` as it returns them, so the files and counts run from the merge base, not from `before_sha`, when the push diverged (section 7, "Compare"); no row when either SHA is all zeros (a branch creation or deletion); `http_status` records the response, so a refused compare is a row with the other columns null (a compare across a force push, `5110f3d...fccb1f5`, returned 200, section 7) | | `repository_activity_compare_files` | `activity_id`, `ordinal`, `filename`, `previous_filename`, `status`, `additions`, `deletions` | Step 4b; PK (`activity_id`, `ordinal`); compare's `files` entries as returned | -| `repository_activity_compare_commits` | `activity_id`, `ordinal`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at` | Step 4b; PK (`activity_id`, `ordinal`); compare's `commits` list in full, each commit stored like a `pr_commits` row (D19), so a commit an event brought joins to its event; written once with its compare and never re-read, so its author link is the one GitHub returned then | +| `repository_activity_compare_commits` | `activity_id`, `ordinal`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at` | Step 4b; PK (`activity_id`, `ordinal`); compare's `commits` list in full, each commit stored like a `pr_commits` row (D19), so a commit an event brought joins to its event; written once with its compare and re-read only when a read version asks for it (5.1, column rules), so its author link is the one GitHub returned then | | `default_branch_commits` | `repository_id`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at`, `committed_at`, `parent_count`, `additions`, `deletions`, `changed_files` | Step 4b; PK (`repository_id`, `sha`); the default branch's history since the floor as GitHub lists it (D30); the author link as in `pr_commits`, a replace keeping the stored link of the row with the same SHA when GitHub returns none (D28); `parent_count` is GitHub's `parents.totalCount`; `changed_files` is GitHub's `changedFilesIfAvailable`, null when GitHub returns none; no message (D19) | | `automation_accounts` | `github_id`, `login`, `reason`, `run_id` | integer primary key; `github_id` unique; replaced from `tracking.toml` at the start of every run, `run_id` naming that run (a reference to `sync_runs`) | | `account_profiles` | `account_id`, `company`, `location`, `website_url`, and an organization's `description`, `email`, `twitter_username`, `is_verified`, `archived_at`, `has_sponsors_listing`, and `pinned_items_remaining`, `refreshed_at` | PK `account_id`, a reference to `github_users` (`ON DELETE CASCADE`); rows only for a `User` or an `Organization` (D31); each lookup that returns the account replaces the whole row with the profile it returned, through the writer and in the transaction that writes `github_users`' login and type, so no earlier value survives a field GitHub now returns as null or a change of type; a lookup that returns `NOT_FOUND` leaves the row unchanged, marked by `github_users.not_found_at` (D28); every field nullable as GitHub's is, the organization fields null for a user and `company` null for an organization; `refreshed_at` as on `github_users` | -| `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting, a lookup, or a commit link, the one `github_users.node_id` holds included, with the first and latest runs that reported it, so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); its migration refuses a file whose `github_users` holds rows, since no file was published before Step 6b (gap 110); consumer | +| `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting, a lookup, or a commit link, the one `github_users.node_id` holds included, with the first and latest runs that reported it (`first_run_id` and `last_run_id`, references to `sync_runs`), so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); its migration refuses a file whose `github_users` holds rows, since no file was published before Step 6b (gap 110); consumer | | `automation_emails` | `email`, `reason`, `run_id` | integer primary key; `email` unique, stored lowercase ASCII (SQLite's `lower()`, which the views apply to commit emails, folds only ASCII letters); replaced the same way | -| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `links_not_found`, `repo_filter`, `deletions_approved` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, else `manual`; `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes and `-unknown` when it fails or times out, gap 105; the package runs from a checkout only when its directory or a parent holds a `.git` entry, and then `git ls-files --error-unmatch` exiting 1 means the package is not tracked, while any other failure of a git call, exit 128 and a missing `git` included, records the package version, or the commit when `rev-parse` answered, followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted; `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `links_not_found` the commit-link accounts a lookup answered `NOT_FOUND` this run, gap 90's lookups included (5.2), and `repo_filter` a JSON array of the `--repo` values as given, sorted, null for a run without `--repo`; `refresh_failures`, like `failures`, counts accepted ones; the migration that adds these three refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | +| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `links_not_found`, `repo_filter`, `deletions_approved`, and from Step 4b `events_deleted` and `history_commits_deleted` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, `manual` when it is unset, and `unknown` for any other value (gap 105); `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes to tracked files, untracked files ignored, and `-unknown` when it fails or times out, gap 105; the package runs from a checkout only when its directory or a parent holds a `.git` entry, and then `git ls-files --error-unmatch` exiting 1 means the package is not tracked, while any other failure of a git call, exit 128 and a missing `git` included, records the package version, or the commit when `rev-parse` answered, followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted, and from Step 4b `events_deleted` and `history_commits_deleted` the activity events and default-branch commits Step 4b deleted (nullable, null for a run before Step 4b, each written in its deletion's transaction); `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `links_not_found` the commit-link accounts a lookup answered `NOT_FOUND` this run, gap 90's lookups included (5.2), and `repo_filter` a JSON array of the `--repo` values as given, sorted, null for a run without `--repo`; `refresh_failures`, like `failures`, counts accepted ones; the migration that adds these three refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | | `repo_sync_state` | `repository_id`, `last_index_started_at`, `index_pr_total`, `index_issue_total`, `last_success_at`, `last_run_id`, `last_outcome`, `halt_reason`, and from Step 4b `activity_read_at`, `activity_first_event_at`, `activity_last_outcome`, `activity_halt_reason`, `history_read_at`, and `default_branch_commits_total` | PK `repository_id`; `activity_first_event_at` is the `timestamp` of the oldest event the latest complete activity read listed, so a consumer can tell a quiet period from a log that does not reach back (gap 106); `activity_read_at` is the GitHub `Date` header of the first page of the latest activity-log read that completed and was written; `history_read_at` and `default_branch_commits_total` are the first page's `Date` header and the `totalCount` of the latest history read that replaced the rows, written only with that replace; `activity_last_outcome` in (`complete`, `time_limited`, `failed`, `halted`); `activity_halt_reason` holds a Step 4b deletion halt's reason and candidate counts and is cleared by a run whose Step 4b did not halt (Step 4b); `last_run_id` is the latest run that attempted the repository, or found it untracked, and `last_outcome` its result, in (`complete`, `time_limited`, `failed`, `incomplete_index`, `halted`, `untracked`), both written by step 8, or by the close of a fatal or interrupted run (5.3 step 8); `last_index_started_at` is written only when that index completed and the repository did not fail later in the run, a time-limited repository included (its unreached items are in `sync_pending`), and never by the close of a fatal or interrupted run; `last_success_at` is the run's `finished_at` when `last_outcome` is `complete` and no item of the repository failed outside `[sync].accepted_failures`; `halt_reason` holds step 7's halt and is cleared by a run whose step 7 ran for the repository and did not halt, a run whose step 7 did not run for it leaving it | | `sync_pending` | `repository_id`, `item_kind`, `number`, `run_id` | PK (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); items a time-limited run planned but did not reach; step 8 replaces the rows of each completely indexed repository that did not fail later in the run (a repository gone during hydration keeps its rows, 5.3 step 4) with this run's unreached items, so a run that reached them all leaves none; consumer, so an in-coverage item not yet hydrated is told apart from one outside coverage | | `sync_failures` | `repository_id`, `item_kind`, `number`, `attempts`, `last_error`, `first_failed_at`, `last_failed_at`, `last_run_id` | unique (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); `attempts` counts the runs in which the item failed: `last_run_id` is the latest such run, and a failure adds one only when its run is another, whatever the runs' clocks read; `last_error` is the error's class and message; row deleted on success, or by step 7 when its repository's in-coverage set no longer lists the item | @@ -393,9 +395,13 @@ Column rules: column a later migration adds to tables that hold rows is null on rows hydrated at an older version until they re-hydrate, and `hydration_version` is a consumer column, so a reader can tell. A table without `hydration_version` (`github_users`, `account_profiles`, the Step 4b compare - tables) to which a later migration adds a GitHub-derived column gains a per-row read version - that the planner compares the same way, or the migration refuses while the table holds rows, so - a column is never left unread on old rows. A migration that adds a table of GitHub-derived data + tables) to which a later migration adds a GitHub-derived column gains a per-row read version: + the refresh (5.2) re-reads an account whose version differs from the code's, and Step 4b's + compare reader re-requests a compare whose version differs, keeping the stored row and + recording the refusal when GitHub refuses it. Before Step 6b the migration may instead refuse + while the table holds rows; from Step 6b every published file holds such rows, so a refusal + would stop every scheduled run until a bootstrap, and only the read version is allowed. So a + column is never left unread on old rows. A migration that adds a table of GitHub-derived data refuses only while the file holds rows no later run reads for it, so no row reads as "none" where nothing was recorded; a table or column every run fills for every stored parent, whose unread state is already marked, does not refuse (Step 4a's issue tables, Step 4b's activity, compare, @@ -416,10 +422,10 @@ PR; `pull_requests (repository_id, state)`, `(repository_id, merged_at)`, `(author_id)`, `(merged_by_id)`, `(merge_commit_oid)`; `issues (repository_id, state)`, `(author_id)`; `pr_closing_issues (issue_repo_github_id, issue_number)`; `repository_activities (repository_id, occurred_at)` and `(after_sha)`, for the D30 -join with `merge_commit_oid`; and every foreign-key column Steps 3b-1 and 4b add (`editor_id`, +join with `merge_commit_oid`; and every foreign-key column Steps 3b-1, 3b-2a-3, and 4b add (`editor_id`, `pr_review_requests.account_id`, `pr_assignees.account_id`, -`repository_activities.actor_id`, the commit tables' `author_id`, and the automation tables' -`run_id`), and `repository_activity_compare_commits (sha)` for D30's commit-to-event join. A test checks that every +`repository_activities.actor_id`, the commit tables' `author_id`, the automation tables' +`run_id`, and `account_node_ids`' `first_run_id` and `last_run_id`), and `repository_activity_compare_commits (sha)` for D30's commit-to-event join. A test checks that every foreign-key column leads an index (a composite primary key counts), except `pr_user_tags.user_tag_id`, which this section keeps unchanged. @@ -499,7 +505,8 @@ GitHub is consumer. The columns that say how current a row is are consumer time of everything the index wrote), `sync_failures`' repository, kind, number, error, and failure times, `sync_pending` (an in-coverage item a time-limited run did not reach), and every `sync_runs` column except the run counters (`graphql_points`, `rest_calls`, `items_*`, -`failures`, `refresh_failures`, `links_not_found`), which with the tag tables are internal. Integer +`failures`, `refresh_failures`, `links_not_found`, and from Step 4b `events_deleted` and +`history_commits_deleted`), which with the tag tables are internal. Integer row ids change on a rebuild, but a pull request's labels, reviews, comments, commits, files, and closing links, and its author and merger, are reachable only through them, so the foreign-key columns of consumer tables and the row ids they reference are consumer, valid within one file; other @@ -547,7 +554,9 @@ same number. `account_node_ids` included. A commit link resolves by its `databaseId`; the node id it carries (GraphQL `GitActor.user.id`, REST compare `author.node_id`) is stored in `account_node_ids` and checked as a sighting's is (gap 103). A commit-link lookup that answers `NOT_FOUND` is counted in - `sync_runs.links_not_found` and listed in the report (D28 stores the commit as before). + `sync_runs.links_not_found` and listed in the report (D28 stores the commit as before); from + Step 6a, a run in which more of its commit-link lookups answer `NOT_FOUND` than the share Step 5 + sets exits 2 (5.3 step 8), since a lookup form GitHub stopped resolving looks the same per link. - Type and login come only from GraphQL: actor sightings (PR author, merger, review author, comment author, editor, requested reviewer, assignee), which carry `__typename`, and `nodes(ids:)` lookups. A commit-linked account (GraphQL `GitActor.user` or the REST compare @@ -718,12 +727,12 @@ an interrupt, which is re-raised. |-------|-------------|--------| | Rate limited | Any HTTP 429 (GraphQL or REST); HTTP 403 with `retry-after`, `x-ratelimit-remaining: 0`, a secondary-limit message, or `errors[].type == "RATE_LIMITED"`; GraphQL HTTP 200 with `RATE_LIMITED` or an error message naming a secondary rate limit (GitHub documents 200 and 403 for GraphQL, 403 and 429 for REST) | Wait for `retry-after`; else, when `x-ratelimit-remaining` is 0 or the error is `RATE_LIMITED`, until `x-ratelimit-reset` or the last `resetAt`; else 60 s. Each further limit before a success waits at least 60 s doubled per repeat, whatever the headers say (GitHub: "If your request continues to fail due to a secondary rate limit, wait for an exponentially increasing amount of time between retries, and throw an error after a specific number of retries"). Never split; not an attempt. A wait that would end after the `--max-minutes` deadline stops queueing instead | | Timeout or resource limit | HTTP 502/504 (GitHub documents both for a request over 10 s); a GraphQL error at any path reporting a timeout or exceeded resource limits (GitHub returns "partial results along with an error indicating that resource limits were exceeded") | Split the batch in half at once; a same-size retry repeats the cost | - | Server or transport error | other 5xx; a GraphQL server fault at any path; connection error; client timeout; a connection the server broke off or a body it encoded wrongly; a GraphQL `errors` that is present and not a list (gap 104); a response without data or errors, without a readable `Date`, or with an error path that is not strings and list indices (never trimmed, since an exact path is what makes an item gone) | Retry up to 3 more times (2, 4, 8 s, with jitter), then split | + | Server or transport error | other 5xx; a GraphQL server fault at any path; connection error; client timeout; a connection the server broke off or a body it encoded wrongly; a GraphQL `errors` key whose value is not a list, `null` included (gap 104); a response without data or errors, without a readable `Date`, or with an error path that is not strings and list indices (never trimmed, since an exact path is what makes an item gone) | Retry up to 3 more times (2, 4, 8 s, with jitter), then split | | Item gone | alias `null` with `NOT_FOUND` | The hydrator reports it gone and does not retry it. In a sync, the run's own index listed the number in coverage, so GitHub's two answers disagree: the engine requests the item once more alone, and a second `NOT_FOUND` is an item failure (step 5, gap 100); an index that no longer lists the number removes the item and its failure (step 7). In verify's lookup by number, removed since sync (5.4) | | Item error | another alias error; partial alias data; a value the typed models reject; a failed check in step 5; HTTP 404/422 on a compare page | Retry the item once alone, then record it in `sync_failures` and write nothing for it | | Repository gone during hydration | a batch's `repository` is null (`RepositoryNotFound`, `RepositoryShapeError`; gap 79) | The repository fails for the rest of the run: its written items stay, step 7 skips it, and its `sync_pending` rows are unchanged | | Local database error | any error the database raises while the writer writes (a lock held past the busy timeout, an unexpected constraint failure; gap 77) | Fatal: roll back the item, close the run, exit 1 | - | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; a 2xx REST response without readable `x-ratelimit-remaining` and `x-ratelimit-reset` (gap 104; other statuses take the rows above whatever their headers); another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | + | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; a 2xx REST response without readable `x-ratelimit-remaining` and `x-ratelimit-reset` (gap 104; other statuses take the other rows of this table whatever their headers); another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | | Unauthorized or forbidden | HTTP 401; a failed credential request (an App token mint answering 4xx without a rate-limit signal); HTTP 403 with no rate-limit signal (a missing permission); an HTTP client failure outside the network (a redirect loop, a proxy or URL problem, a missing auth dependency) | Fatal: close the run, exit 1 | Typed response models check types strictly: GitHub's JSON has exact types, and lax coercion @@ -762,8 +771,9 @@ an interrupt, which is re-raised. issues, user-linked closing issues, review requests, and assignees; every user-linked closing issue is also in the full closing list; every overflow page reports the first page's `totalCount`; a null connection or null node is a failure, never a count of zero; the compare `total_commits` equals the PR's commit - `totalCount`, the fetched compare commits equal `total_commits`, and every commit on the commit - connection's first page is among them (gap 104); the returned number equals + `totalCount`, the fetched compare commits equal `total_commits`, and the commit connection's first + page equals compare's commits at positions `total_commits - 250` to `total_commits - 151`, from + 0, in order (gap 104); the returned number equals the request (the batch names the resolved repository); `state` is `MERGED` exactly when `mergedAt` is set, and a `CLOSED` or `MERGED` PR has `closedAt`, which the coverage rule reads; the entries the database keys (labels, closing issues, user-linked closing issues, reviews, @@ -775,13 +785,16 @@ an interrupt, which is re-raised. another PR fails the item. A PR that closed without merge between index and hydration is written with state `CLOSED` (D3). A hydration that contradicts the run's own index (an alias `NOT_FOUND`, an item outside coverage, an `updatedAt` earlier than the index entry's) is requested once more - alone by the engine, which holds the index, and a second contradiction is an item failure, - recorded before any account lookup for it (gap 100); a deadline that refuses that request + alone by the engine, which holds the index, and a second contradiction is an item failure of class + `IndexContradiction`, whose cause names which of the three it is, recorded before any account + lookup for it (gap 100); a deadline that refuses that request leaves the item in `sync_pending`, and the batch's other checked items are written as at any - deadline. Above 250 commits, only the commit - connection's first page is checked against compare's list: it holds the oldest commits, which a - wrong base adds or drops, and reading the rest would cost continuations a count check already - covers. + deadline. Above 250 commits GraphQL's commit connection holds the newest 250 (section 7, "Commit + cap"), so its first page is compare's 100 commits from position `total_commits - 250`, as the + recording of Prebid.js #8858 shows (254 commits, the page at positions 4-103). Only that page is + read, since the other 150 would cost continuations, and no GraphQL read reaches the oldest + `total_commits - 250` commits, where a wrong base adds or drops commits, so the count checks are + their only check. 6. Write: one batch at a time, on the writer's one connection. The run hydrates a batch, checks it (step 5), looks up its accounts (5.2), and writes each of its items before it requests the next batch; at the deadline it writes the fetched items whose accounts it has and records the rest in @@ -836,12 +849,13 @@ an interrupt, which is re-raised. indexed completely and nothing failed; 2 when an item failed in this run (other than one listed in `[sync].accepted_failures`), a repository failed, had an incomplete index, or halted, a stored repository is untracked in a run without `--repo`, a prune was refused (step 7), a refresh - failed (step 4, other than one an account entry accepts), or the run reached its deadline (status - `time_limited`, whatever it stopped: hydration, the refresh, or a Step 4b read; gap 99); 1 on a - fatal error. The exit code is decided after the prune. An accepted failure is still recorded and - reported; it names the repository and number, or the account, with the error class, the message, - and a reason in `tracking.toml`, so a permanent failure does not turn every run red, and a - failure of another class or message is not accepted (5.5, gap 101). An item entry + failed (step 4, other than one an account entry accepts), the run reached its deadline (status + `time_limited`, whatever it stopped: hydration, the refresh, or a Step 4b read; gap 99), or, from + Step 6a, more of its commit-link lookups answered `NOT_FOUND` than the share Step 5 sets (5.2); 1 + on a fatal error. The exit code is decided after the prune. An accepted failure is still recorded and + reported; it names the repository and number, the account, or a compare, with the error class, the + cause, and a reason in `tracking.toml`, so a permanent failure does not turn every run red, and a + failure of another class or cause is not accepted (5.5, gap 101). An item entry is reported as not having failed when its repository's outcome in this run is `complete`, `time_limited`, or `halted` and its item neither failed in this run nor is pending (5.5); an account entry, only when this run's refresh looked the account up and the lookup succeeded. @@ -936,7 +950,8 @@ the first class that matches, in this order: `subject` is that issue, and a difference in the totals by that count; - removed since sync: a stored item missing from the fresh index whose lookup by number returns `NOT_FOUND`, listed by number (the lookup is a hydration batch: `Gone` is removed since sync, a - hydrated PR an error); + hydrated PR an error, not retried: an index that omits a live PR is the read the sync's step 7 + deletes by, so verify reports it); - unchecked: a differing item whose timeline lookup (5.8) or lookup by number still fails after the retries on a request GitHub did not answer (a transport, server, timeout, or rate-limit failure), or that a lookup had not answered when a repository error stopped it, those of the batch the error @@ -947,7 +962,7 @@ the first class that matches, in this order: An error, an unchecked item, and an unverified repository fail verify in both modes: what verify could not check is a failure, never a pass (D37). `verify --exact` also fails on not synced, pending, halted deletion, removed since sync, and known failures no `[sync].accepted_failures` -entry accepts as 5.5 matches it (repository, number, error class, and message); it is the mode the +entry accepts as 5.5 matches it (repository, number, error class, and cause); it is the mode the Step 3-5 acceptance uses, and drift is reported in both modes. A repository that verify cannot resolve, re-index, or look up is reported as unverified: every repository error the source raises at resolve or index (not found, no default branch, a fetch that still fails after its retries, a @@ -956,9 +971,11 @@ label twice, paging without progress, a number listed twice or out of creation o count mismatch), step 1's own checks (a resolve to a `github_id` other than its stored row's, a later entry resolving to an earlier entry's `github_id`, gap 92, and another `is_tracked` stored repository holding the resolved `full_name`, ignoring case, checked as a dry run checks it, against the stored -names and the names earlier entries resolved to, 5.1), and a repository error during a lookup by -number or a timeline lookup (5.3 step 4), which keeps the classes the repository's comparisons -already gave. An unverified repository fails verify and is reported with its reason; a verify in +names and the names earlier entries resolved to, 5.1), a repository error during a lookup by +number or a timeline lookup (5.3 step 4), and, from Step 4b, verify's own activity or history read +that still fails after its retries, ends before the floor and the log's first event, or whose +count still differs after one rescan; each keeps the classes the repository's comparisons already +gave. An unverified repository fails verify and is reported with its reason; a verify in which every repository it attempts is unverified, and the deadline stopped none, is fatal (exit 1), as a sync is when no configured repository resolves (5.3 step 1). A verify that reaches `--max-minutes` stops with a timeout instead of waiting past it. Exit 0 when no failing class @@ -974,11 +991,12 @@ stops before its comparisons finish. A report it cannot write at exit turns an e because Step 6a reads it (5.6, D36); an interrupt exits 130 either way. The report holds the exit code and whether `--exact` was given; per class, the count of items and their numbers (an item whose differences take two classes counts -in each); per repository, its `last_run_id`, boundary, per-class counts, and per-state counts; +in each); per repository, its `last_run_id`, boundary, per-class counts, per-state counts, and per +child-count check the rows it compared (D39); for each closing-link difference, the issues it names and the stored and fresh totals; from -Step 4b, per repository the compares stored with `files_at_cap` and those GitHub refused, which do -not fail verify (GitHub's limit and GitHub's refusal, which no retry lifts) but are never reported -as complete; the unverified repositories with their reasons; and verify's GraphQL points and REST +Step 4b, per repository the compares stored with `files_at_cap`, those GitHub refused, and those +that still failed, which do not fail verify (GitHub's limit, and answers the sync requests again +on each run, Step 4b) but are never reported as complete; the unverified repositories with their reasons; and verify's GraphQL points and REST calls, counted by its source as a run's are (5.3 step 4), and the seconds from its start to the report's write, since verify writes no `sync_runs` row. A report verify prints that stdout cannot take leaves the exit code verify's, and stderr follows 5.3 step 8, 120 included. @@ -1029,20 +1047,31 @@ strictly typed throughout: an account `github_id` is at least 1, and `login`, `e Python `timedelta` holds, so the planner can build the window), `delete_guard_min_rows` an integer of at least 0, and `delete_guard_percent` a number above 0 and at most 100; an `accepted_failures` entry names -an item (a configured `repository` and a `number` of at least 1) or an account (`github_id`, for a -refresh failure, or `node_id` for an account stored with a null `github_id`), an `error`, and a -`message`: the class and the message of `sync_failures.last_error` (or of the refresh failure's -text), split at its first `": "` and each compared for equality, since several classes carry -more than one cause in their message; the loader refuses a class that is not one of the failure -classes the sync records, and an entry without `message`. Entries are unique by item or account, an -item entry matches by its configured repository, so a rename does not unhook it, and a new cause is -not accepted (gap 101); an entry whose item or account did not fail is reported (5.3 step 8). +an item (a configured `repository` and a `number` of at least 1), an account (`github_id`, for a +refresh failure, or `node_id` for an account stored with a null `github_id`), or from Step 4b a +compare (a configured `repository` and the event's `event_id`), with an `error` and a `message`. +The failure's text (`sync_failures.last_error`, the refresh failure's, or the compare's `error`) +is split at its first `": "`: `error` must equal the class, and `message` the first line of the +rest, the failure's cause. Every failure the sync records puts its cause, in the code's own words, +on that first line, and what can change between runs (GitHub's message and request id, a count, a +typed model's rejection text) on the lines after it, so an entry keeps matching a permanent failure +while a failure with another cause is not accepted; several classes carry more than one cause. The +loader refuses an entry without `message`, and a class whose name is not in the list of recorded +failure classes `tracking.py` keeps, which imports none of the modules that define them; a test +holds that list equal to the concrete subclasses of the failure bases the engine records +(`ItemError`, `PlannerFailure`, `WriteRefused`, and the engine's own `HydratedIdMismatch` and +`IndexContradiction`), `UserLookupFailed` and `LookupConflict` for an account, and the classes +Step 4b's compare reader records. Entries are unique by item, account, or compare, an item or +compare entry matches by its configured repository, so a rename does not unhook it, and a new +cause is not accepted (gap 101); an entry whose item, account, or compare did not fail is reported +(5.3 step 8). `[automation]` is required, its lists may be empty, so a file without it is refused instead of emptying the automation tables (gap 105). The file holds domain policy only. Step 1 added `[[repositories]]`; Step 3b-2a-1 adds `[coverage]` and `[sync]`'s `unsettled_minutes`, `delete_guard_min_rows`, and `delete_guard_percent`, which the planner reads; Step 3b-2a-2a adds `[automation]` and `[sync].accepted_failures`, which the run claim and the exit code read; Step -3b-2a-3 adds an entry's `error`, `message`, and account form, and makes `[automation]` required. +3b-2a-3 adds an entry's `error`, `message`, and account form, and makes `[automation]` required; +Step 4b adds the compare form. ```toml [coverage] @@ -1088,8 +1117,10 @@ Workflow `.github/workflows/sync.yml`: - Triggers: `schedule` at `17 */6 * * *` (start of the hour is a documented high-load time) and `workflow_dispatch` with inputs `bootstrap`, `allow_deletions` and `prune_untracked` (passed to - `ghactivity sync`, 5.3 step 7), and `release` (default `db`; the Step 6 acceptance runs against - `db-test`). A `schedule` run has no inputs, so every step reads `${{ inputs.release || 'db' }}` and + `ghactivity sync`, 5.3 step 7), `repo` (repositories separated by spaces, each validated as a + configured repository and passed as `--repo` to sync and verify; it requires `bootstrap`, the + manifest records it, and a run with it never publishes to release `db`: Step 6b's comparison is + its use), and `release` (default `db`; the Step 6 acceptance runs against `db-test`). A `schedule` run has no inputs, so every step reads `${{ inputs.release || 'db' }}` and treats the other inputs as unset. - `concurrency: { group: db-sync, cancel-in-progress: false }`. A new run replaces a pending one and never cancels a running one. @@ -1144,10 +1175,11 @@ Workflow `.github/workflows/sync.yml`: `actions_run_id`, `sync_run_id`, `sync_finished_at`, the code SHA, the `tracking.toml` digest (`sync_runs.config_sha256`), migration head, `schema_version`, coverage floor, `bootstrap`, per-repository PR and issue counts, per kind the - number of rows whose `hydration_version` differs from the code's, the sync exit code, verify's - exit code, counts per class, unverified repositories, and from Step 4b the compares at the file - cap and those refused (from `--verify-report`; null without - it), and the SHA-256. `snapshot` exits 1 when the report `--verify-report` names is missing or + number of rows whose `hydration_version` differs from the code's, the sync exit code, the sync + run's `repo_filter` and `links_not_found` and from Step 4b its `events_deleted` and + `history_commits_deleted`, verify's exit code, counts per class, unverified repositories, and from + Step 4b the compares at the file cap, those refused, and those that still failed (from + `--verify-report`; null without it), and the SHA-256. `snapshot` exits 1 when the report `--verify-report` names is missing or unreadable, or names a `last_run_id` other than the file's `repo_sync_state`, so a verify that never started, which leaves no report or one from an earlier sync (5.4), fails the job. In the workflow, `snapshot --after-run N --sync-exit C` takes the highest `sync_runs` id the job recorded @@ -1422,7 +1454,8 @@ nested node. Each step is one pull request from a `feature/` branch, except Step 3 (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2), Step 4 (pull requests 4a and 4b), -and Step 6 (pull requests 6a and 6c, with the 6b manual runs between them). A step, or a pull +Step 5 (pull request 5, and 5b when its measurement finds an edit kind that leaves `updatedAt` +unchanged), and Step 6 (pull requests 6a and 6c, with the 6b manual runs between them). A step, or a pull request within one, starts after the previous one merges. ### Step 1: Guardrails @@ -1897,8 +1930,10 @@ check without a test that targets it; CI passes on the exact PR head. Prerequisite: Step 3b-2a-2 has merged. Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh_failures`, -`links_not_found`, and `repo_filter` and the `account_node_ids` table, and refuses a file whose -`sync_runs` or `github_users` holds rows (5.1, gap 110; a development file is rebuilt by a sync): +`links_not_found`, and `repo_filter`, `unknown` to the `trigger` CHECK, and the `account_node_ids` +table, and refuses a file whose `sync_runs` or `github_users` holds rows (5.1, gap 110; a +development file is rebuilt by a sync), with their data dictionary entries and the current +schema-version lock entry's consumer pairs: - a failed 24-hour refresh is counted (accounts), listed in the report, and makes the exit 2 unless an account entry accepts it (5.3 step 4, gap 98); - a run that reaches its deadline exits 2, and the failures a batch made final before the deadline @@ -1906,25 +1941,36 @@ Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh - a hydration that contradicts the run's own index (an alias `NOT_FOUND`, an item outside coverage, an `updatedAt` earlier than the index entry's) is requested once more alone by the engine and then is an item failure (5.3 steps 3, 4, and 5, gap 100); -- an `accepted_failures` entry names the error class and the message it accepts, and may name an - account, by `github_id` or `node_id`, for a refresh failure (5.5, gap 101); +- an `accepted_failures` entry names the error class and the cause it accepts, and may name an + account, by `github_id` or `node_id`, for a refresh failure; every failure the sync records puts + its cause on its first line and what changes between runs on the lines after it; `tracking.py` + keeps the list of recorded failure classes, and the index contradictions take the class + `IndexContradiction` (5.3 step 5, 5.5, gap 101); - a PR whose link `totalCount` exceeds the 10 links the index carries is hydrated every run (5.3 step 3, gap 102); - every node id a sighting, a lookup, or a commit link reports for a stored account is stored in `account_node_ids`, and a commit-link lookup answered `NOT_FOUND` is counted (5.2, gap 103); - the source checks of gap 104: a 2xx REST response without readable rate-limit headers is not - understood, a GraphQL `errors` present and not a list is a server error, a compare above 250 - commits holds every commit on the commit connection's first page, and every index page selects - the repository's `databaseId` and is checked against the resolved `github_id` (the index units are - re-recorded; no `hydration_version` changes); + understood, a GraphQL `errors` whose value is not a list, `null` included, is a server error, a + compare above 250 commits holds the commit connection's first page at positions + `total_commits - 250` to `total_commits - 151`, in order, and every index page selects the + repository's `databaseId` and is checked against the resolved `github_id` (no `hydration_version` + changes; the index document changes, so the GraphQL recordings are re-recorded with `--only + documents` and the first-sync directory with `--only first-sync`, the recording splitter and the + replay keep the repository's `databaseId` beside `pullRequests` in an index unit, and the tests + that assert recorded values follow the new recordings); - the run-record and setup checks of gap 105: `code_version` as 5.1 defines it, with `_git` returning the exit status; a close given an outcome it cannot write raises before writing; `[automation]` is required; the connect listener reads `foreign_keys` back and refuses a connection where it is off; `alembic/env.py` refuses a `stamp` that would change the revision; the wheel check compares the packaged `tracking.toml` bytes with the source file's; the log-file - handler receives every record at its level; and `sync_runs.repo_filter` records `--repo`; + handler receives every record at its level; `sync_runs.repo_filter` records `--repo`; and + `trigger` records `unknown` for a `GITHUB_EVENT_NAME` other than `schedule` and + `workflow_dispatch`; - the data dictionary describes the head-repository, milestone, and team names as of `hydrated_at` (gap 108); +- `docs/development.md`'s list of sync's exit-2 conditions gains the deadline, a failed refresh, + and an index contradiction; - the fixtures these rules break: the scenario model renders `x-ratelimit-*` headers on REST units and the repository `databaseId` on index pages; every test configuration gains `[automation]`; every `accepted_failures` entry in the tests and in the packaged file's comment gains `error` and @@ -1933,36 +1979,44 @@ Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh Tests, each rule on both sides with inputs D39 allows: a refresh request that fails and one that succeeds, a refresh answer with another `databaseId`, an accepted account entry (exit 0) by `github_id` and by `node_id`, and an account entry the refresh did not reach, not reported as not -having failed; a run that +having failed; `refresh_failures` counting each failed refresh, accepted ones included; a run that finishes before its deadline (exit 0) and one whose deadline stops hydration, the refresh, or both (exit 2); a deadline inside a batch after one item's failure became final, which is recorded while the rest wait in `sync_pending`; a `NOT_FOUND` alias for a number the index listed that the retry clears and one it does not, then a later run whose index omits the number and whose step 7 removes the item and its failure; a hydration outside coverage that the retry clears and one it does not; a deadline that refuses the retry, leaving the item in `sync_pending`; a hydrated `updatedAt` -earlier than, equal -to, and later than the index entry's, the earlier one cleared by the retry and not; an accepted -entry whose class and message match, one whose class differs, one whose message differs, and a -loader refusing an unknown class and an entry without `message`; PRs with 10 and with 11 links; a -second node id from a sighting, a lookup, and a REST compare link stored and seen again, and a node -id in `account_node_ids` reported under another `databaseId` failing the item; a commit-link lookup -answered `NOT_FOUND` counted; each source check accepting the shapes GitHub's recordings hold and refusing -each shape its rule names, and a 502 without rate-limit headers still retried; `code_version` outside any `.git` (the -package version), in a checkout with the package untracked (the package version), in a clean and a -dirty checkout (the commit, and `-dirty`), and with `git status`, `ls-files`, and `rev-parse` -failing or timing out inside a checkout (`-unknown`); a close given an outcome with no stored row -leaving the row `running`; a `stamp` to another revision refused with nothing committed and one at -head accepted; the migration refusing a populated file; a connection whose `foreign_keys` reads back -0 refused; the wheel check reporting the SHA-256 of the packaged `tracking.toml` it compared, -equal to the source file's; a record at the log file's level reaching it and one below not; -`repo_filter` written for a `--repo` run and null without one; a file without `[automation]` -refused; and the report listing failed refreshes and commit links answered `NOT_FOUND`. Each is a -regression test that fails against the parent's code. The tests that pinned the old behaviour change -with the rule they pinned: in `tests/sync/test_engine.py` the failed-refresh, deadline (hydration -and refresh), gone-item, leaving-coverage, and 11-link tests; in `tests/sync/test_runs.py` the -skipped-close and `code_version` tests; in `tests/test_tracking_config.py` the defaults without -`[automation]` and the accepted-failure entries; and the deadline-inside-a-batch test, whose -failures are now recorded. +earlier than, equal to, and later than the index entry's, the earlier one cleared by the retry and +not; an accepted entry whose class and cause match, one whose class differs, one whose cause +differs, and one that still matches a failure whose request id, counts, and typed-model text +differ between two runs; a loader refusing an unknown class and an entry without `message`, and +the recorded-class list equal to the classes the engine records; PRs with 10 and with 11 links; a +second node id from a sighting, a lookup, and a REST compare link stored and seen again in a later +run, keeping `first_run_id` and advancing `last_run_id`, and a node id in `account_node_ids` +reported under another `databaseId` failing the item; a commit-link lookup answered `NOT_FOUND` +counted; each source check accepting the shapes GitHub's recordings hold and refusing each shape +its rule names (a 2xx REST response without rate-limit headers; an `errors` of `{}`, `""`, and +`null`; an index page whose repository `databaseId` differs; a commit page from other positions), +the commit page at positions 4-103 of the recording of Prebid.js #8858 accepted, and a 502 +without rate-limit headers still retried; `code_version` outside any `.git` (the package +version), in a checkout with the package untracked (the package version), in a clean checkout, +one holding only an untracked file, and a dirty one (the commit, the commit, and `-dirty`), and +with `git status`, `ls-files`, and `rev-parse` failing or timing out inside a checkout +(`-unknown`); `trigger` for `schedule`, `workflow_dispatch`, another event name, and none; a close +given an outcome with no stored row leaving the row `running`; a `stamp` to another revision and a +`stamp --purge` at head refused with nothing committed, and a `stamp` at head accepted; the +migration refusing a populated file; a connection whose `foreign_keys` reads back 0 refused; the +wheel check reporting the SHA-256 of the packaged `tracking.toml` it compared, equal to the source +file's; a record at the log file's level reaching it and one below not; `repo_filter` written for +a `--repo` run, sorted, with each value as given, and null without one; a file without +`[automation]` refused; and the report listing failed refreshes and commit links answered +`NOT_FOUND`. Each rule's refusing side is a regression test that fails against the parent's code; +its accepting side passes on both. The tests that pinned the old behaviour change with the rule +they pinned: in `tests/sync/test_engine.py` every test that asserts a `time_limited` run exits 0, +the deadline inside a batch among them, whose failures are now recorded, and the failed-refresh, +gone-item, leaving-coverage, and 11-link tests; in `tests/sync/test_runs.py` the skipped-close and +`code_version` tests; and in `tests/test_tracking_config.py` the defaults without `[automation]` +and the accepted-failure entries. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule of this step without a test that targets it; CI passes on the exact PR head. @@ -2023,7 +2077,8 @@ each PR's draft and link event history, which the model gains here, rendering `s `filteredCount` counting the matching entries after its cursor), with the replay matching a timeline unit by its `since` (the timeline recordings join the model's round-trip test), and fake GitHub: each verify class, each overlapping pair of classes taking the first in the 5.4 -order, a stale item, verify's repository set with and without `--repo` (a stored untracked +order, a known failure whose class an entry accepts but whose cause differs, reported as a known +failure and failing `--exact`, and one whose cause matches, passing it, a stale item, verify's repository set with and without `--repo` (a stored untracked repository, a stored row no configured entry names, a renamed repository, and a renamed repository whose configured entry fails to resolve, reported unverified and not untracked, and an entry that resolves to a `github_id` other than its stored row's, with no stored `is_tracked` row reported @@ -2055,10 +2110,9 @@ link totals alone with enough connected events, with only a disconnected event, synthetic disconnected event, an event at the boundary counted and one a second before it not, entries the model renders at a second and at half a second before the boundary, not counted, a stored link no longer indexed on a PR whose index lists every link matched only by an -event naming it, a filtered page whose `totalCount` differs from its entries, a lookup whose entries -differ from its `filteredCount` after its retry and one whose response the typed models reject, -a known failure whose class an entry accepts but whose message differs failing `--exact`, -each an error, one that differs once and matches on its retry, classed by its events, and a +event naming it, a filtered page whose `totalCount` differs from its entries, counted by its +entries, a lookup whose entries differ from its `filteredCount` after its retry and one whose +response the typed models reject, each an error, one that differs once and matches on its retry, classed by its events, and a lookup spanning two pages whose second page's `filteredCount` counts only the entries after its cursor (5.8). A re-recording in the default mode and with `--only documents` keeping the timeline units. @@ -2120,64 +2174,87 @@ field by field. Scope (D30): the repository activity reader, its compares, and the default-branch history reader (5.8); the `repository_activities`, `repository_activity_compares`, `repository_activity_compare_files`, `repository_activity_compare_commits`, and -`default_branch_commits` tables (5.1) in a migration, with a stored `totalCount` for the history +`default_branch_commits` tables (5.1) and the `sync_runs` counters `events_deleted` and +`history_commits_deleted` in a migration, with a stored `totalCount` for the history (`repo_sync_state.default_branch_commits_total`), the as-of times of the log and history reads (`activity_read_at`, `history_read_at`), and the activity read's outcome and halt reason (`activity_last_outcome`: `complete`, `time_limited`, `failed`, or `halted`; `activity_halt_reason`; -5.1); the -planner and writer steps, which run after 5.3 step 6's writes and before step 7. Each run reads the -activity log since the floor in full, following GitHub's `Link` cursors, which the source then -returns with a REST page; a read is complete only when a page has no next link or its oldest event -is before the floor, a deadline gives `time_limited` (exit 2, 5.3 step 8), and any other end fails -the repository's activity for the run (exit 2, gap 106). A complete read upserts the events from -the floor on by `github_id` and records in `activity_first_event_at` the `timestamp` of the oldest -event it listed, one before the floor included, keeping each stored -event's compare, then deletes the stored events the log no longer lists (D7) unless they exceed -max(25, 2%) of the repository's stored events, which halts that deletion as step 7 halts -deletions. It compares each event not yet compared whose `before` and `after` are both non-zero, -storing compare's commits in full (counted against `total_commits`) and its files, with -`files_at_cap` set when compare returned 300 files, GitHub's documented cap for one comparison -(gap 106); a compare GitHub refuses (HTTP 404 or 422) is stored with its `http_status` and requested -again on each later run until it returns 2xx, and verify's report and the manifest count both kinds -(5.4, 5.6). It reads the default branch's history since the -floor in full and replaces that repository's `default_branch_commits` rows only when the read count -equals `totalCount` and the commits it would remove are within the same guard, so a commit a force -push removed leaves as GitHub no longer lists it; a count that differs is rescanned once, and a -second mismatch fails the repository's activity for the run (exit 2, gap 106). A log or history -page, or a compare, that still fails under 5.3 step 4's transient classes makes the repository's -activity `failed` for the run: it writes none of that read, and the run exits 2. The deadline stops -the readers as it stops hydration, a stopped read writes nothing (`time_limited`), and a halt is -`halted` with its reason in `repo_sync_state.activity_halt_reason`; `--allow-deletions` applies its -N to the events and to the history commits separately (5.3 step 7). Event actors and commit authors -are linked like commit links (5.2), and a history replace keeps a commit's stored link when GitHub -returns none (D28). verify compares the stored event ids with the log since the floor and the stored -commits with the history, taking the 5.4 classes, with `activity_read_at` the boundary for events -and `history_read_at` for commits: not synced when the repository's `activity_last_outcome` is not -`complete` or the boundary is null, halted deletion for a halt, drift for an event or commit the -fresh read lists at or after the boundary, removed since sync for a stored event or commit the -fresh read no longer lists, and error otherwise. The data dictionary for the new tables; -recordings from public repositories. +5.1); the compare form of `accepted_failures` (5.5); the planner and writer steps, which run after +5.3 step 6's writes and before step 7. Each run reads the activity log since the floor in full, +following GitHub's `Link` cursors, which the source then returns with a REST page; a read is +complete only when its oldest listed event is before the floor or is the first event a read of the +log in ascending order returns, made in the same run (one REST call), so a listing that ends early +without a next link is not taken for the log's start; a deadline gives `time_limited` (exit 2, 5.3 +step 8), and any other end fails the repository's activity for the run (exit 2, gap 106). A +complete read upserts the events from the floor on by `github_id` and records in +`activity_first_event_at` the `timestamp` of the oldest event it listed, one before the floor +included, keeping each stored event's compare, then deletes the stored events the log no longer +lists (D7), counting them in `sync_runs.events_deleted`, unless they exceed max(25, 2%) of the +repository's stored events, which halts that deletion as step 7 halts deletions. It compares each +event whose `before` and `after` are both non-zero and that has no compare row, or one holding an +`error` or a refusal, storing compare's commits in full (counted against `total_commits`) and the +files of its first page, the only page GitHub lists files on, with `files_at_cap` set when that +page returned 300 files, GitHub's documented cap for one comparison (gap 106). Each compare is +written in its own transaction when it completes, so a deadline keeps the compares already +written. A compare GitHub refuses (HTTP 404 or 422) is stored with its `http_status`; one that +still fails under 5.3 step 4's transient classes, or whose 2xx first page has no `files` list, is +stored with its `error` and makes the repository's activity `failed` (exit 2) unless an +`accepted_failures` entry for the repository and the event's `github_id` accepts it (5.5); both are +requested again on each later run until GitHub answers them, and verify's report and the manifest +count the capped, refused, and failed compares (5.4, 5.6). It reads the default branch's history +since the floor in full and replaces that repository's `default_branch_commits` rows only when the +read count equals `totalCount` and the commits it would remove are within the same guard, counting +them in `sync_runs.history_commits_deleted`, so a commit a force push removed leaves as GitHub no +longer lists it; a count that differs is rescanned once, and a second mismatch fails the +repository's activity for the run (exit 2, gap 106). A log or history page that still fails under +5.3 step 4's transient classes makes the repository's activity `failed` for the run: it writes +none of that read, and the run exits 2. The deadline stops the readers as it stops hydration, a +stopped log or history read writes nothing (`time_limited`), and a halt is `halted` with its +reason in `repo_sync_state.activity_halt_reason`; `--allow-deletions` applies its N to the events +and to the history commits separately (5.3 step 7). Event actors and commit authors are linked +like commit links (5.2), and a history replace keeps a commit's stored link when GitHub returns +none (D28). verify reads the log and the history by the sync's completeness rules; a read that +still fails after its retries, ends before the floor and the log's first event, or whose history +count still differs after one rescan makes the repository unverified (5.4). It compares the stored +event ids with the log since the floor and the stored commits with the history, taking the 5.4 +classes, with `activity_read_at` the boundary for events and `history_read_at` for commits: not +synced when the repository's `activity_last_outcome` is `failed` or `time_limited` or the boundary +is null; halted deletion for a stored event or commit the fresh read no longer lists when it is +`halted`; drift for an event the fresh log lists at or after its boundary, and for a commit the +fresh history lists and the stored rows lack when the fresh log holds an event at or after +`history_read_at` (a merge brings commits with their earlier dates); removed since sync for a +stored event or commit the fresh read no longer lists; and error otherwise. The data dictionary +for the new tables; recordings from public repositories. Tests: an event of each `activity_type` the recordings hold; an event with an all-zero SHA and no compare row; a refused compare stored with its status and requested again on the next run; a -compare of exactly 300 files marked `files_at_cap` and one of 299 not; a two-page activity read -through a `Link` cursor that completes and sets `activity_first_event_at`, one that stops at the -floor, one the deadline stops (`time_limited`, `activity_first_event_at` unchanged), and one that -ends without reaching the floor or the log's end failing the activity; a history count mismatch -rescanned once and, on a second mismatch, failing the activity; a second run sending no compare for -an event already compared; an event deletion past the guard halting; a diverged push whose compare -runs from the merge base; a push of several commits whose compare commits join each commit to its -event; a force push removing commits from `default_branch_commits`; a history read past the guard -halting; a null actor; an event GitHub stops listing removed on the next run; an event the log stops -listing after the stored read classified as removed since sync; `activity_halt_reason` set by a halt -and cleared by the next run that does not halt, `halt_reason` unchanged; `--allow-deletions` -applying N to events and history commits separately; a history replace keeping a commit's stored -author link when GitHub returns none (D28); each verify check and each stored-row check on the recordings, each stored-row check reporting the -rows it compared (D39). +compare of exactly 300 files marked `files_at_cap` and one of 299 not; a compare whose 2xx first +page has no `files` list, and one that still fails under the transient classes, each stored with +its `error`, making the activity `failed`, requested again on the next run, and accepted by a +compare entry; the compares written before a deadline kept; a two-page activity read through a +`Link` cursor that completes and sets `activity_first_event_at`, one that stops at the floor, one +the deadline stops (`time_limited`, `activity_first_event_at` unchanged), and one that ends without +a next link before the first event of the ascending read, failing the activity; a history count +mismatch rescanned once and, on a second mismatch, failing the activity; a second run sending no +compare for an event already compared; an event deletion past the guard halting; deleted events +and history commits counted in `sync_runs.events_deleted` and `history_commits_deleted`; a +diverged push whose compare runs from the merge base; a push of several commits whose compare +commits join each commit to its event; a force push removing commits from +`default_branch_commits`; a history read past the guard halting; a null actor; an event GitHub +stops listing removed on the next run; an event the log stops listing after the stored read +classified as removed since sync; a merge that brings commits dated before `history_read_at`, +classed as drift by its event; a halted repository's unlisted events classed as halted deletion; +verify's own log read failing and ending early, and its history count differing after a rescan, +each making the repository unverified; `activity_halt_reason` set by a halt and cleared by the +next run that does not halt, `halt_reason` unchanged; `--allow-deletions` applying N to events and +history commits separately; a history replace keeping a commit's stored author link when GitHub +returns none (D28); and each verify check and each stored-row check on the recordings, each +stored-row check reporting the rows it compared (D39). Acceptance: on the Step 3b-2 acceptance repositories and on `prebid/github-activity-db`, the stored events equal the log and the stored commits equal the history; each repository's first logged event -is recorded (section 8), with the file count of its largest push's compare; the run records its +is recorded (section 8), with the file count of its largest push's compare and, for Prebid.js, +the page count of its full log read; the run records its REST calls, points, and time. ### Step 5: Full bootstrap and verification @@ -2195,14 +2272,21 @@ failures, each listed in the PR with REST evidence of why it cannot pass; a 50-i matches REST; a difference report against the tracked file classifies every differing item (closed-unmerged PR, file-status rebuild, commit list beyond 250, login rename, changed on GitHub after the old file's sync, in-rule item absent from the old file, schema-only) with per-category counts, zero -unclassified items, and five REST-checked examples per category; the REST sample measures whether a -body edit, a base-ref change, a review dismissal, and (from Step 4a) an issue label event change -`updatedAt`, at least 20 cases of each where the repositories hold them; a kind with a case that -leaves `updatedAt` unchanged, or with fewer than 20 cases, gets an index-side check or a periodic -re-hydration in a pull request of its own before Step 6a, which adds the 5.3 step 3 rule and its -tests (gap 109); the run's time, cost, file -size, and -secondary-limit responses at one request in flight are recorded (and at any wider setting proposed). +unclassified items, and five REST-checked examples per category; a GraphQL timeline read of a +sample measures whether a body edit (`userContentEdits`), a base-ref change +(`BaseRefChangedEvent`), a review dismissal (`ReviewDismissedEvent`), and (from Step 4a) an issue +label event (`LabeledEvent`, `UnlabeledEvent`) change `updatedAt`, at least 20 counted cases of +each where the repositories hold them: a case counts only when no other timeline item or edit lies +between the event and the PR's or issue's `updatedAt`, and an event later than `updatedAt` is a +case that left it unchanged; a kind with such a case, or with fewer than 20 counted cases, gets an +index-side check or a periodic re-hydration in pull request 5b before Step 6a, which adds the 5.3 +step 3 rule and its tests (gap 109); for each index contradiction (an alias `NOT_FOUND`, an item +outside coverage, an earlier `updatedAt`), how many occurred and how many the retry cleared are +recorded, and a kind that recurs gets a delay before its retry, measured and chosen before Step 6a; +the share of commit-link lookups answered `NOT_FOUND` is recorded, and Step 6a's limit is set from +it (5.2); and the run's time, cost, file size, and secondary-limit responses at one request in +flight are recorded (and at any wider setting proposed), with the time the workflow's bootstrap is +projected to take, Step 4b's compares included. ### Step 6: Scheduled sync and publication @@ -2211,39 +2295,54 @@ installed on the tracked repositories, with its app ID, private key, and install repository secrets. Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snapshot`, -`ghactivity verify --db-only` (5.4), and `scripts/compare_reads.py` for 6b, and adds the +`ghactivity verify --db-only` (5.4), and `scripts/compare_reads.py` for 6b, adds the commit-link +`NOT_FOUND` limit with its constant from Step 5's measurement (5.2, 5.3 step 8), and adds the `githubkit[auth-app]` extra to `pyproject.toml` and `uv.lock`. Its tests: `verify --db-only` accepting the gz and manifest `ghactivity snapshot` wrote, and reporting the SHA-256 it computed, the per-repository counts it compared, the revision it read, and the answers of `integrity_check` and `foreign_key_check`, each equal to what the file holds (D39); a release with no complete snapshot pair halting with exit 1; `snapshot --verify-report` exiting 1 on a missing report and on -one from a verify of an earlier sync, whose `last_run_id` differs from the file's, and the manifest listing the report's -unverified repositories (5.6); through -`httpx.MockTransport`, a mocked installation-token response whose `expires_at` lies just over a -minute ahead, a clock moved past it mid-run, and a second token request before the next API request -(githubkit keeps a token until one minute before `expires_at`); and a 401 on an API request still -exiting 1 with the run closed (5.3 step 4); `verify --db-only --out` writing the file and printing -its SHA-256 and highest `sync_runs` id; `snapshot --after-run --sync-exit` refusing no newer row, -two newer rows, a row closed with another exit code, and a configured entry's stored row whose -`last_run_id` is older, and accepting a stored repository the run neither attempted nor marked -(5.6); `snapshot --sync-run` refusing a missing or unclosed row, and neither form or both a usage -error; the manifest carrying `sync_run_id`, `sync_finished_at`, `actions_run_id`, and the Step 4b -compare counts; and a test that reads `sync.yml` and fails unless every `if:` that reads an -exit-code output compares a quoted string and requires it to be non-empty, the job-level -`DATABASE_URL` equals `verify --db-only`'s `--out`, the SHA-256 comparison and the recording of N -come before `alembic upgrade head`, and `snapshot` is given `--after-run` and `--sync-exit`. 6b -creates release `db` and uploads the Step 5 database with its manifest -(`ghactivity snapshot --sync-run` naming Step 5's last run, and `--verify-report` from Step 5's -`verify --exact --report`) as the first snapshot. Before any App run writes to release `db`, 6b -compares the App's reads with the maintainer token's: a `workflow_dispatch` run with -`release: db-test` syncs a sample of the repositories with the App, and at once a local run syncs -the same repositories with the token into a fresh file; `scripts/compare_reads.py` compares the two -files over the hydrated tables, every column and child row except those stamped from `Date` headers -and the run ids. Any difference blocks App runs that publish to `db`, and 6c, until it is explained -or the permission is granted (gap 107). It creates release `db-test` the same way, holding snapshots -this project's code wrote: one built at the revision before the last migration, and one synced with -a `tracking.toml` that listed a repository the current one does not (a sync exit 2). Once the comparison shows no unexplained difference, it runs one -normal manual run and one bootstrap run with `release: db`. 6c enables the schedule and stops +one from a verify of an earlier sync, whose `last_run_id` differs from the file's, and the manifest +listing the report's unverified repositories (5.6); through `httpx.MockTransport`, a mocked +installation-token response whose `expires_at` lies just over a minute ahead, a clock moved past +it mid-run, and a second token request before the next API request (githubkit keeps a token until +one minute before `expires_at`); and a 401 on an API request still exiting 1 with the run closed +(5.3 step 4); `verify --db-only --out` writing the file and printing its SHA-256 and highest +`sync_runs` id; `snapshot --after-run --sync-exit` refusing no newer row, two newer rows, a row +closed with another exit code, and a configured entry's stored row whose `last_run_id` is older, +and accepting a stored repository the run neither attempted nor marked and, after a bootstrap with +`--repo`, the entries it did not name (5.6); `snapshot --sync-run` refusing a missing or unclosed +row, and neither form or both a usage error; the manifest carrying `sync_run_id`, +`sync_finished_at`, `actions_run_id`, `repo_filter`, `links_not_found`, and the Step 4b compare and +deletion counts; the commit-link limit making the exit 2 above its share and not at it; +`scripts/compare_reads.py` on two files the sync wrote from the same recordings, reporting no +difference and the rows and columns it compared per table, equal to the files' counts, and on files +it wrote from recordings of one repository taken at two times, reporting each difference the +recordings hold (D39), and exiting 1 when a file is not at the code's migration head or a compared +table is empty in both; and a test that reads `sync.yml`, lists each exit-code output the `sync` +job declares, and fails unless `publish`'s and `report`'s conditions compare each as a quoted +string and require it to be non-empty, the path parsed from the job-level `DATABASE_URL`, resolved +against the step's working directory, equals `verify --db-only`'s `--out`, the SHA-256 comparison +and the recording of N come before `alembic upgrade head`, the `repo` input is validated and +requires `bootstrap`, and `snapshot` is given `--after-run` and `--sync-exit`. 6b creates release +`db` and uploads the Step 5 database with its manifest (`ghactivity snapshot --sync-run` naming +Step 5's last run, and `--verify-report` from Step 5's `verify --exact --report`) as the first +snapshot. It creates release `db-test` the same way, holding snapshots this project's code wrote: +one built at the revision before the last migration, and one synced with a `tracking.toml` that +listed a repository the current one does not (a sync exit 2). Before any App run writes to release +`db`, 6b compares the App's reads with the maintainer token's: a `workflow_dispatch` run with +`bootstrap`, `release: db-test`, and `repo` naming a sample of the repositories syncs them with the +App into an empty file, and at once a local run syncs the same repositories with the token into a +fresh file. The sample holds a PR with a `Team` review request, one with an `Organization` +assignee, and a private repository other than `prebid-agent-skills` (D33); a case the tracked +repositories do not hold is reported as unmeasured. `scripts/compare_reads.py` compares the two +files over every table the sync writes from GitHub (`repositories`, `repo_labels`, +`github_users`, `account_profiles`, `account_node_ids`, `pull_requests` and its child tables, and +from Step 4 the issue and activity tables), every column and child row except those stamped from +`Date` headers and the run ids, and lists as a difference any value the token read that the App +read as null. Any difference or unmeasured case blocks App runs that publish to `db`, and 6c, until +it is explained or the permission is granted (gap 107). Once the comparison shows no unexplained +difference, it runs one normal manual run and one bootstrap run with `release: db`. 6c enables the schedule and stops tracking `github_activity.db`: it removes the file from the index, the `!/github_activity.db` exception from `.gitignore`, the `check-added-large-files` exclude, the `tracked-db-rollback-journal` and `tracked-db-no-sidecar` hooks, the D26 freeze check, and @@ -2253,7 +2352,9 @@ records the D20 WAL decision for the working file; and documents how to download and the staleness rule 5.6 gives consumers. Acceptance: two consecutive scheduled runs publish, and their manifests list drift, pending, and -failure counts; the bootstrap run publishes, and the next scheduled run starts from its snapshot; a +failure counts; the bootstrap run publishes, and the next scheduled run starts from its snapshot; a bootstrap or +catch-up run that reaches its deadline publishes with the workflow red (D38), and the pull request +records whether the bootstrap run reached it and how many runs it took to complete; a run makes requests more than 60 minutes after its first installation-token mint, and its log records at least two mints; against release `db-test`, the snapshot one migration behind head upgrades and publishes, the snapshot with an untracked repository publishes and the workflow is @@ -2409,7 +2510,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Whether a heartbeat commit counts as repository activity for schedule disabling (undocumented); a disabled schedule fails no run | Not provable inside Step 6; the heartbeat is best effort. A schedule that stops starts no run, so this repository cannot see it; 5.6 states the consumer's staleness rule, and the dashboard's integration builds it (section 9, gap 107) | | Issues read permission for the App | Step 6b's first manual run exercises it with a real token | | GraphQL `rateLimit` readings alternating between two windows (seen with a personal token; GitHub documents that values vary by the region handling a request, but not differing `resetAt` values) | Step 2's guard handles it; Step 6 records App-token readings | -| Whether body edits, base-ref changes, review dismissals, and issue label events always change `updatedAt` (not measured; a state change is caught by the index) | Step 5's REST sample measures each kind, issue label events from Step 4a; a kind not shown to change `updatedAt` gets an index-side check or a periodic re-hydration before Step 6a (gap 109) | +| Whether body edits, base-ref changes, review dismissals, and issue label events always change `updatedAt` (not measured; a state change is caught by the index) | Step 5's GraphQL timeline sample measures each kind, issue label events from Step 4a, counting only cases no later change follows; a kind not shown to change `updatedAt` gets an index-side check or a periodic re-hydration in pull request 5b, before Step 6a (gap 109) | | The error body of a GraphQL secondary limit returned as HTTP 200 (not observed; documented only as "an error message") | Step 2 classifies by type and message; a message it does not recognize is "not understood" (5.3 step 4) and stops the run with exit 1, naming the message; Step 5 records secondary-limit responses | | A PR file list above 3,000 entries (no tracked PR found) | Validation fails the item if `files` stops short of `totalCount`; Step 5 reports any | | Compare fallback on a PR whose head fork was deleted (only a live fork tested) | A count mismatch fails the item; Step 5 reports any | @@ -2425,7 +2526,7 @@ Measured 2026-10-16 unless noted. "Index" means the created-order enumeration in | Whether the scheduled App token can read every resolve field the token used here read (`diskUsage`, `webCommitSignoffRequired`, `hasVulnerabilityAlertsEnabled`, and `planFeatures` included) | Step 6b's App-and-token comparison, before any App run publishes to `db`; a field it cannot read is an error 5.3 step 4 classifies, and a difference blocks App publication (Step 6, gap 107) | | A `DisconnectedEvent`'s `source` and `subject` (none in the 300 PRs read for section 7, "Closing-link timeline events"; GitHub's schema gives it `ConnectedEvent`'s fields) | Step 3b-2b-2 tests a synthetic one; a live one is recorded when seen | | Whether a closing-keyword link can change without the PR's `updatedAt` changing (an issue transfer or deletion, a body edit, a base-ref or default-branch change); 5.4 then reports the difference as an error, unless it is a difference in the totals alone that the connected or disconnected events since the boundary cover (5.4) | Step 3b-2b-2's acceptance lists each such error with its PR and link; Step 5 reports any | -| Whether a pending `reviewRequests` entry for Copilot carries `BOT_kgDOCnlnWA` (the review-request events and the reviews do; no pending request was seen, section 7, "Copilot review requests" and "Step 3b-2a-2c acceptance") | Step 5 reports any Copilot request row; for a `databaseId` reported under a second node id, the writer keeps the stored node id and the run reports the second (5.2) | +| Whether a pending `reviewRequests` entry for Copilot carries `BOT_kgDOCnlnWA` (the review-request events and the reviews do; no pending request was seen, section 7, "Copilot review requests" and "Step 3b-2a-2c acceptance") | Step 5 reports any Copilot request row; for a `databaseId` reported under a second node id, the writer keeps the stored node id, the run reports the second, and `account_node_ids` stores it (5.2, gap 103) | | Whether to collect a `Mannequin`'s `email` ("The mannequin's email on the source instance"; the maintainer's profile decision named users and organizations only, and no mannequin was available to read, so whether GitHub requires a scope for it is unmeasured) | The maintainer decides; a live read on a mannequin comes first | | Whether GitHub returns an `EnterpriseUserAccount`'s `enterprise` to the tokens used (an `Enterprise` object; no tracked repository has such an actor) | Not selected until a live occurrence is read | | Whether GitHub returns an `EnterpriseTeam` requested reviewer's `id`, `fullDatabaseId`, and `slug` to the tokens used (the batch with them dry-runs at 2 points; no tracked repository has such a request) | Step 3b-1 tests a synthetic one; a live occurrence decides, as for `EnterpriseUserAccount.id`: an error at the alias's path fails the item (exit 2) | From ececdb74e64b46bda290b1688386d6e7adc62341 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 18:19:04 -0600 Subject: [PATCH 10/15] docs(plan): close the findings of the review of 12b2452 - D39 says "no flaw is seeded" (valid seed.py rows stay allowed), lists one set of allowed inputs everywhere it is restated, and widens Step 7's replacement to tests that write a planted file, revision, call, connection, or decoy; 5.7's planted import files are marked for that replacement. - The scenario model renders a PR above 250 commits as GitHub does (newest 250), and the commit-page rule is measured live on the acceptance repositories' large PRs before it is fixed. - A 2xx REST response that is not JSON or lacks a Date is a server error whatever its headers; only a readable one without rate-limit headers is not understood. - sync_runs gains link_lookups (the NOT_FOUND limit's denominator, without accounts an earlier lookup answered NOT_FOUND) and entries (each configured entry's github_id), which snapshot --after-run checks. - D25 and publish's condition exclude a run with `repo` from release db; the sync.yml test checks publish's and report's exact values and snapshot's --out and --run-id. - verify --db-only's digest and manifest refusals are tested on snapshots the code wrote; compare_reads maps row ids to natural keys; db-test runs go in a stated order. - Step 4b: a halt exits 2, an empty log is complete when the ascending read is empty, a commit-count mismatch or later-page 404 is a compare error, the refresh runs first, verify reads stored rows before its reads, and a compare entry's not-failed rule. - 5b covers a recurring index contradiction's retry delay; the report lists each contradiction and whether the retry cleared it. --- CLAUDE.md | 15 +-- CONTRIBUTING.md | 11 ++- docs/redesign-plan.md | 206 +++++++++++++++++++++++++++--------------- docs/testing.md | 21 +++-- 4 files changed, 160 insertions(+), 93 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index ffee6fd..54b2ae0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,12 +42,15 @@ uv run pytest # CI also runs Python 3.12, `alembic check`, ## Test Evidence -Nothing is planted, seeded, or tampered, and there is no mutation testing, under any name -(redesign plan D39): no test, review, or acceptance changes a copy of the code, a test, the plan, a -database, or a file to see whether something catches the change, or measures how many such -changes are found. Tests use recorded GitHub responses, stubs of what GitHub or the environment -can send, and databases and files the project's own code wrote, and assert exact results -([Testing](docs/testing.md)). +Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any +name (redesign plan D39): no test, review, or acceptance changes a file or a copy of one, or writes +a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches +it, and none measures how many such flaws are found. Test inputs are GitHub's recorded responses; +stubs of what GitHub or the environment can send, built in memory; calls and configuration the code +must refuse; valid rows (`tests/db/seed.py`); and databases and files the project's own code wrote. +Tests assert exact results; a check that only a defect of this code could fail is tested on its +passing side with the exact count it compared. Tests merged before D39 that plant are replaced in +Step 7 ([Testing](docs/testing.md)). ## Current Scope diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 99b7109..d12575b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -246,10 +246,13 @@ def test_reviewers(migrated_database: Path) -> None: # a copy of the alembic-bu pr = seed.pull_request(seed.repository("prebid/Prebid.js"), 13481, seed.user("author")) ``` -**3. Test on recorded GitHub responses, stubs of what GitHub can send, and files the project's -own code wrote, never on planted or tampered inputs** (redesign plan D39): a check on both sides of -its boundary with the exact result asserted, and a fix with a test that fails against the parent -commit's code ([Testing Guide](docs/testing.md#principles)). +**3. Never plant, tamper, or seed a flaw** (redesign plan D39). Test on GitHub's recorded +responses; stubs of what GitHub or the environment can send, built in memory; calls and +configuration the code must refuse; valid rows (`tests/db/seed.py`); and databases and files the +project's own code wrote, with the exact result asserted. A check GitHub, the environment, or a caller can fail is tested on +both sides of its boundary; one only a defect of this code could fail is tested on its passing side +with the exact count it compared; a fix comes with a test that fails against the parent commit's +code ([Testing Guide](docs/testing.md#principles)). ### Running Tests diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 8f898f5..32219f4 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -37,7 +37,7 @@ audited for silent failures and silently wrong data under D37, and D38 records t | 2 | GitHub source layer | done (PR #7) | | 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | | 4 | Issues, closing-issue links, and default-branch activity (pull requests 4a and 4b) | not started | -| 5 | Full bootstrap and verification (pull request 5, and 5b when Step 5 finds an edit kind that leaves `updatedAt` unchanged) | not started | +| 5 | Full bootstrap and verification (pull request 5, and 5b when Step 5 does not show that every measured edit kind changes `updatedAt`, or finds an index contradiction that recurs) | not started | | 6 | Scheduled sync and publication | not started | | 7 | Documentation rewrite and code audit | not started | @@ -79,7 +79,7 @@ Within scope the database collects and distinguishes data and never interprets i | D22 | The CI test gate allows no skipped tests (`scripts/check_junit.py --max-skipped 0`). A test that must skip in CI needs the allowance raised in the workflow, in the same change. pytest reports `xfail` as skipped, so the same applies to an `xfail`. | | D23 | Closing-issue links are read by the index as well as by hydration, and the stored links are compared with the index on every run, so a link added after merge, which does not change the PR's `updatedAt`, is captured: a PR whose indexed links differ from its stored links is planned for hydration (5.3 step 3). Hydration also reads which links GitHub lists as manually linked (`userLinkedOnly`), at no added cost (section 7). | | D24 | Conversation comments are stored one row per comment, its `github_id`, author, and times and never its text, so a comment by an account the database never stored is a row with a null author, never dropped (D28). Review bodies, inline review comment text, and commit messages are not stored either. | -| D25 | A run, scheduled or manual, publishes when its starting database is valid (the downloaded snapshot passed `verify --db-only`, or `bootstrap` started from an empty database), sync exited 0 or 2, and verify exited 0 (no failing class, 5.4). The manifest records the sync exit code and verify's counts per class. A final job that runs whatever the others did then marks the workflow failed if sync exited 2, verify failed, or nothing was published, so a failure is visible after the data is saved. | +| D25 | A run, scheduled or manual, publishes when its starting database is valid (the downloaded snapshot passed `verify --db-only`, or `bootstrap` started from an empty database), sync exited 0 or 2, verify exited 0 (no failing class, 5.4), and the run had no `repo` input or its release is not `db` (5.6). The manifest records the sync exit code and verify's counts per class. A final job that runs whatever the others did then marks the workflow failed if sync exited 2, verify failed, or nothing was published, so a failure is visible after the data is saved. | | D26 | The Step 5 database is not committed; it becomes the first snapshot of release `db` (Step 6b). From Step 3 until 6c removes it, the tracked `github_activity.db` keeps its pre-Step-3 content: the default `DATABASE_URL` moves to a git-ignored file, and `scripts/check_frozen_db.py` fails when the file's SHA-256 differs from the digest it pins, as a pre-commit hook on the file (so CI's `--all-files` run checks it) and as a test; the tests also fail when git stores the file other than as one regular file, and the `tracked-db-no-sidecar` hook refuses a committed `-journal` or `-wal` file beside it, which SQLite would apply on the next open, and the `-shm` index that goes with a `-wal`. | | D27 | A stored repository no longer listed in `tracking.toml` is reported, with exit 2, on every run without `--repo` (5.3 steps 1 and 8), and its rows stay until `ghactivity sync --prune-untracked` deletes them; the prune, on the operator's explicit flag, removes rows section 1 puts out of scope (repositories not in the tracked list) and is recorded in `sync_runs.deletions_approved` (5.1). | | D28 | The database keeps a person attached to everything it attributed to them. A stored account stays stored after a lookup returns `NOT_FOUND` for it, with its last known login and type, and every reference to it stays: when GitHub later returns that actor as null, as a re-hydration does once GitHub stops returning the account, the writer keeps the stored reference of the same object (the PR's author and merger, the issue's author, the review's reviewer or the comment's author with the same `github_id`, this PR's commit with the same SHA, and the default-branch commit with the same repository and SHA, Step 4b), and the row's `*_reported` column records that GitHub returned no actor in that hydration (5.1). Other actor references (editors, requested reviewers, event actors) take GitHub's current value, null included, and the assignees are GitHub's current `assignedActors` list. `not_found_at` (named `deleted_at` until Step 3b-1) records when a lookup returned `NOT_FOUND`; it changes no reference, the 24-hour refresh skips the account, and a later sighting of the account clears it. GitHub does not say why it returns a null actor (`Prebid.js#13769` is merged, with a null `mergedBy` and an existing author), so a null reference means only that GitHub returned no actor and the object had none stored, and an account never stored has no `github_users` row. A non-null actor GitHub returns replaces the stored one. On a commit, `linked_github_id` holds the account id GitHub linked in the latest hydration whether or not a lookup found the account, so a link whose lookup returned null is told apart from an email linked to no account. The attribution lives in the stored rows and cascades with them, so anything that re-creates a row from GitHub drops it for accounts GitHub no longer returns: a bootstrap (it starts from an empty file), a transferred issue (new `github_id`), and a repository pruned and re-added (D27). | @@ -90,10 +90,10 @@ Within scope the database collects and distinguishes data and never interprets i | D33 | `prebid-agent-skills` is never queried by an agent. Commands an agent runs (sync, verify, the REST sampler, the difference report, recordings, and `ghactivity github test`) name their repositories with `--repo` and leave it out; its rows enter the database through the scheduled App's workflow (Step 6) or a run the maintainer starts. Recordings come from public repositories only (Step 2). | | D34 | Decision of 2026-10-20 (the maintainer): wherever stored data must tell runs apart, a run is its `sync_runs` row id, stored with the data, never a clock reading and never state the writer or the engine keeps in memory, which neither a later run nor a reader of the file can see. `sync_failures.last_run_id` names the latest run in which the item failed, and `attempts` adds one only for a failure in another run, whatever the two runs' clocks read (5.1, gap 91). | | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | -| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, a repository error during a timeline lookup, and from Step 4b a log or history read verify cannot complete make a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and its report records its own points, REST calls, and time (5.4, gap 94). | +| D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, a repository error during a timeline lookup, and from Step 4b a log or history read verify cannot complete make a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and from Step 4b its stored events and default-branch commits in one statement each before the first request of its log and history reads, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | | D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class and one cause, the first line of the failure's message, of an item, an account's refresh, or from Step 4b a compare, so a permanent failure can be accepted without turning every run red while a new cause is not. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the floor or at the log's first event, as a read in ascending order finds it, fails a history read whose count still differs after a rescan, and records a compare that still fails, which an entry can accept. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now counted per run and in the manifest, and from Step 6a a run whose share of link lookups answered `NOT_FOUND` exceeds the one Step 5 sets exits 2, since a lookup form GitHub stopped resolving would look the same per link), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (`items_deleted`, and from Step 4b `events_deleted` and `history_commits_deleted`; the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | -| D39 | Decision of 2026-10-21 (the maintainer): nothing is planted, seeded, or tampered, and there is no mutation testing, under any name. No test, review, or acceptance changes a copy of the code, a test, the plan, review material, a database, a snapshot, or a manifest to see whether something catches the change, and none measures how many such changes a test, a check, or a reviewer finds. A test's inputs are GitHub's recorded responses, stubs of what GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`), calls and configuration the code under test must refuse, valid rows a test writes through the models (`tests/db/seed.py`), and databases and files this project's own code wrote, at any revision; each test asserts the exact result. A check that only a defect of this code could fail is tested on its passing side, with the exact count of the rows or values it compared, so it cannot pass on nothing. A regression test that fails against the parent commit's code remains evidence. Tests merged before D39 that change a copy of a database or a file to see whether a check catches it are replaced in Step 7. | +| D39 | Decision of 2026-10-21 (the maintainer): nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes code, a test, the plan, review material, a recording, a database, a snapshot, a manifest, or any other file, or a copy of one, or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it; none measures how many such flaws a test, a check, or a reviewer finds. A test's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed into that shape; calls and configuration the code under test must refuse; valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or source that only a defect of this code could fail is tested on its passing side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A regression test that fails against the parent commit's code remains evidence. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 7. | ## 3. Coverage rule @@ -340,7 +340,7 @@ and `color`, `pr_user_tags.created_at`). | `issue_labels` | `issue_id`, `name` | PK (`issue_id`, `name`) | | (Step 4a) | | The two issue rows above are the Step 3 plan. Step 4a completes them under D29 the way Step 3b-1 completes the PR rows: `issues.author_reported` (D28), the label id on `issue_labels` (D8), the issue's other free fields (`author_association`, `last_edited_at`, `published_at`, milestone, issue type, the parent and the `duplicateOf` issue by repository id and number), `issue_comments` and `issue_assignees` tables shaped like `pr_comments` and `pr_assignees` with their totals, and their 5.4 child-count checks | | `repository_activities` | `repository_id`, `github_id`, `node_id`, `activity_type`, `ref`, `before_sha`, `after_sha`, `occurred_at`, `actor_github_id`, `actor_id` | Step 4b (D30); `github_id` unique (GitHub's event `id`); `activity_type` is GitHub's value as text (`push`, `force_push`, `branch_creation`, `branch_deletion`, `pr_merge`, `merge_queue_merge` in githubkit 0.14.2's model); `occurred_at` is GitHub's `timestamp`; `actor_github_id` is the REST actor's `id` and `actor_id` its `github_users` row once a lookup resolves it (5.2), both null when GitHub returns no actor | -| `repository_activity_compares` | `activity_id`, `http_status`, `status`, `ahead_by`, `behind_by`, `total_commits`, `merge_base_sha`, `files_at_cap`, `error` | Step 4b; PK `activity_id`; `files_at_cap` is true when compare's first page, the only page GitHub lists files on, returned 300 files, the cap GitHub documents for one comparison, so its file rows may be partial (gap 106); `error` is the class and message of a compare that still failed after its retries or whose 2xx first page had no `files` list (Step 4b), with the other columns null, and null once GitHub answers it; GitHub's REST compare of `before_sha...after_sha` as it returns them, so the files and counts run from the merge base, not from `before_sha`, when the push diverged (section 7, "Compare"); no row when either SHA is all zeros (a branch creation or deletion); `http_status` records the response, so a refused compare is a row with the other columns null (a compare across a force push, `5110f3d...fccb1f5`, returned 200, section 7) | +| `repository_activity_compares` | `activity_id`, `http_status`, `status`, `ahead_by`, `behind_by`, `total_commits`, `merge_base_sha`, `files_at_cap`, `error` | Step 4b; PK `activity_id`; `files_at_cap` is true when compare's first page, the only page GitHub lists files on, returned 300 files, the cap GitHub documents for one comparison, so its file rows may be partial (gap 106); `error` is the class and message of a compare that still failed after its retries, whose 2xx first page had no `files` list, whose fetched commits differ from `total_commits`, or whose later page answered 404 or 422 (Step 4b), with the other columns null, and null once GitHub answers it; GitHub's REST compare of `before_sha...after_sha` as it returns them, so the files and counts run from the merge base, not from `before_sha`, when the push diverged (section 7, "Compare"); no row when either SHA is all zeros (a branch creation or deletion); `http_status` records the response, so a refused compare is a row with the other columns null (a compare across a force push, `5110f3d...fccb1f5`, returned 200, section 7) | | `repository_activity_compare_files` | `activity_id`, `ordinal`, `filename`, `previous_filename`, `status`, `additions`, `deletions` | Step 4b; PK (`activity_id`, `ordinal`); compare's `files` entries as returned | | `repository_activity_compare_commits` | `activity_id`, `ordinal`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at` | Step 4b; PK (`activity_id`, `ordinal`); compare's `commits` list in full, each commit stored like a `pr_commits` row (D19), so a commit an event brought joins to its event; written once with its compare and re-read only when a read version asks for it (5.1, column rules), so its author link is the one GitHub returned then | | `default_branch_commits` | `repository_id`, `sha`, `author_id`, `linked_github_id`, `author_name`, `author_email`, `authored_at`, `committed_at`, `parent_count`, `additions`, `deletions`, `changed_files` | Step 4b; PK (`repository_id`, `sha`); the default branch's history since the floor as GitHub lists it (D30); the author link as in `pr_commits`, a replace keeping the stored link of the row with the same SHA when GitHub returns none (D28); `parent_count` is GitHub's `parents.totalCount`; `changed_files` is GitHub's `changedFilesIfAvailable`, null when GitHub returns none; no message (D19) | @@ -348,7 +348,7 @@ and `color`, `pr_user_tags.created_at`). | `account_profiles` | `account_id`, `company`, `location`, `website_url`, and an organization's `description`, `email`, `twitter_username`, `is_verified`, `archived_at`, `has_sponsors_listing`, and `pinned_items_remaining`, `refreshed_at` | PK `account_id`, a reference to `github_users` (`ON DELETE CASCADE`); rows only for a `User` or an `Organization` (D31); each lookup that returns the account replaces the whole row with the profile it returned, through the writer and in the transaction that writes `github_users`' login and type, so no earlier value survives a field GitHub now returns as null or a change of type; a lookup that returns `NOT_FOUND` leaves the row unchanged, marked by `github_users.not_found_at` (D28); every field nullable as GitHub's is, the organization fields null for a user and `company` null for an organization; `refreshed_at` as on `github_users` | | `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting, a lookup, or a commit link, the one `github_users.node_id` holds included, with the first and latest runs that reported it (`first_run_id` and `last_run_id`, references to `sync_runs`), so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); its migration refuses a file whose `github_users` holds rows, since no file was published before Step 6b (gap 110); consumer | | `automation_emails` | `email`, `reason`, `run_id` | integer primary key; `email` unique, stored lowercase ASCII (SQLite's `lower()`, which the views apply to commit emails, folds only ASCII letters); replaced the same way | -| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `links_not_found`, `repo_filter`, `deletions_approved`, and from Step 4b `events_deleted` and `history_commits_deleted` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, `manual` when it is unset, and `unknown` for any other value (gap 105); `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes to tracked files, untracked files ignored, and `-unknown` when it fails or times out, gap 105; the package runs from a checkout only when its directory or a parent holds a `.git` entry, and then `git ls-files --error-unmatch` exiting 1 means the package is not tracked, while any other failure of a git call, exit 128 and a missing `git` included, records the package version, or the commit when `rev-parse` answered, followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted, and from Step 4b `events_deleted` and `history_commits_deleted` the activity events and default-branch commits Step 4b deleted (nullable, null for a run before Step 4b, each written in its deletion's transaction); `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `links_not_found` the commit-link accounts a lookup answered `NOT_FOUND` this run, gap 90's lookups included (5.2), and `repo_filter` a JSON array of the `--repo` values as given, sorted, null for a run without `--repo`; `refresh_failures`, like `failures`, counts accepted ones; the migration that adds these three refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | +| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `link_lookups`, `links_not_found`, `repo_filter`, `entries`, `deletions_approved`, and from Step 4b `events_deleted` and `history_commits_deleted` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, `manual` when it is unset, and `unknown` for any other value (gap 105); `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes to tracked files, untracked files ignored, and `-unknown` when it fails or times out, gap 105; the package runs from a checkout only when its directory or a parent holds a `.git` entry, and then `git ls-files --error-unmatch` exiting 1 means the package is not tracked, while any other failure of a git call, exit 128 and a missing `git` included, records the package version, or the commit when `rev-parse` answered, followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted, and from Step 4b `events_deleted` and `history_commits_deleted` the activity events and default-branch commits Step 4b deleted (nullable, null for a run before Step 4b, each written in its deletion's transaction); `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `link_lookups` the commit-link accounts this run looked up that no earlier lookup answered `NOT_FOUND` (neither an account stored with `not_found_at`, so none of gap 90's lookups, nor one a stored commit row links by `linked_github_id` without an account), and `links_not_found` those of them a lookup answered `NOT_FOUND` (5.2), `repo_filter` a JSON array of the `--repo` values as given, sorted, null for a run without `--repo`, and `entries` a JSON array, in `tracking.toml` order, of each configured entry's name and the `github_id` 5.3 step 1 resolved or matched it to, null when none, written when the run closes; `refresh_failures`, like `failures`, counts accepted ones; the migration that adds these five refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | | `repo_sync_state` | `repository_id`, `last_index_started_at`, `index_pr_total`, `index_issue_total`, `last_success_at`, `last_run_id`, `last_outcome`, `halt_reason`, and from Step 4b `activity_read_at`, `activity_first_event_at`, `activity_last_outcome`, `activity_halt_reason`, `history_read_at`, and `default_branch_commits_total` | PK `repository_id`; `activity_first_event_at` is the `timestamp` of the oldest event the latest complete activity read listed, so a consumer can tell a quiet period from a log that does not reach back (gap 106); `activity_read_at` is the GitHub `Date` header of the first page of the latest activity-log read that completed and was written; `history_read_at` and `default_branch_commits_total` are the first page's `Date` header and the `totalCount` of the latest history read that replaced the rows, written only with that replace; `activity_last_outcome` in (`complete`, `time_limited`, `failed`, `halted`); `activity_halt_reason` holds a Step 4b deletion halt's reason and candidate counts and is cleared by a run whose Step 4b did not halt (Step 4b); `last_run_id` is the latest run that attempted the repository, or found it untracked, and `last_outcome` its result, in (`complete`, `time_limited`, `failed`, `incomplete_index`, `halted`, `untracked`), both written by step 8, or by the close of a fatal or interrupted run (5.3 step 8); `last_index_started_at` is written only when that index completed and the repository did not fail later in the run, a time-limited repository included (its unreached items are in `sync_pending`), and never by the close of a fatal or interrupted run; `last_success_at` is the run's `finished_at` when `last_outcome` is `complete` and no item of the repository failed outside `[sync].accepted_failures`; `halt_reason` holds step 7's halt and is cleared by a run whose step 7 ran for the repository and did not halt, a run whose step 7 did not run for it leaving it | | `sync_pending` | `repository_id`, `item_kind`, `number`, `run_id` | PK (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); items a time-limited run planned but did not reach; step 8 replaces the rows of each completely indexed repository that did not fail later in the run (a repository gone during hydration keeps its rows, 5.3 step 4) with this run's unreached items, so a run that reached them all leaves none; consumer, so an in-coverage item not yet hydrated is told apart from one outside coverage | | `sync_failures` | `repository_id`, `item_kind`, `number`, `attempts`, `last_error`, `first_failed_at`, `last_failed_at`, `last_run_id` | unique (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); `attempts` counts the runs in which the item failed: `last_run_id` is the latest such run, and a failure adds one only when its run is another, whatever the runs' clocks read; `last_error` is the error's class and message; row deleted on success, or by step 7 when its repository's in-coverage set no longer lists the item | @@ -505,8 +505,8 @@ GitHub is consumer. The columns that say how current a row is are consumer time of everything the index wrote), `sync_failures`' repository, kind, number, error, and failure times, `sync_pending` (an in-coverage item a time-limited run did not reach), and every `sync_runs` column except the run counters (`graphql_points`, `rest_calls`, `items_*`, -`failures`, `refresh_failures`, `links_not_found`, and from Step 4b `events_deleted` and -`history_commits_deleted`), which with the tag tables are internal. Integer +`failures`, `refresh_failures`, `link_lookups`, `links_not_found`, and from Step 4b `events_deleted` +and `history_commits_deleted`), which with the tag tables are internal. Integer row ids change on a rebuild, but a pull request's labels, reviews, comments, commits, files, and closing links, and its author and merger, are reachable only through them, so the foreign-key columns of consumer tables and the row ids they reference are consumer, valid within one file; other @@ -554,9 +554,11 @@ same number. `account_node_ids` included. A commit link resolves by its `databaseId`; the node id it carries (GraphQL `GitActor.user.id`, REST compare `author.node_id`) is stored in `account_node_ids` and checked as a sighting's is (gap 103). A commit-link lookup that answers `NOT_FOUND` is counted in - `sync_runs.links_not_found` and listed in the report (D28 stores the commit as before); from - Step 6a, a run in which more of its commit-link lookups answer `NOT_FOUND` than the share Step 5 - sets exits 2 (5.3 step 8), since a lookup form GitHub stopped resolving looks the same per link. + `sync_runs.links_not_found`, beside `link_lookups`, when no earlier lookup answered `NOT_FOUND` + for it, and listed in the report (D28 stores the commit as before); from Step 6a, a run whose + `links_not_found` exceeds the share of its `link_lookups` Step 5 sets, with at least the minimum + number of lookups Step 5 sets, exits 2 (5.3 step 8), since a lookup form GitHub stopped + resolving looks the same per link. - Type and login come only from GraphQL: actor sightings (PR author, merger, review author, comment author, editor, requested reviewer, assignee), which carry `__typename`, and `nodes(ids:)` lookups. A commit-linked account (GraphQL `GitActor.user` or the REST compare @@ -587,7 +589,7 @@ same number. is the GitHub `Date` header of the response that inserted the row or of its latest lookup that returned the account, the same clock as `hydrated_at`; a `NOT_FOUND` lookup sets `not_found_at` from its own `Date` header and leaves `refreshed_at`. The refresh runs after the last hydration - write and before step 7, oldest `refreshed_at` first, and stops at the deadline; in it, a + write and before Step 4b's readers and step 7, oldest `refreshed_at` first, and stops at the deadline; in it, a `databaseId` that differs from the stored `github_id` is a failed refresh, which makes the exit 2 unless an account entry in `[sync].accepted_failures` accepts it (5.3 step 4). Accounts that no stored row references any more stay stored and refreshed: within one file, `github_users` holds @@ -705,7 +707,7 @@ an interrupt, which is re-raised. A failure the engine detects itself (these two, a hydrated `github_id` other than the index's, a review or comment id stored under another PR, a per-batch lookup's `databaseId` other than the commit link's) is recorded at once, without the solo retry, which would repeat a deterministic - result. Step 5's three contradictions of the run's own index (an alias `NOT_FOUND`, an item + result. The three contradictions of the run's own index that step 5 checks (an alias `NOT_FOUND`, an item outside coverage, an earlier `updatedAt`) are the exception: a lagging read can clear, so the engine requests the item once more alone before it records the failure. The planner is a pure function over snapshots of the stored rows, which `--dry-run` reuses. The @@ -732,7 +734,7 @@ an interrupt, which is re-raised. | Item error | another alias error; partial alias data; a value the typed models reject; a failed check in step 5; HTTP 404/422 on a compare page | Retry the item once alone, then record it in `sync_failures` and write nothing for it | | Repository gone during hydration | a batch's `repository` is null (`RepositoryNotFound`, `RepositoryShapeError`; gap 79) | The repository fails for the rest of the run: its written items stay, step 7 skips it, and its `sync_pending` rows are unchanged | | Local database error | any error the database raises while the writer writes (a lock held past the busy timeout, an unexpected constraint failure; gap 77) | Fatal: roll back the item, close the run, exit 1 | - | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; a 2xx REST response without readable `x-ratelimit-remaining` and `x-ratelimit-reset` (gap 104; other statuses take the other rows of this table whatever their headers); another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | + | Not understood | a GraphQL error without a path, or no data, that matches none of the classes above (a rejected document, an unrecognized limit message); a missing or malformed `rateLimit`; a 2xx REST response whose body is JSON and whose `Date` is readable but which lacks readable `x-ratelimit-remaining` and `x-ratelimit-reset` (gap 104; a 2xx whose body is not JSON or whose `Date` is unreadable is a server error, and other statuses take the other rows of this table, whatever their headers); another status (400, a GraphQL 404 or 422) | Fatal: close the run, exit 1 | | Unauthorized or forbidden | HTTP 401; a failed credential request (an App token mint answering 4xx without a rate-limit signal); HTTP 403 with no rate-limit signal (a missing permission); an HTTP client failure outside the network (a redirect loop, a proxy or URL problem, a missing auth dependency) | Fatal: close the run, exit 1 | Typed response models check types strictly: GitHub's JSON has exact types, and lax coercion @@ -787,7 +789,8 @@ an interrupt, which is re-raised. an item outside coverage, an `updatedAt` earlier than the index entry's) is requested once more alone by the engine, which holds the index, and a second contradiction is an item failure of class `IndexContradiction`, whose cause names which of the three it is, recorded before any account - lookup for it (gap 100); a deadline that refuses that request + lookup for it (gap 100), and the run's report lists each contradiction with its kind and + whether the retry cleared it; a deadline that refuses that request leaves the item in `sync_pending`, and the batch's other checked items are written as at any deadline. Above 250 commits GraphQL's commit connection holds the newest 250 (section 7, "Commit cap"), so its first page is compare's 100 commits from position `total_commits - 250`, as the @@ -847,18 +850,21 @@ an interrupt, which is re-raised. stored, so it makes no exit 2. 8. Record: update `repo_sync_state` and close the `sync_runs` row. Exit 0 when every repository indexed completely and nothing failed; 2 when an item failed in this run (other than one listed - in `[sync].accepted_failures`), a repository failed, had an incomplete index, or halted, a stored + in `[sync].accepted_failures`), a repository failed, had an incomplete index, or halted, a repository's Step 4b activity read failed + or halted, a stored repository is untracked in a run without `--repo`, a prune was refused (step 7), a refresh failed (step 4, other than one an account entry accepts), the run reached its deadline (status `time_limited`, whatever it stopped: hydration, the refresh, or a Step 4b read; gap 99), or, from - Step 6a, more of its commit-link lookups answered `NOT_FOUND` than the share Step 5 sets (5.2); 1 - on a fatal error. The exit code is decided after the prune. An accepted failure is still recorded and + Step 6a, its `links_not_found` exceeded the share of its `link_lookups` Step 5 sets, with at least + the minimum number of lookups Step 5 sets (5.2); 1 on a fatal error. The exit code is decided after the prune. An accepted failure is still recorded and reported; it names the repository and number, the account, or a compare, with the error class, the cause, and a reason in `tracking.toml`, so a permanent failure does not turn every run red, and a failure of another class or cause is not accepted (5.5, gap 101). An item entry is reported as not having failed when its repository's outcome in this run is `complete`, `time_limited`, or `halted` and its item neither failed in this run nor is pending (5.5); an - account entry, only when this run's refresh looked the account up and the lookup succeeded. + account entry, only when this run's refresh looked the account up and the lookup succeeded; a + compare entry, only when this run requested that compare and GitHub answered it with a 2xx first + page holding `files` and every commit (Step 4b). `--max-minutes` stops queueing new hydration, records in `sync_pending` the planned items it did not reach (a repository that failed during hydration keeps its rows, step 4), still runs step 7 for complete repositories and step 8, and sets status `time_limited`; unreached items are not @@ -1058,10 +1064,10 @@ typed model's rejection text) on the lines after it, so an entry keeps matching while a failure with another cause is not accepted; several classes carry more than one cause. The loader refuses an entry without `message`, and a class whose name is not in the list of recorded failure classes `tracking.py` keeps, which imports none of the modules that define them; a test -holds that list equal to the concrete subclasses of the failure bases the engine records -(`ItemError`, `PlannerFailure`, `WriteRefused`, and the engine's own `HydratedIdMismatch` and -`IndexContradiction`), `UserLookupFailed` and `LookupConflict` for an account, and the classes -Step 4b's compare reader records. Entries are unique by item, account, or compare, an item or +holds that one list, which serves every entry form, equal to the concrete subclasses of +`ItemError` and `PlannerFailure`; `ItemConflict`, `AccountMissing`, `LookupConflict`, and +`UserLookupFailed` (an item whose commit-link lookup failed, or a refresh); the engine's +`HydratedIdMismatch` and `IndexContradiction`; and the classes Step 4b's compare reader records. Entries are unique by item, account, or compare, an item or compare entry matches by its configured repository, so a rename does not unhook it, and a new cause is not accepted (gap 101); an entry whose item, account, or compare did not fail is reported (5.3 step 8). @@ -1119,8 +1125,8 @@ Workflow `.github/workflows/sync.yml`: `workflow_dispatch` with inputs `bootstrap`, `allow_deletions` and `prune_untracked` (passed to `ghactivity sync`, 5.3 step 7), `repo` (repositories separated by spaces, each validated as a configured repository and passed as `--repo` to sync and verify; it requires `bootstrap`, the - manifest records it, and a run with it never publishes to release `db`: Step 6b's comparison is - its use), and `release` (default `db`; the Step 6 acceptance runs against `db-test`). A `schedule` run has no inputs, so every step reads `${{ inputs.release || 'db' }}` and + manifest records it, and a run with it never publishes to release `db`, since `publish`'s `if:` requires it to be + empty for `db` (D25): Step 6b's comparison is its use), and `release` (default `db`; the Step 6 acceptance runs against `db-test`). A `schedule` run has no inputs, so every step reads `${{ inputs.release || 'db' }}` and treats the other inputs as unset. - `concurrency: { group: db-sync, cancel-in-progress: false }`. A new run replaces a pending one and never cancels a running one. @@ -1135,7 +1141,8 @@ Workflow `.github/workflows/sync.yml`: `verify --db-only` printed (N; 0 for a bootstrap) and fails unless the file's SHA-256 equals the one it printed (gap 107). It then runs `alembic upgrade head`, asserts the revision is head, runs `ghactivity sync --max-minutes 300` and `ghactivity verify --max-minutes 40 --report - verify.json`, runs `ghactivity snapshot --after-run N --sync-exit C --verify-report verify.json` + verify.json`, runs `ghactivity snapshot --out DIR --run-id ${{ github.run_id }} --after-run N + --sync-exit C --verify-report verify.json` with C the sync's exit code, and hands the snapshot and manifest to `publish` as a workflow artifact, with the sync and verify exit codes as job outputs. The job fails when `verify --db-only` halts, the file's SHA-256 differs, sync exits 1, or `snapshot` refuses. @@ -1176,7 +1183,7 @@ Workflow `.github/workflows/sync.yml`: (`sync_runs.config_sha256`), migration head, `schema_version`, coverage floor, `bootstrap`, per-repository PR and issue counts, per kind the number of rows whose `hydration_version` differs from the code's, the sync exit code, the sync - run's `repo_filter` and `links_not_found` and from Step 4b its `events_deleted` and + run's `repo_filter`, `link_lookups`, and `links_not_found` and from Step 4b its `events_deleted` and `history_commits_deleted`, verify's exit code, counts per class, unverified repositories, and from Step 4b the compares at the file cap, those refused, and those that still failed (from `--verify-report`; null without it), and the SHA-256. `snapshot` exits 1 when the report `--verify-report` names is missing or @@ -1185,8 +1192,8 @@ Workflow `.github/workflows/sync.yml`: workflow, `snapshot --after-run N --sync-exit C` takes the highest `sync_runs` id the job recorded before the sync (0 for a bootstrap) and the sync's exit code, and exits 1 unless exactly one `sync_runs` row is newer than N, that row is closed with exit code C, and it is the - `last_run_id` of the stored row of every entry in the `tracking.toml` that run read (matched as - 5.3 step 1 matches; an entry that never resolved has no row), so a job whose sync did not run + `last_run_id` of the stored row of every `github_id` that run's `sync_runs.entries` records (an + entry that never resolved records none, 5.1), so a job whose sync did not run cannot republish the old file (gap 107). For 6b's first snapshot, `--sync-run ID` names Step 5's last run instead, which must exist and be closed with exit 0 or 2, and supplies the manifest's sync exit code. The manifest records that run as `sync_run_id` and `sync_finished_at`, and @@ -1237,7 +1244,8 @@ enabled afterwards. fails when another module does. It scans `src/` outside `source/`, `scripts/`, and `alembic/` for imports at any scope, `TYPE_CHECKING` blocks included, and for `importlib.import_module` and `__import__` called with a string literal, and fails when it scans no file; planted files test each -spelling, and a string decoy is not flagged. Tests are exempt, since they build githubkit's errors. A +spelling, and a string decoy is not flagged (a test from before D39, which Step 7 replaces with one +on its passing side that asserts the exact count of files and imports it scanned). Tests are exempt, since they build githubkit's errors. A second test fails when `source/` imports SQLAlchemy or `db/`, so fetchers cannot touch the database. | Disposition | Modules | @@ -1454,8 +1462,8 @@ nested node. Each step is one pull request from a `feature/` branch, except Step 3 (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2), Step 4 (pull requests 4a and 4b), -Step 5 (pull request 5, and 5b when its measurement finds an edit kind that leaves `updatedAt` -unchanged), and Step 6 (pull requests 6a and 6c, with the 6b manual runs between them). A step, or a pull +Step 5 (pull request 5, and 5b when its measurements call for the 5.3 step 3 rule or the retry +delay Step 5 names), and Step 6 (pull requests 6a and 6c, with the 6b manual runs between them). A step, or a pull request within one, starts after the previous one merges. ### Step 1: Guardrails @@ -1575,8 +1583,9 @@ every boundary the design states (the coverage floor, the 10-minute unsettled wi past `deadline_at` for the lock, the 24-hour refresh, the delete guard's max(25, 2%)), taken from GitHub's recorded responses, stubs of what GitHub or the environment can send, and databases this project's code wrote, with the exact result asserted, and regression tests that fail against the -parent's code. No test changes a copy of the code, a test, a database, or a file to see whether a -check catches it, and there is no mutation testing, under any name. Each wait on a thread or +parent's code. No test changes a file or a copy of one, or writes a file, a row, a revision, a call +site, or a decoy holding a flaw, to see whether a check catches it, and there is no mutation +testing, under any name. Each wait on a thread or subprocess has an explicit timeout. #### Step 3a: New schema and cutover @@ -1930,7 +1939,7 @@ check without a test that targets it; CI passes on the exact PR head. Prerequisite: Step 3b-2a-2 has merged. Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh_failures`, -`links_not_found`, and `repo_filter`, `unknown` to the `trigger` CHECK, and the `account_node_ids` +`link_lookups`, `links_not_found`, `repo_filter`, and `entries`, `unknown` to the `trigger` CHECK, and the `account_node_ids` table, and refuses a file whose `sync_runs` or `github_users` holds rows (5.1, gap 110; a development file is rebuilt by a sync), with their data dictionary entries and the current schema-version lock entry's consumer pairs: @@ -1949,7 +1958,8 @@ schema-version lock entry's consumer pairs: - a PR whose link `totalCount` exceeds the 10 links the index carries is hydrated every run (5.3 step 3, gap 102); - every node id a sighting, a lookup, or a commit link reports for a stored account is stored in - `account_node_ids`, and a commit-link lookup answered `NOT_FOUND` is counted (5.2, gap 103); + `account_node_ids`, and a commit-link lookup answered `NOT_FOUND` is counted in `links_not_found` + beside `link_lookups` (5.2, gap 103); - the source checks of gap 104: a 2xx REST response without readable rate-limit headers is not understood, a GraphQL `errors` whose value is not a list, `null` included, is a server error, a compare above 250 commits holds the commit connection's first page at positions @@ -1959,6 +1969,12 @@ schema-version lock entry's consumer pairs: documents` and the first-sync directory with `--only first-sync`, the recording splitter and the replay keep the repository's `databaseId` beside `pullRequests` in an index unit, and the tests that assert recorded values follow the new recordings); +- before the commit-page rule is fixed, the GraphQL first page and the compare of every in-coverage + PR above 250 commits in the three Step 3b-2a-2c acceptance repositories are read live and the + rule checked on each; section 7 records how many hold merge commits, whether each page is + compare's positions `total_commits - 250` to `total_commits - 151` in order, and the points and + REST calls spent, and if any is not, the rule is chosen from that measurement before the step + merges; - the run-record and setup checks of gap 105: `code_version` as 5.1 defines it, with `_git` returning the exit status; a close given an outcome it cannot write raises before writing; `[automation]` is required; the connect listener reads `foreign_keys` back and refuses a @@ -1972,7 +1988,16 @@ schema-version lock entry's consumer pairs: - `docs/development.md`'s list of sync's exit-2 conditions gains the deadline, a failed refresh, and an index contradiction; - the fixtures these rules break: the scenario model renders `x-ratelimit-*` headers on REST units - and the repository `databaseId` on index pages; every test configuration gains `[automation]`; + and the repository `databaseId` on index pages, and renders a pull request it made above 250 + commits as GitHub does (section 7, "Commit cap"): its `commits` connection keeps `totalCount` + and pages over the newest 250, so its first page is compare's positions `total_commits - 250` to + `total_commits - 151`, while a recorded pull request keeps its recorded page; `docs/testing.md` + and the model's refusal of a push to Prebid.js #8858 (`tests/source/scenario.py`, + `tests/source/test_scenario.py`) say that its GraphQL page is compare's positions 4-103, in + order, recorded separately, so a push cannot keep both; `tests/sync/expected_state.py` computes + `account_node_ids` from the model's history, adds `refresh_failures`, `link_lookups`, and + `links_not_found` to its run counters, and computes each run row's `repo_filter` and `entries`; + every test configuration gains `[automation]`; every `accepted_failures` entry in the tests and in the packaged file's comment gains `error` and `message`; `tests/test_cli_sync_report.py` follows the account and message forms. @@ -1994,11 +2019,14 @@ the recorded-class list equal to the classes the engine records; PRs with 10 and second node id from a sighting, a lookup, and a REST compare link stored and seen again in a later run, keeping `first_run_id` and advancing `last_run_id`, and a node id in `account_node_ids` reported under another `databaseId` failing the item; a commit-link lookup answered `NOT_FOUND` -counted; each source check accepting the shapes GitHub's recordings hold and refusing each shape +counted in `link_lookups` and `links_not_found`, and one for an account an earlier run's lookup +answered `NOT_FOUND` counted in neither; the report listing each index contradiction with its kind +and whether the retry cleared it; `entries` recording each configured entry's `github_id`, a +renamed repository's included, and null for one that never resolved; each source check accepting the shapes GitHub's recordings hold and refusing each shape its rule names (a 2xx REST response without rate-limit headers; an `errors` of `{}`, `""`, and `null`; an index page whose repository `databaseId` differs; a commit page from other positions), -the commit page at positions 4-103 of the recording of Prebid.js #8858 accepted, and a 502 -without rate-limit headers still retried; `code_version` outside any `.git` (the package +the commit page at positions 4-103 of the recording of Prebid.js #8858 accepted, and a 502, a 200 +whose body is not JSON, and a 200 without `Date`, none with rate-limit headers, still retried; `code_version` outside any `.git` (the package version), in a checkout with the package untracked (the package version), in a clean checkout, one holding only an untracked file, and a dirty one (the commit, the commit, and `-dirty`), and with `git status`, `ls-files`, and `rev-parse` failing or timing out inside a checkout @@ -2014,8 +2042,8 @@ a `--repo` run, sorted, with each value as given, and null without one; a file w its accepting side passes on both. The tests that pinned the old behaviour change with the rule they pinned: in `tests/sync/test_engine.py` every test that asserts a `time_limited` run exits 0, the deadline inside a batch among them, whose failures are now recorded, and the failed-refresh, -gone-item, leaving-coverage, and 11-link tests; in `tests/sync/test_runs.py` the skipped-close and -`code_version` tests; and in `tests/test_tracking_config.py` the defaults without `[automation]` +gone-item, leaving-coverage, and 11-link tests; in `tests/sync/test_runs.py` the skipped-close, +`trigger`, and `code_version` tests; and in `tests/test_tracking_config.py` the defaults without `[automation]` and the accepted-failure entries. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no rule of @@ -2052,7 +2080,9 @@ before its first REST request (D33); the sampler takes its repositories with `-- `prebid-agent-skills` before any request (D33); run from `scripts/`, reaching GitHub through `GitHubSource.transport`; Steps 3b-2b-2, 4, and 5 reuse it. -Tests: the offline tests the scope lists; a list read across pages, one of them ending on a full +Tests: the offline tests the scope lists; a stub REST body with the recorded `updated_at` and one +compared value GitHub changes without moving `updatedAt` (`author_association`, a label name) +reported as a mismatch with its PR, field, stored value, and REST value; a list read across pages, one of them ending on a full page followed by an empty one; the recorder mode sending no request for a repository `PRIVATE` lists and refusing a private repository before any REST request; a re-recording in the default mode and with `--only documents` leaving the mode's directory, REST and GraphQL units, as it is; and the @@ -2177,14 +2207,16 @@ Scope (D30): the repository activity reader, its compares, and the default-branc `default_branch_commits` tables (5.1) and the `sync_runs` counters `events_deleted` and `history_commits_deleted` in a migration, with a stored `totalCount` for the history (`repo_sync_state.default_branch_commits_total`), the as-of times of the log and history reads -(`activity_read_at`, `history_read_at`), and the activity read's outcome and halt reason +(`activity_read_at`, `history_read_at`), the log's oldest listed event (`activity_first_event_at`), +and the activity read's outcome and halt reason (`activity_last_outcome`: `complete`, `time_limited`, `failed`, or `halted`; `activity_halt_reason`; 5.1); the compare form of `accepted_failures` (5.5); the planner and writer steps, which run after -5.3 step 6's writes and before step 7. Each run reads the activity log since the floor in full, +5.3 step 6's writes and the refresh (5.2) and before step 7. Each run reads the activity log since the floor in full, following GitHub's `Link` cursors, which the source then returns with a REST page; a read is complete only when its oldest listed event is before the floor or is the first event a read of the log in ascending order returns, made in the same run (one REST call), so a listing that ends early -without a next link is not taken for the log's start; a deadline gives `time_limited` (exit 2, 5.3 +without a next link is not taken for the log's start, and a read that lists no event is complete +when the ascending read also returns none, `activity_first_event_at` staying null; a deadline gives `time_limited` (exit 2, 5.3 step 8), and any other end fails the repository's activity for the run (exit 2, gap 106). A complete read upserts the events from the floor on by `github_id` and records in `activity_first_event_at` the `timestamp` of the oldest event it listed, one before the floor @@ -2196,9 +2228,10 @@ event whose `before` and `after` are both non-zero and that has no compare row, files of its first page, the only page GitHub lists files on, with `files_at_cap` set when that page returned 300 files, GitHub's documented cap for one comparison (gap 106). Each compare is written in its own transaction when it completes, so a deadline keeps the compares already -written. A compare GitHub refuses (HTTP 404 or 422) is stored with its `http_status`; one that -still fails under 5.3 step 4's transient classes, or whose 2xx first page has no `files` list, is -stored with its `error` and makes the repository's activity `failed` (exit 2) unless an +written. A compare whose first page GitHub refuses (HTTP 404 or 422) is stored with its +`http_status`; one that still fails under 5.3 step 4's transient classes, whose 2xx first page has +no `files` list, whose fetched commits differ from `total_commits`, or whose later page answers 404 +or 422 is stored with its `error` and makes the repository's activity `failed` (exit 2) unless an `accepted_failures` entry for the repository and the event's `github_id` accepts it (5.5); both are requested again on each later run until GitHub answers them, and verify's report and the manifest count the capped, refused, and failed compares (5.4, 5.6). It reads the default branch's history @@ -2209,11 +2242,13 @@ longer lists it; a count that differs is rescanned once, and a second mismatch f repository's activity for the run (exit 2, gap 106). A log or history page that still fails under 5.3 step 4's transient classes makes the repository's activity `failed` for the run: it writes none of that read, and the run exits 2. The deadline stops the readers as it stops hydration, a -stopped log or history read writes nothing (`time_limited`), and a halt is `halted` with its -reason in `repo_sync_state.activity_halt_reason`; `--allow-deletions` applies its N to the events +stopped log or history read writes nothing (`time_limited`), and a halt is `halted` (exit 2, 5.3 +step 8) with its reason in `repo_sync_state.activity_halt_reason`; `--allow-deletions` applies its N to the events and to the history commits separately (5.3 step 7). Event actors and commit authors are linked like commit links (5.2), and a history replace keeps a commit's stored link when GitHub returns -none (D28). verify reads the log and the history by the sync's completeness rules; a read that +none (D28). verify reads the repository's stored events and default-branch commits in one statement +each before the first request of its log and history reads, and reads the log and the history by +the sync's completeness rules; a read that still fails after its retries, ends before the floor and the log's first event, or whose history count still differs after one rescan makes the repository unverified (5.4). It compares the stored event ids with the log since the floor and the stored commits with the history, taking the 5.4 @@ -2229,9 +2264,12 @@ for the new tables; recordings from public repositories. Tests: an event of each `activity_type` the recordings hold; an event with an all-zero SHA and no compare row; a refused compare stored with its status and requested again on the next run; a compare of exactly 300 files marked `files_at_cap` and one of 299 not; a compare whose 2xx first -page has no `files` list, and one that still fails under the transient classes, each stored with -its `error`, making the activity `failed`, requested again on the next run, and accepted by a -compare entry; the compares written before a deadline kept; a two-page activity read through a +page has no `files` list, one whose fetched commits differ from `total_commits`, one whose later +page answers 404, and one that still fails under the transient classes, each stored with its +`error`, making the activity `failed`, requested again on the next run, and accepted by a compare +entry, which a later run that GitHub answers reports as not having failed; a log read that lists no +event, with an ascending read that returns none, complete; an activity or history halt making the +exit 2; the compares written before a deadline kept; a two-page activity read through a `Link` cursor that completes and sets `activity_first_event_at`, one that stops at the floor, one the deadline stops (`time_limited`, `activity_first_event_at` unchanged), and one that ends without a next link before the first event of the ascending read, failing the activity; a history count @@ -2282,9 +2320,9 @@ case that left it unchanged; a kind with such a case, or with fewer than 20 coun index-side check or a periodic re-hydration in pull request 5b before Step 6a, which adds the 5.3 step 3 rule and its tests (gap 109); for each index contradiction (an alias `NOT_FOUND`, an item outside coverage, an earlier `updatedAt`), how many occurred and how many the retry cleared are -recorded, and a kind that recurs gets a delay before its retry, measured and chosen before Step 6a; -the share of commit-link lookups answered `NOT_FOUND` is recorded, and Step 6a's limit is set from -it (5.2); and the run's time, cost, file size, and secondary-limit responses at one request in +recorded, and a kind that recurs gets a delay before its retry, measured, and added with its tests in pull +request 5b before Step 6a; `link_lookups` and `links_not_found` are recorded, and Step 6a's share +and minimum are set from them (5.2); and the run's time, cost, file size, and secondary-limit responses at one request in flight are recorded (and at any wider setting proposed), with the time the workflow's bootstrap is projected to take, Step 4b's compares included. @@ -2300,21 +2338,28 @@ Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snap `githubkit[auth-app]` extra to `pyproject.toml` and `uv.lock`. Its tests: `verify --db-only` accepting the gz and manifest `ghactivity snapshot` wrote, and reporting the SHA-256 it computed, the per-repository counts it compared, the revision it read, and the answers of `integrity_check` -and `foreign_key_check`, each equal to what the file holds (D39); a release with no complete -snapshot pair halting with exit 1; `snapshot --verify-report` exiting 1 on a missing report and on -one from a verify of an earlier sync, whose `last_run_id` differs from the file's, and the manifest +and `foreign_key_check`, each equal to what the file holds; halting with exit 1 and naming its check +on an asset digest that names another file (a stub of GitHub's release-asset response) and on one +snapshot's gz given with another snapshot's manifest, both written by `snapshot`, its +decompression, `integrity_check`, `foreign_key_check`, and unknown-revision refusals tested on their +passing side (D39); a release with no complete snapshot pair halting with exit 1; `snapshot +--verify-report` exiting 1 on a missing report, on a file that is not a verify report (the gz +`snapshot` wrote), and on one from a verify of an earlier sync, whose `last_run_id` differs from the file's, and the manifest listing the report's unverified repositories (5.6); through `httpx.MockTransport`, a mocked -installation-token response whose `expires_at` lies just over a minute ahead, a clock moved past -it mid-run, and a second token request before the next API request (githubkit keeps a token until +installation-token response whose `expires_at` lies just over a minute ahead, the wall clock +githubkit's token cache reads (`datetime.now` in `githubkit.auth.app` and +`githubkit.cache.mem_cache`) stubbed past it mid-run, and a second token request before the next API request (githubkit keeps a token until one minute before `expires_at`); and a 401 on an API request still exiting 1 with the run closed (5.3 step 4); `verify --db-only --out` writing the file and printing its SHA-256 and highest `sync_runs` id; `snapshot --after-run --sync-exit` refusing no newer row, two newer rows, a row -closed with another exit code, and a configured entry's stored row whose `last_run_id` is older, +closed with another exit code, a configured entry's stored row whose `last_run_id` is older, and a renamed repository's row so +older, found through `entries`, and accepting a stored repository the run neither attempted nor marked and, after a bootstrap with `--repo`, the entries it did not name (5.6); `snapshot --sync-run` refusing a missing or unclosed row, and neither form or both a usage error; the manifest carrying `sync_run_id`, -`sync_finished_at`, `actions_run_id`, `repo_filter`, `links_not_found`, and the Step 4b compare and -deletion counts; the commit-link limit making the exit 2 above its share and not at it; +`sync_finished_at`, `actions_run_id`, `repo_filter`, `link_lookups`, `links_not_found`, and the Step +4b compare and deletion counts; the commit-link limit making the exit 2 above its share, not at it, +and not below its minimum number of lookups; `scripts/compare_reads.py` on two files the sync wrote from the same recordings, reporting no difference and the rows and columns it compared per table, equal to the files' counts, and on files it wrote from recordings of one repository taken at two times, reporting each difference the @@ -2324,12 +2369,19 @@ job declares, and fails unless `publish`'s and `report`'s conditions compare eac string and require it to be non-empty, the path parsed from the job-level `DATABASE_URL`, resolved against the step's working directory, equals `verify --db-only`'s `--out`, the SHA-256 comparison and the recording of N come before `alembic upgrade head`, the `repo` input is validated and -requires `bootstrap`, and `snapshot` is given `--after-run` and `--sync-exit`. 6b creates release +requires `bootstrap`, `publish`'s condition is exactly the sync output `'0'` or `'2'` and the +verify output `'0'` with `inputs.repo` empty when the release is `db`, `report`'s requires both +outputs `'0'` and `publish`'s success, and `snapshot` is given `--out`, `--run-id`, `--after-run`, +and `--sync-exit`. 6b creates release `db` and uploads the Step 5 database with its manifest (`ghactivity snapshot --sync-run` naming Step 5's last run, and `--verify-report` from Step 5's `verify --exact --report`) as the first snapshot. It creates release `db-test` the same way, holding snapshots this project's code wrote: one built at the revision before the last migration, and one synced with a `tracking.toml` that -listed a repository the current one does not (a sync exit 2). Before any App run writes to release +listed a repository the current one does not (a sync exit 2). Runs against `db-test` go in this +order, each prepared snapshot written and uploaded after the previous `db-test` publication so +that it is the latest pair: the snapshot one migration behind head and a run from it; the snapshot +with an untracked repository and a run from it; a run from a branch whose `tracking.toml` names a +repository that does not resolve; and the App comparison below. Before any App run writes to release `db`, 6b compares the App's reads with the maintainer token's: a `workflow_dispatch` run with `bootstrap`, `release: db-test`, and `repo` naming a sample of the repositories syncs them with the App into an empty file, and at once a local run syncs the same repositories with the token into a @@ -2339,7 +2391,8 @@ repositories do not hold is reported as unmeasured. `scripts/compare_reads.py` c files over every table the sync writes from GitHub (`repositories`, `repo_labels`, `github_users`, `account_profiles`, `account_node_ids`, `pull_requests` and its child tables, and from Step 4 the issue and activity tables), every column and child row except those stamped from -`Date` headers and the run ids, and lists as a difference any value the token read that the App +`Date` headers, the run ids, and row ids, each foreign key compared as the natural key of the row +it names, as `tests/sync/expected_state.py` does, and lists as a difference any value the token read that the App read as null. Any difference or unmeasured case blocks App runs that publish to `db`, and 6c, until it is explained or the permission is granted (gap 107). Once the comparison shows no unexplained difference, it runs one normal manual run and one bootstrap run with `release: db`. 6c enables the schedule and stops @@ -2358,8 +2411,10 @@ records whether the bootstrap run reached it and how many runs it took to comple run makes requests more than 60 minutes after its first installation-token mint, and its log records at least two mints; against release `db-test`, the snapshot one migration behind head upgrades and publishes, the snapshot with an untracked repository publishes and the workflow is -marked failed, and a re-run of `publish` after an interrupted upload completes; `verify --db-only`'s -refusals are covered by the 6a tests of what it compared (D39). +marked failed, and a re-run of `publish` after an interrupted upload completes; the run from a branch +whose `tracking.toml` names a repository that does not resolve publishes nothing, verify reporting +that repository unverified, and is marked failed; `verify --db-only`'s refusals are covered by the +6a tests. ### Step 7: Documentation rewrite and code audit @@ -2370,9 +2425,12 @@ counts. A final audit of the whole repository, made once Steps 1-6 have landed, - dependencies in `pyproject.toml` that nothing imports; - scripts, hooks, and CI steps for mechanisms a step removed; - tests and fixtures of removed paths, and fixtures no test reads; -- tests merged before D39 that change a copy of code, a database, or a file to see whether a check - catches it (`tests/db/test_schema.py`'s planted differences and `tests/db/test_migrations.py`'s - planted failure among them), each replaced under D39's rule; +- tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, + a connection, or a decoy holding a flaw, to see whether a check catches it (among them + `tests/db/test_schema.py`'s planted differences, `tests/db/test_migrations.py`'s planted failure, + foreign-key violation, and view-alter revisions, the planted calls and decoys of the placeholder + check and the suppression counter, and 5.7's planted import files), each replaced under D39's + rule; - every note that defers something until a step that has landed (in code, docs, `.gitignore`, and hook comments). diff --git a/docs/testing.md b/docs/testing.md index c9b06f6..2445cb0 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -11,13 +11,15 @@ How the test suite is built and what it counts as evidence. The plan of record i over `httpx.MockTransport`, answering from responses recorded from public repositories. Databases are real SQLite files built by `alembic upgrade head`. - **Direct evidence, nothing planted** (redesign plan D39). A test's inputs are GitHub's recorded - responses, stubs of what GitHub or the environment can send, calls and configuration the code - must refuse, and databases and files the project's own code wrote; it asserts the exact result. - No test changes a copy of the code, a test, a database, or a file to see whether a check catches - it, and none measures how many such changes are caught: there is no mutation testing, under any - name. A check that only a defect of this code could fail is tested on its passing side, with the - exact count of what it compared. A fix comes with a regression test that fails against the parent - commit's code. Tests merged before D39 that change a copy of a database or a file are replaced in + responses; stubs of what GitHub or the environment can send, which a test may build in memory + from a recorded unit; calls and configuration the code must refuse; valid rows (`tests/db/seed.py`); + and databases and files the project's own code wrote. Each test asserts the exact result. No + test changes a file or a copy of one, or writes a file, a row, a revision, a call site, or a decoy + holding a flaw, to see whether a check catches it, and none measures how many such flaws are + caught: there is no mutation testing, under any name. A check that only a defect of this code + could fail is tested on its passing side, with the exact count of what it compared; one GitHub, + the environment, or a caller can fail is tested on both sides. A fix comes with a regression test + that fails against the parent commit's code. Tests merged before D39 that plant are replaced in Step 7. - **No skipped tests** (D22). CI runs `scripts/check_junit.py --max-skipped 0`, which fails a run where any test was skipped (pytest reports `xfail` as skipped) or none ran. @@ -80,14 +82,15 @@ tests/ - **The real environment.** Migration tests run the repository's `alembic.ini`, `env.py`, and revisions. `write_revision` adds a test-only revision after the head through `version_locations`, which is how a planted failure, a foreign-key violation, and a batch alter - under the views are run through `env.py`. `env.py` is loaded fresh by every command, so its + under the views are run through `env.py` (tests from before D39, which Step 7 replaces). `env.py` is loaded fresh by every command, so its `MigrationRefused` is matched by class name. - **Seeding.** `tests/db/seed.py` writes rows through the models' tables with every column a test does not care about filled with a valid default; GitHub ids come from one process-wide counter, so separate transactions never collide. - **Schema comparison.** `schema_of` reads columns (`PRAGMA table_xinfo`), indexes, foreign keys, and every item of each table's definition, which is where constraint names and CHECK text - live; each reading is tested against planted differences. + live; each reading is tested against planted differences (tests from before D39, which Step 7 + replaces). - **Dictionary and schema version.** `tests/test_data_dictionary.py` compares `docs/data-dictionary.md` with a migrated database in both directions. `tests/schema_versions.json` maps each schema version (`PRAGMA user_version`) to its consumer From a2fed22e7392c1a6f97107ebd45b06bac818b38f Mon Sep 17 00:00:00 2001 From: chrishuie Date: Wed, 21 Oct 2026 18:42:47 -0600 Subject: [PATCH 11/15] docs(plan): one D39 text everywhere it is restated; settle the last gaps - D39's rule is one text, copied word for word into Step 3, CLAUDE.md, docs/testing.md, and CONTRIBUTING.md; it allows stubs built in memory from a recorded unit, valid rows written in SQL, and configuration naming a state GitHub can produce, for tests and acceptances. - Step 3b-2a-3: the readable-2xx REST rule in its summaries, and how each refusing shape is built (replay.modify or the scenario model's set). - sync_runs.entries records the matched stored row's github_id, else the resolved one; snapshot skips an id with no stored row; a snapshot's T is the time snapshot writes it. - 6a states why verify --db-only's unknown-revision refusal is tested on its passing side; the acceptance names the refusals the 6a tests cover. - D38 and 6a state the commit-link limit as 5.2 does; Step 7 names only the view-alter revision that leaves a view in place. --- CLAUDE.md | 26 +++++++++++------ CONTRIBUTING.md | 25 +++++++++++----- docs/redesign-plan.md | 66 +++++++++++++++++++++++++++---------------- docs/testing.md | 28 ++++++++++-------- 4 files changed, 93 insertions(+), 52 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 54b2ae0..a4137d6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,15 +42,23 @@ uv run pytest # CI also runs Python 3.12, `alembic check`, ## Test Evidence -Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any -name (redesign plan D39): no test, review, or acceptance changes a file or a copy of one, or writes -a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches -it, and none measures how many such flaws are found. Test inputs are GitHub's recorded responses; -stubs of what GitHub or the environment can send, built in memory; calls and configuration the code -must refuse; valid rows (`tests/db/seed.py`); and databases and files the project's own code wrote. -Tests assert exact results; a check that only a defect of this code could fail is tested on its -passing side with the exact count it compared. Tests merged before D39 that plant are replaced in -Step 7 ([Testing](docs/testing.md)). +Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name +(redesign plan D39). No test, review, or acceptance changes a file or a copy of one (other than a +stub the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a +flaw, to see whether something catches it, and none measures how many such flaws are found. A test's +or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the +environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a +test may build in memory from a recorded unit changed into that shape; calls the code under test +must refuse, and configuration it must refuse or that names a state GitHub can produce (a repository +that does not resolve, one no longer listed); valid rows a test writes (through the models with +`tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any +revision. Each test asserts the exact result. A check over stored data, files, or source that only a +defect of this code could fail is tested on its passing side, with the exact count of the rows, +files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or +a caller can fail is tested on both sides. A fix comes with a regression test that fails against the +parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, +a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are +replaced in Step 7. See [Testing](docs/testing.md). ## Current Scope diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d12575b..952fc62 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -246,13 +246,24 @@ def test_reviewers(migrated_database: Path) -> None: # a copy of the alembic-bu pr = seed.pull_request(seed.repository("prebid/Prebid.js"), 13481, seed.user("author")) ``` -**3. Never plant, tamper, or seed a flaw** (redesign plan D39). Test on GitHub's recorded -responses; stubs of what GitHub or the environment can send, built in memory; calls and -configuration the code must refuse; valid rows (`tests/db/seed.py`); and databases and files the -project's own code wrote, with the exact result asserted. A check GitHub, the environment, or a caller can fail is tested on -both sides of its boundary; one only a defect of this code could fail is tested on its passing side -with the exact count it compared; a fix comes with a test that fails against the parent commit's -code ([Testing Guide](docs/testing.md#principles)). +**3. Never plant, tamper, or seed a flaw** (redesign plan D39). Nothing is planted or tampered, no +flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance +changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a row, +a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none +measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded +responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an +interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed +into that shape; calls the code under test must refuse, and configuration it must refuse or that +names a state GitHub can produce (a repository that does not resolve, one no longer listed); valid +rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files +this project's own code wrote, at any revision. Each test asserts the exact result. A check over +stored data, files, or source that only a defect of this code could fail is tested on its passing +side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; +a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with +a regression test that fails against the parent commit's code. Tests merged before D39 that change a +file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, +to see whether a check catches it are replaced in Step 7. See the [Testing +Guide](docs/testing.md#principles). ### Running Tests diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 32219f4..0e02663 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -92,8 +92,8 @@ Within scope the database collects and distinguishes data and never interprets i | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, a repository error during a timeline lookup, and from Step 4b a log or history read verify cannot complete make a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and from Step 4b its stored events and default-branch commits in one statement each before the first request of its log and history reads, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | -| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class and one cause, the first line of the failure's message, of an item, an account's refresh, or from Step 4b a compare, so a permanent failure can be accepted without turning every run red while a new cause is not. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter and the commit-link lookups GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the floor or at the log's first event, as a read in ascending order finds it, fails a history read whose count still differs after a rescan, and records a compare that still fails, which an entry can accept. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now counted per run and in the manifest, and from Step 6a a run whose share of link lookups answered `NOT_FOUND` exceeds the one Step 5 sets exits 2, since a lookup form GitHub stopped resolving would look the same per link), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (`items_deleted`, and from Step 4b `events_deleted` and `history_commits_deleted`; the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | -| D39 | Decision of 2026-10-21 (the maintainer): nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes code, a test, the plan, review material, a recording, a database, a snapshot, a manifest, or any other file, or a copy of one, or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it; none measures how many such flaws a test, a check, or a reviewer finds. A test's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed into that shape; calls and configuration the code under test must refuse; valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or source that only a defect of this code could fail is tested on its passing side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A regression test that fails against the parent commit's code remains evidence. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 7. | +| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class and one cause, the first line of the failure's message, of an item, an account's refresh, or from Step 4b a compare, so a permanent failure can be accepted without turning every run red while a new cause is not. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter, each configured entry's `github_id`, and its first commit-link lookups and those GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the floor or at the log's first event, as a read in ascending order finds it, fails a history read whose count still differs after a rescan, and records a compare that still fails, which an entry can accept. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now counted per run and in the manifest, and from Step 6a a run whose `links_not_found` exceeds the share of its `link_lookups` Step 5 sets, with at least Step 5's minimum number of lookups, exits 2 (an account an earlier lookup answered `NOT_FOUND` counted in neither), since a lookup form GitHub stopped resolving would look the same per link), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (`items_deleted`, and from Step 4b `events_deleted` and `history_commits_deleted`; the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | +| D39 | Decision of 2026-10-21 (the maintainer). Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed into that shape; calls the code under test must refuse, and configuration it must refuse or that names a state GitHub can produce (a repository that does not resolve, one no longer listed); valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or source that only a defect of this code could fail is tested on its passing side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 7. | ## 3. Coverage rule @@ -348,7 +348,7 @@ and `color`, `pr_user_tags.created_at`). | `account_profiles` | `account_id`, `company`, `location`, `website_url`, and an organization's `description`, `email`, `twitter_username`, `is_verified`, `archived_at`, `has_sponsors_listing`, and `pinned_items_remaining`, `refreshed_at` | PK `account_id`, a reference to `github_users` (`ON DELETE CASCADE`); rows only for a `User` or an `Organization` (D31); each lookup that returns the account replaces the whole row with the profile it returned, through the writer and in the transaction that writes `github_users`' login and type, so no earlier value survives a field GitHub now returns as null or a change of type; a lookup that returns `NOT_FOUND` leaves the row unchanged, marked by `github_users.not_found_at` (D28); every field nullable as GitHub's is, the organization fields null for a user and `company` null for an organization; `refreshed_at` as on `github_users` | | `account_node_ids` | `account_id`, `node_id`, `first_run_id`, `last_run_id` | Step 3b-2a-3; PK (`account_id`, `node_id`); `node_id` unique; every node id GitHub reported for the account in an actor sighting, a lookup, or a commit link, the one `github_users.node_id` holds included, with the first and latest runs that reported it (`first_run_id` and `last_run_id`, references to `sync_runs`), so GitHub's second node id for one `databaseId` is stored, not dropped (5.2, gap 103); its migration refuses a file whose `github_users` holds rows, since no file was published before Step 6b (gap 110); consumer | | `automation_emails` | `email`, `reason`, `run_id` | integer primary key; `email` unique, stored lowercase ASCII (SQLite's `lower()`, which the views apply to commit emails, folds only ASCII letters); replaced the same way | -| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `link_lookups`, `links_not_found`, `repo_filter`, `entries`, `deletions_approved`, and from Step 4b `events_deleted` and `history_commits_deleted` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, `manual` when it is unset, and `unknown` for any other value (gap 105); `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes to tracked files, untracked files ignored, and `-unknown` when it fails or times out, gap 105; the package runs from a checkout only when its directory or a parent holds a `.git` entry, and then `git ls-files --error-unmatch` exiting 1 means the package is not tracked, while any other failure of a git call, exit 128 and a missing `git` included, records the package version, or the commit when `rev-parse` answered, followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted, and from Step 4b `events_deleted` and `history_commits_deleted` the activity events and default-branch commits Step 4b deleted (nullable, null for a run before Step 4b, each written in its deletion's transaction); `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `link_lookups` the commit-link accounts this run looked up that no earlier lookup answered `NOT_FOUND` (neither an account stored with `not_found_at`, so none of gap 90's lookups, nor one a stored commit row links by `linked_github_id` without an account), and `links_not_found` those of them a lookup answered `NOT_FOUND` (5.2), `repo_filter` a JSON array of the `--repo` values as given, sorted, null for a run without `--repo`, and `entries` a JSON array, in `tracking.toml` order, of each configured entry's name and the `github_id` 5.3 step 1 resolved or matched it to, null when none, written when the run closes; `refresh_failures`, like `failures`, counts accepted ones; the migration that adds these five refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | +| `sync_runs` | `started_at`, `deadline_at`, `finished_at`, `status`, `exit_code`, `trigger`, `code_version`, `config_source`, `config_sha256`, `graphql_points`, `rest_calls`, `items_planned`, `items_hydrated`, `items_deleted`, `failures`, `refresh_failures`, `link_lookups`, `links_not_found`, `repo_filter`, `entries`, `deletions_approved`, and from Step 4b `events_deleted` and `history_commits_deleted` | `status` in (`running`, `complete`, `time_limited`, `failed`, `aborted`), set as 5.3 step 8 says; `deadline_at` is `started_at` plus `--max-minutes`, null without it; `trigger` is `schedule` or `workflow_dispatch` from `GITHUB_EVENT_NAME`, `manual` when it is unset, and `unknown` for any other value (gap 105); `code_version` is `GITHUB_SHA`, else the commit of the checkout the package runs from (with `-dirty` appended when `git status` lists changes to tracked files, untracked files ignored, and `-unknown` when it fails or times out, gap 105; the package runs from a checkout only when its directory or a parent holds a `.git` entry, and then `git ls-files --error-unmatch` exiting 1 means the package is not tracked, while any other failure of a git call, exit 128 and a missing `git` included, records the package version, or the commit when `rev-parse` answered, followed by `-unknown`; the package runs from a checkout when git tracks its `__init__.py`), else the package version; `config_source` is `packaged` or the `TRACKING_CONFIG_PATH` path made absolute (symlinks not followed), and `config_sha256` the digest of the `tracking.toml` bytes the run read (gap 88); `items_planned` counts the distinct items planned for hydration (in-place updates excluded), `items_hydrated` the hydrated items written, and `items_deleted` the item rows step 7 and the prune deleted, and from Step 4b `events_deleted` and `history_commits_deleted` the activity events and default-branch commits Step 4b deleted (nullable, null for a run before Step 4b, each written in its deletion's transaction); `failures` counts this run's item failures, accepted ones included, and `refresh_failures` the accounts whose refresh failed this run (5.3 step 4), `link_lookups` the commit-link accounts this run looked up that no earlier lookup answered `NOT_FOUND` (neither an account stored with `not_found_at`, so none of gap 90's lookups, nor one a stored commit row links by `linked_github_id` without an account), and `links_not_found` those of them a lookup answered `NOT_FOUND` (5.2), `repo_filter` a JSON array of the `--repo` values as given, sorted, null for a run without `--repo`, and `entries` a JSON array, in `tracking.toml` order, of each configured entry's name and the `github_id` of the stored row 5.3 step 1 matched it to, else the `github_id` it resolved to, null when it resolved to none, written when the run closes; `refresh_failures`, like `failures`, counts accepted ones; the migration that adds these five refuses a file whose `sync_runs` holds rows (gap 110); `deletions_approved` is null or JSON with `allow_deletions` (the repository, the approved N, and for each guarded set, step 7's items and from Step 4b the activity events and the default-branch commits, its candidate count and whether its deletion ran) and `prune_untracked` (each pruned repository's `github_id`, `full_name`, and items deleted), written as `{"allow_deletions": {"repository", "github_id", "approved", "sets": {"items": {"candidates", "deleted"}}}, "prune_untracked": [{"github_id", "full_name", "items_deleted"}]}` with sorted keys, in the transaction of the deletion it records, as `items_deleted` is; every column is consumer except the run counters (5.1, consumer interface) | | `repo_sync_state` | `repository_id`, `last_index_started_at`, `index_pr_total`, `index_issue_total`, `last_success_at`, `last_run_id`, `last_outcome`, `halt_reason`, and from Step 4b `activity_read_at`, `activity_first_event_at`, `activity_last_outcome`, `activity_halt_reason`, `history_read_at`, and `default_branch_commits_total` | PK `repository_id`; `activity_first_event_at` is the `timestamp` of the oldest event the latest complete activity read listed, so a consumer can tell a quiet period from a log that does not reach back (gap 106); `activity_read_at` is the GitHub `Date` header of the first page of the latest activity-log read that completed and was written; `history_read_at` and `default_branch_commits_total` are the first page's `Date` header and the `totalCount` of the latest history read that replaced the rows, written only with that replace; `activity_last_outcome` in (`complete`, `time_limited`, `failed`, `halted`); `activity_halt_reason` holds a Step 4b deletion halt's reason and candidate counts and is cleared by a run whose Step 4b did not halt (Step 4b); `last_run_id` is the latest run that attempted the repository, or found it untracked, and `last_outcome` its result, in (`complete`, `time_limited`, `failed`, `incomplete_index`, `halted`, `untracked`), both written by step 8, or by the close of a fatal or interrupted run (5.3 step 8); `last_index_started_at` is written only when that index completed and the repository did not fail later in the run, a time-limited repository included (its unreached items are in `sync_pending`), and never by the close of a fatal or interrupted run; `last_success_at` is the run's `finished_at` when `last_outcome` is `complete` and no item of the repository failed outside `[sync].accepted_failures`; `halt_reason` holds step 7's halt and is cleared by a run whose step 7 ran for the repository and did not halt, a run whose step 7 did not run for it leaving it | | `sync_pending` | `repository_id`, `item_kind`, `number`, `run_id` | PK (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); items a time-limited run planned but did not reach; step 8 replaces the rows of each completely indexed repository that did not fail later in the run (a repository gone during hydration keeps its rows, 5.3 step 4) with this run's unreached items, so a run that reached them all leaves none; consumer, so an in-coverage item not yet hydrated is told apart from one outside coverage | | `sync_failures` | `repository_id`, `item_kind`, `number`, `attempts`, `last_error`, `first_failed_at`, `last_failed_at`, `last_run_id` | unique (`repository_id`, `item_kind`, `number`); `item_kind` in (`pull_request`, `issue`); `attempts` counts the runs in which the item failed: `last_run_id` is the latest such run, and a failure adds one only when its run is another, whatever the runs' clocks read; `last_error` is the error's class and message; row deleted on success, or by step 7 when its repository's in-coverage set no longer lists the item | @@ -1175,8 +1175,8 @@ Workflow `.github/workflows/sync.yml`: `[--verify-report PATH]`, writes the copy with `VACUUM INTO`, checks its header for rollback-journal mode (the check `scripts/check_db_journal.py` makes), gzips it, and writes the manifest, in the workflow and for 6b's first snapshot (`run_id` null). Names are - `github_activity-.db.gz` and `github_activity-.manifest.json`, with `T` as - `YYYYMMDDTHHMMSSZ`, because GitHub renames asset names that contain special characters. The latest + `github_activity-.db.gz` and `github_activity-.manifest.json`, with `T` the UTC time + `snapshot` writes the copy, as `YYYYMMDDTHHMMSSZ`, because GitHub renames asset names that contain special characters. The latest snapshot is the newest `T` whose two assets are both in state `uploaded`; `verify --db-only` on that pair decides, and a mismatch halts, with no fallback to an older pair. The manifest records `actions_run_id`, `sync_run_id`, `sync_finished_at`, the code SHA, the `tracking.toml` digest @@ -1193,7 +1193,8 @@ Workflow `.github/workflows/sync.yml`: before the sync (0 for a bootstrap) and the sync's exit code, and exits 1 unless exactly one `sync_runs` row is newer than N, that row is closed with exit code C, and it is the `last_run_id` of the stored row of every `github_id` that run's `sync_runs.entries` records (an - entry that never resolved records none, 5.1), so a job whose sync did not run + entry that never resolved records none, 5.1; a recorded `github_id` with no stored row has + nothing to check), so a job whose sync did not run cannot republish the old file (gap 107). For 6b's first snapshot, `--sync-run ID` names Step 5's last run instead, which must exist and be closed with exit 0 or 2, and supplies the manifest's sync exit code. The manifest records that run as `sync_run_id` and `sync_finished_at`, and @@ -1578,15 +1579,27 @@ httpx declaration (gap 76) have merged. Every Step 3 test of a GitHub interaction runs through the real `GitHubSource` over `httpx.MockTransport`. Databases come from one template per test session, built by `alembic upgrade head` and copied for each test; `metadata.create_all` is used only by the 5.1 -schema comparison. Evidence is direct and nothing is planted (D39): inputs at and on each side of -every boundary the design states (the coverage floor, the 10-minute unsettled window, 10 minutes -past `deadline_at` for the lock, the 24-hour refresh, the delete guard's max(25, 2%)), taken from -GitHub's recorded responses, stubs of what GitHub or the environment can send, and databases this -project's code wrote, with the exact result asserted, and regression tests that fail against the -parent's code. No test changes a file or a copy of one, or writes a file, a row, a revision, a call -site, or a decoy holding a flaw, to see whether a check catches it, and there is no mutation -testing, under any name. Each wait on a thread or -subprocess has an explicit timeout. +schema comparison. Evidence is direct, with inputs at and on each side of every boundary the design +states (the coverage floor, the 10-minute unsettled window, 10 minutes past `deadline_at` for the +lock, the 24-hour refresh, the delete guard's max(25, 2%)) and regression tests that fail against +the parent's code. D39 governs every test. Nothing is planted or tampered, no flaw is seeded, and +there is no mutation testing, under any name. No test, review, or acceptance changes a file or a +copy of one (other than a stub the inputs below allow), or writes a file, a row, a revision, a call +site, or a decoy holding a flaw, to see whether something catches it, and none measures how many +such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a +response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing +`git`, a clock), which a test may build in memory from a recorded unit changed into that shape; +calls the code under test must refuse, and configuration it must refuse or that names a state GitHub +can produce (a repository that does not resolve, one no longer listed); valid rows a test writes +(through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own +code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or +source that only a defect of this code could fail is tested on its passing side, with the exact +count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, +the environment, or a caller can fail is tested on both sides. A fix comes with a regression test +that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of +one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a +check catches it are replaced in Step 7. Each wait on a thread or subprocess has an explicit +timeout. #### Step 3a: New schema and cutover @@ -1960,8 +1973,8 @@ schema-version lock entry's consumer pairs: - every node id a sighting, a lookup, or a commit link reports for a stored account is stored in `account_node_ids`, and a commit-link lookup answered `NOT_FOUND` is counted in `links_not_found` beside `link_lookups` (5.2, gap 103); -- the source checks of gap 104: a 2xx REST response without readable rate-limit headers is not - understood, a GraphQL `errors` whose value is not a list, `null` included, is a server error, a +- the source checks of gap 104: a 2xx REST response whose body is JSON and whose `Date` is readable + but without readable rate-limit headers is not understood, a GraphQL `errors` whose value is not a list, `null` included, is a server error, a compare above 250 commits holds the commit connection's first page at positions `total_commits - 250` to `total_commits - 151`, in order, and every index page selects the repository's `databaseId` and is checked against the resolved `github_id` (no `hydration_version` @@ -2023,8 +2036,10 @@ counted in `link_lookups` and `links_not_found`, and one for an account an earli answered `NOT_FOUND` counted in neither; the report listing each index contradiction with its kind and whether the retry cleared it; `entries` recording each configured entry's `github_id`, a renamed repository's included, and null for one that never resolved; each source check accepting the shapes GitHub's recordings hold and refusing each shape -its rule names (a 2xx REST response without rate-limit headers; an `errors` of `{}`, `""`, and -`null`; an index page whose repository `databaseId` differs; a commit page from other positions), +its rule names (a 2xx REST response whose body is JSON and whose `Date` is readable but without +rate-limit headers; an `errors` of `{}`, `""`, and +`null`; an index page whose repository `databaseId` differs; a commit page from other positions; each built in memory from a recorded unit with +`replay.modify` or by the scenario model's `set`), the commit page at positions 4-103 of the recording of Prebid.js #8858 accepted, and a 502, a 200 whose body is not JSON, and a 200 without `Date`, none with rate-limit headers, still retried; `code_version` outside any `.git` (the package version), in a checkout with the package untracked (the package version), in a clean checkout, @@ -2334,15 +2349,16 @@ repository secrets. Scope: 6a merges `sync.yml` with `workflow_dispatch` only, adds `ghactivity snapshot`, `ghactivity verify --db-only` (5.4), and `scripts/compare_reads.py` for 6b, adds the commit-link -`NOT_FOUND` limit with its constant from Step 5's measurement (5.2, 5.3 step 8), and adds the +`NOT_FOUND` limit with its share and minimum from Step 5's measurement (5.2, 5.3 step 8), and adds the `githubkit[auth-app]` extra to `pyproject.toml` and `uv.lock`. Its tests: `verify --db-only` accepting the gz and manifest `ghactivity snapshot` wrote, and reporting the SHA-256 it computed, the per-repository counts it compared, the revision it read, and the answers of `integrity_check` and `foreign_key_check`, each equal to what the file holds; halting with exit 1 and naming its check on an asset digest that names another file (a stub of GitHub's release-asset response) and on one snapshot's gz given with another snapshot's manifest, both written by `snapshot`, its -decompression, `integrity_check`, `foreign_key_check`, and unknown-revision refusals tested on their -passing side (D39); a release with no complete snapshot pair halting with exit 1; `snapshot +decompression, `integrity_check`, and `foreign_key_check` refusals, which once the digest matches +only a defect of `snapshot` could reach, tested on their passing side (D39), and its unknown-revision +refusal on its passing side too, since no code at 6a's head writes a revision after its own head; a release with no complete snapshot pair halting with exit 1; `snapshot --verify-report` exiting 1 on a missing report, on a file that is not a verify report (the gz `snapshot` wrote), and on one from a verify of an earlier sync, whose `last_run_id` differs from the file's, and the manifest listing the report's unverified repositories (5.6); through `httpx.MockTransport`, a mocked @@ -2413,8 +2429,8 @@ records at least two mints; against release `db-test`, the snapshot one migratio upgrades and publishes, the snapshot with an untracked repository publishes and the workflow is marked failed, and a re-run of `publish` after an interrupted upload completes; the run from a branch whose `tracking.toml` names a repository that does not resolve publishes nothing, verify reporting -that repository unverified, and is marked failed; `verify --db-only`'s refusals are covered by the -6a tests. +that repository unverified, and is marked failed; `verify --db-only`'s digest and manifest refusals are +covered by the 6a tests. ### Step 7: Documentation rewrite and code audit @@ -2428,7 +2444,7 @@ counts. A final audit of the whole repository, made once Steps 1-6 have landed, - tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it (among them `tests/db/test_schema.py`'s planted differences, `tests/db/test_migrations.py`'s planted failure, - foreign-key violation, and view-alter revisions, the planted calls and decoys of the placeholder + foreign-key violation, and the view-alter revision that leaves a view in place, the planted calls and decoys of the placeholder check and the suppression counter, and 5.7's planted import files), each replaced under D39's rule; - every note that defers something until a step that has landed (in code, docs, `.gitignore`, diff --git a/docs/testing.md b/docs/testing.md index 2445cb0..062f22d 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -10,17 +10,23 @@ How the test suite is built and what it counts as evidence. The plan of record i - **Real components, faked boundaries.** GitHub is the only fake: the real `GitHubSource` runs over `httpx.MockTransport`, answering from responses recorded from public repositories. Databases are real SQLite files built by `alembic upgrade head`. -- **Direct evidence, nothing planted** (redesign plan D39). A test's inputs are GitHub's recorded - responses; stubs of what GitHub or the environment can send, which a test may build in memory - from a recorded unit; calls and configuration the code must refuse; valid rows (`tests/db/seed.py`); - and databases and files the project's own code wrote. Each test asserts the exact result. No - test changes a file or a copy of one, or writes a file, a row, a revision, a call site, or a decoy - holding a flaw, to see whether a check catches it, and none measures how many such flaws are - caught: there is no mutation testing, under any name. A check that only a defect of this code - could fail is tested on its passing side, with the exact count of what it compared; one GitHub, - the environment, or a caller can fail is tested on both sides. A fix comes with a regression test - that fails against the parent commit's code. Tests merged before D39 that plant are replaced in - Step 7. +- **Direct evidence, nothing planted** (redesign plan D39). Nothing is planted or tampered, no flaw + is seeded, and there is no mutation testing, under any name. No test, review, or acceptance + changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a + row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and + none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's + recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a + timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded + unit changed into that shape; calls the code under test must refuse, and configuration it must + refuse or that names a state GitHub can produce (a repository that does not resolve, one no longer + listed); valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and + databases and files this project's own code wrote, at any revision. Each test asserts the exact + result. A check over stored data, files, or source that only a defect of this code could fail is + tested on its passing side, with the exact count of the rows, files, or values it compared, so it + cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on + both sides. A fix comes with a regression test that fails against the parent commit's code. Tests + merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a + connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 7. - **No skipped tests** (D22). CI runs `scripts/check_junit.py --max-skipped 0`, which fails a run where any test was skipped (pytest reports `xfail` as skipped) or none ran. - **Time.** Source tests pass a `FakeClock`, so waits and retries are asserted, never slept. From 54428dc0df36eae0db3101ca95f0eb24c6f09c47 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Thu, 22 Oct 2026 05:24:14 -0600 Subject: [PATCH 12/15] docs(plan): state verify's child-count and timeline tests without planted inputs Verify's child-count checks and Step 4a's issue child-count checks are tested on their passing side, each reporting the rows it compared, equal to the counts the scenario model or the stored totals give. The timeline lookup's entries before the boundary are events the scenario model renders. --- docs/redesign-plan.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 23b9710..938cc2f 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -1848,7 +1848,8 @@ order, a stale item, verify's repository set with and without `--repo` (a stored repository, a stored row no configured entry names, a renamed repository, and a renamed repository whose configured entry fails to resolve, reported unverified and not untracked, and an entry that resolves to a `github_id` other than its stored row's, with no stored `is_tracked` row reported -untracked), its child-count checks (each by a planted row deletion), an unverified repository for +untracked), its child-count checks, each reporting the rows it compared, equal to the counts the +scenario model gives, an unverified repository for each failure 5.4 lists, one made unverified by a lookup by number and one by a timeline lookup, each keeping the classes it already had and leaving unchecked the items no lookup had answered, those of the batch the error ended included, an unchecked item and an unverified repository each making a @@ -1874,8 +1875,7 @@ another issue, a link difference made by a closing keyword with no event, a diff link totals alone with enough connected events, with only a disconnected event, and with too few, a synthetic disconnected event, an event at the boundary counted and one a second before it not, entries -planted in the lookup's response at a second and at half a second before the boundary and not -counted, a stored link no longer indexed on a PR whose index lists every link matched only by an +the model renders at a second and at half a second before the boundary, not counted, a stored link no longer indexed on a PR whose index lists every link matched only by an event naming it, a filtered page whose `totalCount` differs from its entries, a lookup whose entries differ from its `filteredCount` after its retry and one whose response the typed models reject, each an error, one that differs once and matches on its retry, classed by its events, and a @@ -1928,8 +1928,8 @@ Tests: issue first sync, update, close, and reopen; unsettled re-hydration and v issues; an issue's stored author kept when GitHub returns it as null (D28); issue comments and assignees written and replaced; a transfer arriving with a new id; an old number returning `NOT_FOUND`, treated as gone; a snapshot one migration behind head upgrades and syncs; verify's -child-count checks for `issue_labels`, `issue_comments`, and `issue_assignees`, each by a planted -row deletion. +child-count checks for `issue_labels`, `issue_comments`, and `issue_assignees`, each reporting the +rows it compared, equal to the stored totals. Acceptance: `verify --exact` for issues on the same repositories; a REST sample of issues matches field by field. From f55ae747b2dbf1fe811dce12046fe3c5bd801007 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Thu, 22 Oct 2026 05:27:14 -0600 Subject: [PATCH 13/15] docs(plan): take planting out of every step; replace the planted tests next (3b-2a-2d) - New Step 3b-2a-2d, ahead of Step 3b-2a-3: replace every merged test that changes a file or a copy of one, or writes a file, revision, call, connection, or decoy holding a flaw, to see whether a check catches it; a check GitHub, the environment, or a caller can fail keeps a refusing side built from D39's inputs, and one only a defect could fail is tested on its passing side with the exact count it compared. Step 7's item for this is removed, and D39 and its copies name 3b-2a-2d. - The history of Steps 1, 2, 3a, 3b-1, 3b-2a-2a, and 3b-2a-2c, gaps 52, 68, and 80, and 5.7 no longer describe planted, decoy, changed-copy, or edited-entry tests, and Steps 1 and 2 no longer accept on code removed or a document changed to see a test fail. - CLAUDE.md lists Step 3b-2a-2d. --- CLAUDE.md | 9 ++-- CONTRIBUTING.md | 2 +- docs/redesign-plan.md | 101 ++++++++++++++++++++++++------------------ docs/testing.md | 6 +-- 4 files changed, 66 insertions(+), 52 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index a4137d6..850b7d5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,8 +57,7 @@ defect of this code could fail is tested on its passing side, with the exact cou files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, -a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are -replaced in Step 7. See [Testing](docs/testing.md). +a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 3b-2a-2d. See [Testing](docs/testing.md). ## Current Scope @@ -82,11 +81,13 @@ replaced in Step 7. See [Testing](docs/testing.md). - `ghactivity sync` (Step 3b-2a-2c, `cli/sync.py`): the engine's run with the options of plan 5.7; a run that never starts exits 1, and a run that starts exits with the engine's 0, 2, or 1 -**Not implemented yet:** D37 in the sync (Step 3b-2a-3, D38), the REST +**Not implemented yet:** tests without planted inputs (Step 3b-2a-2d, D39), D37 in the sync +(Step 3b-2a-3, D38), the REST sampler (Step 3b-2b-1), and `verify` (Step 3b-2b-2). The tracked `github_activity.db` stays frozen at the first schema (D26). -**Planned in the redesign plan:** D37 in the sync (Step 3b-2a-3), the REST sampler (Step +**Planned in the redesign plan:** tests without planted inputs (Step 3b-2a-2d), D37 in the sync +(Step 3b-2a-3), the REST sampler (Step 3b-2b-1), and verify (Step 3b-2b-2); GitHub Issues, closing-issue links, and default-branch activity (Step 4); the full bootstrap (Step 5); scheduled sync and release publication (Step 6). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 952fc62..6cc1c3a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -262,7 +262,7 @@ side, with the exact count of the rows, files, or values it compared, so it cann a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, -to see whether a check catches it are replaced in Step 7. See the [Testing +to see whether a check catches it are replaced in Step 3b-2a-2d. See the [Testing Guide](docs/testing.md#principles). ### Running Tests diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 0e02663..c81ff42 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -35,7 +35,7 @@ audited for silent failures and silently wrong data under D37, and D38 records t |------|-------|--------| | 1 | Guardrails | done (PR #5) | | 2 | GitHub source layer | done (PR #7) | -| 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | +| 3 | New schema, PR sync engine, cutover (pull requests 3a, 3b-1, 3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-2d, 3b-2a-3, 3b-2b-1, and 3b-2b-2) | in progress: 3a done (PR #11), 3b-1 done (PR #14), 3b-2a-1 done (PR #19), 3b-2a-2a done (PR #21), 3b-2a-2b-1 done (PR #24), 3b-2a-2b-2 done (PR #25), 3b-2a-2c done (PR #26) | | 4 | Issues, closing-issue links, and default-branch activity (pull requests 4a and 4b) | not started | | 5 | Full bootstrap and verification (pull request 5, and 5b when Step 5 does not show that every measured edit kind changes `updatedAt`, or finds an index contradiction that recurs) | not started | | 6 | Scheduled sync and publication | not started | @@ -93,7 +93,7 @@ Within scope the database collects and distinguishes data and never interprets i | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, a repository error during a timeline lookup, and from Step 4b a log or history read verify cannot complete make a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and from Step 4b its stored events and default-branch commits in one statement each before the first request of its log and history reads, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | | D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class and one cause, the first line of the failure's message, of an item, an account's refresh, or from Step 4b a compare, so a permanent failure can be accepted without turning every run red while a new cause is not. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter, each configured entry's `github_id`, and its first commit-link lookups and those GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the floor or at the log's first event, as a read in ascending order finds it, fails a history read whose count still differs after a rescan, and records a compare that still fails, which an entry can accept. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now counted per run and in the manifest, and from Step 6a a run whose `links_not_found` exceeds the share of its `link_lookups` Step 5 sets, with at least Step 5's minimum number of lookups, exits 2 (an account an earlier lookup answered `NOT_FOUND` counted in neither), since a lookup form GitHub stopped resolving would look the same per link), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (`items_deleted`, and from Step 4b `events_deleted` and `history_commits_deleted`; the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | -| D39 | Decision of 2026-10-21 (the maintainer). Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed into that shape; calls the code under test must refuse, and configuration it must refuse or that names a state GitHub can produce (a repository that does not resolve, one no longer listed); valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or source that only a defect of this code could fail is tested on its passing side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 7. | +| D39 | Decision of 2026-10-21 (the maintainer). Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed into that shape; calls the code under test must refuse, and configuration it must refuse or that names a state GitHub can produce (a repository that does not resolve, one no longer listed); valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or source that only a defect of this code could fail is tested on its passing side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 3b-2a-2d. | ## 3. Coverage rule @@ -182,7 +182,7 @@ Every gap found in the audit, and where it is resolved. | 49 | An empty `TRACKING_CONFIG_PATH` became the working directory, and a directory or unreadable file raised a raw `OSError` | Step 1: empty means the packaged file; read errors raise `TrackingConfigError` | | 50 | `actions/checkout` left the job token in `.git/config` while the built-wheel job ran build-backend code from PyPI | Step 1: `persist-credentials: false` on every checkout | | 51 | `.env.example`, which the setup instructions copy, was deleted in `7899671` | Step 1: restored | -| 52 | The migration test passed while migrating `DATABASE_URL` (by default the tracked database) if `alembic/env.py` ignored its URL override; two `tracking.toml` guards (unknown top-level keys, empty values) had no test | Step 1: decoy URL and head-revision check; guard tests | +| 52 | The migration test passed while migrating `DATABASE_URL` (by default the tracked database) if `alembic/env.py` ignored its URL override; two `tracking.toml` guards (unknown top-level keys, empty values) had no test | Step 1: the migration test checks the file it migrates and the head revision; guard tests | | 53 | Suppression counting missed `.pyi` files, `@no_type_check`, and `# isort: skip_file` / `# isort: off`; the Any-alias hook missed `type X = Any` | Step 1 | | 54 | `check-added-large-files` checked nothing in CI: with `--all-files` no file is newly added | Step 1: `--enforce-all`; `github_activity.db` exempt until Step 6 | | 55 | Build inputs not pinned: the hatchling build backend and its dependencies, `pre-commit-hooks` by tag, wheel-job dependencies installed without hashes | Step 1: hatchling pinned in `pyproject.toml` and, with its dependencies, hash-pinned in `build-constraints.txt` (a test keeps the two pins equal); `pre-commit-hooks` by SHA; `--require-hashes` installs | @@ -198,7 +198,7 @@ Every gap found in the audit, and where it is resolved. | 65 | `docs/development.md` said `build_engine` sets the rollback journal; `docs/roadmap.md` Phases 2-3 contradict this plan; other docs describe removed or nonexistent modules, commands, and counts | The plan audit corrected `docs/development.md` and marked `docs/roadmap.md` as history; Step 7 rewrites the rest | | 66 | The rollback-journal guard does not see a hot `-journal` beside the tracked file, which a commit would separate from it | Step 3 pins the tracked file's content (D26), so CI rejects any change to it; 6c removes it | | 67 | The JUnit skip allowance (D22) is a count, so once raised any test may use it | Accepted while the allowance is 0; the first raised allowance names its tests instead | -| 68 | `scripts/count_suppressions.py` does not count ruff's `# ruff: disable[...]` range suppression, which ruff 0.14.11 honors only in preview | Dormant while `preview` is off (`pyproject.toml` sets none). Resolved before Step 3: the counter counts each `# ruff: disable` in any letter case, never `# ruff: enable`, with tests for the spellings section 7 lists as honored, a string decoy, and an `enable` decoy (section 7, "Range suppressions"). It refuses (exit 1) a file it may read differently from ruff: any non-UTF-8 coding declaration, a lone CR, a file pre-commit's file tagging (identify) marks python or pyi without a lowercase `.py` or `.pyi` suffix, such as `module.PY`, `app.wsgi`, or an executable with a `#!/usr/bin/env python3` shebang, or a file identify cannot read (section 7, "Counter file forms") | +| 68 | `scripts/count_suppressions.py` does not count ruff's `# ruff: disable[...]` range suppression, which ruff 0.14.11 honors only in preview | Dormant while `preview` is off (`pyproject.toml` sets none). Resolved before Step 3: the counter counts each `# ruff: disable` in any letter case, never `# ruff: enable`, and counts the spellings section 7 lists as honored (section 7, "Range suppressions"). It refuses (exit 1) a file it may read differently from ruff: any non-UTF-8 coding declaration, a lone CR, a file pre-commit's file tagging (identify) marks python or pyi without a lowercase `.py` or `.pyi` suffix, such as `module.PY`, `app.wsgi`, or an executable with a `#!/usr/bin/env python3` shebang, or a file identify cannot read (section 7, "Counter file forms") | | 69 | click exits 2 on every usage error (an unknown option or command, an out-of-range value), the code D25 publishes on, so a mistyped workflow input would publish the unchanged database | Step 3a: a run that never starts exits 1 (5.3 step 8) | | 70 | Under alembic's SQLite defaults each revision commits on its own and schema changes are not transactional: a revision that fails after dropping a table leaves it dropped, with the previous revision stamped. A statement executed on the migration connection before `context.configure()` makes alembic treat the transaction as external and never commit: `upgrade` exits 0 with no revision stamped | Step 3a: one transaction per upgrade, nothing executed before `context.configure()` (5.1) | | 71 | A batch alter of a table that a view reads fails ("error in view ...: no such table") | Step 3a: migrations drop and recreate the dependent views (5.1) | @@ -210,7 +210,7 @@ Every gap found in the audit, and where it is resolved. | 77 | In rollback-journal mode a read left half-consumed on another connection blocks a commit until the busy timeout, which then raises "database is locked"; the comment on `SQLITE_BUSY_TIMEOUT_MS` in `db/engine.py` says a lock fails the item. Through the aiosqlite dialect a query fetches every row and closes its cursor inside `execute`, so only a server-side cursor leaves a result half-read; a reader in another process (an open `sqlite3` shell) can still hold the lock | Step 3b-2a-1: no code in `src/` reads through a server-side cursor (`stream`, `stream_scalars`, `stream_results=True`, `yield_per`), the snapshot reader first; Step 3b-2a-2b-2: a database error in the writer is fatal (5.3 step 4); Step 3b-2a-2a: the comment is corrected | | 78 | `source/users.py` checks a looked-up node's id against the requested node id, but not its `databaseId` against the commit link's, which only `sync/` holds | Step 3b-2a-2b-2: a mismatch fails the items waiting on the link (5.2) | | 79 | Hydration raises `RepositoryNotFound` or `RepositoryShapeError` when a batch's `repository` is null; 5.3 assigned repository failures only to resolve and index | Step 3b-2a-2b-2: a row of the 5.3 step 4 table | -| 80 | Gap 44's rule (new code uses `{}` placeholders) has no gate, and ruff cannot be one: PLE1205 flags a correct `logger.warning("brace {}", value)` and passes `logger.warning("percent %s", value)` | Step 3a: an AST check over `src/`, run as a pre-commit hook (so CI's `--all-files` run checks it) and tested with planted calls and a string decoy | +| 80 | Gap 44's rule (new code uses `{}` placeholders) has no gate, and ruff cannot be one: PLE1205 flags a correct `logger.warning("brace {}", value)` and passes `logger.warning("percent %s", value)` | Step 3a: an AST check over `src/`, run as a pre-commit hook (so CI's `--all-files` run checks it) | | 81 | At `6fedb05`, `v_pr_reviewers` and `v_pr_commenters` leave out the PR author's own reviews and comments, accounts of type `Bot`, accounts listed in `[automation]`, and reviews and comments with a null author, and `v_pr_commenters` also leaves out commenters who reviewed the PR | Step 3b-1: row-level views that keep every row and carry each condition as a column (5.1, D29) | | 82 | `v_pr_reviewers.final_verdict` reduces a reviewer's reviews to one state by a rule the database chose, ordered by review id, which does not follow submission order (section 7, "Review order") | Step 3b-1: removed; GitHub's own `reviewDecision` is stored on the PR (D29) | | 83 | `is_automation` merges GitHub's type `Bot` with the maintainer's lists into one flag, computed from the latest run's copy of `tracking.toml` | Step 3b-1: the account's type and the matching `[automation]` entry's reason as separate columns, and the run that copied the lists (5.2) | @@ -1244,9 +1244,8 @@ enabled afterwards. `source/` is the only package that imports githubkit; from Step 3, when `github/` is gone, a test fails when another module does. It scans `src/` outside `source/`, `scripts/`, and `alembic/` for imports at any scope, `TYPE_CHECKING` blocks included, and for `importlib.import_module` and -`__import__` called with a string literal, and fails when it scans no file; planted files test each -spelling, and a string decoy is not flagged (a test from before D39, which Step 7 replaces with one -on its passing side that asserts the exact count of files and imports it scanned). Tests are exempt, since they build githubkit's errors. A +`__import__` called with a string literal, and fails when it scans no file; its test asserts the exact +count of the files and imports it scanned (D39, Step 3b-2a-2d). Tests are exempt, since they build githubkit's errors. A second test fails when `source/` imports SQLAlchemy or `db/`, so fetchers cannot touch the database. | Disposition | Modules | @@ -1461,7 +1460,8 @@ nested node. ## 6. Execution steps Each step is one pull request from a `feature/` branch, except Step 3 (pull requests 3a, 3b-1, -3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-3, 3b-2b-1, and 3b-2b-2), Step 4 (pull +3b-2a-1, 3b-2a-2a, 3b-2a-2b-1, 3b-2a-2b-2, 3b-2a-2c, 3b-2a-2d, 3b-2a-3, 3b-2b-1, and 3b-2b-2), Step +4 (pull requests 4a and 4b), Step 5 (pull request 5, and 5b when its measurements call for the 5.3 step 3 rule or the retry delay Step 5 names), and Step 6 (pull requests 6a and 6c, with the 6b manual runs between them). A step, or a pull @@ -1496,12 +1496,9 @@ Scope: Tests: pragma values and a rejected foreign-key violation, on a directly built engine and through the application's `get_session()`; `alembic check` on a fresh database; config loading for valid, invalid, and default cases; the JUnit gate for each condition, including a report written by a -real all-skipped pytest run; the suppression counter for each spelling, string decoys, and each cap -at and over its limit. +real all-skipped pytest run; and the suppression counter. -Acceptance: CI passes on the exact PR head with a non-empty job list; removing the model index makes -CI fail; removing the pragma listener makes its test fail; routing `get_engine()` around -`build_engine()` makes a test fail; a wheel without `tracking.toml` fails the built-wheel job. +Acceptance: CI passes on the exact PR head with a non-empty job list. ### Step 2: GitHub source layer @@ -1541,20 +1538,19 @@ between two windows; each query document's point cost, computed from GitHub's do equals its 5.8 figure. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no -invariant check without a test that targets it; changing a query document without re-recording -fails the suite; the lock check fails on an edited or removed entry (tested against a real git -repository) and runs on the exact PR head; CI passes on the exact PR head. +invariant check without a test that targets it; the lock check runs on the exact PR head; CI passes +on the exact PR head. ### Step 3: New schema, PR sync engine, cutover -Ten pull requests. 3a lands the schema and removes the old pipeline, which imports the old models +Eleven pull requests. 3a lands the schema and removes the old pipeline, which imports the old models and so cannot outlive them; 3b-1 adds what the sync collects under D29 (the query documents, typed records, schema, and row-level views of D3-D6, D8, D14, and D28); 3b-2a-1 adds the planner and the configuration it reads; 3b-2a-2a adds the writer and the run records on that schema, so the writer is built against the final tables; 3b-2a-2b-1 adds the scenario model the engine's tests run against; 3b-2a-2b-2 adds the sync engine over the planner and the writer, `--dry-run` -included; 3b-2a-2c adds `ghactivity sync`, the seeded runs, and the live acceptance; 3b-2a-3 -applies D37 to the sync (D38); 3b-2b-1 adds +included; 3b-2a-2c adds `ghactivity sync`, the seeded runs, and the live acceptance; 3b-2a-2d +replaces the tests that planted inputs (D39); 3b-2a-3 applies D37 to the sync (D38); 3b-2b-1 adds the REST sampler that checks what the sync wrote; and 3b-2b-2 adds verify, whose live acceptance runs the sampler. "Step 3b-2" elsewhere in this plan means 3b-2a-1, the parts of 3b-2a-2, 3b-2a-3, and the two parts of 3b-2b together; "Step 3b-2a" means 3b-2a-1, 3b-2a-2, and 3b-2a-3; "Step @@ -1598,7 +1594,7 @@ count of the rows, files, or values it compared, so it cannot pass on nothing; a the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a -check catches it are replaced in Step 7. Each wait on a thread or subprocess has an explicit +check catches it are replaced in Step 3b-2a-2d. Each wait on a thread or subprocess has an explicit timeout. #### Step 3a: New schema and cutover @@ -1632,29 +1628,22 @@ to the dictionary); `docs/architecture.md`, `docs/data-model.md`, `docs/rate-lim and `docs/github-api-quirks.md` are marked as describing removed code until Step 7. Tests: the migration aborting when a tag table holds rows; its success on a seeded database at -`b3c4d5e6f7g8` and on a scratch copy of the tracked file; a planted failure in the second of two -revisions applied by one upgrade leaving the file at the revision the upgrade started from, with -neither applied; the migration connection reporting `foreign_keys=0`, and the upgrade refused with -nothing applied when env.py's connect listener reads anything else (the test plants, through -env.py's own engine with its DBAPI connect function patched, a connection already inside a -transaction with `foreign_keys=ON`; env.py accepts no caller-supplied connection, which would bypass -the listener and the one-transaction setup); a batch alter of `pull_requests` keeping its child +`b3c4d5e6f7g8` and on a scratch copy of the tracked file; the migration connection reporting +`foreign_keys=0`; a batch alter of `pull_requests` keeping its child rows, through env.py with a test-only revision that drops and recreates the views; env.py refusing the tracked file; `downgrade()` raising; `PRAGMA user_version` holding `schema_version`, the -`schema_version` lock failing on a consumer column or view column missing at head and on a consumer -column at head missing from the current entry, and the CI lock -step failing on a pair removed from a base entry (against a real git repository); the revision +`schema_version` lock and the CI lock step; the revision tuple equal to the chain; the schema comparison (5.1), CHECK names included; every foreign-key column leading an index; the timestamp type rejecting a naive value and converting an aware one to UTC; view rows, on seeded data, for a null-author PR, a null reviewer, `final_verdict` (a later `DISMISSED`, a later `COMMENTED`, order by review id), and `v_pr_commenters` excluding reviewers; the dictionary in both directions; only `source/` importing githubkit, and `source/` importing neither SQLAlchemy nor `db/`; `scripts/check_frozen_db.py` on -the pinned file and on a changed copy; `LOGGING__ROTATION` overriding `Settings().logging.rotation`; +the pinned file; `LOGGING__ROTATION` overriding `Settings().logging.rotation`; a usage error, an invalid `tracking.toml` (printing no traceback), and a missing token exiting 1 through the installed entry point; an unhandled exception through the installed entry point printing no local variable values, and every loguru handler configured with `diagnose` off (gap 75); -`github rate-limit` and `github test`; the placeholder check on planted calls and a string decoy; +`github rate-limit` and `github test`; the placeholder check; the default `DATABASE_URL` and `.env.example` naming `./github_activity.local.db`; and the anchored `.gitignore` exception (`github_activity.local.db` and a `github_activity.db` in a subdirectory ignored). @@ -1705,8 +1694,7 @@ and a commit linked to no account told apart from one whose lookup returned `NOT (`last_edited_at` set) with no editor kept as an editor row, a request row carrying its request ids, a reference D28 kept carrying `*_reported` false, and a commit row an `[automation].emails` entry matches carrying its `email_automation_reason`; the lock -failing on a changed -view digest and on a removed view (against a real git repository); the dictionary in both directions. +with view digests; the dictionary in both directions. Acceptance: an adversarial review of the change, by reviewers who did not write it, finds no invariant check without a test that targets it, no consumer surface that does what D29 forbids, @@ -1791,7 +1779,7 @@ writer. Tests: the writer gets direct tests, over SQLite files built by `alembic upgrade head` and seeded rows, with records built from recorded or synthetic units; the planner's are 3b-2a-1's. Cases: each write rule of 5.1 and 5.3 step 6 on both sides (D28's kept references for each kind of reference, -GitHub's value replacing the others, child rows replaced, and the validated totals), a planted +GitHub's value replacing the others, child rows replaced, and the validated totals), a `pr_user_tags` row surviving re-hydration and removed when its PR is deleted (D17), every item write on the writer's one connection, a re-hydration that returns a null editor clearing the stored one while a null author is kept, the merge state and stack written from a hydration, review @@ -1923,7 +1911,7 @@ returns, and the report a dry run prints. The seeded scenario runs below, and re Tests: `ghactivity sync` exiting 2 through a child interpreter that runs the installed package (`python -I -c`, the Step 3a pattern) with the replay patched in, while a usage error still exits 1, as a refused claim does with no `sync_runs` row; crash mid-run then resume with committed -items intact, by a child process that sends itself SIGKILL at a planted point under +items intact, by a child process that sends itself SIGKILL at a set point under `subprocess.run(timeout=...)`, which leaves its row `running`. One test runs a first sync of go-gdpr from recordings made for it, without the model, and checks exit 0 and each stored item's child rows against the recording's totals and node lists. A few fixed seeds each drive several runs over random @@ -1947,9 +1935,40 @@ syncs, the pull request records that PR's `updatedAt` and `mergeStateStatus` fro 8). An adversarial review of the change, by reviewers who did not write it, finds no invariant check without a test that targets it; CI passes on the exact PR head. +#### Step 3b-2a-2d: Tests without planted inputs (D39) + +Prerequisite: Step 3b-2a-2c has merged, and D39 is on `main`. + +Scope: tests and `docs/testing.md`; nothing under `src/`, `scripts/`, or `alembic/` changes, and no +rule loses its test. Every test merged before D39 that changes a file or a copy of one, or writes a +file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it, +is replaced under D39: a check that GitHub, the environment, or a caller can fail keeps a refusing +side built from D39's inputs, and a check that only a defect of this code could fail is tested on +its passing side, with the exact count of what it compared. Among them: the schema comparison's +changed copies of a migrated database (`tests/db/test_schema.py`); the migration tests' revisions +holding a failure, a foreign-key violation, or a batch alter that leaves a view in place, and their +connection put inside a transaction (`tests/db/test_migrations.py`); the frozen-file check on a +changed copy; the schema-version lock, the query lock, and the CI lock step on edited or removed +entries; the placeholder check's planted calls and decoy, the suppression counter's planted +spellings, decoys, and caps, and the import-boundary test's planted files (5.7); and the planted or +tampered inputs a search finds in `tests/test_cli_sync.py`, `tests/test_cli_app.py`, +`tests/test_declared_dependencies.py`, `tests/test_server_side_cursors.py`, +`tests/test_data_dictionary.py`, `tests/sync/test_writer.py`, and `tests/sync/test_connection.py`. +`docs/testing.md` drops its descriptions of the replaced tests. + +Tests: each rule a replaced test held has a test D39 allows, on both sides where GitHub, the +environment, or a caller can fail it, or on its passing side with the exact count it compared; the +pull request lists each test it replaces or removes, with the rule it held and the test that now +holds it. + +Acceptance: a search of `tests/` for `plant`, `tamper`, `decoy`, `corrupt`, and `mutat` finds none, +and each hit of a wider search the pull request runs is listed with why D39 allows it; an +adversarial review of the change, by reviewers who did not write it, finds no rule that lost its +test; CI passes on the exact PR head. + #### Step 3b-2a-3: D37 in the sync -Prerequisite: Step 3b-2a-2 has merged. +Prerequisite: Step 3b-2a-2d has merged. Scope: the sync rules D38 changes, with a migration that adds `sync_runs.refresh_failures`, `link_lookups`, `links_not_found`, `repo_filter`, and `entries`, `unknown` to the `trigger` CHECK, and the `account_node_ids` @@ -2441,12 +2460,6 @@ counts. A final audit of the whole repository, made once Steps 1-6 have landed, - dependencies in `pyproject.toml` that nothing imports; - scripts, hooks, and CI steps for mechanisms a step removed; - tests and fixtures of removed paths, and fixtures no test reads; -- tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, - a connection, or a decoy holding a flaw, to see whether a check catches it (among them - `tests/db/test_schema.py`'s planted differences, `tests/db/test_migrations.py`'s planted failure, - foreign-key violation, and the view-alter revision that leaves a view in place, the planted calls and decoys of the placeholder - check and the suppression counter, and 5.7's planted import files), each replaced under D39's - rule; - every note that defers something until a step that has landed (in code, docs, `.gitignore`, and hook comments). diff --git a/docs/testing.md b/docs/testing.md index 062f22d..6c8fa1d 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -26,7 +26,7 @@ How the test suite is built and what it counts as evidence. The plan of record i cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a - connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 7. + connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 3b-2a-2d. - **No skipped tests** (D22). CI runs `scripts/check_junit.py --max-skipped 0`, which fails a run where any test was skipped (pytest reports `xfail` as skipped) or none ran. - **Time.** Source tests pass a `FakeClock`, so waits and retries are asserted, never slept. @@ -88,14 +88,14 @@ tests/ - **The real environment.** Migration tests run the repository's `alembic.ini`, `env.py`, and revisions. `write_revision` adds a test-only revision after the head through `version_locations`, which is how a planted failure, a foreign-key violation, and a batch alter - under the views are run through `env.py` (tests from before D39, which Step 7 replaces). `env.py` is loaded fresh by every command, so its + under the views are run through `env.py` (tests from before D39, which Step 3b-2a-2d replaces). `env.py` is loaded fresh by every command, so its `MigrationRefused` is matched by class name. - **Seeding.** `tests/db/seed.py` writes rows through the models' tables with every column a test does not care about filled with a valid default; GitHub ids come from one process-wide counter, so separate transactions never collide. - **Schema comparison.** `schema_of` reads columns (`PRAGMA table_xinfo`), indexes, foreign keys, and every item of each table's definition, which is where constraint names and CHECK text - live; each reading is tested against planted differences (tests from before D39, which Step 7 + live; each reading is tested against planted differences (tests from before D39, which Step 3b-2a-2d replaces). - **Dictionary and schema version.** `tests/test_data_dictionary.py` compares `docs/data-dictionary.md` with a migrated database in both directions. From 6dbd2848112d7cb1ddc00660d31a813c35a014d3 Mon Sep 17 00:00:00 2001 From: chrishuie Date: Thu, 22 Oct 2026 06:00:08 -0600 Subject: [PATCH 14/15] docs(plan): close the review of a2fed22 and f55ae74, and of #27's D37 - Step 3b-2a-2d names every test file that plants by file (the engine's patched writer and prune, user_version and alembic_version edits, dropped indexes, the checkers' planted files, the wheel copy, edited query documents), states which inputs stay as D39 inputs, tests env.py's file-held violation refusal on its passing side, and accepts on a list of every test that writes or patches, with a word search over tests/**/*.py. - D39 and its four copies except configuration as well as stubs. - 6a tests verify --db-only's unknown-revision refusal on a file from a flawless test-only revision past head. - 3b-2a-3: the foreign_keys read-back through a fake DBAPI connection, and entries for an entry that resolves to another github_id. - D25 and 5.4: an unverified repository makes the exit 2; a rate limit only waits; association drift is a deliberate exception (D38); the timeline test's half-second entry states the model's comparison order. - 3b-2a-2b-2's history, 5.7, and the Step 3 overview follow. --- CLAUDE.md | 3 +- CONTRIBUTING.md | 2 +- docs/redesign-plan.md | 89 +++++++++++++++++++++++++++---------------- docs/testing.md | 2 +- 4 files changed, 60 insertions(+), 36 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 850b7d5..a531734 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,8 +43,7 @@ uv run pytest # CI also runs Python 3.12, `alembic check`, ## Test Evidence Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name -(redesign plan D39). No test, review, or acceptance changes a file or a copy of one (other than a -stub the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a +(redesign plan D39). No test, review, or acceptance changes a file or a copy of one (other than a stub or configuration the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6cc1c3a..54cb981 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -248,7 +248,7 @@ def test_reviewers(migrated_database: Path) -> None: # a copy of the alembic-bu **3. Never plant, tamper, or seed a flaw** (redesign plan D39). Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance -changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a row, +changes a file or a copy of one (other than a stub or configuration the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index c81ff42..221fdaf 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -79,7 +79,7 @@ Within scope the database collects and distinguishes data and never interprets i | D22 | The CI test gate allows no skipped tests (`scripts/check_junit.py --max-skipped 0`). A test that must skip in CI needs the allowance raised in the workflow, in the same change. pytest reports `xfail` as skipped, so the same applies to an `xfail`. | | D23 | Closing-issue links are read by the index as well as by hydration, and the stored links are compared with the index on every run, so a link added after merge, which does not change the PR's `updatedAt`, is captured: a PR whose indexed links differ from its stored links is planned for hydration (5.3 step 3). Hydration also reads which links GitHub lists as manually linked (`userLinkedOnly`), at no added cost (section 7). | | D24 | Conversation comments are stored one row per comment, its `github_id`, author, and times and never its text, so a comment by an account the database never stored is a row with a null author, never dropped (D28). Review bodies, inline review comment text, and commit messages are not stored either. | -| D25 | A run, scheduled or manual, publishes when its starting database is valid (the downloaded snapshot passed `verify --db-only`, or `bootstrap` started from an empty database), sync exited 0 or 2, verify exited 0 (no failing class, 5.4), and the run had no `repo` input or its release is not `db` (5.6). The manifest records the sync exit code and verify's counts per class. A final job that runs whatever the others did then marks the workflow failed if sync exited 2, verify failed, or nothing was published, so a failure is visible after the data is saved. | +| D25 | A run, scheduled or manual, publishes when its starting database is valid (the downloaded snapshot passed `verify --db-only`, or `bootstrap` started from an empty database), sync exited 0 or 2, verify exited 0 (no failing class and no unverified repository, 5.4), and the run had no `repo` input or its release is not `db` (5.6). The manifest records the sync exit code and verify's counts per class. A final job that runs whatever the others did then marks the workflow failed if sync exited 2, verify failed, or nothing was published, so a failure is visible after the data is saved. | | D26 | The Step 5 database is not committed; it becomes the first snapshot of release `db` (Step 6b). From Step 3 until 6c removes it, the tracked `github_activity.db` keeps its pre-Step-3 content: the default `DATABASE_URL` moves to a git-ignored file, and `scripts/check_frozen_db.py` fails when the file's SHA-256 differs from the digest it pins, as a pre-commit hook on the file (so CI's `--all-files` run checks it) and as a test; the tests also fail when git stores the file other than as one regular file, and the `tracked-db-no-sidecar` hook refuses a committed `-journal` or `-wal` file beside it, which SQLite would apply on the next open, and the `-shm` index that goes with a `-wal`. | | D27 | A stored repository no longer listed in `tracking.toml` is reported, with exit 2, on every run without `--repo` (5.3 steps 1 and 8), and its rows stay until `ghactivity sync --prune-untracked` deletes them; the prune, on the operator's explicit flag, removes rows section 1 puts out of scope (repositories not in the tracked list) and is recorded in `sync_runs.deletions_approved` (5.1). | | D28 | The database keeps a person attached to everything it attributed to them. A stored account stays stored after a lookup returns `NOT_FOUND` for it, with its last known login and type, and every reference to it stays: when GitHub later returns that actor as null, as a re-hydration does once GitHub stops returning the account, the writer keeps the stored reference of the same object (the PR's author and merger, the issue's author, the review's reviewer or the comment's author with the same `github_id`, this PR's commit with the same SHA, and the default-branch commit with the same repository and SHA, Step 4b), and the row's `*_reported` column records that GitHub returned no actor in that hydration (5.1). Other actor references (editors, requested reviewers, event actors) take GitHub's current value, null included, and the assignees are GitHub's current `assignedActors` list. `not_found_at` (named `deleted_at` until Step 3b-1) records when a lookup returned `NOT_FOUND`; it changes no reference, the 24-hour refresh skips the account, and a later sighting of the account clears it. GitHub does not say why it returns a null actor (`Prebid.js#13769` is merged, with a null `mergedBy` and an existing author), so a null reference means only that GitHub returned no actor and the object had none stored, and an account never stored has no `github_users` row. A non-null actor GitHub returns replaces the stored one. On a commit, `linked_github_id` holds the account id GitHub linked in the latest hydration whether or not a lookup found the account, so a link whose lookup returned null is told apart from an email linked to no account. The attribution lives in the stored rows and cascades with them, so anything that re-creates a row from GitHub drops it for accounts GitHub no longer returns: a bootstrap (it starts from an empty file), a transferred issue (new `github_id`), and a repository pruned and re-added (D27). | @@ -92,8 +92,8 @@ Within scope the database collects and distinguishes data and never interprets i | D35 | Engine decisions of 2026-10-20 (the maintainer): Step 3b-2a-2b is two pull requests, the scenario model and the engine with `--dry-run` (section 6). A run can stop at any point: every write is its own transaction and every close updates its row only while it is `running`, so whatever ends a run without its close leaves the row `running` for the stale-row rule or `--break-lock` to recover, and nothing shields a writer transaction from an interrupt (5.3, gap 93). The deadline is reached when a request or a wait is refused for it (5.3 step 4), and the engine hydrates and writes one batch at a time (5.3 step 6). An `AccountMissing` refusal is an item failure, so an engine defect still publishes (5.3 step 6, D25); two entries resolving to one repository fail only the later one (5.3 step 1, gap 92); and D27's exit 2 applies to runs without `--repo`. | | D36 | Verify decisions of 2026-10-21: Step 3b-2b is two pull requests, the REST sampler and then verify with the live acceptance (section 6). Verify writes nothing to the database and claims no run: before any request it reads `sync_runs` and exits 1 while a row holds the lock by the claim's rule, so a stale or crashed run stays for the next claim or `sync --break-lock` (5.3). `--repo` limits verify to the repositories it names, as it limits a sync (D27). A verify that never starts writes no report; once it starts, `--report` is written on every exit verify's own code handles, and one that cannot be written turns an exit 0 or 2 into 1, because Step 6a reads it, while an interrupt exits 130 either way; verify's printed output follows sync's rule (5.3 step 8). Every repository failure 5.3 gives a resolve, an index, or a hydration batch, a repository error during a timeline lookup, and from Step 4b a log or history read verify cannot complete make a repository unverified, and a verify in which every repository it attempts is unverified, the deadline having stopped none, is fatal. verify reads each repository's stored rows in two statements before indexing it, and from Step 4b its stored events and default-branch commits in one statement each before the first request of its log and history reads, and its report records its own points, REST calls, and time (5.4, gap 94). | | D37 | Decision of 2026-10-21 (the maintainer): an error is better than a silent failure or silently wrong data. Where the code cannot check something, or cannot accept what GitHub returned, it reports a failure that stops publication or turns the run red, never a pass. For verify: an unchecked item and an unverified repository fail verify in both modes, not only `--exact`, and a lookup whose answer verify cannot accept is an error (5.4, 5.8, gap 97), so D25 does not publish a database verify could not check. | -| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class and one cause, the first line of the failure's message, of an item, an account's refresh, or from Step 4b a compare, so a permanent failure can be accepted without turning every run red while a new cause is not. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter, each configured entry's `github_id`, and its first commit-link lookups and those GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the floor or at the log's first event, as a read in ascending order finds it, fails a history read whose count still differs after a rescan, and records a compare that still fails, which an entry can accept. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now counted per run and in the manifest, and from Step 6a a run whose `links_not_found` exceeds the share of its `link_lookups` Step 5 sets, with at least Step 5's minimum number of lookups, exits 2 (an account an earlier lookup answered `NOT_FOUND` counted in neither), since a lookup form GitHub stopped resolving would look the same per link), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (`items_deleted`, and from Step 4b `events_deleted` and `history_commits_deleted`; the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | -| D39 | Decision of 2026-10-21 (the maintainer). Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed into that shape; calls the code under test must refuse, and configuration it must refuse or that names a state GitHub can produce (a repository that does not resolve, one no longer listed); valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or source that only a defect of this code could fail is tested on its passing side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 3b-2a-2d. | +| D38 | Decisions of 2026-10-21 applying D37 to the sync and to the steps not yet built (gaps 98-110). The sync exits 2 when a 24-hour account refresh fails, when the run reaches its deadline, and when a hydration contradicts the run's own index after one retry (Step 3b-2a-3); the failures a batch made final before a deadline are recorded, not left pending. An `accepted_failures` entry accepts one error class and one cause, the first line of the failure's message, of an item, an account's refresh, or from Step 4b a compare, so a permanent failure can be accepted without turning every run red while a new cause is not. A PR with more closing links than the index carries is hydrated every run. Every node id a sighting, a lookup, or a commit link reports for a stored account is stored, and `sync_runs` records the `--repo` filter, each configured entry's `github_id`, and its first commit-link lookups and those GitHub answered `NOT_FOUND`. The source refuses what it cannot read (gap 104), and the run records refuse what they cannot write and mark `-unknown` what they cannot determine (gap 105). Step 4b marks a compare file list at GitHub's 300-file cap and counts it in verify's report and the manifest, completes an activity read only at the floor or at the log's first event, as a read in ascending order finds it, fails a history read whose count still differs after a rescan, and records a compare that still fails, which an entry can accept. Step 5 measures which edits leave `updatedAt` unchanged. Step 6 compares the App's reads with the token's before any App run publishes to `db`, pins the database file the job checked, tests that every exit-code `if:` compares a non-empty string, and ties a snapshot to the job's own sync. A schedule that stops is the one silent case this repository cannot see: no run starts, so nothing turns red; 5.6 states the staleness rule a consumer applies, and building it belongs to the dashboard's integration (section 9). Left as they are, because the stored data already says what happened: a commit-linked account a lookup answers `NOT_FOUND` (D28; now counted per run and in the manifest, and from Step 6a a run whose `links_not_found` exceeds the share of its `link_lookups` Step 5 sets, with at least Step 5's minimum number of lookups, exits 2 (an account an earlier lookup answered `NOT_FOUND` counted in neither), since a lookup form GitHub stopped resolving would look the same per link), a compare at GitHub's 300-file cap and one GitHub refuses (marked by `files_at_cap` and `http_status`, counted in verify's report and the manifest), a REST git author without a name or email (stored null, as GitHub reports none), deletions recorded as a count (`items_deleted`, and from Step 4b `events_deleted` and `history_commits_deleted`; the snapshots retained for 7 days hold the identities; older ones are not kept), which accounts failed a refresh (printed; the count is stored, the run is red unless an account entry accepts the failure, and the accounts are retried), an `author_association` difference (verify reports it as drift with its own count: GitHub computes association when read and changes it without moving `updatedAt`, section 7, so no read tells a change from a defect), and the merge state as of `hydrated_at` (D32). New tables and columns of GitHub-derived data on a populated file follow gap 110's rule (5.1). | +| D39 | Decision of 2026-10-21 (the maintainer). Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes a file or a copy of one (other than a stub or configuration the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing `git`, a clock), which a test may build in memory from a recorded unit changed into that shape; calls the code under test must refuse, and configuration it must refuse or that names a state GitHub can produce (a repository that does not resolve, one no longer listed); valid rows a test writes (through the models with `tests/db/seed.py`, or in SQL); and databases and files this project's own code wrote, at any revision. Each test asserts the exact result. A check over stored data, files, or source that only a defect of this code could fail is tested on its passing side, with the exact count of the rows, files, or values it compared, so it cannot pass on nothing; a check that GitHub, the environment, or a caller can fail is tested on both sides. A fix comes with a regression test that fails against the parent commit's code. Tests merged before D39 that change a file or a copy of one, or write a file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it are replaced in Step 3b-2a-2d. | ## 3. Coverage rule @@ -959,7 +959,8 @@ the first class that matches, in this order: hydrated PR an error, not retried: an index that omits a live PR is the read the sync's step 7 deletes by, so verify reports it); - unchecked: a differing item whose timeline lookup (5.8) or lookup by number still fails after the - retries on a request GitHub did not answer (a transport, server, timeout, or rate-limit failure), + retries on a request GitHub did not answer (a transport, server, or timeout failure; a rate limit only + waits, and a wait the deadline refuses ends verify as a timeout), or that a lookup had not answered when a repository error stopped it, those of the batch the error ended included, listed by number; - error: anything else, a lookup whose answer verify cannot accept included (a timeline lookup @@ -985,7 +986,7 @@ gave. An unverified repository fails verify and is reported with its reason; a v which every repository it attempts is unverified, and the deadline stopped none, is fatal (exit 1), as a sync is when no configured repository resolves (5.3 step 1). A verify that reaches `--max-minutes` stops with a timeout instead of waiting past it. Exit 0 when no failing class -occurred, 2 when one did, 1 on a fatal error or a timeout, and 130 on an interrupt, the CLI's code +occurred and no repository is unverified, 2 otherwise, 1 on a fatal error or a timeout, and 130 on an interrupt, the CLI's code (5.3). A verify that never starts exits 1 before any request, writes no report, and leaves PATH as it was: a usage error (`--repo` naming no configured repository, a single-valued option given twice), an invalid `tracking.toml`, missing credentials, the head check, a held lock, a database @@ -1244,8 +1245,8 @@ enabled afterwards. `source/` is the only package that imports githubkit; from Step 3, when `github/` is gone, a test fails when another module does. It scans `src/` outside `source/`, `scripts/`, and `alembic/` for imports at any scope, `TYPE_CHECKING` blocks included, and for `importlib.import_module` and -`__import__` called with a string literal, and fails when it scans no file; its test asserts the exact -count of the files and imports it scanned (D39, Step 3b-2a-2d). Tests are exempt, since they build githubkit's errors. A +`__import__` called with a string literal, and fails when it scans no file; from Step 3b-2a-2d its test +asserts the exact count of the files and imports it scanned (D39). Tests are exempt, since they build githubkit's errors. A second test fails when `source/` imports SQLAlchemy or `db/`, so fetchers cannot touch the database. | Disposition | Modules | @@ -1554,7 +1555,7 @@ replaces the tests that planted inputs (D39); 3b-2a-3 applies D37 to the sync (D the REST sampler that checks what the sync wrote; and 3b-2b-2 adds verify, whose live acceptance runs the sampler. "Step 3b-2" elsewhere in this plan means 3b-2a-1, the parts of 3b-2a-2, 3b-2a-3, and the two parts of 3b-2b together; "Step 3b-2a" means 3b-2a-1, 3b-2a-2, and 3b-2a-3; "Step -3b-2a-2" means 3b-2a-2a, the two parts of 3b-2a-2b, and 3b-2a-2c; "Step 3b-2a-2b" means 3b-2a-2b-1 +3b-2a-2" means 3b-2a-2a, the two parts of 3b-2a-2b, 3b-2a-2c, and 3b-2a-2d; "Step 3b-2a-2b" means 3b-2a-2b-1 and 3b-2a-2b-2; and "Step 3b-2b" means 3b-2b-1 and 3b-2b-2. The REST sampler and its recorder mode are 3b-2b-1's; verify (5.4), its timeline lookup, and the scenario model's timeline responses are 3b-2b-2's; the planner, its snapshot reader, step 7's candidate, guard, and `--allow-deletions N` @@ -1564,7 +1565,8 @@ file's two comment changes are 3b-2a-1's; `[automation]` and `[sync].accepted_fa loading, every write the sync makes, run records, the run lock, the head check, and the busy-timeout comment are 3b-2a-2a's; the scenario model and its round-trip test are 3b-2a-2b-1's; the command with its usage errors and the runs that never start, the seeded runs, the go-gdpr recordings, and -the live acceptance are 3b-2a-2c's; the changes D38 makes to the sync, its exit code included, +the live acceptance are 3b-2a-2c's; the D39 test replacements are 3b-2a-2d's; the changes D38 +makes to the sync, its exit code included, are 3b-2a-3's; and the rest, `--dry-run` and the exit code of a run that starts included, is 3b-2a-2b-2's. Until 3b-2a-2c `main` has no `sync` command and the tracked database stays frozen (D26). No sync has written the new schema before 3b-2a-2, so 3b-1's changes touch no @@ -1580,7 +1582,7 @@ states (the coverage floor, the 10-minute unsettled window, 10 minutes past `dea lock, the 24-hour refresh, the delete guard's max(25, 2%)) and regression tests that fail against the parent's code. D39 governs every test. Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance changes a file or a -copy of one (other than a stub the inputs below allow), or writes a file, a row, a revision, a call +copy of one (other than a stub or configuration the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a timeout, an interrupt, a failing @@ -1868,7 +1870,7 @@ rules are tested in 3b-2a-1 and 3b-2a-2a; these cases show the engine joins them review id stored under another PR, a PR number stored under another `github_id`); a duplicate repository label id; a repository gone during hydration; a database error in the writer and any other exception after the claim, each closing the run `failed`; an `AccountMissing` refusal - recorded as an item failure (the writer replaced by a fake that raises it), and no other engine + recorded as an item failure, and no other engine test leaving one in `sync_failures`; - accounts (D28): a bot seen first through a commit; an account deleted after it was stored (`not_found_at` set; its references kept through a re-hydration that returns it as null, each @@ -1944,16 +1946,31 @@ rule loses its test. Every test merged before D39 that changes a file or a copy file, a revision, a call, a connection, or a decoy holding a flaw, to see whether a check catches it, is replaced under D39: a check that GitHub, the environment, or a caller can fail keeps a refusing side built from D39's inputs, and a check that only a defect of this code could fail is tested on -its passing side, with the exact count of what it compared. Among them: the schema comparison's -changed copies of a migrated database (`tests/db/test_schema.py`); the migration tests' revisions -holding a failure, a foreign-key violation, or a batch alter that leaves a view in place, and their -connection put inside a transaction (`tests/db/test_migrations.py`); the frozen-file check on a -changed copy; the schema-version lock, the query lock, and the CI lock step on edited or removed -entries; the placeholder check's planted calls and decoy, the suppression counter's planted -spellings, decoys, and caps, and the import-boundary test's planted files (5.7); and the planted or -tampered inputs a search finds in `tests/test_cli_sync.py`, `tests/test_cli_app.py`, -`tests/test_declared_dependencies.py`, `tests/test_server_side_cursors.py`, -`tests/test_data_dictionary.py`, `tests/sync/test_writer.py`, and `tests/sync/test_connection.py`. +its passing side, with the exact count of what it compared. Among them, by file: +`tests/db/test_schema.py` (changed copies of a migrated database); `tests/db/test_migrations.py` (revisions holding a failure, a foreign-key violation, a `COMMIT` or +`END`, or a batch alter that leaves a view in place; the connection put inside a transaction; the +copies changed for the definition reading); `tests/db/test_tracked_database.py` (the frozen file's +changed copy); `tests/db/test_views.py` (indexes dropped on a copy); `tests/sync/test_engine.py` (a +writer patched to raise `AccountMissing` or a `RuntimeError`, a prune patched to raise, and +`alembic_version` set to a revision that does not exist); `tests/sync/test_runs.py` (`user_version` +set on a migrated copy); `tests/sync/test_writer.py` (a wrong `targets_default_branch` written into +a row); `tests/test_cli_sync.py` (`alembic_version` set to `'older'`); `tests/test_cli_app.py` (the +command written to raise `boom`); the schema-version lock, the query lock, and the CI lock step on +edited or removed entries (`tests/test_data_dictionary.py`, `tests/test_check_query_lock.py`, and +`tests/source/test_recordings.py`'s edited query documents); and the planted calls, spellings, +decoys, alias files, subprocess calls, and package copies of the checkers' tests +(`tests/test_check_log_placeholders.py`, `tests/test_count_suppressions.py`, +`tests/test_check_any_aliases.py`, `tests/test_suite_rules.py`, `tests/test_import_boundaries.py`, +`tests/test_server_side_cursors.py`, `tests/test_declared_dependencies.py`, and +`tests/test_wheel_smoke.py`). A file at an older schema comes from a real earlier revision, as +`tests/sync/test_runs.py` builds one at `33f8570f6540`. env.py's refusal of a file that already +holds a foreign-key violation is tested on its passing side: a command over a file of seeded rows +succeeds, and the test counts the rows of that file the check read. These are D39 inputs and keep +their tests, renamed where a name says planted: a recorded unit `replay.modify` changes into a shape +GitHub can send (`FAIL_ONE_ITEM`), a kill or an interrupt at a set point, an exception a caller +raises in a transaction body (`tests/sync/test_connection.py`), the CLI's exit-code test commands, +valid tables in a scratch database (`tests/test_data_dictionary.py`'s `planted.db`), the tracked-file +check's WAL, non-database, symlink, and sidecar inputs, and a size limit patched to a small value. `docs/testing.md` drops its descriptions of the replaced tests. Tests: each rule a replaced test held has a test D39 allows, on both sides where GitHub, the @@ -1961,10 +1978,13 @@ environment, or a caller can fail it, or on its passing side with the exact coun pull request lists each test it replaces or removes, with the rule it held and the test that now holds it. -Acceptance: a search of `tests/` for `plant`, `tamper`, `decoy`, `corrupt`, and `mutat` finds none, -and each hit of a wider search the pull request runs is listed with why D39 allows it; an -adversarial review of the change, by reviewers who did not write it, finds no rule that lost its -test; CI passes on the exact PR head. +Acceptance: the pull request lists every test under `tests/**/*.py` that writes a source file, +a revision, a row, a database, or a package copy, changes a recorded unit, or patches a call or a +constant, with the D39 input it uses or the test that replaced it; a search of `tests/**/*.py` +(recorded GitHub text under `tests/fixtures/` is outside it) for `plant`, `tamper`, `decoy`, +`corrupt`, and `mutat` finds none; an adversarial review of the change, by reviewers who did not +write it, finds no rule that lost its test and no test outside that list that holds a flaw; CI +passes on the exact PR head. #### Step 3b-2a-3: D37 in the sync @@ -2054,7 +2074,8 @@ reported under another `databaseId` failing the item; a commit-link lookup answe counted in `link_lookups` and `links_not_found`, and one for an account an earlier run's lookup answered `NOT_FOUND` counted in neither; the report listing each index contradiction with its kind and whether the retry cleared it; `entries` recording each configured entry's `github_id`, a -renamed repository's included, and null for one that never resolved; each source check accepting the shapes GitHub's recordings hold and refusing each shape +renamed repository's included, null for one that never resolved, and the stored row's +`github_id` for an entry that resolves to another; each source check accepting the shapes GitHub's recordings hold and refusing each shape its rule names (a 2xx REST response whose body is JSON and whose `Date` is readable but without rate-limit headers; an `errors` of `{}`, `""`, and `null`; an index page whose repository `databaseId` differs; a commit page from other positions; each built in memory from a recorded unit with @@ -2067,7 +2088,9 @@ with `git status`, `ls-files`, and `rev-parse` failing or timing out inside a ch (`-unknown`); `trigger` for `schedule`, `workflow_dispatch`, another event name, and none; a close given an outcome with no stored row leaving the row `running`; a `stamp` to another revision and a `stamp --purge` at head refused with nothing committed, and a `stamp` at head accepted; the -migration refusing a populated file; a connection whose `foreign_keys` reads back 0 refused; the +migration refusing a populated file; a connection whose `foreign_keys` reads back 0 refused, through a fake DBAPI connection answering +`PRAGMA foreign_keys` with 0, as SQLite built without foreign-key support does (an environment +stub); the wheel check reporting the SHA-256 of the packaged `tracking.toml` it compared, equal to the source file's; a record at the log file's level reaching it and one below not; `repo_filter` written for a `--repo` run, sorted, with each value as given, and null without one; a file without @@ -2172,8 +2195,9 @@ timeline lookup through the replay: a connected event naming the differing issue another issue, a link difference made by a closing keyword with no event, a difference in the link totals alone with enough connected events, with only a disconnected event, and with too few, a synthetic -disconnected event, an event at the boundary counted and one a second before it not, entries -the model renders at a second and at half a second before the boundary, not counted, a stored link no longer indexed on a PR whose index lists every link matched only by an +disconnected event, an event at the boundary counted and one a second before it not, an entry +the model holds half a second before the boundary, which it compares with `since` before writing +its time in whole seconds, not counted, a stored link no longer indexed on a PR whose index lists every link matched only by an event naming it, a filtered page whose `totalCount` differs from its entries, counted by its entries, a lookup whose entries differ from its `filteredCount` after its retry and one whose response the typed models reject, each an error, one that differs once and matches on its retry, classed by its events, and a @@ -2377,7 +2401,8 @@ on an asset digest that names another file (a stub of GitHub's release-asset res snapshot's gz given with another snapshot's manifest, both written by `snapshot`, its decompression, `integrity_check`, and `foreign_key_check` refusals, which once the digest matches only a defect of `snapshot` could reach, tested on their passing side (D39), and its unknown-revision -refusal on its passing side too, since no code at 6a's head writes a revision after its own head; a release with no complete snapshot pair halting with exit 1; `snapshot +refusal on a file `snapshot` wrote after an upgrade through a test-only revision past head that +holds no flaw, as newer code writes one, halting with exit 1 and naming the revision check; a release with no complete snapshot pair halting with exit 1; `snapshot --verify-report` exiting 1 on a missing report, on a file that is not a verify report (the gz `snapshot` wrote), and on one from a verify of an earlier sync, whose `last_run_id` differs from the file's, and the manifest listing the report's unverified repositories (5.6); through `httpx.MockTransport`, a mocked @@ -2448,8 +2473,8 @@ records at least two mints; against release `db-test`, the snapshot one migratio upgrades and publishes, the snapshot with an untracked repository publishes and the workflow is marked failed, and a re-run of `publish` after an interrupted upload completes; the run from a branch whose `tracking.toml` names a repository that does not resolve publishes nothing, verify reporting -that repository unverified, and is marked failed; `verify --db-only`'s digest and manifest refusals are -covered by the 6a tests. +that repository unverified, and is marked failed; `verify --db-only`'s digest, manifest, and revision refusals +are covered by the 6a tests. ### Step 7: Documentation rewrite and code audit diff --git a/docs/testing.md b/docs/testing.md index 6c8fa1d..e721b7c 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -12,7 +12,7 @@ How the test suite is built and what it counts as evidence. The plan of record i Databases are real SQLite files built by `alembic upgrade head`. - **Direct evidence, nothing planted** (redesign plan D39). Nothing is planted or tampered, no flaw is seeded, and there is no mutation testing, under any name. No test, review, or acceptance - changes a file or a copy of one (other than a stub the inputs below allow), or writes a file, a + changes a file or a copy of one (other than a stub or configuration the inputs below allow), or writes a file, a row, a revision, a call site, or a decoy holding a flaw, to see whether something catches it, and none measures how many such flaws are found. A test's or an acceptance's inputs are GitHub's recorded responses; stubs of a response GitHub or the environment can send (a status, a body, a From 8f0a75998022f165aac9f2c3b1ab5d06d9b238da Mon Sep 17 00:00:00 2001 From: chrishuie Date: Thu, 22 Oct 2026 06:20:35 -0600 Subject: [PATCH 15/15] docs(plan): close the convergence check of 6dbd284 - 3b-2a-3: SQLite built without foreign-key support answers `PRAGMA foreign_keys` with no row, not 0; the listener refuses a missing read-back, tested with a fake DBAPI connection that returns no row. - 3b-2a-2d names test_snapshot.py's half-read cursor control, test_views.py's row-id projection control, the authorizer and ATTACH revisions, and the revision-rule checker's planted files; the acceptance lists every test that runs a call or query holding a flaw, and its review finds no test that holds one; the patched size limit becomes an oversized in-memory unit; the patched prune's interrupt case stays as an interrupt at a set point. - 3b-2b-2: no model entry at half a second; under the measured since and whole-second times verify's createdAt filter is tested on its passing side. - 5.4: without --max-minutes a rate-limit wait over the named constant is fatal. --- docs/redesign-plan.md | 39 +++++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 16 deletions(-) diff --git a/docs/redesign-plan.md b/docs/redesign-plan.md index 221fdaf..78bf06e 100644 --- a/docs/redesign-plan.md +++ b/docs/redesign-plan.md @@ -959,8 +959,9 @@ the first class that matches, in this order: hydrated PR an error, not retried: an index that omits a live PR is the read the sync's step 7 deletes by, so verify reports it); - unchecked: a differing item whose timeline lookup (5.8) or lookup by number still fails after the - retries on a request GitHub did not answer (a transport, server, or timeout failure; a rate limit only - waits, and a wait the deadline refuses ends verify as a timeout), + retries on a request GitHub did not answer (a transport, server, or timeout failure; a rate limit waits: + a wait the deadline refuses ends verify as a timeout, and without `--max-minutes` a wait longer + than the named constant is fatal, 5.3 step 4), or that a lookup had not answered when a repository error stopped it, those of the batch the error ended included, listed by number; - error: anything else, a lookup whose answer verify cannot accept included (a timeline lookup @@ -1948,10 +1949,12 @@ is replaced under D39: a check that GitHub, the environment, or a caller can fai side built from D39's inputs, and a check that only a defect of this code could fail is tested on its passing side, with the exact count of what it compared. Among them, by file: `tests/db/test_schema.py` (changed copies of a migrated database); `tests/db/test_migrations.py` (revisions holding a failure, a foreign-key violation, a `COMMIT` or -`END`, or a batch alter that leaves a view in place; the connection put inside a transaction; the +`END`, a statement the authorizer refuses, an `ATTACH`, or a batch alter that leaves a view in +place; the revision-rule checker's planted files; the connection put inside a transaction; the copies changed for the definition reading); `tests/db/test_tracked_database.py` (the frozen file's -changed copy); `tests/db/test_views.py` (indexes dropped on a copy); `tests/sync/test_engine.py` (a -writer patched to raise `AccountMissing` or a `RuntimeError`, a prune patched to raise, and +changed copy); `tests/db/test_views.py` (indexes dropped on a copy, and a row id projected under a consumer name); +`tests/sync/test_snapshot.py` (a server-side cursor left half-read); `tests/sync/test_engine.py` (a +writer patched to raise `AccountMissing` or a `RuntimeError`, a prune patched to raise a `RuntimeError`, and `alembic_version` set to a revision that does not exist); `tests/sync/test_runs.py` (`user_version` set on a migrated copy); `tests/sync/test_writer.py` (a wrong `targets_default_branch` written into a row); `tests/test_cli_sync.py` (`alembic_version` set to `'older'`); `tests/test_cli_app.py` (the @@ -1970,7 +1973,9 @@ their tests, renamed where a name says planted: a recorded unit `replay.modify` GitHub can send (`FAIL_ONE_ITEM`), a kill or an interrupt at a set point, an exception a caller raises in a transaction body (`tests/sync/test_connection.py`), the CLI's exit-code test commands, valid tables in a scratch database (`tests/test_data_dictionary.py`'s `planted.db`), the tracked-file -check's WAL, non-database, symlink, and sidecar inputs, and a size limit patched to a small value. +check's WAL, non-database, symlink, and sidecar inputs. `tests/source/test_recorder.py`'s file-size +limit patched to a small value is replaced by a recorded unit given, in memory, a body over +`MAX_FILE_BYTES`. `docs/testing.md` drops its descriptions of the replaced tests. Tests: each rule a replaced test held has a test D39 allows, on both sides where GitHub, the @@ -1979,11 +1984,12 @@ pull request lists each test it replaces or removes, with the rule it held and t holds it. Acceptance: the pull request lists every test under `tests/**/*.py` that writes a source file, -a revision, a row, a database, or a package copy, changes a recorded unit, or patches a call or a -constant, with the D39 input it uses or the test that replaced it; a search of `tests/**/*.py` +a revision, a row, a database, or a package copy, changes a recorded unit, patches a call or a +constant, or runs a call or a query holding a flaw, with the D39 input it uses or the test that +replaced it; a search of `tests/**/*.py` (recorded GitHub text under `tests/fixtures/` is outside it) for `plant`, `tamper`, `decoy`, `corrupt`, and `mutat` finds none; an adversarial review of the change, by reviewers who did not -write it, finds no rule that lost its test and no test outside that list that holds a flaw; CI +write it, finds no rule that lost its test and no test that holds a flaw; CI passes on the exact PR head. #### Step 3b-2a-3: D37 in the sync @@ -2030,7 +2036,8 @@ schema-version lock entry's consumer pairs: - the run-record and setup checks of gap 105: `code_version` as 5.1 defines it, with `_git` returning the exit status; a close given an outcome it cannot write raises before writing; `[automation]` is required; the connect listener reads `foreign_keys` back and refuses a - connection where it is off; `alembic/env.py` refuses a `stamp` that would change the revision; + connection where it is off or where the read-back returns no row (SQLite built without + foreign-key support); `alembic/env.py` refuses a `stamp` that would change the revision; the wheel check compares the packaged `tracking.toml` bytes with the source file's; the log-file handler receives every record at its level; `sync_runs.repo_filter` records `--repo`; and `trigger` records `unknown` for a `GITHUB_EVENT_NAME` other than `schedule` and @@ -2088,9 +2095,9 @@ with `git status`, `ls-files`, and `rev-parse` failing or timing out inside a ch (`-unknown`); `trigger` for `schedule`, `workflow_dispatch`, another event name, and none; a close given an outcome with no stored row leaving the row `running`; a `stamp` to another revision and a `stamp --purge` at head refused with nothing committed, and a `stamp` at head accepted; the -migration refusing a populated file; a connection whose `foreign_keys` reads back 0 refused, through a fake DBAPI connection answering -`PRAGMA foreign_keys` with 0, as SQLite built without foreign-key support does (an environment -stub); the +migration refusing a populated file; a connection whose `foreign_keys` read-back returns no row refused, through a fake DBAPI connection +that answers `PRAGMA foreign_keys` with no row, as SQLite built without foreign-key support does +(an environment stub), and one that reads back 1 accepted; the wheel check reporting the SHA-256 of the packaged `tracking.toml` it compared, equal to the source file's; a record at the log file's level reaching it and one below not; `repo_filter` written for a `--repo` run, sorted, with each value as given, and null without one; a file without @@ -2195,9 +2202,9 @@ timeline lookup through the replay: a connected event naming the differing issue another issue, a link difference made by a closing keyword with no event, a difference in the link totals alone with enough connected events, with only a disconnected event, and with too few, a synthetic -disconnected event, an event at the boundary counted and one a second before it not, an entry -the model holds half a second before the boundary, which it compares with `since` before writing -its time in whole seconds, not counted, a stored link no longer indexed on a PR whose index lists every link matched only by an +disconnected event, an event at the boundary counted and one a second before it not (under the measured `since` +and whole-second times, verify's own `createdAt` filter drops no entry the lookup returns, so it +is tested on its passing side, every returned entry counted), a stored link no longer indexed on a PR whose index lists every link matched only by an event naming it, a filtered page whose `totalCount` differs from its entries, counted by its entries, a lookup whose entries differ from its `filteredCount` after its retry and one whose response the typed models reject, each an error, one that differs once and matches on its retry, classed by its events, and a