Skip to content

Reuse publication metadata: 15.7% less complete-check time (no κ) - #6162

Merged
Mister-Meeseeks merged 3 commits into
CrocSwap:mainfrom
petalas:research/w11-1022ak-boundary-obstruction
Oct 28, 2026
Merged

Mister-Meeseeks merged 3 commits into
CrocSwap:mainfrom
petalas:research/w11-1022ak-boundary-obstruction

Conversation

@petalas

@petalas petalas commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Measured verifier improvement with a scoped acceptance-equivalence argument and finite controls; no new kappa. Reuse the committed SOURCES.json bytes already authenticated by the complete publication byte/history check.

 exact working/committed byte comparison and final reader-status check
- reread committed SOURCES.json through two extra Git processes
+ reuse its already compared immutable payload, bounded by 16 MiB
 unchanged metadata validation and raw source-pin ancestry checks

On the unchanged selected P104 package, median complete check() time falls from 120.33 ms to 101.49 ms, a 15.66% reduction. On a small stage-zero package it falls from 121.65 ms to 99.36 ms, an 18.33% reduction. Nine alternating runs per arm on each package have separated ranges and identical complete receipts; all 36 timed checks succeed. These are warm-process complete-function measurements on CPython 3.12.15/macOS arm64. Imports, baseline extraction and CLI startup are excluded from both arms. No scientific verifier is timed or audited.

This contribution was produced by Nick Petalas. Credit ywz65's byte preflight, Nick Petalas's earlier history check, roger-sato's immutable-source isolation, Douglas Colkitt's package runner and the retained reader contributors. The reuse technique is ordinary; the contribution is its bounded integration, failure controls and measured complete-check improvement. Advance CLAIM.

Evidence

  • Before: duplicate metadata reading accounts for about 21 ms in a separate 130 ms profile, dominated by synchronous Git operations. For the measured one-pin packages the change removes two of nine Git processes.
  • After: 42 tests pass normally and under optimized Python, with no skips. The final registered commands take 9.78 s and 10.02 s against 120 s limits. Complete hash-bound receipts agree. Controls cover byte drift, malformed committed metadata, final-reader failure ordering, exact size bounds, older refs, cross-call invalidation, unavailable/noncommit/side-branch pins, shallow history, replacements, grafts, modes and Git errors/timeouts.
  • PASS_REGISTERED_RESEARCH_PACKAGE, committed bytes, executable modes and declared source-history checks pass. The publication preflight checks 16 files at 6021973.
  • Official export and all nine declared input declarations pass. The exported-tree verifier also passes all 42 tests outside the project checkout, without project Git metadata. Git remains available for synthetic test fixtures.
  • Static inspection confirms that the extracted metadata/history validator's AST is unchanged. Strict-warning syntax/control checks pass. No additional linter is installed locally.

From the worktree root:

/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/publication-metadata-reuse/verify.py
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -O -W error -B research/publication-metadata-reuse/verify.py
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/publication-metadata-reuse
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/publication-metadata-reuse --check-source-history --check-modes
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/publication-metadata-reuse/benchmark.py --scratch /absolute/external/scratch --repeats 9

Elsewhere substitute CPython 3.12. The optional benchmark needs full history, an existing external scratch directory, both named benchmark packages and their VERIFY.json inputs materialized first. It obtains the original checker from canonical base 721c03822c421b7f7fcbb2598993b947c4d88419 and emits every sample, complete receipt and implementation hash. The generated original measurement receipt is retained below.

Limitations and unrun obligations

The stable-checkout contract remains required. At most 16 MiB of committed metadata remains live for one check; the existing full-blob reader allocation is unchanged. The standalone source_history() boundary still fetches its own committed metadata. The two removed Git operations no longer have failure events; errors from retained operations still propagate. The proof assumes successful immutable object reads, rather than identical subprocess traces.

No second-host performance, native Windows mode execution, selected scientific cold replay, physical multiplier, supplier admission or complete multiplication theorem is claimed. Independent human and formal proof-kernel review are unclaimed. The five selected framework assumptions remain.

Open next questions

Measure the same complete-check workload on a second operating system and Git implementation, retaining complete receipts and alternating samples. A second host and matching materialized inputs are required. No combined speedup with separate preflight optimizations is claimed.

Related open work

  • #6155 removes eager mismatch-only SHA256 work in the same check() function in scripts/check_research_commit.py. Its comparison hunk around lines 130–139 overlaps this PR's hunk around 134–144. Whichever lands second will likely need a merge-conflict resolution. The mechanisms affect different cost stages; fixtures and timed boundaries differ, so their percentages are not compared or combined.
  • #6156 bounds working-source comparison memory in scripts/research_package.py. It changes another shared verifier script under the same replay-runtime card; this PR retains that module and claims no combined memory or timing benefit.
  • #6161 adds a numeric-reference prefilter in scripts/find_overlap.py, also under replay-runtime. It measures overlap search rather than publication checking. No common performance number is compared or combined.
  • #6164 supplies an opt-in adapter that batches source-commit resolution and ancestry queries. This PR removes duplicate committed-metadata reads while retaining those queries. No composed speedup is claimed.

The completion refresh checked this account's open PRs created since the CLAIM. The related shared-verifier submissions are listed above. No additional computations were run to compare their results.

CI notes

Uncertified by CI at head 60219737c80280a0bab577fe80f10b8c7b4fa22b. In run 40726988089, plan was cancelled after reaching the workflow's timeout-minutes: 10 limit. Its recorded lifetime is 16:32:40–16:42:42 UTC on October 28, 2026. The package replay was skipped and did not run in CI.

The job metadata and log locate the cancellation in actions/checkout during the repository fetch. Onboarding helper tests is marked skipped, not failed. Thus this run does not show whether the edits to scripts/check_research_commit.py pass CI's onboarding tests. No claim that this change introduced no onboarding failure follows from this cancelled run. The checkout timeout is independent of the checker implementation.

Separately, canonical main 721c03822c421b7f7fcbb2598993b947c4d88419, the PR6030 kappa-admission commit pushed with [skip ci], changed the selected record without updating its documentation contracts. Even without the timeout, the unchanged main baseline would fail these two onboarding tests:

  • test_research_current_status.CurrentStatusTests.test_current_section_matches_selected_record expects standalone **0.003125804** in docs/research/current-status.md; the document instead bolds 0.003125804 = 781451/250000000 as one expression.
  • test_research_current_status.CurrentCiNavigationTests.test_current_ci_section_matches_selected_record expects make frontier6030-verify; the actual test reads docs/ci-verification.md, whose current CI section still names Raise conditional kappa to 0.00310045 with independent helper splits #5599 and make frontier5599-verify.

Both PR #6150 run 40724809789 and PR #6120 run 40691776027 report exactly those two failing tests. Their logs were read; they are separate evidence, not results of this PR's cancelled onboarding step. Open PR #6050 targets the documentation fix.

A maintainer re-run is needed after main is fixed. This hosted run certified none of the package checks. The 42 normal/optimized local controls, registered replay, committed-byte/mode/source-history checks and exported replay remain separate passing local evidence. No push, CI re-run or main-document edit was made after this cancellation.

Merge Danger

Door: two-way. Blast Radius: publication-checker. The production diff retains one bounded immutable payload and extracts the unchanged history validator. Byte-only checking and standalone history checking retain their interfaces. Predecessor source pins and the selected frontier are unchanged.

Generated complete-check benchmark receipt
{
  "base_commit": "721c03822c421b7f7fcbb2598993b947c4d88419",
  "baseline_sha256": "63026ec3805f2a3236cc071fc78cf1c54bc6528c43e2f3d82548dc68bf346bce",
  "candidate_sha256": "4cbbf428194fcb7e0fe0845dc798ae65ec3b015e858701275106bb3d4c1082ef",
  "order": "baseline,candidate repeated",
  "platform": "macOS-27.0.1-arm64-arm-64bit",
  "python": "3.12.15 (main, Oct 14 2026, 00:55:37) [Clang 23.1.3 ]",
  "results": [
    {
      "complete_receipt": {
        "commit": "721c03822c421b7f7fcbb2598993b947c4d88419",
        "files": 6,
        "package": "research/stage-zero-forced-translation",
        "source_history_commits": [
          "2a13eb1827784206202db90e5dc8327cf8a250db"
        ]
      },
      "errors": 0,
      "package": "research/stage-zero-forced-translation",
      "samples": {
        "baseline": {
          "median_seconds": 0.12165470817126334,
          "range_seconds": [
            0.11786337499506772,
            0.13298737513832748
          ],
          "seconds": [
            0.13298737513832748,
            0.12591566587798297,
            0.12056225002743304,
            0.12284633284434676,
            0.12115795817226171,
            0.12165470817126334,
            0.11969016608782113,
            0.11786337499506772,
            0.12853012513369322
          ]
        },
        "candidate": {
          "median_seconds": 0.09935891698114574,
          "range_seconds": [
            0.09792166599072516,
            0.10272229090332985
          ],
          "seconds": [
            0.10272229090332985,
            0.09801095817238092,
            0.09861549991182983,
            0.09792166599072516,
            0.09847941691987216,
            0.10203716694377363,
            0.09935891698114574,
            0.10074541694484651,
            0.10141620808281004
          ]
        },
        "fraction_reduced": 0.18327109180788936
      },
      "successful_checks": 18
    },
    {
      "complete_receipt": {
        "commit": "721c03822c421b7f7fcbb2598993b947c4d88419",
        "files": 16,
        "package": "research/p104-helper470-twin-rung-removal",
        "source_history_commits": [
          "19fdb086ea96af9a9f87936a2b888e10af4ed810"
        ]
      },
      "errors": 0,
      "package": "research/p104-helper470-twin-rung-removal",
      "samples": {
        "baseline": {
          "median_seconds": 0.12033345806412399,
          "range_seconds": [
            0.11824208311736584,
            0.12616533297114074
          ],
          "seconds": [
            0.12616533297114074,
            0.1256263330578804,
            0.12033345806412399,
            0.12272116588428617,
            0.11878012493252754,
            0.12238483293913305,
            0.1201730000320822,
            0.11937200021930039,
            0.11824208311736584
          ]
        },
        "candidate": {
          "median_seconds": 0.10148520790971816,
          "range_seconds": [
            0.09892558283172548,
            0.1039622500538826
          ],
          "seconds": [
            0.1039622500538826,
            0.10345308296382427,
            0.10156391700729728,
            0.1007089160848409,
            0.09955858415924013,
            0.10287175001576543,
            0.09906270797364414,
            0.10148520790971816,
            0.09892558283172548
          ]
        },
        "fraction_reduced": 0.15663349543534155
      },
      "successful_checks": 18
    }
  ],
  "runner_sha256": "84a45190a6a96c20993a0d4a1045fbf24de660d2e6fef8f8a447bb3d1aa78db6",
  "schema": "publication-metadata-reuse-benchmark/v1",
  "timed_region": "complete check(); imports and baseline extraction excluded"
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants