chore: update generated-code verification and preserve SDK helpers

.castiron.stats.yml

@@ -1,8 +1,8 @@
 schema_version: 1
-generation_id: 491e58ee-627a-4a57-91d7-2d5261cecfe9
+generation_id: a993ea45-7f4f-4d48-9eea-27e2f709ca7c
 openapi_spec_hash: 1f2d498f08bdd6dacffeb41cee8f44e5
 openapi_transformed_spec_hash: 77075630a8655ccbb8d1f790f19f5731
-config_hash: d356dda7ff9ca3fbc614107eb546de9f
-codegen_sha: eeef8c0a76fd3885ff1da6b83ed2ba0ef6531414
-codegen_hash: ccac0bcc0ccfd20617a6e93498d22157f63843c1288cf775b7f3d5d393e49358
-public_codegen_sha: 40b2e7c300353a5cc5c8f8992e6616e999f4ecad
+config_hash: 7a8181d81e1b9225f8ee6f09171d712e
+codegen_sha: 6f2038ca576a63929de017d9145a842c3a09beb1
+codegen_hash: a761b2892108918f3ad49d4929a6b756fa023b3192025bbc2bbcf618dea6c13a
+public_codegen_sha: 9fdcc5c60f9247fe11ec25aaa0f486033f738421

.github/workflows/castiron-custom-code.yml

@@ -14,11 +14,11 @@ permissions:
   contents: read
 
 concurrency:
-  group: castiron-custom-code-${{ github.event.pull_request.number || github.event.merge_group.head_ref }}
+  group: castiron-custom-code-${{ github.event_name }}-${{ github.event.pull_request.number || github.event.merge_group.head_ref }}
   cancel-in-progress: false
 
 env:
-  REPORTER_SHA256: a52a1aabf4047f4c20e96d1d788c25a4aa78c1987b4fa73a13d22c92ef213a3f
+  REPORTER_SHA256: c243015818640b1e7068605c5352fcb770d406202ee4e75316fe0ca85d06a1cc
 
 jobs:
   queue-signal:

docs/castiron-generated-code-transition.md

@@ -6,7 +6,25 @@ Eliminate handwritten patches in generated-owned files while preserving every ex
 
 No removals or renames of public symbols, new required arguments, narrower input/return types, dependency or Python-floor changes, altered defaults, new payload limits, changed warnings, or changed sync/async behavior are authorized by this transition. Deprecated methods and compatibility import paths remain supported.
 
-## Current public baseline and measured final candidate — 2026-10-22
+## Verified public baseline — 2026-10-26
+
+The strict public report at `904ed3f6ba0f2351af67c9456e9f020401a8a7f8` measured **0 generated files with custom edits / 0 custom patch lines**. These counts carry forward to public main `06dbb5d019d7b6870b5898bf5df579ca60b61d28`: a full-tree comparison found only a change to the SDK-owned `create-releases.yml` workflow, outside generated ownership. All other paths, including all **2,025 generated manifest entries**, retain identical bytes and Git modes. The report verified the generated snapshot hash; its two omitted generated workflows were separately authenticated against the manifest and private checkpoint.
+
+The original `cfa0a79b6395bb0561b28879122a9f90a87cdada` audit measured **41 mixed files / 7,409 custom patch lines**, comprising 4,890 additions and 2,519 deletions. Patch lines measure differences from generated output, including removed lines; they are not the total size of handwritten runtime modules or compiler templates.
+
+## Current incremental plan
+
+The mixed-file count reached zero before this follow-up. Remaining work simplifies how the compiler expresses the preserved SDK interfaces: ordinary templates consume native compiler facts, while runtime algorithms remain in wholly handwritten modules. Do not introduce a new facade framework or developer-facing breaking changes.
+
+The first slice covers Chat parsing/streaming and chunked uploads, with separate compiler reviews followed by private SDK adoption and public promotion where there is an actual SDK change. Subsequent small slices cover Files/Artifacts, Sessions, Threads/Runs, Responses, and client/package wiring, followed by removal of unused declaration machinery after a compatibility audit.
+
+The remaining inventory contains nine declaration blocks totaling **16,555 authored YAML lines**, plus **75 residual binding lines** tracked separately. These measure compiler configuration still to simplify, not custom patches in SDK files.
+
+For each slice, compare generated output against the existing implementation using the same API/configuration inputs. Preserve method signatures, annotations, defaults, imports, sync/async behavior, and runtime algorithms. Review any other generated drift separately, run the relevant compatibility and repository checks, keep the verified mixed-file and patch-line counts at zero, and satisfy ordinary review and merge gates.
+
+The dated audits and original work tracks below are historical records. Their pending-delivery statements and proposed declaration machinery describe the earlier approach; this incremental plan supersedes them.
+
+## Historical public baseline and candidate — 2026-10-22
 
 The released baseline `cfa0a79b6395bb0561b28879122a9f90a87cdada` (3.37.0) contains **41 mixed files / 7,409 custom patch lines** (4,890 additions and 2,519 deletions), spanning **39,960 physical lines**. Its public generated checkpoint is `992b9cbba312b4bf10128b9f1d36112b48340363`.
 

scripts/castiron/CUSTOM_CODE.md

@@ -244,6 +244,16 @@ for its own initialization. A malformed base pin cannot enter this path; other
 changes without a base pin fail closed. Ordinary pin updates execute under the
 previous approved version and cannot include SDK/tooling changes.
 
+An existing unpinned `next` may instead adopt the complete authenticated current-main
+snapshot when next is an ancestor of main and main already uses strict ownership
+with zero custom lines. The canonical `sdk prepare-next-baseline` command preserves
+all main bytes, modes and existing public checkpoints, adding only the exact
+same-main policy pin. This exception cannot include additional edits, a different
+pin, a relaxed policy or multiple queue constituents. Ordinary checkpoint, manifest,
+zero-budget and repository review checks remain mandatory. Main must still match
+at final publication; if it moves, prepare and qualify a fresh candidate. This is
+adoption of already-public output, not regeneration or a budget increase.
+
 Keep required status activation separate until hosted tests have exercised main
 and next PRs, forks, retargeting, policy changes, and both merge queues. Require
 the target-specific next contexts after observing their genuine producers;

scripts/castiron/custom_code_budget.py

@@ -33,6 +33,7 @@
 read_policy = policy.read_policy
 changed_paths = policy.changed_paths
 queued_entries = policy.queued_entries
+is_main_snapshot_bootstrap = policy.is_main_snapshot_bootstrap
 
 POLICY = ".castiron-ratchet.json"
 NEXT_POLICY_VERSION = 1
@@ -341,6 +342,8 @@ def evaluate(
     measurement: tuple[dict[str, Any], bytes] | None = None,
     policy_sha: str | None = None,
     target_branch: str = "main",
+    main_snapshot: str | None = None,
+    repository: str | None = None,
 ) -> tuple[dict[str, Any], bytes]:
     """In a queue, isolate each PR but budget the complete merged candidate."""
     base, head = report.require_sha(base), report.require_sha(head)
@@ -360,12 +363,36 @@ def evaluate(
         read_budget(repo, head)
         if pull_heads is not None and not pull_heads:
             raise ValueError("merge group has no verified constituent PRs")
+        if main_snapshot is not None and (
+            target_branch != "next"
+            or main_snapshot != policy
+            or repository is None
+            or (pull_heads is not None and len(pull_heads) != 1)
+            or not is_main_snapshot_bootstrap(
+                repo,
+                repository,
+                base,
+                head,
+                policy,
+                constituent=pull_heads[0]
+                if pull_heads is not None and len(pull_heads) == 1
+                else None,
+            )
+            or any(
+                not is_main_snapshot_bootstrap(repo, repository, base, candidate, policy)
+                for candidate in (pull_heads if pull_heads is not None else [head])
+            )
+        ):
+            raise ValueError("invalid authenticated main snapshot bootstrap")
         for pr_head in pull_heads if pull_heads is not None else [head]:
-            proposed = check_isolation(repo, base, pr_head)
+            proposed = 0 if main_snapshot is not None else check_isolation(repo, base, pr_head)
             if proposed is not None:
                 proposed_limits.append(proposed)
         checks["isolation"] = outcome(
-            "success", "Budget changes are isolated; human review is required for increases."
+            "success",
+            "Exact authenticated strict-zero main snapshot adoption."
+            if main_snapshot is not None
+            else "Budget changes are isolated; human review is required for increases.",
         )
     except (report.ReportError, ValueError, UnicodeError) as exc:
         checks["isolation"] = outcome("failure", str(exc))
@@ -638,6 +665,35 @@ def github_evaluate(
                 pull_heads.append(candidate)
         if not pull_heads:
             raise ValueError("cannot verify merge-group PR membership; retry the current group")
+    main_snapshot = selection.get("main_snapshot") or None
+    if main_snapshot is not None:
+        if (
+            branch != "next"
+            or main_snapshot != main
+            or report.api("GET", f"{root}/git/ref/heads/main")["object"]["sha"] != main
+        ):
+            raise ValueError("authenticated main snapshot changed; retry bootstrap")
+        # This evaluator has its own bare object store, including when reusing
+        # the PR report. SHA fetches do not create the selector's named main ref.
+        report.git(
+            repo,
+            "fetch",
+            "--quiet",
+            "--no-tags",
+            "origin",
+            "+refs/heads/main:refs/remotes/origin/main",
+        )
+        if report.git(repo, "rev-parse", "refs/remotes/origin/main").decode().strip() != main:
+            raise ValueError("authenticated main changed during snapshot fetch; retry")
+        if not is_main_snapshot_bootstrap(
+            repo,
+            repository,
+            base,
+            head,
+            main,
+            constituent=pull_heads[0] if pull_heads is not None and len(pull_heads) == 1 else None,
+        ):
+            raise ValueError("authenticated main snapshot changed; retry bootstrap")
     if branch == "next":
         assert selection is not None
         base_pin = read_policy(repo, base).get("next_policy")
@@ -646,7 +702,11 @@ def github_evaluate(
             if (
                 selection.get("bootstrap") is not True
                 or "next_policy" not in read_policy(repo, head)
-                or (pull_heads is None and changed_paths(repo, base, head) != {POLICY.encode()})
+                or (
+                    main_snapshot is None
+                    and pull_heads is None
+                    and changed_paths(repo, base, head) != {POLICY.encode()}
+                )
             ):
                 raise ValueError("unreviewed next policy initialization")
         elif selection.get("bootstrap") is not False or base_pin != {
@@ -664,16 +724,150 @@ def github_evaluate(
         measurement=measurement,
         policy_sha=main,
         target_branch=branch,
+        main_snapshot=main_snapshot,
+        repository=repository,
     )
     result["target_branch"] = branch
     return result, patch
 
 
+def prepare_main_snapshot(
+    repo: Path, repository: str, base: str, authenticated_main: str, *, fetch: bool = False
+) -> str:
+    """Create and verify an adoption commit without altering a ref, index or metadata."""
+    base, authenticated_main = map(report.require_sha, (base, authenticated_main))
+    strict = {"schema_version": 2, "max_custom_patch_lines": 0, "generated_files": "strict"}
+    if read_policy(repo, authenticated_main) != strict or "next_policy" in read_policy(repo, base):
+        raise ValueError("main snapshot preparation requires strict-zero main and unpinned next")
+    if policy.merge_base(repo, base, authenticated_main) != base:
+        raise ValueError("protected next must be an ancestor of authenticated main")
+    value = {**strict, "next_policy": {"repository": repository, "main_sha": authenticated_main}}
+    blob = report.git(
+        repo,
+        "hash-object",
+        "-w",
+        "--stdin",
+        input_bytes=(json.dumps(value, indent=2) + "\n").encode(),
+    ).strip()
+    entries = report.git(repo, "ls-tree", "-z", authenticated_main).split(b"\0")
+    # Replace only the root policy blob, preserving every other entry and mode.
+    replacement = b"100644 blob " + blob + b"\t" + POLICY.encode()
+    tree = (
+        report.git(
+            repo,
+            "mktree",
+            "-z",
+            input_bytes=b"\0".join(
+                replacement if entry.partition(b"\t")[2] == POLICY.encode() else entry
+                for entry in entries
+            ),
+        )
+        .decode()
+        .strip()
+    )
+    head = (
+        report.git(
+            repo,
+            "commit-tree",
+            tree,
+            "-p",
+            base,
+            "-p",
+            authenticated_main,
+            "-m",
+            "Adopt verified strict-zero main baseline on next",
+        )
+        .decode()
+        .strip()
+    )
+    if not is_main_snapshot_bootstrap(repo, repository, base, head, authenticated_main):
+        raise ValueError("prepared snapshot does not match authenticated main")
+    result, _ = evaluate(
+        repo,
+        base,
+        head,
+        public=True,
+        fetch=fetch,
+        target_branch="next",
+        policy_sha=authenticated_main,
+        main_snapshot=authenticated_main,
+        repository=repository,
+    )
+    if not all(check["state"] == "success" for check in result["checks"].values()):
+        raise ValueError("main snapshot verification failed: " + json.dumps(result["checks"]))
+    return cast(str, head)
+
+
+def read_public_api(path: str) -> Any:
+    # Reuse the local gh login without reading or printing its credential.
+    output = subprocess.run(
+        ["gh", "api", "--hostname", "github.com", "--method", "GET", path],
+        capture_output=True,
+        check=True,
+        timeout=60,
+    )
+    return json.loads(output.stdout)
+
+
+def prepare_next_baseline(repo: Path, repository: str, branch: str) -> dict[str, str]:
+    """Authenticate public refs and create one new local review branch; never push."""
+    if not report.REPOSITORY.fullmatch(repository) or not branch.startswith("codex/"):
+        raise ValueError("expected a repository and a new codex/ review branch")
+    report.git(repo, "check-ref-format", "--branch", branch)
+    if (
+        report.git(repo, "remote", "get-url", "origin").decode().strip()
+        != f"https://github.com/{repository}.git"
+    ):
+        raise ValueError("preparation requires the public repository as origin")
+    root = f"repos/{repository}"
+    metadata = read_public_api(root)
+    if (
+        metadata["private"]
+        or metadata["default_branch"] != "main"
+        or metadata["full_name"] != repository
+    ):
+        raise ValueError("preparation requires a public repository with default main")
+
+    def ref(name: str) -> str:
+        return cast(
+            str,
+            report.require_sha(read_public_api(f"{root}/git/ref/heads/{name}")["object"]["sha"]),
+        )
+
+    main, base = ref("main"), ref("next")
+    report.git(
+        repo,
+        "fetch",
+        "--quiet",
+        "--no-tags",
+        "origin",
+        "+refs/heads/main:refs/remotes/origin/main",
+        "+refs/heads/next:refs/remotes/origin/next",
+    )
+    for name, expected in (("main", main), ("next", base)):
+        if (
+            report.git(repo, "rev-parse", f"refs/remotes/origin/{name}").decode().strip()
+            != expected
+        ):
+            raise ValueError("public refs changed during preparation; retry")
+    head = prepare_main_snapshot(repo, repository, base, main, fetch=True)
+    if ref("main") != main or ref("next") != base:
+        raise ValueError("public refs changed during verification; retry")
+    # Compare-and-create only. Existing review branches and working trees are preserved.
+    report.git(repo, "update-ref", f"refs/heads/{branch}", head, "0" * 40)
+    return {"head_sha": head, "base_sha": base, "main_sha": main, "branch": branch}
+
+
 def main() -> int:
     if sys.argv[1:2] == ["select-policy"]:
         return cast(int, policy.main(sys.argv[1:]))
     parser = argparse.ArgumentParser(description=__doc__)
     commands = parser.add_subparsers(dest="command", required=True)
+    commands.add_parser("review-rules")
+    prepare = commands.add_parser("prepare-next-baseline")
+    prepare.add_argument("--repo", type=Path, required=True)
+    prepare.add_argument("--repository", required=True)
+    prepare.add_argument("--branch", required=True)
     generated = commands.add_parser("verify-generated")
     generated.add_argument("--repo", type=Path, required=True)
     generated.add_argument("--policy-ref", required=True)
@@ -697,6 +891,17 @@ def main() -> int:
         command.add_argument("--repo", type=Path, required=True)
         command.add_argument("--out", type=Path, required=True)
     args = parser.parse_args()
+    if args.command == "review-rules":
+        print(json.dumps(policy.HUMAN_REVIEW_RULES, separators=(",", ":")))
+        return 0
+    if args.command == "prepare-next-baseline":
+        try:
+            preparation = prepare_next_baseline(args.repo, args.repository, args.branch)
+        except (ValueError, OSError, report.ReportError, subprocess.SubprocessError) as exc:
+            print(str(exc), file=sys.stderr)
+            return 1
+        print(json.dumps(preparation, sort_keys=True))
+        return 0
     if args.command == "verify-generated":
         # Local pipeline integration pins policy and checkpoint before merging.
         # Legacy SDKs without a policy retain their existing behavior.