Repository navigation
Conversation
This was referenced Nov 1, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The distributed probe did not expose its actual normalization multiplier or window token counts to TrainCheck. This adds an opt-in native API observation at the actual loss multiplication and evaluates a separately authored Python invariant offline, making the missing objective contract directly checkable.
A fresh agent received the intended global-token mean objective, DDP semantics, and a human-selected scalar interface. It derived
applied_scale ≈ world_size / global_tokens. The rule was frozen before native-trace evaluation; the final equation/tolerance were not tuned on faulty traces. This is one targeted authoring pilot, not built-in inference or autonomous bug discovery. Access isolation was instructional, not a formal blinded experiment.The checker validates successful pre/post pairs, complete rank/update/microbatch coverage, and cross-rank count consistency. It preserves raw records and identities; malformed or incomplete evidence exits with an error and unknown violation count. The adapter retains the original multiplication and temporary tensor lifetimes. The predicate runs outside training.
Measured results
Final runtime revision:
fe482640d2979481419040fe3395b894cb8600fd. Ring18, NVIDIA A16, PyTorch 2.2.2+cu121, FP32 5,504-parameter Transformer, four updates per run; one trial per configuration.All nine training phases completed. All three faulty configurations first violate at update 1 in the offline traces. The independent gradient oracle agrees with every healthy/faulty classification. Plain/observed complete checkpoint comparisons pass for healthy and faulty accumulation-2 runs: 4 rank-file pairs, 548 tensor pairs, matching whole-file hashes, without dropping fields.
Validation and limits
26c77ceworkload and pass after correction; preliminary runs are preserved separately and excluded from final claims.c138f5e7e050939182ade7f60b79baad9a2c5833ca42ad4c36a237d6a549f09b.Full commands, provenance, results, and limitations:
docs/measurements/agent-authored-normalization-20261101.md.Stack dependency: targets #167 (
codex/parameter-observation-time), which depends on #166 and #164. This PR is not merged.