dream(evaluation-adapters): luator entrypoint at commit `9f4d8a9aadae67735273f124a4d73c5 - #11
Closed
jjohare wants to merge 1 commit into
Closed
dream(evaluation-adapters): luator entrypoint at commit `9f4d8a9aadae67735273f124a4d73c5#11jjohare wants to merge 1 commit into
jjohare wants to merge 1 commit into
Conversation
jjohare
added a commit
that referenced
this pull request
Sep 7, 2026
On 2026-09-07 the REQUIRED darwin evaluator returned outcome=PASSED while its leaderboard listed five candidates in generation 2 (g2_v0..g2_v4), against the documented <=4/generation bound. The bound existed only as prose in the compiled prompt (packages/compile/src/index.ts:269); nothing on the evaluator path read it, and the gate recorded no veto. Adds packages/cli/src/darwinBounds.ts — DARWIN_BOUNDS policy constants, parseLeaderboardRows, checkDarwinBounds, checkDarwinBoundsFromStdout — dependency-free and additive. The real 09-07 leaderboard is pinned as a regression fixture. Lands PR #11 with three corrections from the integration review: - the ADR is renumbered 0057 -> 0003 and moved to docs/adrs/ with this repo's section shape; the assumed ADR-0056 predecessor came from another corpus; - checkDarwinBounds is itself fail-visible: ok can never be true while parseStatus is 'unparsable'. That branch used to live only in the checkDarwinBoundsFromStdout wrapper, so a caller reaching for the exported pure function got a silent pass on unreadable input — the exact failure mode the module exists to eliminate; - the generation bound constrains DEPTH, not the count of distinct labels. A sparse g1/g3/g5 run is five generations deep but showed cardinality 3 and passed. Distinct labels are kept as generationsObserved, for reporting only. The ADR states plainly that this is detection, not enforcement: the nightly gate lives in the external annexe runner, not this repo, so nothing here can veto a night. npm test: 145/145 (was 135/135); 10 new tests, 2 of them one per correction. Co-Authored-By: jjohare <github@thedreamlab.uk> Claude-Session: https://claude.ai/code/session_01EzN6LxAjkxypXeZ3fTQhz8
Contributor
Author
|
Landed on main via operator integration 2026-09-07 (with ADR renumbered to 0003, fail-visible pure checker, depth-based generation bound). |
jjohare
added a commit
that referenced
this pull request
Sep 7, 2026
An operator row, not a night: no hypothesis was tested and no evaluator ran, so the verdict is INCONCLUSIVE and the witness is "operator" rather than a computed triple — the same shape as the 2026-08-28 fate-reconciliation row. The fate is recorded as #11:MERGED although GitHub shows the PR closed. The ledger's fate vocabulary tracks whether the change landed, not the button pressed: #11's content is on main, corrected. That follows the 08-28 precedent, which recorded #7/#9 as MERGED after they were consolidated by other means. The finding reads "guard in verify-entrypoint" rather than "wired into" only because the latter is 84 characters and the row contract caps findings at 80. npm test 149 of 149, row contract included. Co-Authored-By: jjohare <github@thedreamlab.uk> Claude-Session: https://claude.ai/code/session_01EzN6LxAjkxypXeZ3fTQhz8
jjohare
added a commit
that referenced
this pull request
Sep 7, 2026
…ht tokens Not every ledger row is a night. The 2026-08-28 fate reconciliation and the 2026-09-07 PR #11 integration are both operator actions with no hypothesis behind them, and lacking a token for that, both were filed INCONCLUSIVE. That is not a cosmetic mislabel. The engine derives a dry streak from the ledger, counts INCONCLUSIVE toward it and ignores every other token, and a long enough streak parks the repo. Operator rows filed as INCONCLUSIVE therefore push a repo toward being parked over nights nobody attempted — the ledger's own signal channel poisoned by rows carrying no experimental content. Adds BLOCKED-ENV, HANDOFF and OPERATOR alongside the three night outcomes, and n/a to the Evaluated column. ADR-0001's three-state invariant is qualified in scope, not overturned: a night that ran still ends in exactly one of ACCEPT, REJECT or INCONCLUSIVE, and the compiled prompt still says so verbatim. Scope grew once past the regex, for a reason worth recording. verifyLedger gated on its own private copy of the vocabulary, so the row this change exists to permit would have passed the row contract and then been rejected by "ledger verify" — and evaluated n/a would have failed the same way. Both now import the same constants from rowContract, which is the single definition. A validator that disagrees with the verifier is worse than either alone, because a row passes one gate and fails the other with nowhere single to look. The ACCEPT-only rules are deliberately not widened: a non-night row legitimately carries no PR and no computed witness, and there is a test asserting the vocabulary change did not drag those two rules along with it. ADR-0004 records the split. ADR-0001 is amended by follow-on rather than edited in place, per the rule in docs/adrs/INDEX.md — which also keeps this change off a file another session is holding. Known gaps, stated rather than papered over: verdictStats still buckets the non-night tokens into "other" and the TUI still counts every row as a night, both already true of the 08-28 row; and this repo's CLI accepts BLOCKED-ENV and HANDOFF on read before it can mint them on write. npm test 159 of 159, up from 149. Lint and build clean. Co-Authored-By: jjohare <github@thedreamlab.uk> Claude-Session: https://claude.ai/code/session_01EzN6LxAjkxypXeZ3fTQhz8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft PR opened by the dream engine on a GATED ACCEPT night (2026-09-07-dream-machine, run
ab4ced4e48b76e83).Candidate tree
c80a8578199c7576d4339a7c88485d97b1f80d60was applied in isolation and the required evaluators re-run against it: bench=PASSED, darwin=PASSED. Witnesscae9a2589672.A human decides the merge — evaluation is not promotion.
Finding: Given the darwin evaluator entrypoint at commit
9f4d8a9aadae67735273f124a4d73c5a30674620run with--sandbox mock, when an evolution run emits a generation containing five candidates (g2_v0..g2_v4), then the required-evaluator gate reportsoutcome=PASSEDwith no bound violation raised, because the evaluation-adapter layer enforces no per-generation candidate cap.Gate: ACCEPT upheld: 2 required evaluator(s) passed on candidate tree c80a8578199c