Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
919c9a8
test(oracle-gaps): law 15 owes a stream log that the first line names
KyleAMathews Nov 13, 2026
17ea634
test(oracle-gaps): a refusal after a check started names its stream l…
KyleAMathews Nov 13, 2026
ceeeaf6
feat(oracle-gaps): write the stream to <out>/stream.log and name it f…
KyleAMathews Nov 13, 2026
23b47d5
Merge origin/main (#48, #54) into feat/gap-cli-stream-log
KyleAMathews Nov 13, 2026
5a92a3e
fix(oracle-gaps): own the stream log in the supervisor; refuse a seco…
KyleAMathews Nov 13, 2026
fd5655d
wip(oracle-gaps): checkpoint at the pre-restructure CLI with the orac…
KyleAMathews Nov 13, 2026
4d6cc15
fix(oracle-gaps): claim the output directory atomically and release r…
KyleAMathews Nov 13, 2026
a2d51b9
fix(oracle-gaps): install the stop handlers before any project module…
KyleAMathews Nov 13, 2026
2a0fdda
test(oracle-gaps): the spawn stop signals once the CLI has started, d…
KyleAMathews Nov 14, 2026
f5b7db8
Merge remote-tracking branch 'origin/main' into feat/gap-cli-stream-log
KyleAMathews Nov 14, 2026
76522ff
test(oracle-gaps): one decision for whether a refused run owes its note
KyleAMathews Nov 14, 2026
a784642
fix(oracle-gaps): install the stop handlers before the started marker
KyleAMathews Nov 14, 2026
d338cb1
fix(oracle-gaps): take over a stale claim under a mutex, and name res…
KyleAMathews Nov 14, 2026
cb6caf7
Merge remote-tracking branch 'origin/main' into feat/gap-cli-stream-log
KyleAMathews Nov 14, 2026
6222e25
ci(oracles): place the new Windows fixed-seed units in shards
KyleAMathews Nov 14, 2026
5fa74e0
chore(oracle-gaps): drop the unsupervised POSIX seam
KyleAMathews Nov 14, 2026
9560e86
fix(oracle-gaps): create the takeover mutex with its pid, and never t…
KyleAMathews Nov 14, 2026
c8352b8
docs(oracle-gaps): document every test-only ORACLE_GAPS_TEST hook
KyleAMathews Nov 14, 2026
7db86c5
test(oracle-gaps): hold the exit at a named point, and delete the wal…
KyleAMathews Nov 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/gap-cli-stream-log.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/oracles': patch
---

The `oracle-gaps` CLI now writes the lines it streams on stderr to `stream.log` in the output directory too, in order, cold and through `--daemon`, and the first line of a check on both streams names that file: `oracle-gaps: note log <out>/stream.log`. An agent that starts a cold check in the background can read the run's progress and its done line there while it runs. A check through `--daemon` prints its lines at the end of its check, so its progress is readable there only after it ends.
22 changes: 17 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,13 +123,25 @@ agent.
to the process's stdout and stderr goes to `test-output.log` in the output
directory, which a note names; `summary.md` holds the report for reading.
A run through the daemon prints the same lines, at the end of its check.
The first line of a check, on both streams, is
`oracle-gaps: note log <out>/stream.log`: the run writes its stderr lines
to that file too, in order, so a check started in the background can be
read there while it runs and after it ends. A cold run's lines reach the
file as they are printed; a run through `--daemon` prints its lines at the
end of its check, so the file fills then.
`--help` is not a check, so its usage text has no prefix.

For contributors: two environment variables in the `ORACLE_GAPS_TEST_`
namespace are test-only hooks for the CLI's own oracle, not settings for
users. `ORACLE_GAPS_TEST_HOLD_EXIT_MS` holds the run's exit after its done
line, and `ORACLE_GAPS_TEST_WARM_PROBE` makes the first config warm-up
wait where a stop signal can reach it.
For contributors: the `ORACLE_GAPS_TEST_` environment variables are test-only
hooks for the CLI's own oracle, not settings for users. Each is inert unless
set; a hook may pause at one named point and never changes an outcome.
`ORACLE_GAPS_TEST_STARTED` names a file that the CLI writes as its first project statement, before any project module loads, so a test can tell a stop that came before the CLI started from one that came during its loads.
`ORACLE_GAPS_TEST_PAUSE_AFTER_STARTED` (milliseconds) blocks the CLI for that long right after the started file is written, so a stop sent on seeing the file lands inside the pause.
`ORACLE_GAPS_TEST_PAUSE_AT` names one point at which `ORACLE_GAPS_TEST_PAUSE_MS` (milliseconds) blocks the CLI: `takeover` (inside a stale-claim takeover, after the stale pid is read) and `mutex-create` (inside the takeover mutex's creation).
`ORACLE_GAPS_TEST_STOP` names a file whose appearance makes the CLI take a stop as a SIGINT would.
`ORACLE_GAPS_TEST_HOLD_UNTIL` names a file: after its done line the run keeps
its exit, and its output claim, until the file exists, so a test can send a
stop or start a second run before the run ends. `ORACLE_GAPS_TEST_WARM_PROBE` makes the first config warm-up wait where a
stop signal can reach it.

A few endings print no done line, or one more line: a SIGKILL of the CLI,
or a signal that comes while Node is still starting the CLI, before its
Expand Down
4 changes: 3 additions & 1 deletion packages/oracles/oracle-gaps/ending.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,10 @@ export function endRun(error, out = error?.out) {
// The last line says the run refused, and where its report is once the
// run knows its output directory (the stream's done line, stream.mjs);
// it ends stdout too (index.mjs).
// A reportless refusal (another run uses the directory) writes nothing
// there: the directory's files are the other run's.
const done = write(
out,
error.reportless ? undefined : out,
results,
`Refused: ${message}\n\nFix: ${error.fix}\n`,
)
Expand Down
161 changes: 137 additions & 24 deletions packages/oracles/oracle-gaps/gap-report-model-oracle.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -769,6 +769,73 @@ const drawn = fc.record({
}),
// Who names the importer's function in an owns entry, if any.
importer: fc.constantFrom(`none`, `a`, `b`),
// Where the run writes: the output directory `--out` names (`explicit`), or
// the one the CLI picks when `--out` is left out (`default`, args.mjs), whose
// stream log, results, and done line must still agree (law 15).
outMode: fc.constantFrom(`explicit`, `explicit`, `explicit`, `default`),
// The run's `<out>/stream.log` is a directory, so the log cannot be written
// (a problem the author can fix: the run refuses before it checks).
logBlocked: fc.constantFrom(
false,
false,
false,
false,
false,
false,
false,
false,
true,
),
// A second run starts on the first run's output directory while the first
// is live (the coordinator's decision of 2026-11-13 on F4): the second
// refuses, and the first's stream log stays its own.
overlap: fc.constantFrom(
false,
false,
false,
false,
false,
false,
false,
false,
false,
false,
true,
),
// The stop witness alone (`w_stop`), which a run the driver stops needs. A
// field of its own, so the witness options above keep their weights.
stopWitness: fc.constantFrom(
false,
false,
false,
false,
false,
false,
false,
false,
true,
),
// A stop sent as soon as the CLI spawns, before its own run has started (P1:
// the note is written before the child exists, so the stopped line follows
// it). The driver adds this stop to the run's stops.
spawnStop: fc.constantFrom(
false,
false,
false,
false,
false,
false,
false,
false,
true,
),
// The signal of a spawn stop and when it lands, 0 to 150 ms after the CLI
// spawns (the spawn-delay law, review of #57).
spawnSignal: fc.constantFrom(`SIGINT`, `SIGINT`, `SIGTERM`),
spawnDelay: fc.integer({ min: 0, max: 150 }),
// The spawn stop waits for the CLI's started marker, then signals at once (the
// deterministic witness, event-driven rather than timed).
spawnOnStart: fc.constantFrom(false, false, false, true),
})

// The edit sequence over the witness module, the one list of steps that a
Expand All @@ -780,6 +847,10 @@ const drawn = fc.record({
// nothing changed, which reuses the confirmations that the first check ran.
const witnessSteps = [`test`, `helper`, `revert`]
const budgetSteps = [`recheck`]
// The stop witness alone takes no steps, so it counts only where the history
// has none; a history with steps and the field is a legal history without it.
const stopsOnly = (history) =>
history.stopWitness === true && history.steps === null
const keepsSteps = (history) => {
if (history.witness === false || history.steps === null) return true
const kept = history.witness === `budget` ? budgetSteps : witnessSteps
Expand Down Expand Up @@ -2469,14 +2540,15 @@ function resolveHistory(history) {
...exported,
...importer,
...dotGen,
...(history.witness
...(history.witness || stopsOnly(history)
? witness
.filter(
(w) =>
// The budget witness costs two 10 s confirmations per mutant,
// so only histories of the `budget` mode carry it.
(history.witness === true && w.id !== `w_slowpair`) ||
(history.witness === `capture` && captureWitness.has(w.id)) ||
(stopsOnly(history) && w.id === `w_stop`) ||
(history.witness === `budget` && w.id === `w_slowpair`),
)
.map((w) => ({
Expand Down Expand Up @@ -2837,7 +2909,26 @@ function resolveHistory(history) {
depSubpath: history.depSubpath === true,
unbuiltDep: history.unbuiltDep === true,
budget: history.budget ?? `two`,
outMode: history.outMode ?? `explicit`,
// Both name the run's `--out` directory before it starts, so they need
// an explicit one (a default directory is chosen by the CLI).
logBlocked:
history.logBlocked === true && outModeOf(history) === `explicit`,
// A second run needs a live first run, which has its note and stream log.
// A blocked log refuses the first run before it is live, and a refusing
// fault (an argument, declaration, or config fault) ends it before its note
// too, so the overlap is generated only for a run that reaches its check
// (decision of 2026-11-13, review of #57: O1 and O2).
overlap:
history.overlap === true &&
outModeOf(history) === `explicit` &&
history.logBlocked !== true &&
(history.fault ?? `none`) === `none`,
stopRuns: history.stopRuns ?? [],
spawnStop: history.spawnStop === true,
spawnSignal: history.spawnSignal ?? `SIGINT`,
spawnDelay: history.spawnDelay ?? 0,
spawnOnStart: history.spawnOnStart === true,
onlyRun: history.onlyRun === true,
pragma: history.pragma === true,
cliPath: history.cliPath ?? `plain`,
Expand Down Expand Up @@ -4083,10 +4174,23 @@ function refusal(plan) {
if (plan.fault === `brokenVitest`) return { cause: `vitest`, runtime: true }
if (plan.fault === `projects`)
return { cause: `test.projects`, runtime: true }
// A stream log that cannot be written is refused before the check starts
// (the CLI opens the log before its first line, index.mjs), so its stdout
// owes its done line alone.
if (plan.logBlocked === true)
return { cause: `the stream log`, arguments: true }
if (plan.fault === `badEntry`) return { cause: `${SOURCE}#nope` }
// An owns entry that names a missing file (a test file's declaration) is
// found by the analysis, after the check has started, so this refusal
// owes the stream log's note. The argument-stage case, a `--source` that
// is not a file of the package, is the static oracle's `absentSource`.
if (plan.fault === `missingSource`) return { cause: `src/missing.ts` }
if (plan.fault === `nonLiteral`) return { cause: `b.test.ts` }
if (plan.fault === `outsideSource`) return { cause: `outside the package` }
// A `--source` outside the package is an argument the CLI refuses before
// it starts a check (the argument rules that the static oracle judges),
// so its stdout has no line that names a stream log.
if (plan.fault === `outsideSource`)
return { cause: `outside the package`, arguments: true }
// The analysis reads the test files that the package's Vitest config
// includes (index.mjs, step 2), so a config that does not load is named
// before anything the analysis would find. The cause names the config
Expand Down Expand Up @@ -4183,6 +4287,7 @@ const declarationFaults = [
`nonLiteral`,
`outsideSource`,
]
const outModeOf = (history) => history.outMode ?? `explicit`
const reachesRun = (history) =>
!declarationFaults.includes(history.fault ?? `none`)

Expand Down Expand Up @@ -4235,21 +4340,22 @@ function model(plan) {
// fault, a Vitest that does not load, a config that does not load, and a
// config that defines test.projects, which a loaded context shows. Every
// other run names it once, refused or not.
const configLines = [
`outsideSource`,
`brokenVitest`,
`configThrows`,
`configNoDefault`,
`configNull`,
`configSyntax`,
`projects`,
].includes(plan.fault)
? []
: [
plan.configPath === undefined
? `oracle-gaps: config none, Vitest's default discovery`
: `oracle-gaps: config ${plan.configPath}`,
]
const configLines =
[
`outsideSource`,
`brokenVitest`,
`configThrows`,
`configNoDefault`,
`configNull`,
`configSyntax`,
`projects`,
].includes(plan.fault) || plan.logBlocked === true
? []
: [
plan.configPath === undefined
? `oracle-gaps: config none, Vitest's default discovery`
: `oracle-gaps: config ${plan.configPath}`,
]
if (refused) return { refused, configLines }
const sites = sitesOf(plan)
const owners = ownersOf(plan)
Expand Down Expand Up @@ -5493,8 +5599,13 @@ function endingOf(exit, stderr, out, name = (text) => text, stdout = ``) {
* The first way a run's ending breaks the refusal law, or undefined. The
* cause must name `cause`; with `outKnown`, the run knew its output
* directory, so the block names it and the directory holds the refusal.
* `logNote` is the line that names the run's stream log, owed first on
* stdout when the run refused after it started a check, and undefined for
* a refusal of its arguments, before any check (the maintainer's decision
* of 2026-11-12 that every run that starts a check names its stream log
* first).
*/
function refusalBreach(ending, { cause, outKnown }) {
function refusalBreach(ending, { cause, outKnown, logNote }) {
if (ending.exit !== 2) return `the refusal exited ${ending.exit}, not 2`
if (ending.stackFrames) return `the refusal printed a stack trace`
if (ending.cause === undefined)
Expand All @@ -5513,12 +5624,14 @@ function refusalBreach(ending, { cause, outKnown }) {
!/^refused(?:, report .+)?$/.test(ending.done)
)
return `the refusal's last line is not \`oracle-gaps: done refused\``
// The done line ends stdout too, as its only line.
if (
ending.stdout.length !== 1 ||
ending.stdout[0] !== `oracle-gaps: done ${ending.done}`
)
return `the refusal's stdout is ${JSON.stringify(ending.stdout)}, not its done line`
// The done line ends stdout too, as its only line after the line that
// names the stream log, if the run started a check.
const owedStdout = [
...(logNote === undefined ? [] : [logNote]),
`oracle-gaps: done ${ending.done}`,
]
if (JSON.stringify(ending.stdout) !== JSON.stringify(owedStdout))
return `the refusal's stdout is ${JSON.stringify(ending.stdout)}, not ${JSON.stringify(owedStdout)}`
if (!outKnown) return undefined
if (!ending.namesOutput)
return `the refusal does not name its report in its output directory: done ${ending.done}`
Expand Down
Loading
Loading