fix: meaningful diagnostics, S-first layering, worker retry, representability surfacing - #74
Conversation
…ayered build failures, representability, worker retry
New assertions for the ratified diagnostics + robustness slice, written and
run BEFORE any implementation. Failing output on the pre-change tree:
pnpm --filter @dspack-studio/composer-core test:
❯ src/diagnostics.test.ts (7 tests | 7 failed) 8ms
× catalogGateFindings … a failing gate with errorDetails yields ONE finding per instance …
→ catalogGateFindings is not a function
× catalogGateFindings … without errorDetails the single finding keeps the caller's target, caps the joined message …
→ catalogGateFindings is not a function
× catalogGateFindings … three or fewer errors are never marked truncated …
→ catalogGateFindings is not a function
× buildFailure … identical catalog-gate errors under both A2UI versions render ONCE …
→ expected [ { gate: 'A3', …(3) }, …(3) ] to have a length of 2 but got 4
× buildFailure … identical-message repetition within one version is capped to one row …
→ expected [ { gate: 'A3', …(3) }, …(3) ] to have a length of 2 but got 4
× buildFailure … when S-gate failures explain the turn, A-gate reasons collapse to one summary …
→ expected [ { gate: 'A3', …(3) }, …(7) ] to have a length of 2 but got 8
× buildFailure … a failed-lint-exhausted whose last attempt carries a representability refusal …
→ expected undefined to deeply equal { pass: false, …(1) }
Test Files 1 failed | 2 passed (3)
Tests 7 failed | 61 passed (68)
pnpm --filter composer test:
❯ worker/propose.test.mjs (5 tests | 4 failed) 9ms
× retries ONCE when the provider returns prose instead of JSON, then succeeds
→ expected 502 to be 200 // Object.is equality
× a truncated fenced payload also gets exactly one retry …
→ expected 502 to be 200 // Object.is equality
× two unusable outputs return the existing 502 provider-unavailable shape — and there is NO third call
→ expected "spy" to be called 2 times, but got 1 times
✓ provider rate-limiting (429 shape) is NEVER retried — one call, classified busy
× the model-call+parse helper is exported and retries exactly once (unit, injected fake env.AI)
→ callModel is not a function
Test Files 1 failed | 4 passed (5)
Tests 4 failed | 25 passed (29)
The one passing new test (429 never retried) deliberately pins EXISTING
behavior that must survive the retry change — the zone limit is not hammered.
Notes on scaffolding: catalogGateFindings is reached via a namespace import
so its absence fails per-test (TypeError) instead of failing the module at
load time and masking the behavioral buildFailure failures; the folded
attempt's representability access is cast (as any) until the field exists.
Both are flipped to direct typed access in the implementation commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… failures, representability, worker bounded retry
Makes the fail-first suite from the previous commit green. Four defects from
the field report, one principle: genuine errors are layered, never dropped.
composer-core findings — catalogGateFindings (shared by browser + agent):
* dspack-emit >= 0.7 errorDetails (feature-detected via Array.isArray) yield
ONE finding per invalid component instance, target `Component#id` — the
hardcoded "a2ui@0.9.1" target (and its dead deep-link) is gone from the
detail path. Message = first 3 error strings + " (+N more)"; the COMPLETE
raw strings ride the new optional ComposerFinding.evidence field.
* Without errorDetails (0.6): the single finding keeps the caller's version
target, caps the joined message the same way (the measured 3,278-char
table cell), and keeps ALL raw strings on evidence.
composer-core build — buildFailure presentation:
* emitGateReasons dedupes A-gate reasons by (gate, code, message): a reason
under both A2UI versions renders once targeted "both A2UI versions";
identical-message repetition within one version collapses too (the ~100-row
panel becomes the distinct facts; the full audit report stays on the turn).
* S-first layering: when the last attempt's S-gate reasons exist, A-gate
reasons collapse to one summary ("N catalog-gate findings — see Checks for
detail") — S findings already explain the failure.
* dspack-gen >= 0.4 attempts[].representability (feature-detected) is kept
by the fold on TurnAttempt and surfaced as a structured reason for
failed-lint-exhausted turns (stoppedAt "attempt N · representability")
instead of "unknown … without structured evidence".
apps/composer validation.ts + apps/agent project.ts / record-fixture.ts:
the twin gate-mapping loops now call catalogGateFindings (cross-referenced
comments); record-fixture prints one line per instance the same way.
worker propose.mjs — bounded retry on unusable OUTPUT only:
* callModel = runModel (one provider call; rejection classified, never
retried — 429/busy/timeout/kill-switch must not hammer the zone limit)
+ parseProposal (fence-strip + JSON.parse; throw = unusable output).
Exactly one retry on unusable output; second failure rides the existing
502 provider-unavailable path. Exported for unit tests; the race timer is
now cleared (no dangling 45s handle).
Also flips the fail-first scaffolding to direct typed imports (namespace
access + `as any` were only there so the pre-change tree could run the file).
pnpm --filter @dspack-studio/composer-core test: 68 passed (68)
pnpm --filter composer test: 29 passed (29)
pnpm --filter agent test: 44 passed (44)
pnpm test (all packages) + pnpm -r typecheck: green
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…es collapsed; evidence expanders; representability labels
Checks (validate-view):
* Layering: error findings first, then warnings, then infos (stable sort, so
gate order holds within a severity) — the view no longer opens on a
fidelity-warning wall (measured ~67 rows on a real project; 193 on the
shipped shadcn example) before anything project-specific.
* Fidelity findings live behind a native <details> disclosure, COLLAPSED by
default: "N fidelity notes — how this contract projects onto A2UI". They
are the projection record, not unresolved work; nothing is hidden — one
click away, rows rendered by the same FindingRow.
* Findings carrying `evidence` (capped catalog-gate messages) get a
"full gate output (N errors)" expander: monospace, scrollable, complete.
* Every e2e-pinned testid survives verbatim: finding-${gate}-${code},
acknowledged-${target}, emit-status, validate-status, run-validate,
run-emit (grepped e2e/ — only composer-agent.spec.ts touches this view).
Build (build-view):
* An attempt whose folded record carries representability { pass: false }
(dspack-gen >= 0.4, feature-detected upstream in composer-core) shows
"representability FAIL" beside its gates (refusal as title text) and its
repair discloses as "repair sent — representability". Repair message
rendering itself is unchanged — the text already flows.
Verified live against the dev server (both example projects): errors with
their ACKNOWLEDGED chips sort to the top, 28/193 fidelity notes collapse to
one summary line, the disclosure opens on click with finding-fidelity-*
testids intact.
pnpm -r typecheck green; pnpm test + composer (29) + agent (44) all passing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
dspack-studio-composer | a7871e3 | Aug 10 2026, 10:17 PM |
There was a problem hiding this comment.
Pull request overview
This PR improves diagnostics fidelity and presentation across composer-core, the Composer app, and the hosted Worker by introducing structured, per-instance catalog-gate findings (when available), deduplicating/collapsing build-failure reasons, surfacing representability refusals, and adding a bounded retry for model-output parse failures.
Changes:
- Add
catalogGateFindings()to generate per-component-instance A-gate findings with capped messages and full rawevidence, and adopt it in browser + agent emit reporting. - Improve build-failure summarization (A-gate dedupe across versions, S-first layering) and surface dspack-gen representability refusals in turn attempts/failures.
- Update Checks UI to sort by severity, collapse fidelity notes, and add an expander for full raw evidence; add Worker
callModel()with a single retry on unusable provider output plus tests.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/composer-core/src/index.ts | Re-export catalogGateFindings and related types for shared use. |
| packages/composer-core/src/findings.ts | Add evidence to findings and implement catalogGateFindings() (per-instance + capped message). |
| packages/composer-core/src/diagnostics.test.ts | Add fail-first tests for catalog-gate findings and build-failure layering/dedup/representability surfacing. |
| packages/composer-core/src/build.ts | Deduplicate A-gate reasons, collapse A-gate walls when S-gates explain failure, and surface representability refusal. |
| apps/composer/worker/propose.test.mjs | Add tests for bounded retry behavior on unusable provider output (and no retry on provider failures). |
| apps/composer/worker/propose.mjs | Implement runModel (timer cleanup) + callModel with one retry on parse-only failures; export helper for tests. |
| apps/composer/app/views/validate-view.tsx | Sort findings by severity, collapse fidelity notes, and add evidence expander rendering. |
| apps/composer/app/views/build-view.tsx | Display representability failures and annotate repair details accordingly. |
| apps/composer/app/validation.ts | Use catalogGateFindings() for browser emit gate reporting. |
| apps/agent/src/record-fixture.ts | Use catalogGateFindings() to print per-instance gate diagnostics in fixture recording. |
| apps/agent/src/project.ts | Use catalogGateFindings() for agent emit gate reporting. |
Suppressed comments (2)
packages/composer-core/src/findings.ts:155
gate.errorsis treated asstring[], but the input is a looseCatalogGateLikecoming from external reports. Iferrorsis unexpectedly not an array, this path will attach a non-array value asevidenceand breakcappedMessage/UI consumers. Guard withArray.isArray.
});
}
const evidence = gate.errors ?? [];
return [withEvidence(finding(gateId, code, "error", fallbackTarget, cappedMessage(evidence, code || "gate failed")), evidence)];
}
apps/composer/app/views/validate-view.tsx:52
data-testidfor the evidence block only usesgate+code, which can collide across multiple findings (especially with per-instance A-gate findings). Includetarget(or another stable discriminator) so each evidence expander is uniquely addressable.
</summary>
<pre
data-testid={`finding-evidence-${f.gate}-${f.code}`}
style={{ fontFamily: "var(--mono)", fontSize: 11, whiteSpace: "pre-wrap", maxHeight: 200, overflow: "auto", color: "var(--fg-dim)", margin: "4px 0 0" }}
>
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| for (const error of (gate.errors ?? ["gate failed"]) as string[]) { | ||
| const key = `${gate.gate} ${gate.name ?? ""} ${error}`; | ||
| const entry = merged.get(key); | ||
| if (entry) { | ||
| if (version) entry.versions.add(version); | ||
| continue; | ||
| } | ||
| merged.set(key, { | ||
| reason: { gate: String(gate.gate), code: String(gate.name ?? ""), message: error }, | ||
| versions: new Set(version ? [version] : []), | ||
| }); | ||
| } |
| const target = | ||
| typeof detail?.component === "string" && typeof detail?.id === "string" | ||
| ? `${detail.component}#${detail.id}` | ||
| : fallbackTarget; | ||
| const evidence = (detail?.errors ?? []).map(detailErrorString); | ||
| return withEvidence(finding(gateId, code, "error", target, cappedMessage(evidence, code || "gate failed")), evidence); |
| return ( | ||
| <tr style={{ borderTop: "1px solid var(--line-soft)" }} data-testid={`finding-${f.gate}-${f.code}`}> | ||
| <td style={{ padding: "6px 8px", fontFamily: "var(--mono)", fontSize: 12 }}> |
| {rows.map((f, i) => ( | ||
| <FindingRow key={i} f={f} /> | ||
| ))} |
Phase 1 presentation + robustness from the ratified P1+P2 milestone. Feature-detected against dspack-emit 0.7 / dspack-gen 0.4 — no dependency bumps in this PR (bump follows once emit 0.7.0 is on npm).
catalogGateFindings()(composer-core) consumes emit 0.7'serrorDetailswhen present — one finding per component instance with an honestComponent#idtarget (the hardcodeda2ui@0.9.1target is gone), message capped at 3 errors (+N more), full raw strings preserved onfinding.evidence. Fallback for emit 0.6 caps the joined message and keeps everything onevidence. Agent twin (project.ts,record-fixture.ts) uses the same helper.failed-lint-exhaustedturns whose last attempt carriesrepresentabilitynow surface the emitter refusal instead of "unknown".evidenceget a raw-evidence expander.🤖 Generated with Claude Code