Skip to content

Release 0.6.3: reliable repository tools, repair feedback, and evidence handoff - #36

Merged
pkieltyka merged 1 commit into
masterfrom
release/0.6.3
Sep 25, 2026
Merged

pkieltyka merged 1 commit into
masterfrom
release/0.6.3

Conversation

@pkieltyka

Copy link
Copy Markdown
Collaborator

Summary

Prepare 0.6.3 with more reliable repository investigation, structured-output recovery, and final reporting. Reviews could previously stop useful source discovery at a local limit, lose evidence between worker attempts and later stages, or produce a fallback report without making synthesis failure sufficiently clear. This release gives investigations bounded headroom, preserves source provenance, improves corrective feedback, and exposes reporting limitations.

Changes

  • Repository tools: validate raw arguments before SDK coercion; require inclusive read_range bounds and preserve EOF/range metadata; guide unknown-path discovery; make unsupported-language outlines explicit and include complete small-file text when possible. Preserve structured cache results and disclose bounded search/file-list omissions.
  • Investigation budgets: use configured local values as soft targets with fixed 2× hard ceilings for aggregate calls, rounds, and result characters. Report remaining targets before and during investigation. Refused/invalid requests do not consume executed-call allowances, while refusal and round limits still bound repeated mistakes. Explicit spending caps, per-result caps, provider context limits, and repair deadlines remain bounded.
  • Evidence handoff: retain successful reads across worker retries, failure, and cancellation with worker/attempt/revision provenance; pass collected source into verification; improve question-based reconciliation selection and reference consistency; recognize valid repository paths outside changed-file context.
  • Structured recovery: include the latest rejected keys/types in custom and replacement repair prompts. Detect XML parameter syntax outside JSON strings and provide bounded schema-specific JSON guidance. Malformed drafts remain rejected; complete assembled submissions still require strict validation within existing attempt/deadline limits.
  • Reports: require explicit resolved/narrowed/unresolved decisions for supplied concerns, preserve unsupported questions honestly, consolidate equivalent fallback findings without collapsing distinct diagnoses, and place failed-synthesis notices before findings while preserving accurate coverage.
  • Release and tests: set package/workflow examples to 0.6.3, document the behavior and Plan 124, and add repository-tool, evidence-retention, repair, rendering, and synthetic regression coverage.

Validation

  • pnpm test: 1,557 tests across 68 files, including workflow validation.
  • make evals: 39 deterministic synthetic checks across 2 files, with no model calls.
  • pnpm run typecheck, pnpm run build, and git diff --check: passed.

Completed OMSX reviews on the same repository revision (df1125f56a):

Model Time Calls Cost Structured repairs Outcome
OpenRouter Opus 5 high 4m26s 56 $4.2763 0 Complete; 1 finding
OpenRouter DeepSeek 4.1 Flash max 10m13s 90 $0.2581 0 Complete; 3 findings

Neither run hit a budget refusal or required composition fallback. Both still reported unresolved questions. Their success does not establish that XML repair feedback improved live recovery, because neither needed a repair.

Remaining follow-up

Reconciliation can still omit decisive, already-collected source excerpts and leave answered questions in the report. Severity calibration, preserving concrete recommendation expressions, and report concision also need further work. This release improves mechanics and disclosure; it does not claim to eliminate semantic review errors.

Merge this PR before tagging/publishing 0.6.3. This PR does not publish the release.

… handoff

Repository investigations could exhaust local limits before reading decisive
source, treat unsupported-language outlines as complete symbol inventories,
and lose structured tool metadata on cache hits. Repair attempts could repeat
unhelpful feedback, while successful source observations and valid repository
paths could disappear from downstream verification and reporting.

- Raise and document local investigation targets, expose remaining targets to
  models, and derive fixed 2x hard ceilings for aggregate calls, rounds, and
  result characters across packet review, system review, and verification.
  Keep per-result caps, explicit spending limits, and repair deadlines bounded;
  separate invalid/refused requests from executed-call accounting.
- Require valid inclusive read_range bounds before SDK coercion, preserve
  actual delivered ranges and EOF behavior, and guide missing-file recovery
  through path discovery. Make unsupported-language outlines explicit and
  include complete small-file source when it fits.
- Pack searches and file lists on complete entry boundaries, retain omission
  metadata, and preserve complete structured results through the tool cache.
- Retain successful source observations across worker retries and failures
  with attempt/revision provenance; supply bounded collected source to the
  verifier and improve per-question reconciliation evidence selection.
- Validate attention paths against repository-wide known paths, require an
  explicit reconciliation disposition for each supplied concern, and retain
  unresolved questions without accepting unsupported resolution references.
- Feed repairs the latest bounded key/type diagnostics even with replacement
  conversations and custom prompts. Detect XML parameter syntax outside JSON
  strings, provide targeted schema/format guidance, and keep malformed drafts
  rejected until a complete submission passes strict validation.
- Consolidate equivalent cross-file fallback findings without merging distinct
  diagnoses, preserve publication identities and source provenance, and show
  failed synthesis prominently without misrepresenting completed coverage.
- Add Plan 124, regression coverage, and deterministic text-tool synthetics;
  update package version and workflow examples to 0.6.3.

Validation: pnpm test (1,557 tests, 68 files, including workflow validation),
make evals (39 checks, 2 files), pnpm run typecheck, pnpm run build, and
git diff --check all pass.

Latest OMSX runs completed without structured-output repairs or budget
refusals: OpenRouter Opus 5 high in 4m26s ($4.2763), and DeepSeek 4.1 Flash max
in 10m13s ($0.2581). These runs establish successful execution, not causality
for the XML repair improvements, which were not exercised. Evidence selection
can still omit decisive excerpts, and report severity/concision remain
follow-up work; do not claim these remaining quality issues are resolved.
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

🧞 Codegenie review failed (llm_call_failed).

HTTP 400: You have reached your specified API usage limits. You will regain access on 2026-10-01 at 00:00 UTC.

— View Workflow Job

@pkieltyka
pkieltyka merged commit 8cbe1ce into master Sep 25, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant