generic-chat adapter: strong error signals depend solely on explicit `is_error` flag — document the contract or widen detection

[stwhwing]

Summary

While wiring a custom runtime (Pi session v3) into evolver via the generic-chat format, auto-distill silently produced zero genes ("not enough to distill — need ≥1 strong signal") even though the session contained real tool failures.

Root cause (source-level, 2.0.30)

In evolver-runtime-adapters/adapters.js (chatMessageToTurns), turn.errorMessage is set for a role=tool message only when is_error/isError === true (or text starts with Exit code: N≠0) — the code comment states: "messages have no universal error flag, so we honor an explicit is_error/isError when the producer sets one". Meanwhile the strong-signal extractor (STRONG_TEXT = /(^|\n)\s*(Error:|Traceback|Exception|FAILED|panic:|\bexit code [1-9])/) only tests the turn's text fields. A toolResult carrying a text-prefixed failure (e.g. ERROR (tool failed): ...) sets neither errorMessage nor matches STRONG_TEXT, so the strong-signal count is always 0 and draftGeneCandidate returns null.

Fix we applied (validated end-to-end)

Our adapter now emits "is_error": true on failed toolResults and on bash executions with non-zero exit codes — after which auto-distill immediately drafts genes (2× strong/error_result + 1× success_prose). Reference implementation:
https://github.com/stwhwing/pi-evox-lab/blob/main/code/pi_session_adapter.js

Suggestion

• Document the is_error contract explicitly in the generic-chat format docs — it is load-bearing for the entire distill path;
• Optionally: recognize common failure markers in toolResult text (Traceback / UnicodeDecodeError / JSONDecodeError / non-zero exit codes) as a fallback, since many transcript exporters won't set the flag.

Full report: https://github.com/stwhwing/pi-evox-lab/blob/main/docs/experiment-report.md (§13)