Skip to content

cycle execute/verify is fail-closed for every runtime; no plugin mechanism for third-party runners (e.g. Pi) #627

Description

@stwhwing

Summary

We tried to use the official evolver cycle path to materialize Capsules offline and hit a hard wall (all on 2.0.30, re-verified today):

  • evolver cycle --help: --runner enumerates only claude|codex|gemini;
  • evolver cycle capabilities --json: all nine runtimes report execute/verify unsupported or experimental (only gemini is experimental; generic-chat — which we ingest from — is fully unsupported for execute);
  • evolver solidify --dry-run --json{"deprecated":true,"replacement":"evolver cycle; evolver review; evolver publish"};
  • evolver cycle --runner gemini --limit 1action=fail status=refused reason=details-redacted.

Impact

Fully offline environments (or any runtime outside the three-name allowlist) cannot exercise the official cycle path at all. In our project we ended up building an equivalent offline orchestrator (run → independent validation → guarded strategy injection → cross-round metrics): https://github.com/stwhwing/pi-evox-lab

Suggestion

  • Allow a pluggable external runner (a command/container contract plus --validation-cmd, similar to the existing runner test seam in autoDistillTranscript), so third-party runtimes can participate in cycle under their own containment;
  • Or expose a documented "validation-only" cycle mode for offline harnesses.

Full report: https://github.com/stwhwing/pi-evox-lab/blob/main/docs/experiment-report.md (§19)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions