Skip to content

seidroid-review: re-dispatch a review that never reached the agent (exit 6) #105

Description

@devin-ai-integration

Reviews die with driver exit 6 — "transport to the agent failed" — often enough to cost roughly a quarter of runs in the sampled repos (PLT-1234). Exit 6 means the run decided nothing: no verdict, no position, no model quota spent on one. The job fails and the pull request gets no review.

The driver half landed in sei-protocol/sei-internal-skills#427: the run-key session lookup now retries a request that never reached the server, so a blackholed listing inside one dispatch no longer ends the run. What it cannot recover is a connection that stays dead for the whole process — that needs a new process, which only this workflow can start.

The change

One new input, transport-redispatches (number, default 2). 0 restores today's single-dispatch behaviour.

Wrap the driver invocation in Drive session + collect verdict. It currently runs the binary once:

set +e
"$BIN" "${args[@]}"
rc=$?
set -e

It should re-dispatch on exit 6 up to the allowance:

dispatch=1
while :; do
  set +e; "$BIN" "${args[@]}"; rc=$?; set -e
  if [ "$rc" -ne 6 ] || [ "$dispatch" -gt "${REDISPATCHES:-0}" ]; then break; fi
  echo "::warning::dispatch $dispatch did not reach the agent (exit 6); re-dispatching"
  sleep $(( dispatch * 10 ))
  dispatch=$(( dispatch + 1 ))
done
echo "dispatches=$dispatch" >> "$GITHUB_OUTPUT"

Plus a $GITHUB_STEP_SUMMARY line when dispatch > 1, so the failure rate this absorbs stays visible: a job that goes green on its second dispatch is the only place a transport failure gets reported.

Three things that are load-bearing

  • Only exit 6 buys another dispatch. Every other non-zero code means the run decided something. Re-running those either duplicates a verdict or repeats a real failure.
  • Same arguments, therefore the same run key (repository + pull request). That is what makes dispatch 2 adopt dispatch 1's session through the lookup in #427 rather than open a second one and review the pull request twice. No nonce, no timestamp in the args.
  • The downstream gate stays as it is. It keys on verdict.md being non-empty, not on the exit code, so a run that reached a verdict still publishes and a no-verdict run still posts nothing.

A re-dispatch is a new process on a fresh connection, which is the point — retrying inside the driver would repeat the request on the same dead connection, and in-run recovery has already failed by the time exit 6 is returned. The cost is a job's worth of runner time against a failure rate measured in whole percent.

Testing

test/seidroid-review/redispatch.sh drives the step against a stub driver taking its exit codes from STUB_EXITS, with sleep shadowed so the backoff is asserted rather than waited out. It reads the allowance out of the workflow YAML rather than repeating the default.

Ready-made patch

The change is written and its harness passes locally, but Devin has no push access to this repository (git push returns 403 through the git proxy), so it could not be opened as a pull request. The patch is available from the Devin session below and applies with git am — 6 files: the workflow, the new harness, its stubs, .gitignore, and the test README.

Written by Devin

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions