Skip to content

fix(supervisor): bound pending exec stdin and cancel stalled writers - #3846

Open
shiju-nv wants to merge 3 commits into
NVIDIA:mainfrom
shiju-nv:fix/exec-input-bounds
Open

shiju-nv wants to merge 3 commits into
NVIDIA:mainfrom
shiju-nv:fix/exec-input-bounds

Conversation

@shiju-nv

Copy link
Copy Markdown
Collaborator

Summary

Bound pending stdin to 4 MiB per ordinary SSH exec channel, including the active write, and release stalled input writers on channel close or disconnect. Overflow reports exit code 74 and requests termination of the affected exec while keeping the SSH callback responsive to other channel traffic.

Related Issue

No issue required: this corrects an unbounded queue and detached input writer in the existing exec path.

Changes

  • Replace the unbounded queue and detached OS thread with a byte buffer and cancellable async writer. Coalesce small packets and reject excess input before copying it.
  • Preserve EOF draining and reuse capacity as input is written. Slow consumers that exceed the pending-input budget receive a failed exec; the supervisor logs the reason.
  • Keep overflow handling independent of output credit. Do not enqueue diagnostic stderr that could delay another channel's completion.
  • Replace tests that duplicated the old writer loop with coverage through the real SSH handler, and document the overload behavior.

Testing

  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

Preserve upstream architecture documentation removal while retaining the published stdin overload contract.

Signed-off-by: Shiju <shiju@nvidia.com>
Keep the pending-input limit beside the RPC lifecycle contract so the streaming CLI documentation can merge independently.

Signed-off-by: Shiju <shiju@nvidia.com>
@johntmyers johntmyers added gator:in-review Gator is reviewing or awaiting PR review feedback test:e2e Requires end-to-end coverage labels Sep 29, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for 2595b40. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@johntmyers johntmyers left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gator-agent

PR Review Status

The initial code-only review found no blocking findings. The supervisor-process change is project-valid as a localized resource-bound and cancellation fix, its user-visible overload behavior is documented, and the required E2E workflow is queued for this head.

Blocking findings:

  • No blocking findings remain

Carried findings:

  • None

Non-blocking suggestions:

  • None
Gator metadata
  • Validation: Localized supervisor correctness fix with handler-level regression coverage
  • Docs: Fern documentation updated in docs/how-it-works/sandboxes/overview.mdx
  • Checks: Current-head baseline checks completed; required E2E is queued
  • E2E: test:e2e applied and Branch E2E Checks run 36630140527 rerun queued
  • Head SHA: 2595b40f6f060437ccb5b29c1145b499e9d80bcd
  • Base SHA: 33a8eac196a21aff3125dea46462a4591bcb5d0b
  • Merge base SHA: 33a8eac196a21aff3125dea46462a4591bcb5d0b
  • Patch ID: a296c71d0634a50b6252e3a70c1dd620c426e2c6
  • Gator payload: 9
  • Review mode: initial
  • Previous reviewed SHA: none
  • Review budget exhausted: no
  • Maintainer decision required: no
  • Next state: gator:watch-pipeline

@johntmyers johntmyers added gator:watch-pipeline Gator is monitoring PR CI/CD status gator:blocked Gator is blocked by process or repository gates gator:approval-needed Gator completed review; maintainer approval needed and removed gator:in-review Gator is reviewing or awaiting PR review feedback gator:watch-pipeline Gator is monitoring PR CI/CD status gator:blocked Gator is blocked by process or repository gates labels Sep 29, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gator:approval-needed Gator completed review; maintainer approval needed test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants