Skip to content

Fix SSE buffering in Claude relayed proxy - #361

Merged
lilly-luo merged 1 commit into
mainfrom
fix/claude-relayed-sse-buffering
Aug 20, 2026
Merged

Fix SSE buffering in Claude relayed proxy#361
lilly-luo merged 1 commit into
mainfrom
fix/claude-relayed-sse-buffering

Conversation

@rohita5l

Copy link
Copy Markdown
Collaborator

Summary

  • Forward upstream SSE/network chunks immediately instead of buffering them to 8 KiB.
  • Add opt-in, metadata-only relayed-proxy transport diagnostics.
  • Add an artifact-focused relayed-auth reproduction harness and focused tests.

Root cause

Passing a fixed 8192-byte chunk size to httpx Response.iter_raw caused httpx to accumulate small SSE heartbeat frames. During long Claude artifact or document generation, Claude Code could receive no downstream bytes for minutes and report that the response stopped arriving, then retry.

Using the default raw iterator forwards each upstream network chunk as it arrives, preserving heartbeat delivery and keeping the downstream stream active.

Validation

  • Focused tests: 33 passed.
  • Focused Ruff check: passed.
  • Live A/B against main.jasmine.rohit-test:
    • Before: HTTP 200, zero downstream bytes, client disconnect at 284s, no artifact.
    • After: first byte at 1.263s, 1,812,360 bytes across 4,025 chunks, completed in 358.5s; valid 85 KB Markdown artifact with all 60 sections and ARTIFACT_TEST_DONE.
  • Three additional fixed-proxy artifact trials completed with zero suspicious markers:
    • 333.9s, valid 86,031-byte artifact, 1,777,033 streamed bytes across 3,299 chunks.
    • 337.3s, valid 82,312-byte artifact, 1,704,030 streamed bytes across 4,195 chunks.
    • 280.1s, valid 70,102-byte artifact, 1,445,877 streamed bytes across 3,226 chunks.

Each additional artifact contains one title plus exactly 60 requested sections and the ARTIFACT_TEST_DONE marker.

Comment thread src/ucode/gateway_proxy.py
@lilly-luo
lilly-luo enabled auto-merge (squash) August 20, 2026 16:56
@lilly-luo
lilly-luo merged commit 3ac4088 into main Aug 20, 2026
2 checks passed
@lilly-luo
lilly-luo deleted the fix/claude-relayed-sse-buffering branch August 20, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants