Repository navigation
streamable-http client: no size bound before JSONRPCMessage.model_validate_json — one large server message can OOM the client #3330
Description
Summary
StreamableHTTPTransport parses every inbound SSE event and JSON response body with JSONRPCMessage.model_validate_json / model_validate_json(content) and no size bound. Because pydantic validation of a large JSON document allocates several times the wire size in live Python objects, a single oversized message from a server can exhaust the client's memory. There is no hook to inspect or reject a message before it is parsed.
We hit this in production: a client connected to ~30 MCP servers, one of which answered tools/list with a 21.9 MB single SSE event. Measured with tracemalloc at the parse site:
#1 663.4 MB in 9,957,044 blocks
mcp/client/streamable_http.py:217
message = JSONRPCMessage.model_validate_json(sse.data)
#2 46.9 MB in 3 blocks
httpx_sse/_decoders.py:61 (whole event buffered as one string)
#3 46.9 MB in 2 blocks
httpx_sse/_decoders.py:120 (copy from slicing)
Roughly a 7× amplification of the wire size in live objects, transient but concurrent with other parses. A trivial request that triggered only catalogue loading peaked at 2.35 GB RSS from a 56 MB baseline; heavier concurrent work reached 4.2 GB. Removing that one server dropped the same request's peak to 456 MB. Versions: mcp 1.27.1, Python 3.14.
Why a client-side bound is needed
The client cannot know in advance that a server will return a huge payload, and a misbehaving or misconfigured server should not be able to OOM its client. Today the only outcome is process death with nothing identifying the responsible server — the failure surfaces as an unexplained kill rather than an actionable error.
What we did as a workaround, and why it was awkward
We wrapped StreamableHTTPTransport._handle_sse_event and _handle_json_response to measure sse.data / the response body and reject anything over a configured cap before parsing. Two things made this harder than expected, and both seem worth addressing upstream:
- Raising from the handler is not viable. Every call site wraps it in
except Exception: logger.debug(...)and then reconnects withLast-Event-ID, which replays the same oversized payload. The raise is swallowed and the pending request hangs. - Delivering a bare
Exceptionon the read stream does not fail the request either. InBaseSession._receive_loop,isinstance(message, Exception)routes to_handle_incoming, which for the default message handler is a no-op; only aJSONRPCResponse/JSONRPCErrormatching_response_streams[request_id]completessend_request, and that awaits withtimeout=None. To fail the request we had to synthesize aJSONRPCErrorand recover the request id from the raw payload with a regex, since parsing it is exactly what we were trying to avoid.
Suggested improvements
- An optional client-side maximum message size (constructor argument and/or environment variable) enforced before
model_validate_json, on both the SSE andapplication/jsonpaths. - When it trips, fail the corresponding pending request with a distinct error identifying the endpoint and the observed size, rather than letting the reconnect path replay the payload.
- Failing that, a documented hook to inspect a raw message before parsing would let clients implement this without patching private methods.
- Independently:
_handle_sse_eventreturningFalsefor a parse failure causes the caller to treat the stream as ended and reconnect withLast-Event-ID; for a deterministic failure (such as a payload that will always be too large, or malformed JSON) this retries something that cannot succeed.
Happy to open a PR if a maintainer indicates the preferred shape (constructor arg vs. env var vs. pre-parse hook).
Activity
Ben-Home commentedon Aug 18, 2026
We hit the same class of issue in production with a large connector fleet, and the fix that works is to put the size gate at the transport boundary, not at the validator. model_validate_json on a big document is the wrong layer to defend: pydantic will allocate the full object graph regardless, so by the time validation runs the memory is already spent.
Practical pattern:
One subtlety: rejecting after a partial read is fine for HTTP, but for SSE you should drop the whole event, not resync mid-stream, because the next data field continues the same logical message.
Our MCP gateway at corpusiq.io enforces a default per-message cap at the boundary and we have not had a client OOM since; the pattern is roughly 20 lines in the transport layer, and it also gives you a clean error path to surface to the user ("server sent an oversized response") instead of a killed process.
tarunag10 commentedon Aug 20, 2026
I checked this against draft #3338. That PR intentionally restores unbounded SSE compatibility by passing
max_event_size=None; it is useful groundwork for consistent decoder configuration, but it does not solve this OOM issue and leaves JSON unbounded too.Before implementing a public API, I propose this contract:
max_inbound_message_size, rather than an SSE-onlymax_event_size;application/jsonresponse bytes instead of materializing an unlimited body first;Last-Event-IDdoes not replay the same oversized event;litellm_trace_id-style session usability after a request-scoped rejection where the underlying stream remains safe.Suggested acceptance matrix: exactly-at-limit and limit+1 for SSE and JSON; multibyte payloads counted as wire bytes; assert
model_validate_jsonis never called after rejection; no reconnect/replay for deterministic oversize; correlated request completion; standalone GET-stream behavior; and a subsequent normal request proving session recovery. A default and v1/v2 exposure policy are the remaining maintainer decisions. If this shape is acceptable, I can implement it after assignment in line with the repository contribution policy.Vish12345678673 commentedon Sep 18, 2026
Hi! I would like to work on this issue.
I reviewed the Streamable HTTP receive path and the reported memory-amplification failure. I would like to reproduce the behavior locally and implement a focused pre-parse message-size guard with regression coverage, while preserving the existing reconnect and pending-request semantics.
I have not started a PR yet and will wait for maintainer confirmation before proceeding.
Suhas-123-cell commentedon Oct 29, 2026
The stdio client has the same gap, one layer earlier, so it may be worth covering in whatever option this issue settles on.
In
mcp/client/stdio.py(stdio_client→stdout_reader, mcp 2.2.0):buffergrows until the server writes a newline, with no limit. A server that writes a long run of output without one grows the client's memory until the process is killed, and since every chunk re-concatenates and re-splits the whole buffer, the time spent is quadratic in the line length as well. As with HTTP, the client can only check a message's size after it has arrived whole, and there is no hook to stop it earlier short of copying the transport or patchingTextReceiveStream.This matters most for clients that sit between an agent and servers they don't fully trust (gateways, proxies, guards), where one misbehaving server should cost one failed connection, not the client process.
A transport-neutral
max_inbound_message_size, as proposed above, would cover it: on stdio, once the pending line passes the cap, fail the connection the same way a broken stream is handled, so the session sees it closed.