[Bug?]: server-fn chunk header is read before it has fully arrived — any chunk boundary in the first 10 bytes of a frame corrupts the response

[binnodon]

Duplicates

• I have searched the existing issues

Latest version

• I have tested the latest version

Current behavior 😯

SerovalChunkReader.next() waits for more data only when the buffer is completely empty. It guards the payload length with a while loop, but never checks that the 12-byte frame header has fully arrived before reading it.

A ReadableStream gives no guarantee that a chunk boundary won't fall inside those 12 bytes. When it does, subarray(1, 11) returns a truncated length field, Number.parseInt reads the wrong value (or NaN), and the response is either rejected or mis-framed.

This only shows up when the response is delivered in more than one chunk, which is why it is effectively invisible locally and appears in production behind a proxy/tunnel/CDN — anywhere TCP segmentation or a small MTU splits the body.

Expected behavior 🤔

When running the 'Steps to reproduce' the payload should round-trips regardless of where the stream is chunked. Framing is a transport concern; chunk boundaries are not the sender's to control.

Steps to reproduce 🕹

Self-contained: no dependencies, no network, no seroval. Both classes are transcribed verbatim from @solidjs/start@2.0.1 dist/fns/serialization.js. Save as repro.mjs and run node repro.mjs (Node 18+).

// --- server side: createChunk(), verbatim ---------------------------------
function createChunk(data) {
  const encodeData = new TextEncoder().encode(data);
  const bytes = encodeData.length;
  const baseHex = bytes.toString(16);
  const totalHex = "00000000".substring(0, 8 - baseHex.length) + baseHex;
  const head = new TextEncoder().encode(`;0x${totalHex};`);
  const chunk = new Uint8Array(12 + bytes);
  chunk.set(head);
  chunk.set(encodeData, 12);
  return chunk;
}

// --- client side: SerovalChunkReader, verbatim ----------------------------
class SerovalChunkReader {
  constructor(stream) {
    this.reader = stream.getReader();
    this.buffer = new Uint8Array(0);
    this.done = false;
  }
  async readChunk() {
    const chunk = await this.reader.read();
    if (!chunk.done) {
      const newBuffer = new Uint8Array(this.buffer.length + chunk.value.length);
      newBuffer.set(this.buffer);
      newBuffer.set(chunk.value, this.buffer.length);
      this.buffer = newBuffer;
    } else {
      this.done = true;
    }
  }
  async next() {
    if (this.buffer.length === 0) {
      if (this.done) return { done: true, value: undefined };
      await this.readChunk();
      return await this.next();
    }
    const head = new TextDecoder().decode(this.buffer.subarray(1, 11));
    const bytes = Number.parseInt(head, 16);
    if (Number.isNaN(bytes)) throw new Error("Malformed server function stream.");
    while (bytes > this.buffer.length - 12) {
      if (this.done) throw new Error("Malformed server function stream.");
      await this.readChunk();
    }
    const partial = new TextDecoder().decode(this.buffer.subarray(12, 12 + bytes));
    this.buffer = this.buffer.subarray(12 + bytes);
    return { done: false, value: partial };
  }
}

// --- deliver one frame in two chunks, varying the split point -------------
const PAYLOAD = '($R[0]={"hello":"world"})';
const framed = createChunk(PAYLOAD);

const streamSplitAt = (bytes, at) =>
  new ReadableStream({
    start(c) {
      c.enqueue(bytes.subarray(0, at)); // one segment
      c.enqueue(bytes.subarray(at));    // the next
      c.close();
    },
  });

for (let at = 1; at <= 13; at++) {
  const reader = new SerovalChunkReader(streamSplitAt(framed, at));
  const out = [];
  let verdict;
  try {
    for (;;) {
      const r = await reader.next();
      if (r.done) break;
      out.push(r.value);
    }
    const got = out.join("");
    verdict = got === PAYLOAD ? "ok" : `CORRUPTED -> ${JSON.stringify(got)}`;
  } catch (e) {
    verdict = `THREW -> ${e.message}`;
  }
  console.log(`split at ${String(at).padStart(2)} | ${verdict}`);
}

Running the above, the output is:

split at  1 | THREW -> Malformed server function stream.
split at  2 | THREW -> Malformed server function stream.
split at  3 | THREW -> Malformed server function stream.
split at  4 | THREW -> Malformed server function stream.
split at  5 | THREW -> Malformed server function stream.
split at  6 | THREW -> Malformed server function stream.
split at  7 | THREW -> Malformed server function stream.
split at  8 | THREW -> Malformed server function stream.
split at  9 | THREW -> Malformed server function stream.
split at 10 | THREW -> Malformed server function stream.
split at 11 | ok
split at 12 | ok
split at 13 | ok

Splits at 11+ are fine because bytes 1–10 (the 0x + 8 hex digits) have all arrived by then. Any earlier boundary breaks the frame.

Context 🔦

My production environment chunks the streams which leads to regular stream failures

Root cause

dist/fns/serialization.js (2.0.1), in next():

if (this.buffer.length === 0) {          // only guards "nothing at all"
  ...
  await this.readChunk();
  return await this.next();
}
const head = new TextDecoder().decode(this.buffer.subarray(1, 11));  // may be short
const bytes = Number.parseInt(head, 16);
if (Number.isNaN(bytes)) throw new Error("Malformed server function stream.");
while (bytes > this.buffer.length - 12) { ... }   // payload IS guarded

With a 3-byte buffer, subarray(1, 11) clamps to subarray(1, 3) → "0x" → NaN. With a 10-byte buffer it yields "0x0000001", a plausible-looking but wrong length, so the payload slice is taken at the wrong offset.

Suggested fix

Mirror the existing payload guard, before reading the header:

while (this.buffer.length < 12) {
  if (this.done) throw new Error("Malformed server function stream.");
  await this.readChunk();
}
const head = new TextDecoder().decode(this.buffer.subarray(1, 11));

Affected versions

Confirmed identical in 1.1.7, 1.3.2 and 2.0.1 (current latest). I did not check further back than 1.1.7.

Additional context

How it surfaces in 1.x. In the 1.x line next() evaluates the payload inline (value: deserialize(partial)), so a header truncated to a small-but-valid number produces an empty slice that evaluates to undefined without complaint. The stream then resyncs onto the middle of the payload and the failure appears later, as:

Uncaught (in promise) SyntaxError: Unexpected end of input
    at eval (<anonymous>)
    at Hn (...)        <- seroval's deserialize
    at no.next (...)
    at async je (...)

That trace is misleading — this is not a seroval bug. seroval owns value↔string serialisation and has no length-prefix framing at all (no subarray(1, 11), no readChunk). It is handed a truncated source string by the un-framer above and evaluates it faithfully. I mention it because anyone hitting only the SyntaxError variant would reasonably file against seroval first, which may be why this doesn't appear to have been reported.

Before 1.1.5 this was a hang, not an error. Without the Number.isNaN guard (added in 1.1.5), a NaN length makes every comparison false, so subarray(12 + NaN) is subarray(0) — the buffer never advances, done is never set, and drain()'s for (;!(await this.next()).done;) spins on already-resolved promises. That is an infinite microtask loop, so it starves the event loop completely: no timers, no paint, no input, and DevTools cannot break in. We spent a while chasing an unresponsive tab that turned out to be this.

Diagnosability. The 1.1.x–1.3.x message includes the offending header text (Malformed server function stream header: 0x), which is what let us identify the truncation. 2.0.1 uses the generic "Malformed server function stream." for both branches. Keeping the header value — and distinguishing "header incomplete" from "payload incomplete" — would make this much easier to diagnose in the field.

Possibly related: #2295 (server-fn stream responses and proxies). Different defect, but the same subsystem and the same "only reproduces behind a proxy" character.

Your environment 🌎