`engine: julia` fails with bare `ERROR: Internal Error` on large documents — `writeJuliaCommand` does not handle a partial socket write

[plessl]

Bug description

writeJuliaCommand in the Julia engine sends the whole notebook payload with a
single conn.write() and treats anything but a full write as unreachable:

quarto-cli/src/resources/extension-subtrees/julia-engine/src/julia-engine.ts

Lines 835 to 838 in eb126f9

	const bytesWritten = await conn.write(messageBytes);
	if (bytesWritten !== messageBytes.length) {
	throw new Error("Internal Error");
	}

const bytesWritten = await conn.write(messageBytes);
if (bytesWritten !== messageBytes.length) {
  throw new Error("Internal Error");
}

Deno.Conn.write() has POSIX write(2) semantics: on a TCP socket it writes as
much as the send buffer can absorb and returns a short count. That is
documented, expected behaviour — the caller is meant to loop (or use
writeAll). The engine imports no write helper and does not loop, so any
document whose encoded payload outgrows the socket send buffer dies with a bare
Error("Internal Error") naming no file, no line and no cell.

The Julia side is left holding a message with no terminating newline and logs a
JSON UnexpectedEOF, which is the first thing a user finds and is a red
herring — it points at whatever content happens to sit at the truncation offset
rather than at the writer.

This is not exotic: a slide deck or book chapter with a dozen or so inlined SVG
figures reaches a few hundred KB easily, because {{< include >}} is resolved
before execution, so the whole inlined document is what gets sent.

Suggested fix

Loop until the buffer is drained:

let offset = 0;
while (offset < messageBytes.length) {
  offset += await conn.write(messageBytes.subarray(offset));
}

or await writeAll(conn, messageBytes) from @std/io.

Verified against the same slow-reader harness as below — 400 000, 1 000 000 and
4 000 000 byte messages all arrive intact (5 400 000 sent, 5 400 000 received)
where the unpatched call short-wrote at 173 708.

Worth considering alongside it: throw new Error("Internal Error") gives the
user nothing to act on. Even "failed to write N bytes to the Julia server (wrote
M)" would have made this self-diagnosing.

How this was diagnosed

The document that first hit this was a lecture deck with fifteen inlined
figures. It rendered with eight and failed with fifteen, while every Julia cell
in it executed fine in isolation — which is what sent the search towards payload
size rather than cell content.

The diagnosis and the patch above were produced with Claude Code: bisecting the
document by figure to bracket the threshold, reading writeJuliaCommand against
the Deno.Conn.write() contract, and building the slow-reader harness that
shows the short write with no Quarto in the picture. Every number quoted in this
report was then reproduced on the machine described under My environment
rather than taken on trust.

Steps to reproduce

No project, extensions or figures needed — one Julia cell plus enough raw
content to exceed the socket buffer.

python3 - <<'PY'
fence = "`" * 3
body = "\n".join(
    f'<span class="x{i}">Lorem ipsum dolor sit amet, '
    f'consectetur adipiscing elit.</span>'
    for i in range(4000))
open("big.qmd", "w").write(
    "---\ntitle: repro\nengine: julia\n---\n\n"
    + fence + "{julia}\n#| echo: false\n1 + 1\n" + fence + "\n\n"
    + fence + "{=html}\n" + body + "\n" + fence + "\n")
PY

quarto render big.qmd

big.qmd comes out at 334 978 bytes and fails; the same file built with
range(1500) is 124 978 bytes and renders fine, so the two bracket the
threshold on this machine.

Independent confirmation that conn.write short-writes

The engine's assumption can be checked without Quarto at all:

const listener = Deno.listen({ port: 0, hostname: "127.0.0.1" });
const addr = listener.addr as Deno.NetAddr;
(async () => {                      // deliberately slow reader
  const c = await listener.accept();
  await new Promise((r) => setTimeout(r, 3000));
  const buf = new Uint8Array(65536);
  while (await c.read(buf) !== null) {}
})();
const conn = await Deno.connect({ port: addr.port, hostname: "127.0.0.1" });
for (const size of [64_000, 260_000, 400_000, 1_000_000]) {
  const written = await conn.write(new Uint8Array(size).fill(65));
  console.log(size, "->", written, written === size ? "ok" : "SHORT WRITE");
}

    64000 ->     64000 ok
   260000 ->    260000 ok
   400000 ->    173708 SHORT WRITE
  1000000 ->    343180 SHORT WRITE

Actual behavior

ERROR: Internal Error

Stack trace:
    at writeJuliaCommand (…/julia-engine/julia-engine.js:1155:11)
    at async executeJulia (…/julia-engine/julia-engine.js:1102:20)
    at async Object.execute (…/julia-engine/julia-engine.js:741:20)
    at async renderExecute (…/quarto.js:136835:25)
    …

and, from quarto call engine julia log:

┌ Error: Failed to parse json message.
│   error =
│    ArgumentError: invalid JSON at byte position 327214 while parsing type String: UnexpectedEOF
│    /span>\n<span class=\\
│
└ @ QuartoNotebookRunner ~/.julia/packages/QuartoNotebookRunner/evCNi/src/socket.jl:197

The offset lands in the middle of a <span> — that is the truncation point, not
a defect in the content, and it is what makes the message misleading.

The truncation offset is reproducible for a given machine and workload, but it
is a socket buffer size, not a protocol constant: on macOS
net.inet.tcp.sendspace starts at 128 KiB and auto-tunes towards
net.inet.tcp.autosndbufmax, and how much gets through also depends on how
quickly the server drains. Users on different platforms will see different
thresholds, which is part of why this is hard to recognise.

Expected behavior

The payload is written in full regardless of size, and the document renders. If
a write genuinely cannot complete, the error should name the document rather
than being an unqualified Internal Error.

My environment

• macOS 26.6.2 (build 25G83), Apple silicon
• Julia 1.12.7
• QuartoNotebookRunner 0.17.4
• net.inet.tcp.sendspace = 131072, net.inet.tcp.autosndbufmax = 4194304

Bug is present in the shipped julia-engine.js of Quarto 1.10.18 and in
src/julia-engine.ts on main (checked at eb126f9, where it is still a single
unlooped conn.write with no writeAll anywhere in the file).

Quarto check output

Quarto 1.10.18
[✓] Checking environment information...
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.10.0: OK
      Dart Sass version 1.101.0: OK
      Deno version 2.7.14: OK
      Typst version 0.15.1: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.10.18
      Path: /Applications/quarto/bin
[✓] Checking tools....................OK
[✓] Checking LaTeX....................OK
      Using: Installation From Path
[✓] Checking Chrome Headless....................OK
[✓] Checking basic markdown render....OK
[✓] Checking R installation...........(None)
[✓] Checking Python 3 installation....OK
      Version: 3.14.6

[gordonwoodhull]

cc @jkrumbiegel