Windows: killing one endpoint's process takes down the shared daemon and every attached connection ("Connection closed") #2796

Version

codebase-memory-mcp 0.12.1

Platform

Windows x64 (10.0.26300)

Install channel

GitHub release archive (windows-amd64.zip)

Binary variant

standard

What happened, and what did you expect?

Short version: when an MCP client kills one of its codebase-memory-mcp.exe child processes (e.g. an idle-connection eviction sweep), the shared daemon dies with it, and every other endpoint exe that was attached to that daemon exits with code 1. The next tool call on any surviving pooled connection fails with Connection closed. Reconnect heals it, but every in-flight/idle connection breaks at once.

Observed with the ZCode desktop client (Electron), whose MCP pool keeps several stdio connections per server and evicts idle ones ~30s after their lease refcount drops to 0 — the eviction path terminates the child process. Two failures on the first evening of multi-session usage, both with the identical signature.

Topology observed (v0.12.1, Windows):

• Each stdio spawn is a thin endpoint; endpoints share one detached daemon that listens on 127.0.0.1:9749 (default UI port) and owns the cache/index.
• Daemon lifetime is managed by a "last committed client disconnected" heuristic (runtime_stopping → lifetime_end).

Evidence from both sides (same incident):

• Client pool log: victim connection idle-but-leased; death discovered at next tools/call → instant connection_lost → Connection closed. In both incidents the victim died 4s / 20s after a sibling pooled connection was closed by the pool.
• Server cbm-daemon.log: zero entries for the victim endpoints' death — no runtime_stopping, no lifetime_end, no error. They vanished without running any graceful path (consistent with external termination). The failed tool calls never reached a handler (no tools/call line).

Reproduction

Deterministic lab repro (Node, Windows — two pooled endpoints + kill the daemon):

// A and B = two spawned `codebase-memory-mcp --tool-profile=analysis` stdio endpoints
// (initialize handshake done on both; a tools/call completed on each)
// daemon = the process listening on 127.0.0.1:9749 (spawned by A's launch)
killProcess(daemonPid)          // simulate a client sweep killing a child
// → B's exe exits with code 1 immediately
// → B's next JSON-RPC request hits EOF  → client reports "Connection closed"

Control: closing endpoint A's stdin cleanly leaves the daemon and B unharmed (daemon survives, B keeps serving). Only abrupt process termination of the daemon cascades.

Full repro script and analysis artifacts: I can attach on request (client-side pool events with traceIds, server log timeline, process-table correlation).

Logs

Server-side (C:/cbm-cache/logs/cbm-daemon.log) around an incident — note the absence of any shutdown trail for the victim:

level=info msg=mcp.request protocol=jsonrpc method=tools/call tool=list_projects status=ok duration_ms=1678
level=info msg=mcp.request protocol=jsonrpc method=tools/call tool=index_status status=ok duration_ms=109
... 15 minutes of no server-side activity (client idle) ...
[victim endpoint dies: NO log line at all — no runtime_stopping, no lifetime_end]
level=info msg=mcp.request protocol=jsonrpc method=tools/call tool=get_architecture status=ok duration_ms=1219   <- after client reconnect

Client-side pool events (ZCode zcode-<date>.jsonl), same incident:

12:06:25Z info  mcp.pool.lease...  (idle begins)
12:20:14Z info  mcp.pool.lease.released  (sibling connection, ref=0)
12:20:45Z info  mcp.server.closed (sibling, status=completed)
12:21:05Z info  tool.call.started          <- my next call on the victim connection
12:21:05Z warn  mcp.server.connection_lost status=failed
12:21:05Z error tool.call.failed  "Connection closed"

Diagnosis and suggested fix directions

Root cause chain: the daemon is spawned (detached) by an endpoint, so on Windows it remains inside the spawning client's job/process tree; a client that reaps its children with a kill sweep takes down the shared daemon that other, still-live connections depend on. Endpoints treat daemon loss as fatal (exit 1) rather than re-attaching.

Two candidate fixes (either one breaks the cascade; both is best):

1. Escape the sweep (Windows, minimal): spawn the daemon with CREATE_BREAKAWAY_FROM_JOB / assign it to its own job object, so a client-side kill-on-close sweep of its tree cannot reach a daemon that serves multiple clients.
2. Endpoint resilience: on daemon connection loss, attempt to spawn/re-attach to a daemon before giving up, instead of exiting.

Workaround currently deployed

A "keeper" process outside the client's tree holds an initialized MCP connection, so the daemon is owned by the keeper and client sweeps can't reach it. Works reliably; happy to share as a script if useful.

Diagnostics trajectory (memory / performance / leak issues)

Not a memory/leak issue; lifecycle only. Side observation: killed endpoints leak as orphaned codebase-memory-mcp.exe processes (several MB each) — related to #2400's "multiple exe" report, but the cascade above is the core defect.