fix(engine-cuda): bound Qwen3 shape-plan cache with a 2-entry LRU

[Qiiks]

Problem

QwenContext retains every distinct (batch, seq) ShapePlan for the worker's lifetime. Each plan owns a persistent arena (scales with batch*seq*hidden + batch*heads*seq^2, see #2314-style arithmetic), up to 64 MiB of cuBLASLt workspace, a cudaGraph and a cudaGraphExec. A long-lived worker that embeds diverse batch shapes therefore accumulates one arena per distinct shape.

Observed live: the production gateway serving AFT/MC traffic logged repeated cudaMalloc ... out of memory engine crashes (cuda_family_common.cuh:73) after accumulating shapes like 64x172 (arena 646 MB), 16x1117 (1.59 GB), and dozens of 1xN shapes — on a 6 GiB RTX 4050, where persistent f16 weights alone are ~1.1 GiB.

Fix

Bound the cache: retain at most two warm plans per context (max_plans = 2, LRU order).

• Evict before allocate: a cache miss with a full cache evicts the LRU victim before allocating the new arena, so a third full arena never raises the peak.
• Evict while idle: eviction happens after cudaStreamSynchronize on the context's single stream, so victim buffers are freed while no kernel references them; the just-executed plan is promoted to MRU and never the victim.
• RAII teardown: erasing a map entry destroys the unique_ptr<ShapePlan>, whose destructor chain frees graph exec, graph, and every DeviceAllocation (cudaFree). Eviction is what bounds VRAM.
• Also restores the plans member declaration that the eviction edit had dropped (the tree previously failed to compile with 7 undefined-identifier errors).

Verification (live RTX 4050, production gateway lane)

• Full release build of synapse-worker-cuda (--features cuda) green.
• Four distinct shapes (1x34, 1x66, 1x98, 1x130) each captured a plan (captured_exact=true in worker logs).
• Replay of the evicted first shape after the fourth returned a bit-identical vector (max abs delta 0.0), norms all 1.0 — correctness preserved across eviction.
• Steady serving VRAM ~1.9 GiB with no growth across shape churn (previously unbounded, ultimately OOM-crashing).

Notes

• 2 (not larger) was chosen deliberately: embedding workloads are dominated by a couple of hot shapes; per The owned-CUDA capability floor is env-only with no probe and no docs — a supported machine refuses until the operator guesses two variable names #13/The owned-CUDA worker's packaging contract: no release asset, DLL-missing exe fails the acceptance probe at PE load, and no worker_bin default #14 the worker's lifetime is session-scoped and idle-unload already covers VRAM release between sessions. A config knob is deliberately not added (YAGNI); raise max_plans if a workload shows pathological eviction thrash.
• The pad-budget chunker on the gateway side (max 256 items / 4096 padded tokens) already bounds any single plan's arena at ~0.7 GiB worst case; this PR bounds the number of plans.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

---

Summary by cubic

Bounds the Qwen3 shape-plan cache to two entries with LRU eviction, preventing VRAM exhaustion on workers that see many distinct batch shapes.

• Evicts the least-recently-used plan before allocating a new one, after the CUDA stream is idle, so no kernel references the freed buffers.
• Restores the plans member declaration that was accidentally dropped, fixing a compile error.

^{Written for commit d409db1. Summary will update on new commits.}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[cubic-dev-ai]

No issues found across 1 file

_{Re-trigger cubic}