Polish GPT-2 demo: multi-head fusion, KV cache, fewer launches

[tetsuo-cpp]

Summary

Improve the GPT-2 attention demo from a correctness spike into a more realistic inference path: multi-head (or fused) kernels, KV cache support, and fewer launches per layer.

Motivation

Current demo limitations (demo/README.md):

• Batch size 1
• No KV cache (use_cache=False) — full recompute every step
• Max sequence length 1024 (shared allocation)
• 12 kernel launches per layer (one per head)

That is enough to prove WarpForth can run real attention, but not enough to showcase competitive structure or exercise multi-buffer / multi-head patterns.

Proposed improvements (incremental)

1. Fused multi-head kernel or grid mapping that covers all heads in one launch
2. KV cache: accept K/V cache tensors and only compute the new row; update host patch to use_cache=True
3. Batch > 1 (even batch=2) to force batch indexing discipline
4. Use Python package / metadata APIs when available instead of bespoke PyCUDA glue
5. Document tokens/s before vs after on a fixed GPU SKU

Acceptance criteria

• At least one of: fused multi-head or KV cache path working end-to-end
• Demo README updated with new flags and remaining limits
• Numerical parity check vs Hugging Face reference on a short prompt
• Launch count per layer documented and reduced if fusion lands

Non-goals

• Matching FlashAttention performance
• Full training support

Related

• Attention kernel (Naive attention kernel in Forth #44 closed), GPT-2 demo (GPT-2 interactive inference demo with WarpForth attention #46 closed)
• Python package / metadata issues
• Warp reductions (Warp-level primitives: shuffle and reductions #10) for faster softmax