Skip to content

perf(tui): bound live thinking stream rendering to O(1) tail - #118

Merged
kingsword09 merged 2 commits into
mainfrom
perf-1
Sep 2, 2026
Merged

perf(tui): bound live thinking stream rendering to O(1) tail#118
kingsword09 merged 2 commits into
mainfrom
perf-1

Conversation

@kingsword09

@kingsword09 kingsword09 commented Sep 2, 2026

Copy link
Copy Markdown
Owner
  • Bound active thinking stream rendering to a constant-depth tail buffer (max 24 lines / 4096 chars)
  • Avoid re-parsing the entire accumulated reasoning trace as Markdown on every delta
  • Preserve the complete reasoning trace for expansion (Ctrl+O) and full-text search
  • Protect surrogate pair boundaries during tail trimming
  • Add unit tests for live tail bounding, full trace retrieval, and UTF-16 safety

Fixes #117

   - Bound active thinking stream rendering to a constant-depth tail buffer (max 24 lines / 4096 chars)
   - Avoid re-parsing the entire accumulated reasoning trace as Markdown on every delta
   - Preserve the complete reasoning trace for expansion (Ctrl+O) and full-text search
   - Protect surrogate pair boundaries during tail trimming
   - Add unit tests for live tail bounding, full trace retrieval, and UTF-16 safety
Adds scripts/bench-tui-thinking.ts (bun run bench:tui-thinking) with three
modes for issue #117:

- default: compares HEAD vs pre-fix ThinkingView (7f3d73b) on the same corpus
- --verify: flatness gates (last bucket vs first bucket headroom, absolute
  per-frame ceiling, total ceiling) usable as a regression gate
- --stress: full 100k-line stream through HEAD with CPU/RSS/heap deltas,
  plus geometric checkpoint probes on the legacy component to extrapolate
  its quadratic end-of-stream frame cost
@kingsword09

Copy link
Copy Markdown
Owner Author

Benchmark evidence for issue #117

Added scripts/bench-tui-thinking.ts (commit 5a2ed28, runnable via bun run bench:tui-thinking). Three modes:

Mode Purpose Runtime
bun scripts/bench-tui-thinking.ts compare HEAD vs pre-fix 7f3d73b on the same corpus ~1s
bun scripts/bench-tui-thinking.ts --verify regression gate: flatness + absolute ceilings ~2s
bun scripts/bench-tui-thinking.ts --stress 100k-line stream + CPU/RSS/heap + legacy checkpoint probes ~35s

1. Compare mode — reproduces the reported O(n²)

Same corpus (mixed CJK prose, bold/code, fenced TS blocks, lists), same machine:

trace size legacy last-bucket (ms/frame) HEAD last-bucket (ms/frame) legacy total HEAD total
200 lines (~11 KB) 8.4 0.40 739 ms 93 ms
500 lines (~27 KB) 41.5 0.34 8.2 s 166 ms
1000 lines (~54 KB) 168 0.33 62.9 s 322 ms
2000 lines (~109 KB) 625 0.33 7.9 min 665 ms

At ~50 KB (the issue's "saturation" point) the legacy component is already at ~168 ms/frame (~6 fps). At 109 KB it hits 625 ms/frame.

2. Stress mode — 100k-line stream (5.6 MB trace)

HEAD completes the full 100,000-line stream:

  • wall 34.5 s / CPU 34.9 s (single core)
  • RSS 261 MB (delta +188 MB), heapUsed 24 MB, external 9.3 MB
  • last 100 frames average 0.344 ms/frame, max 0.521 ms/frame
  • getSearchText() retains all 5,594,601 characters for /search and expansion

Legacy cannot complete 100k lines — extrapolated to ~8.2 hours. Instead it is probed at 7 geometric checkpoints (each in a fresh view so total benchmark time stays linear):

checkpoint legacy end-frame (ms)
500 47.6
1,000 192.3
2,000 686.0
4,000 2,572.0
8,000 9,967.4
16,000 39,654.6
32,000 158,946.1

Linear regression slope 4.927 ms/line, R² ≈ 1 (every doubling of trace → ~4× per-frame cost, confirming O(n²)). Extrapolated end-of-stream frame:

  • 50k lines: 246 s (4 min stuck on one frame)
  • 100k lines: 493 s (8.2 min stuck on one frame)

3. --verify regression gate (usable in CI)

All four trials pass (exit 0):

  • last-bucket ≤ first-bucket + 0.5 ms (flatness)
  • largest trial last bucket ≤ 4 ms/frame
  • 2000-line total ≤ 1500 ms

If someone ever regresses the tail buffer back to full-trace rendering, the last bucket jumps to hundreds of ms and the gate fails immediately.


Summary

metric legacy (7f3d73b) HEAD (3920d74)
per-frame cost vs trace length O(n) → O(n²) total O(1) per frame
100k-line end-of-stream frame ~493,000 ms (extrapolated) 0.344 ms
100k-line total render ~8.2 hours (extrapolated) 34.5 s
speedup at 100k lines ~856×
tail-buffer memory ceiling unbounded trace in Markdown 24 lines / 4 KB cap
full trace retained for search/expand yes yes (this.text + getSearchText())

The numbers reproduce the issue's reported "25% CPU / 594 MB / decaying throughput on a 50 KB thinking trace" and confirm the fix keeps active-stream rendering bounded regardless of trace length.

@kingsword09
kingsword09 merged commit 2a63d1d into main Sep 2, 2026
1 of 2 checks passed
@kingsword09
kingsword09 deleted the perf-1 branch September 2, 2026 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

O(n²) thinking-stream rendering: ThinkingView re-parses the whole growing trace as Markdown on every reasoning delta

1 participant