v1.8.0: AI/LLM real-time token streaming engine - #14
Merged
Merged
Conversation
…ith frame-aligned throttling, transient syntax auto-repair, typing carets, and auto-scroller
…mapper, stream pause/resume/cancel controls, and LaTeX math & markdown link transient syntax auto-repair
…for AI streaming demo
…nd 1000-token throughput benchmark suite
… publish script, html normalizer, and update docs for v1.8.0 release
…am package resolution
…ll, and fallback gates Code review of the v1.8.0 streaming feature found several bugs where HyperViewer's streaming path didn't wire into existing state correctly: - didUpdateWidget missed streamingController swaps and autoRepairSyntax toggles, so neither triggered a reparse until the next stream tick. - The content fade restarted on every throttled streaming tick (as often as 16ms), faster than its own 300ms duration, causing perpetual flicker instead of a single settle-in fade. - fallbackBuilder's complexity gate checked widget.content, which .streaming() hardcodes to '', so it could never fire regardless of how complex the streamed HTML became. - The virtualized ListView.builder bound widget.controller?.scrollController directly instead of the existing _effectiveScrollController fallback, so autoScrollToBottom silently no-op'd in virtualized/auto mode without an external controller.
…appends and add throttle backoff - append() now rejects tokens after error() (previously only guarded against completed), so a stray append can no longer silently resume a stream that already failed and erase its error state without a reset() call. - bindStream/bindCustomStream now wrap onData (and the mapper call) in try/catch and route genuine failures into error(), matching the "catches errors" dartdoc — a synchronous throw from append() inside onData previously bypassed onError entirely and surfaced as an unhandled zone exception. A failure racing an already-terminal state is swallowed instead of overwriting a legitimate completed/error value. - Added adaptive throttle backoff (_effectiveThrottleDuration): the notification interval widens as the buffer grows past 10,000 and 50,000 chars (up to a new maxThrottleDuration, default 200ms), bounding the total cost of full-document reparsing on every tick over a long stream. Short/typical streams keep the default 16ms cadence. Adds fake_async as a dev dependency to deterministically test the adaptive throttle without wall-clock timing.
…ences normalizeMarkdown's backtick/tilde/asterisk/math parity checks and its incomplete-link repair scanned the entire accumulated buffer, so an odd character count or unmatched bracket sitting inside an already-closed, fully-rendered code fence earlier in the document was miscounted as unclosed — corrupting unrelated trailing prose with a spurious appended character on every subsequent render tick. Both now scan only the text after the last closed fence (safeStart), which is provably safe to slice on since the character just before it is always part of the fence delimiter itself, never an escape character or half of a multi-char marker. normalizeHtml's truncated-tag detection compared the last tag-open position against the last '>' anywhere in the whole document, so a literal '>' inside a still-unterminated tag's quoted attribute value could be mistaken for that tag's real closing bracket, letting a genuinely truncated tag through unstripped. Replaced with a quote-aware forward scan from the tag's start that ignores '>' while inside a single- or double-quoted attribute value.
…claims about v1.8.0 CHANGELOG.md's 1.8.0 section didn't mention any of the 3 follow-up fix commits (10 bugs across HyperViewer, HyperStreamingController, StreamSyntaxNormalizer). Added a ### 🐛 Fixes subsection in this repo's established style, folded into the still-unpublished 1.8.0 entry rather than bumped to 1.8.1. Also corrected claims that no longer matched (or never matched) actual behavior, following the project's existing practice of self-correcting doc over-claims: - Root README's "zero UI jank" and the core README's streaming sample (flat 16ms with no mention of the adaptive backoff added in this session's throttle fix) both implied unconditional 16ms throttling regardless of stream length. - ROADMAP.md's v1.8.0 "Incremental Delta-Append Engine" section described an architecture (token-level UDT append, tail-only layout invalidation) that was never built — what shipped is full-document reparse with adaptive throttle backoff, a real but different mitigation. A feasibility review of RenderHyperBox (immutable TextNode.text, no cross-parse node identity, no incremental parser, part-file architecture with no encapsulation boundary between layout/paint/selection/accessibility) found true tail-only layout to be a separate, multi-subsystem epic — not a scoped patch to fold into this release. The roadmap now says so explicitly instead of describing unbuilt work as done. - The "Live KaTeX & Syntax Highlighting in Streaming Mode" roadmap item is marked verified-non-issue: hyper_render_highlight's lexer doesn't throw on malformed/incomplete code, and flutter_math_fork's Math.tex wraps both parse and build stages in a catch-all fallback, so a delimiter-balanced-but-internally-malformed LaTeX fragment (which StreamSyntaxNormalizer intentionally never brace-balances) safely falls through to the red-text fallback. No production code change was needed; regression tests were added instead (separate commit).
…-mode and error-simulation controls The sample AI response text rendered a fake "Engine Benchmark Performance" table (Memory Footprint, Re-parse Overhead, FPS during Burst) with numbers that were never measured, including a literally false "Incremental Tail" re-parse-overhead claim — the engine does a full document reparse on every tick, just with an adaptively-backed-off notification frequency for large buffers. Shipping fabricated benchmark numbers inside the example app itself (not just docs) is worse than a doc over-claim: a developer running the demo sees it rendered as if it were real telemetry. Replaced with a plain description of the actual throttle-backoff mechanism. The demo also only ever exercised HyperRenderMode.sync with no way to trigger an error, so it never demonstrated two of the behaviors fixed this session: autoScrollToBottom in virtualized mode, and the error() -> reset() lifecycle. Added a Mode dropdown (sync/virtualized) and a "Simulate Error" button that calls controller.error(...), so the existing ERROR status badge has a real path to reach it.
… on transient streaming syntax A review of whether streamed code fences (highlighted via codeHighlighter) or math blocks (rendered via hyper_render_math's MathNodePlugin/flutter_math_fork) could crash or silently swallow exceptions while content is transiently malformed mid-stream found no reproducible bug — flutter_highlight's lexer is best-effort and doesn't throw on incomplete code, and Math.tex's onErrorFallback wraps both its parse and build stages in a catch-all. Adding regression tests for both so this stays true instead of relying only on that investigation's conclusion: - test/streaming_viewer_test.dart: streams a Markdown code fence through every intermediate state (open, StreamSyntaxNormalizer's synthetic auto-close, and the real close) with a real DefaultCodeHighlighter wired in, asserting no exception at any tick. - packages/hyper_render_math/test/math_node_plugin_test.dart: builds and pumps MathNodePlugin with structurally malformed LaTeX (an unbalanced brace — the one case StreamSyntaxNormalizer intentionally never repairs, since it only balances $/$$ delimiters), asserting it falls through to the red-text fallback instead of throwing.
…ription and README The package description and README's migration guide/feature matrix asserted specific unverified numbers (60 FPS, 5x faster RenderObject speed) that contradict the project's own doc/COMPARISON_MATRIX.md, which explicitly states 3 of 6 layout fixtures currently exceed their 16ms budget in the project's own debug-mode harness and "we do not claim a verified 60 FPS figure." These numbers were never measured in release/AOT mode and had no reproducible source. Replaced with the architectural claims that are actually true and verifiable by reading the source: single-RenderObject rendering instead of a nested widget tree, CSS float support no mainstream competitor has, crash-free selection at any document size. No external/competitor data was added to keep these claims static-and- durable rather than reintroducing the kind of decaying number this project has already had to walk back twice before (see COMPARISON_MATRIX.md's own history).
~25 comments across lib/, packages/hyper_render_core/lib/, and example/lib/ carried internal-tracker prefixes (CRIT-NN:, BUG-NN:, MED-NN:) with no meaning to external readers of a published pub.dev package. Removed just the prefix token in each case — the technical explanation that follows was already self-contained and reads identically well without it. No comment's substance changed; a few bare trailing "// CRIT-02" tags with no attached explanation (referring back to a fuller comment elsewhere in the same file) were removed entirely since they carried no content on their own.
Both docs were stale (CSS_PROPERTIES_MATRIX.md's header/footer disagreed with each other — v1.4.0 vs v1.5.0 — and both predated v1.8.0 by 3 minor versions; LIMITATIONS.md was stamped v1.5.x-unreleased). Bumped both to today's date and v1.8.0. Also added a new LIMITATIONS.md entry for the AI/LLM streaming feature: it re-parses the full accumulated buffer on every throttled tick (mitigated by adaptive backoff, not eliminated), not a true tail-only incremental append — matching the correction already made to ROADMAP.md's v1.8.0 section after a feasibility review found the original tail-only-layout description didn't match what shipped.
prepare_publish.sh never mentioned hyper_render_epub anywhere (publish order comment, dependency_overrides removal loop, publish_to removal loop, or the printed manual-publish instructions) — it predates that package being added to the monorepo. scripts/publish.sh already handles it correctly, but having two divergent publish scripts is itself the risk: whichever one an operator reaches for determines whether epub's dev-only dependency_overrides block (path: ../../, path: ../hyper_render_core) gets stripped before publish. This is the same bug class that already caused a real incident (v1.5.0 sub-packages published with leftover dependency_overrides, commit f30ff5c). Added epub to both removal loops and its own test step, and corrected the final publish-order instructions: epub must publish AFTER root (it depends on hyper_render itself), unlike the other six sub-packages which publish before it. Also fixed hyper_render_html/CHANGELOG.md: a `## [Unreleased]` section sat out of chronological order between 1.6.0 and 1.5.1, holding two bugfixes (nbsp whitespace, keyframes regex crash) that shipped in 1.6.0 per git history but were never relabeled with a real version heading. Folded into the 1.6.0 section where they belong.
CLAUDE.md claimed a test/fuzz/ directory with "43 fuzz cases for HTML/Markdown/Sanitizer parsers" that did not actually exist — the only file with "fuzz" in its name tested CSS animation color/timing parsing, unrelated to what was documented. Added the real thing: test/fuzz/parser_fuzz_test.dart seeds 8 HTML and 6 Markdown documents, applies randomized structural mutations (char insert/delete, tag truncation, quote stripping, slice duplication) with a fixed seed for reproducibility, and asserts none of 330 generated mutants plus 9 hand-picked adversarial shapes (dangling quotes, null-byte smuggling shapes, deep unclosed nesting) throws through the HTML adapter, Markdown adapter, or HtmlSanitizer. All 339 cases pass — no bugs found, which is itself a useful signal, not just a placeholder suite. Also closed a genuine gap the security-test audit flagged: no test verified the sanitizer's regex/URL-matching logic stays fast under adversarial long input (ReDoS). Added 4 timing-bounded tests to test/html_sanitizer_test.dart — very long attribute values, many whitespace-separated URL tokens, alternating quote/backslash patterns, and a 10x-input-size linearity check. Confirmed no catastrophic backtracking risk exists today (the sanitizer's only regex and UrlSafety's control-character strip are both simple character classes with no nested quantifiers) — these are regression guards against one being introduced later, not a fix for an active vulnerability.
The crash-net smoke test that mounts every demo on the home screen and asserts it opens without a FlutterError never included AiStreamingDemo — the screen for this session's entire focus, the v1.8.0 streaming feature. Added it; passes cleanly.
Hero line still said "1 980+ tests" (pre-dates this session's fuzz suite and ReDoS tests — real count is now 2,460+ across root+core+html+ epub). The hyper_render_epub install snippet also still pinned ^0.1.0, three patch versions behind its actual current version (0.1.2).
Contributor
✅ Layout Regression — All fixtures within 60 FPS budget
One or more fixtures exceeded the 16 ms budget.
No action required. |
Contributor
❌ Visual Regression Detected27 golden test(s) failed on Flutter 3.41.5 / ubuntu-22.04. The rendered output no longer matches the reference images. If the change is intentional, regenerate the goldens on the docker run --rm \
-v $(pwd):/workspace -w /workspace \
ghcr.io/cirruslabs/flutter:3.41.5 \
bash -c "apt-get update -qq && \
apt-get install -y fonts-noto fonts-noto-cjk fonts-roboto && \
flutter pub get && \
flutter test test/golden/ --update-goldens"
git add test/golden/goldens/
git commit -m "chore: update golden references (Flutter 3.41.5)"
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
HyperStreamingController,StreamSyntaxNormalizer,HyperTypingCaret, andHyperViewer.streaming(...)— a token-by-token streaming render path for AI/LLM chat responses (Gemini/OpenAI/Claude-style SSE or WebSocket feeds), with frame-aligned throttling that adaptively backs off for very long responses, transient syntax auto-repair (unclosed code fences/bold/links/tables), and a typing caret.Before merge, this branch went through a dedicated code-review pass and a full production-readiness sweep (see commit history below) — 10 real bugs found and fixed in the initial implementation, followed by a documentation/demo honesty pass and a broader pre-publish QA sweep across all 9 packages in the monorepo.
What shipped, in order
e557111,ecf1c16,c980b01,cc0dcee): streaming controller, syntax normalizer, typing caret,HyperViewer.streaming(), TPS telemetry,bindCustomStream, pause/resume/cancel, LaTeX/markdown transient repair, throughput benchmark suite.1b4097e,99712a7): auto-scroll, package versions, publish script, HTML normalizer.3aab42e,be6259c,fe97306):didUpdateWidgetmissing reparse triggers, content-fade flicker,fallbackBuildernever firing during streaming,autoScrollToBottomnot attaching in virtualized mode,append()allowed aftererror(), a synchronous throw insidebindStream'sonDatabypassingonError, andStreamSyntaxNormalizerscans corrupting content past an already-closed code fence.2a274d7,3f33d0b,faedaab,86fc22f): CHANGELOG now documents all 10 fixes; ROADMAP.md corrected — the "Incremental Delta-Append Engine" (true tail-only layout) was never built, only a full-reparse + adaptive-throttle mitigation was, and a feasibility review found the real thing to be a separate multi-subsystem epic; removed a fabricated benchmark table from the demo app's sample content; dropped unverified "60 FPS"/"5x faster" claims from the package description and README, matching this project's owndoc/COMPARISON_MATRIX.mddisclaimer.627c0a5..cb94c38): stripped ~25 internal ticket-ID comment prefixes; refreshed stale docs (CSS_PROPERTIES_MATRIX.md,LIMITATIONS.md); fixedscripts/prepare_publish.sh, which never accounted forhyper_render_epub(same bug class as a real incident inf30ff5c); fixed an orphaned out-of-order changelog section inhyper_render_html; added a real fuzz suite (339 seeded-random cases across the HTML/Markdown adapters and sanitizer — 0 crashes found) plus ReDoS/timing regression tests; added the streaming demo to the example app's crash-net smoke test (it was missing); refreshed stale test-count/version claims in the root README.Verification
flutter test test/ packages/hyper_render_core/test/ --exclude-tags golden→ 2340 passing, 0 failing(cd packages/hyper_render_html && flutter test)→ 73 passing(cd packages/hyper_render_epub && flutter test)→ 48 passingflutter analyzeclean (root + all sub-packages);dart format --set-exit-if-changed .cleanflutter build macos --debugforexample/succeeds; app launches without a crash (verified via captured stdout/stderr, no crash report generated)example/test/all_demos_smoke_test.dart— all 24 demo screens (including the newly-addedAiStreamingDemo) open without aFlutterErrordart pub publish --dry-runclean forhyper_render_core(0 warnings); the other 6 already-publishable sub-packages show only a harmless "git state modified" advisory from the dependency_overrides strip step. Root andhyper_render_epubcorrectly fail to resolve pre-publish since they depend on versions not yet live on pub.dev — expected, matches the documented publish order.Not in this PR
dart pub publish— this PR only merges tomain; publishing to pub.dev is a separate, explicit step after this merge.