Skip to content

Open a conversation at the bottom, keep it still while a Bot writes, and add a rail of turns - #882

Merged
guidovizoso merged 51 commits into
mainfrom
guido/chat-polish-4-stream
Oct 29, 2026
Merged

guidovizoso merged 51 commits into
mainfrom
guido/chat-polish-4-stream

Conversation

@guidovizoso

@guidovizoso guidovizoso commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

What this changes

A conversation now opens already at the bottom, holds still while a Bot writes, and streams smoothly. The work was built in five stacked stages. They ship as one PR because each stage builds on the one before it.

1. Opening. A channel's history is shown once it has all loaded, already at the bottom, instead of loading and then gliding down.

  • Voice chats are waited for, so they don't push the page after it is drawn.
  • Messages that were there when the conversation opened appear without a cascade. Only messages that arrive afterwards animate.
  • Group chats open at the bottom the same way.
  • A message typed while history is still loading shows under the placeholder.

2. Drawing. Charts, tables, cards and decisions a Bot draws grow in place while they stream.

  • A tool row keeps its key for its whole life, so it is never remounted.
  • Gallery cards wait for grants instead of flashing "not available".
  • A card that failed on half-sent arguments tries again on the next chunk.
  • Half-arrived numbers, links and colours are held back until they are complete.
  • A streaming table draws its complete rows while its last row is still arriving.

3. Scrolling. We kept @shadcn/react's MessageScroller. Its prepend path already keeps the reader's position when rows are added above, and its autoScroll prop is the only part we needed to control.

  • Opening a long conversation: only the newest page of rows is drawn. Older pages, 20 rows each, are drawn one per frame as the reader nears the top, without moving what they are reading.
  • Following: whether the view follows a reply now depends on what the person does, through a pure reducer, nextFollow. Where the view happens to be no longer decides it.
    • It stops following on an upward wheel, a scroll key, a touch, a scrollbar drag, or selecting transcript text.
    • It follows again when the person comes back near the end, presses the arrow, or sends.
    • Keys typed into a field and layout clamps don't count.

4. Streaming.

  • Live replies: a reply that arrives while the conversation is open is revealed at a steady pace, a word at a time, catching up within 400ms after a burst. The rest is out 300ms after the stream ends.
    • Text with no spaces (Chinese or Japanese, a long link) reveals progressively instead of waiting for a space.
    • The last word shows once the text stops growing, even if the run is still busy.
  • History is drawn whole, in Streamdown's static mode.
  • Finished rows stay still: a finished tool group no longer re-renders on every token. A running tool row still redraws as its arguments stream in: rows are built from copies of the agent's tool calls, because the agent client grows arguments in place.
  • Voice-chat cards keep their message objects between renders.
  • Generative UI on a cold load: an activity drawn before its renderer was registered appears once the renderer arrives.
  • Reduced motion shows text as it arrives.

5. A rail of turns. A rail in the left margin has one mark per turn, like the one in T3 Code. The turn being read is drawn longer and darker.

  • Preview: hovering or focusing a mark shows what the person sent and the start of the reply, as plain words.
  • Jump: choosing a mark goes to that turn, leaving the transcript's usual 24px above it. A turn in older history that isn't drawn yet gets every page down to it drawn first (drawTo), and the jump happens a frame later, once the scroller has put the reader back.
  • Following: jumping stops following, but only once the pages are in place. Drawing pages above makes the scroller nudge the scroll position down while the reader is at the end, and that nudge would otherwise count as the person scrolling back to the end and switch following on again.
  • Which turn is current: the last turn to start above a third of the way down the view. At the end of the conversation it is the last turn, even a short one.
  • What counts as a turn: files sent with a caption are one turn with it, and routine runs start none.
  • When it shows: only with at least two turns, and only when the scroller is wide enough to leave a margin (@3xl).

Nothing to do for these:

  • Code-block highlighting: OpenBot installs no Streamdown code plugin, and Streamdown draws the same box before and after its code loads.
  • Image sizes: attachment thumbnails are already a fixed size-32.

Where it runs

Only in the browser. No server code, schema or API changes.

  • New state that outlives a request? None. All new state is React state inside a page.
  • What happens on the second replica? Nothing different. The server isn't involved.
  • Anything serialised? None.
  • Anything fanned out to a browser? None new.
  • New listener, port, or schedule? None.

Boundary and audit

  • Every acting call still goes through the gateway: resolve, decide, audit, then act. (Unchanged.)
  • New refusals and new failures each write a row. (There are none.)
  • Nothing new is trusted from the client that the server can resolve itself. (Unchanged.)

Changelog

  • CHANGELOG.md → Unreleased → "A conversation opens at the bottom and stays still while a Bot writes".

Proof

Tests: new tests cover every stage:

  • transcript-pages, follow-intent and transcript-follow for scrolling;
  • paced-reveal and transcript-stream for the reveal;
  • transcript-rerender for rows that stay still;
  • transcript-turns and transcript-rail for the rail;
  • late-renderers and voice-archive for the rest.

Full app suite: 2065 pass, 1 fail. The failure is the existing serve.test.ts "a stalled handshake times out…", which needs the network. Typecheck, biome lint --error-on-warnings and biome format are clean.

Measured in the browser (a Chromium script against a running app):

  • Paging: drawing older pages moved what the reader was looking at by at most 0.5px when scrolled by code, and 0.4px under the mouse wheel. Jumping to the top adds one page per frame, with no errors on a 100+ page conversation.

  • Following: every rule held mid-stream (wheel, key, select, return to the end, the arrow, send). The 72px new-turn peek is the same as stage 1.

  • Opening: shows the placeholder, then the bottom in one frame.

  • A reply with a web search and a long answer:

    • Earlier rows had 0 DOM changes while it streamed.
    • Text grew by at most about a word per frame.
    • The only height drops came from the scroller's own spacer, with the total height steady, so no extra reserved height was needed.
  • The rail: on a 13-turn conversation, the current mark matched what was on screen and hovering showed the right preview. Jumping to the first turn drew the 17 older rows and landed on it, 24px below the top, with following off. On five conversations the last mark was selected on open.

Known limits

  • Prepend and send in one commit: if older rows are drawn above in the same commit as a new send, the scroller can align to the new turn a beat late. This is documented in chat-transcript.tsx.
  • Live messages keep streaming mode after they finish, so the renderer never swaps under the reader. They become static on the next opening. Footnotes and reference-style links can render slightly differently between the two.
  • Voice cards that arrive late: history waits at most 1.5s for voice chats. A card that arrives after that is placed after the message before the call, which can be among rows already on screen. It draws at rest, since the call happened before the conversation opened (ec5d335). One case is left: if the conversation ends on the person's own unanswered message, the scroller can read that message as new and move it to the top of the view. Fixing that needs the scroller to recognise new rows by id instead of by position. This only happens when the voice service is slow.
  • A message that starts streaming after it mounted with the history (reattaching to a run) stays in static mode. This is rare.
  • Not tested:
    • A real scrollbar-thumb drag: headless Chrome turns it into a text selection. Detecting a press in the scrollbar's gutter was checked by measurement.
    • Screen reader announcements of per-frame text growth.
    • How a rail jump interacts with following needs real scrolling, which happy-dom doesn't have, so it was checked in the browser only.

…e it

A lone foldable call is now a group of one under a group: key and is drawn as the same line it was. Existing grouping tests that asserted a lone call's own id or an unprefixed group id now assert the group: id.
Comment thread app/src/lib/copilot/gallery-tools.tsx Resolved
Comment thread app/src/lib/voice/archive.ts Resolved

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for a few correctness issues in the inline comments. The direction is good: opening at the bottom, holding still while a Bot writes, and the turn rail are all welcome.

Must fix

  • Charts and tables can lose their last value for good (gallery-tools.tsx:202). A call counts as still streaming unless its status is complete. A run that was stopped, or whose page closed before the handler answered, reloads as inProgress permanently. So settledArgs keeps dropping its last number, link or colour, and DataTable hides the final row.
  • Old messages replay (transcript-pages.ts:67). split and from are row indices fixed when the conversation opens. A history merge or tool steps regrouping shifts rows across the boundary. They remount as live, so their entrance plays again and they type themselves out from zero. They can also make the scroller jump to an old turn.
  • The view stops following a streaming reply in four cases:
    • a downward wheel or End press at the bottom, because the library leaves following and nothing switches it back (follow-intent.tsx:221);
    • a scrollbar drag with overlay or RTL scrollbars (follow-intent.tsx:243);
    • clicking the rail's mark for the current last turn (chat-transcript.tsx:1951);
    • the first message sent in a transcript with no earlier message from the person (chat-transcript.tsx:1693).

Should fix

  • The reveal's timer resets on every token (paced-reveal.ts:111). It falls behind, then dumps the rest in the finish.
  • Opening waits on the voice-chat query (channel-chat.tsx:572). A slow or failing /api/voice/sessions holds every conversation on the skeleton, and anything sent meanwhile isn't drawn.
  • The red "Couldn't show" box flashes between chunks for a card that throws on partial arguments (tool-boundary.tsx:42).

The rest are smaller and up to you:

  • per-chunk work in the rail and readActiveTurn;
  • the per-row late-renderer subscriptions;
  • the dropped cleanup in useDecisionTool;
  • stale comments;
  • the duplicated ToolRow branch.

Details and failure scenarios are on each line.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fast turnaround. 13 of the 15 are fixed, and the other two are partly fixed and fine to merge as they are. The full app suite passes at eefed68 (2083 pass), and typecheck and lint are clean. One new bug from the fixes, so one more round:

Must fix: a decision card that fails is now invisible while the run waits on it (785c3ea, tool-boundary.tsx:59-67).

  • Why it happens: the boundary renders nothing when a card failed, the call isn't final, and the row can still receive arguments. For a tool row, final means a result exists (final={result !== undefined}, chat-transcript.tsx:1410), and a decision has no result until the person answers. The run stays busy for the whole wait, so the decision stays the active last row.
  • Example: a choice whose options arrives as a string throws at (args.options ?? []).map (decisions.tsx:180). At 53686a9 that showed "Couldn't show Choice". Now nothing is drawn, and the turn hangs until someone presses Stop.
  • Also affected: a gallery card for as long as a slow executing round trip lasts.
  • Suggested fix: treat the call as final once its arguments are complete, either by checking that args parses or by passing the SDK status through. Add a regression test for a decision that throws while it waits.

Worth a look (not blocking)

  • Late voice cards land among the live rows (bc3d2be). The reply says they go into the history region, but withVoiceChats places each card right after its anchor (archive.ts:228-250), and only rows above the opening split are history. In a channel of 20 rows or fewer, the split is row 0. So a voice card that arrives after the 1.5s cap plays its entrance mid-conversation, and can read to the scroller as a new row. It only happens when the voice service is slow. Showing history without voice chats is the right tradeoff; the description is just off.
  • Two misplaced comments from the delta:
    • useDecisionTool's doc block now sits above useForgetRenderOnUnmount (gallery-tools.tsx:235-262);
    • late-renderers.tsx:63 points to LateRenderersScope, which doesn't exist.
  • Late renderers are now one watch per transcript, which is the main thing. It still polls up to 10 frames because SDK 1.83.0's setRenderActivityMessages notifies nobody. Fine for now.
  • Before release: a manual pass on a Mac trackpad for the downward flick at the end and the overlay-scrollbar drag. Both are fixed in the code, but tested only with simulated input.

@guidovizoso

Copy link
Copy Markdown
Collaborator Author

Fixed the late voice cards in ec5d335. A voice card for a call that started before the conversation opened now draws at rest. A card for a call made while the conversation is open still animates in. Both cases are covered in transcript-late-voice.test.tsx.

One case is left, now listed under known limits: if the conversation ends on the person's own unanswered message, a late card pushes that message into the slot the scroller checks for new rows, and it can move the message to the top of the view. Fixing that needs the scroller to recognise new rows by id instead of by position.

I also merged main (#871). The new spokeAfter is passed through ToolRow and checked in its memo, with a test. App suite: 2128 pass, lint and types clean.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The blocker is fixed: toolCallFinal treats a call as final once its arguments parse. Tool arguments are always an object, which only parses once it's closed, so a decision that throws on complete arguments now shows its failure while the run waits. Half-arrived arguments still draw nothing. Both cases are pinned in tool-boundary.test.tsx. The two misplaced doc comments are back where they belong. Late voice cards from before the conversation opened now draw at rest (transcript-late-voice.test.tsx). CI is green.

One thing still worth a manual check before release, as noted last round: a real Mac trackpad for the downward flick at the end and the overlay-scrollbar drag, since both are tested with simulated input only.

@guidovizoso
guidovizoso merged commit ab9855c into main Oct 29, 2026
19 checks passed
@guidovizoso
guidovizoso deleted the guido/chat-polish-4-stream branch October 29, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants