Skip to content

Add the Session API crate - #1593

Open
MaxHeimbrock wants to merge 38 commits into
max/room-restart-event-orderfrom
max/session-api
Open

MaxHeimbrock wants to merge 38 commits into
max/room-restart-event-orderfrom
max/session-api

Conversation

@MaxHeimbrock

@MaxHeimbrock MaxHeimbrock commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

New livekit-session crate with Session. Similar to the other session APIs in https://docs.livekit.io/frontends/build/sessions/.

Look at the example or my PR in the Rust dev client for usage.

Disclaimer: I mostly used Claude to write this. I am still not a Rust expert. I tried to find wholes as much as possible, look at the developer facing APIs and test it myself through the Rust dev client.

Decisions I took:

The platform audio is activated for the user by default, opt out via the session options (with_platform_audio = false).

If you pass a cached token source, you will run into issues reusing the same token. This is a footgun we accept, since invalidating tokens after use is not easy to expose in the current design. Also fixed token sources are allowed, which cause the same problem.

We don't plan to expose this Rust session via FFI for now.

Using the actor pattern according to AGENT.md.

Public API

Every enum and the options/state structs are #[non_exhaustive]. Message is not, so it can be built for restore_messages.

Item Kind Purpose
Session struct, Clone A conversation with an agent. Handles share one session; dropping the last one closes the room
Session::new(token_source, options) constructor Credentials from a TokenSourceFixed, e.g. literal
Session::with_fetch_options(token_source, fetch_options, options) constructor Credentials from a TokenSourceConfigurable with full TokenSourceFetchOptions
Session::with_agent(token_source, agent_name, options) constructor Shorthand for with_fetch_options that dispatches agent_name
Session::start(&self) -> Result<(), SessionStartError> async Fetches credentials, connects, and publishes the platform microphone; resolves at room connect, not agent connect
Session::end(&self) async Closes the room, releasing the audio devices, or cancels a pending start; idempotent
Session::state(&self) -> watch::Receiver<SessionState> method Current state (borrow) and changes (changed, wait_for)
Session::room(&self) -> Option<Arc<Room>> method The room while connected or reconnecting, for what the session does not do itself, e.g. publishing a camera
Session::agent_audio_track(&self) -> Option<RemoteAudioTrack> method The agent's microphone track, once subscribed, for an app that wants its frames
Session::agent_avatar_video_track(&self) -> Option<RemoteVideoTrack> method The avatar worker's camera track, once subscribed
Session::send_text(&self, text: &str) -> Result<Message, MessageSendError> async Sends chat on lk.chat and adds it to the feed once sent
Session::messages(&self) -> Vec<Message> method The feed, ordered by seq
Session::restore_messages(&self, messages: &[Message]) method Replaces the feed in the given order, renumbered from zero
Session::subscribe(&self) -> mpsc::UnboundedReceiver<SessionEvent> method Changes to the message feed
SessionOptions { agent_connect_timeout, room_options, platform_audio } struct, Default Agent timeout (20 s default), RoomOptions, and whether start uses the platform audio devices (true by default: the microphone is published and the agent's audio played); builders with_agent_connect_timeout, with_room_options, with_platform_audio
SessionState { connection, agent } struct, Copy, Default Snapshot published through state()
SessionConnectionState enum Disconnected, Connecting, Connected, Reconnecting; is_connected()
AgentState enum, Copy Disconnected, Connecting, Initializing, Idle, Listening, Thinking, Speaking, Failed(AgentFailure), Unknown; is_connected(), is_pending(), is_finished()
AgentFailure enum Timeout, Left
SessionEvent enum MessageUpserted(Message), MessagesRestored
Message { id, seq, timestamp, origin, participant_identity, text, is_final } struct One feed entry; identified by id + participant_identity
MessageOrigin enum AgentTranscript, UserTranscript, UserInput, Chat
SessionStartError error AlreadyStarted, Cancelled, Token(TokenSourceError), Connect(RoomError), PlatformAudio(AudioError) (no audio devices), Microphone(RoomError) (the microphone failed to publish; the room is closed again)
MessageSendError error NotConnected, Stream(StreamError)

Commits

Group 1. Crate scaffolding

  1. feat(session): add the livekit-session crate manifest and module layout
  2. chore(workspace): register livekit-session with cargo, knope and CODEOWNERS
  3. IMPORTANT(session): add the agent interop constants
  4. feat(session): add the start and send error types

Group 2. Message feed
5. feat(session): add the Message model and the MessageStore feed
6. test(session): cover MessageStore ordering, upsert and restore
7. IMPORTANT(session): aggregate lk.transcription streams into transcript messages
8. test(session): cover transcript aggregation

Group 3. Agent projection
9. feat(session): add AgentState and AgentFailure
10. feat(session): find the agent, its avatar worker and their tracks
11. IMPORTANT(session): project AgentState with timeout and leave detection
12. test(session): cover agent discovery and state parsing
13. test(session): cover agent projection across reconnects, leaves and timeouts

Group 4. Session runtime
14. feat(session): add the Dispatcher fan-out
15. test(session): cover Dispatcher pruning
16. feat(session): add the state shared between a session and its actor
17. feat(session): add SessionOptions, SessionState and SessionEvent
18. feat(session): add the Session handle and its constructors
19. IMPORTANT(session): add the session actor and its agent-state projection
20. IMPORTANT(session): read chat and transcript streams in the actor
21. IMPORTANT(session): start and end a session
22. feat(session): expose state, room, agent tracks and the message feed
23. test(session): assert with_agent fetches with the agent name

Group 5. End-to-end tests
24. test(session): add the e2e harness with a stand-in agent
25. test(session): e2e agent state follows joins, leaves and reconnects
26. test(session): e2e agent connect timeout
27. test(session): e2e start publishes the platform microphone
28. test(session): e2e the feed merges chat and transcripts
29. test(session): e2e agent audio and avatar video tracks
30. test(session): e2e restart after a disconnect and a rejected token
31. test(session): e2e end and drop cancel a pending start
32. test(session): e2e a start cancelled during the handshake leaves nothing behind
33. test(session): e2e memory lifecycle

Group 6. Docs, example and changeset
34. docs(session): add the crate README
35. feat(example): add the session voice and text chat example
36. chore(session): add the changeset

@MaxHeimbrock MaxHeimbrock changed the title livekit-session: add the Session API crate Add the Session API crate Oct 28, 2026
@MaxHeimbrock
MaxHeimbrock added this pull request to stack #1598 October 29, 2026 14:57
@github-actions

Copy link
Copy Markdown
Contributor

Changeset ✓

This PR includes a changeset covering all affected packages:

Package Bump
livekit-session patch

@xianshijing-lk
xianshijing-lk requested review from 1egoman and removed request for 1egoman October 30, 2026 23:18
@xianshijing-lk

Copy link
Copy Markdown
Contributor

Just found out it is still a draft, please let us know if it is ready.

@MaxHeimbrock
MaxHeimbrock removed this pull request from stack #1598 November 2, 2026 10:15
@MaxHeimbrock
MaxHeimbrock changed the base branch from max/token-source-invalidate-dispatch to max/room-restart-event-order November 2, 2026 10:15
@MaxHeimbrock
MaxHeimbrock added this pull request to stack #1621 November 2, 2026 10:15
@MaxHeimbrock

MaxHeimbrock commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor Author

Just found out it is still a draft, please let us know if it is ready.

I am still looking into a few things, weighing short term vs long term design decisions:

Do we want to expose this session implementation via FFI or have a platform specific session implementation for Rust client languages?

This is related to:

  • How do we expose the room, if the session api is not enough and we need it (e.g. register rpc calls, mute local audio track, ...)
  • TokenSource is currently also implemented on each platform, so we can not use the Rust constructors as is from FFI
  • Currently, the Rust session does not start with audio by default, as Platform audio is not always used from client languages

@1egoman

It would be easiest for now to just focus on Rust and ignore the FFI questions, or not expose it via FFI in the future.

This commit adds the manifest of the new `livekit-session` crate and
its root module.

The manifest declares the crate metadata, the features, and the
dependencies. Each TLS feature forwards to `livekit` and to
`livekit-token-source`. The two crates then use the same TLS backend.
The `__lk-e2e-test` feature enables the end-to-end tests. These tests
need a running `livekit-server --dev`.

The feature-combinations table uses the curated mode. The comment
above the table gives the reason.

The root module declares the submodules and re-exports the public
types. The submodules follow in the next commits.
…OWNERS

This commit registers the new crate in the workspace.

The root `Cargo.toml` adds the crate as a workspace member and as a
workspace dependency at version 0.1.0. `Cargo.lock` adds the lock
entry of the crate.

`knope.toml` adds the release package, so knope can bump the version
and write the changelog. `CODEOWNERS` assigns the crate to its owner.
`AGENTS.md` adds the crate to the list of curated crates in the
feature-combinations section.
This commit adds the constants that the session shares with
`livekit-agents`.

The constants are the text-stream topics for chat and transcription,
the participant attributes for the agent state and the avatar worker,
the text-stream attributes of a transcript segment, and the default
agent-connect timeout.

These strings are the contract between the session and the agent. A
wrong string breaks the session without an error. Compare each value
with the `livekit-agents` source before you approve this commit.
This commit adds the two error enums of the crate.

`SessionStartError` tells why `Session::start` failed. The variants
cover an already started session, a cancelled start, a failed token
fetch, a failed room connect, a failed platform-audio start, and a
failed microphone publish.

`MessageSendError` tells why `Session::send_text` failed. The variants
cover a session without a room and a failed text stream.

Both enums are `non_exhaustive`, so later changes can add variants
without a breaking change.
This commit adds the public `Message` and `MessageOrigin` types, and
the private `MessageUpdate` and `MessageStore`.

A `Message` is one entry of the feed. Its `id` and its
`participant_identity` together identify it. Its `seq` gives the
position in the feed.

A `MessageUpdate` is a message as a source reports it, before the
store assigns a `seq`.

The `MessageStore` keeps the messages in receive order. `upsert` adds
a new message or updates an existing message in place. The updated
message keeps its `seq`. `restore` replaces the feed and numbers the
messages from zero.
This commit adds the unit tests of `MessageStore`.

The tests show that an update keeps the `seq` of the message, that the
feed uses the receive order and not the sender timestamp, and that
`restore` keeps the given order and numbers the messages again from
zero.
…t messages

This commit adds `TranscriptStream`, `parse_final` and
`TranscriptAggregator`.

Agents send transcripts in two shapes on the `lk.transcription` topic.
Agent speech is one stream. Each chunk appends to the text, and the
trailer of the stream finalizes the segment. User speech is a new
stream for each interim result. Each stream carries the full text, and
the last stream is marked final.

The aggregator handles both shapes with one rule. A chunk appends when
it arrives on the stream that last wrote the segment. A chunk replaces
the text when it arrives on a new stream.

A segment is keyed by the segment id and the participant. A new
segment from a participant drops the older partial segments of that
participant.

`on_stream_end` finalizes a segment, unless the stream is marked not
final or a newer stream took the segment over.

Review this logic with care. It decides what text the user sees and
when a transcript is complete.
This commit adds the unit tests of `TranscriptAggregator` and
`parse_final`.

The tests cover appending on one stream, replacing on a new stream,
the segment id fallback, the parsing of the final attribute,
finalization without the attribute, a superseded stream, empty chunks,
the same segment from two participants, and the drop of older
partials.
This commit adds the public `AgentState` and `AgentFailure` enums.

`AgentState` tells what the agent is doing, as the session projects it
from the room. The predicates `is_connected`, `is_pending` and
`is_finished` group the states for most callers.

`from_attribute` parses the `lk.agent.state` attribute. A missing
attribute means `Idle`. An unknown value means `Unknown`.

`AgentFailure` gives the reason of a `Failed` state: a timeout, or an
agent that left.
This commit adds the `AgentCandidate` trait and the lookup functions
of the agent module.

`AgentCandidate` abstracts a remote participant, so the unit tests can
use fake participants.

`find_agent` returns the participant of kind agent that does not
publish on behalf of another participant. With several agents, the
lowest identity wins. `find_avatar_worker` returns the participant
that publishes on behalf of the agent.

`agent_audio_track` and `agent_avatar_video_track` return the agent's
microphone track and the avatar worker's camera track, once
subscribed. `activity` reads the agent's state attribute.
This commit adds `AgentTracker`.

The tracker projects the `AgentState` from the connection state and
the remote participants. It keeps two facts across calls: whether an
agent was ever seen, and whether the agent-connect timeout fired.

Without an agent, the projection is `Failed(Left)` when an agent was
seen before and the room is connected. A full reconnect removes every
participant, so a missing agent while reconnecting is not `Left`.

`time_out` records that the agent-connect timeout fired. The timeout
holds only for the agent present at that time. An agent that joins
later reports its own activity.

A `Failed` state is not final. An agent that becomes conversational
replaces it.

Review the state transitions with care. The UI shows them to the user.
This commit adds `Shared` and the `lock` helper.

`Shared` holds the session state, the message store and the event
dispatcher. A `Session` and its actor both hold an `Arc<Shared>`.
`Shared` holds no reference back to either, so it keeps neither alive.

Each `start` begins a new generation. `publish` writes a state only
when its generation is current. An actor of an earlier start that is
still shutting down cannot overwrite the state of a newer start.

`upsert` dispatches the message event under the store lock.
Subscribers see updates in the order the store applied them.

The `history` field exists only in tests. It records every published
state, so tests can assert on states a `watch` receiver would skip.

`lock` locks a mutex and recovers from poisoning.
This commit adds the public option, state and event types of the
session.

`SessionOptions` holds the agent-connect timeout, the room options,
and the `platform_audio` flag. The flag defaults to `true`. The
builder methods set each field.

`SessionConnectionState` and `SessionState` describe the session at
one point in time. `SessionEvent` reports a change to the message
feed.

All four types are `non_exhaustive`.
This commit adds the `Session` handle and the private types behind it.

`SessionTokenSource` wraps a fixed or a configurable token source. The
constructor that the caller uses selects the variant.

`Session` is a cheap handle around `Inner`. `Inner` holds the token
source, the options, the shared state and the `Lifecycle`.

`Lifecycle` is `Idle`, `Starting` or `Running`. `Running` holds the
room, the stop sender and the actor task. `join` drops the stop sender
and waits for the actor.

`Session::new` takes a fixed source. `Session::with_fetch_options`
takes a configurable source and fetch options. `Session::with_agent`
is a shorthand that sets the agent name.
This commit adds `SessionActor` and the `Connection` it runs.

The actor runs one connected room for one `start`. It holds the room,
the platform audio, the room events, the stop receiver, the agent
deadline and the `AgentTracker`. It holds no `Session` handle, so
dropping the last handle stops the actor.

`run` selects between the stop channel, the room events and the agent
deadline. A `Disconnected` room event or a closed stop channel ends
the loop.

On exit, the actor closes the stop receiver first. A `start` that
follows then finds the actor gone. The actor closes the room when the
session stopped it. It then publishes the default state.

`on_room_event` tracks the connection state and refreshes the
projection on participant and track changes. `on_agent_deadline`
records the timeout in the tracker and refreshes the projection.

Review the shutdown order and the ownership with care. See the
"Memory lifecycle" section of AGENTS.md.
This commit adds the text-stream reading to `SessionActor`.

`on_text_stream` takes the reader of a stream on the `lk.chat` or
`lk.transcription` topic. It leaves streams on other topics to the
app. A chat stream is read in full. A transcription stream is read
chunk by chunk, followed by an end marker.

The readers are polled in the actor loop, through a `SelectAll`. The
`TranscriptAggregator` then has one writer.

`on_reader_event` turns each chunk, end or chat into a
`MessageUpdate` and upserts it into the shared store.
This commit adds `Session::start`, `Session::end`, `connect`,
`publish_microphone` and `StartGuard`.

`start` fails with `AlreadyStarted` while the session is starting or
running. It begins a new generation, joins a stale actor, and then
fetches credentials and connects the room. `end` can cancel this
phase.

Platform audio is created before the room connects. After the
connect, `start` publishes the microphone. This step is not
cancellable. A failed publish closes the room.

`start` spawns the actor only when the lifecycle still holds its
generation. Otherwise `end` was called during the connect. `start`
then closes the room and returns `Cancelled`.

`StartGuard` resets the lifecycle and the state when `start` fails or
is dropped before the actor runs.

`end` closes the room through the actor, or cancels a pending start.
It does nothing when there is nothing to end.

Review the cancellation paths with care. Each path must leave no room,
task or audio device behind.
This commit adds the accessor methods of `Session`.

`state` returns a `watch` receiver of the session state. `room`
returns the room while it is connected or reconnecting.

`agent_audio_track` and `agent_avatar_video_track` return the agent's
media tracks, once subscribed.

`send_text` sends chat to the agent on `lk.chat` and adds the text to
the feed once sent. `messages`, `restore_messages` and `subscribe`
read, replace and observe the feed.
This commit adds a unit test of `Session::with_agent`.

A recording token source captures the fetch options. The test shows
that the options carry the agent name.
This commit adds the end-to-end test module and its harness. The
tests run against a `livekit-server --dev` and need the
`__lk-e2e-test` feature.

`TestEnv` builds tokens for a unique room. It joins a second room as
the agent, as an avatar worker, or as an observer. The agent room sets
the agent state attribute.

The helper functions wait for a session state, poll a condition, and
read the private lifecycle of a session.

The root module declares the test module behind the feature.
This commit adds the first end-to-end tests of the agent projection.

The tests show that a started session reports the agent once it
joins, that `end` disconnects and is idempotent, that an agent that
leaves is reported as `Left`, and that a full reconnect is not
reported as `Left`.
This commit adds the end-to-end tests of the agent-connect timeout.

The tests show that the timeout fires without an agent, that a late
agent recovers the state, that the timeout is armed without an agent
dispatch, and that the timeout fires for an agent stuck in
`initializing`.
This commit adds the end-to-end test of the default platform audio.

The test shares the platform-audio handle with the session. It shows
that `start` publishes the microphone track and that `end` releases
the audio devices. The test skips on a machine without a microphone.
This commit adds the end-to-end tests of the message feed.

The first test shows that user input reaches the agent on `lk.chat`,
that an agent transcript grows in place and is finalized by the
trailer, that chat from the agent joins the feed, and that a failed
send adds nothing.

The second test shows that the session leaves streams on other topics
to the app.
This commit adds the end-to-end tests of the agent media accessors.

The helpers publish a silent audio track or a blank video track from
the agent room.

The tests show that only the microphone is the agent's audio, that
only the avatar worker's camera is the avatar video, and that an
avatar worker of kind agent is not taken for the agent.
This commit adds two end-to-end tests of the start path.

The first test removes the user from the room through the server API.
It shows that the session reports `Disconnected`, that a new `start`
releases the old room, and that `end` after a disconnect is a no-op.

The second test shows that a rejected token fails `start` with
`Connect` and leaves the session idle.
This commit adds the end-to-end tests of a start that waits on the
token source.

`PendingSource` never answers. The tests show that `end` cancels the
start with `Cancelled`, and that a dropped `start` future resets the
session.
…ing behind

This commit adds the end-to-end tests of a start that waits in
`Room::connect`.

`SilentServer` accepts one connection and never answers. The tests
cancel the start with `end` or by a drop of the future. They show
that the client hung up, that no actor was spawned, and that the task
count returns to the baseline.
This commit adds the end-to-end lifecycle tests that AGENTS.md asks
for.

The tests show that dropping the last `Session` handle closes the
room, that repeated start and end cycles leave no task or subscriber
behind, and that failed starts spawn nothing.

Each test checks observable destruction: a `Weak` that no longer
upgrades, a strong count of one, or the task count of the runtime.
This commit adds the README of the crate.

The README gives a usage example and explains the credentials, the
lifecycle, the state table, the message feed, and the things the
session owns.
This commit adds `examples/session` and registers it in the
workspace.

The example creates a session from a pre-generated token or from a
development token server. It prints the session state and the
completed messages. Lines typed on stdin go to the agent.

By default, the example uses the platform audio. The `--text` flag
turns it off.

`Cargo.lock` adds the lock entry of the example.
This commit adds the knope changeset for the new crate.

The summary is one line of prose, as the "Documenting changes"
section of AGENTS.md requires.
@MaxHeimbrock
MaxHeimbrock marked this pull request as ready for review November 3, 2026 17:09
@MaxHeimbrock
MaxHeimbrock requested a review from ladvoc as a code owner November 3, 2026 17:09

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 2 potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

Comment thread livekit-session/src/session.rs Resolved
Comment on lines +51 to +52
livekit = { workspace = true }
livekit-token-source = { workspace = true }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Disabling defaults still enables native backends

Building livekit-session without default features still enables native networking. Both dependencies activate native by default, so the crate's native switch cannot disable either backend.

Suggested change
livekit = { workspace = true }
livekit-token-source = { workspace = true }
livekit = { workspace = true, default-features = false }
livekit-token-source = { workspace = true, default-features = false }

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Comment thread livekit-session/src/agent.rs Outdated
Comment on lines +33 to +45
Disconnected,
/// The session is connecting, or is connected and no agent has joined yet.
Connecting,
/// The agent joined and is starting up.
Initializing,
/// The agent joined but is not in a conversation.
Idle,
/// The agent is listening to the user.
Listening,
/// The agent is preparing a reply.
Thinking,
/// The agent is replying.
Speaking,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Maybe it makes sense to put any agent state within each enum entry?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope I understood your suggestion correctly. I looked at Swift and saw the nested structure there. Have a look at my changes at 1acf3e8 and tell me if this is what you wanted to see :) .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea this is moving in the right direction!

Comment thread livekit-session/src/session.rs Resolved
Comment on lines +239 to +244
/// options, such as agent metadata.
pub fn with_agent(
token_source: impl TokenSourceConfigurable + Send + Sync + 'static,
agent_name: &str,
session_options: SessionOptions,
) -> Self {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: It could be interesting to expose a builder type api here maybe? Session::new_configurable(configurable token source).with_agent("agent name").with_options(...) or something...

Comment on lines +436 to +443
/// A receiver of changes to the message feed.
///
/// Unbounded, like [`Room::subscribe`]: the session must never block on a
/// slow subscriber, and the feed's rate is bounded by the conversation.
/// Drop the receiver to unsubscribe; it is removed on the next event.
pub fn subscribe(&self) -> mpsc::UnboundedReceiver<SessionEvent> {
self.inner.shared.subscribe()
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: very interesting, this could play well into some stuff blaze has been talking about to make uniffi events more stream based...

Comment on lines +504 to +506
/// What the agent is doing.
pub agent: AgentState,
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: We can talk about this in our 1:1, but some original thinking was done back in the day to try to make session able to support multiple agents in the future with minimal updates. It would be nice to try to carry over those decisions in here as well.

Comment thread AGENTS.md Resolved

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

}
// Dropped while the actor was publishing the microphone. Dropping its
// `stop` sender stops it.
drop(lifecycle.take_running(self.generation));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Cancelled starts detach room tasks

Dropping start() during microphone publication drops the Running task handle without joining it. The actor closes in the background, but no owner observes its completion or failure.

Learn more

A running session keeps the actor's JoinHandle in Running, and Running::join normally drops the stop sender and awaits that handle. Here take_running removes the only handle, then drops it without awaiting. Tokio detaches the actor, so no owner can observe a failed shutdown or guarantee a join. The same unowned task path occurs when the lifecycle changes after spawning the actor.

Example: A caller times out session.start() while the microphone publication is pending. Its guard signals stop but discards the task handle; the actor closes the room asynchronously without a joiner.

Recommended fix: Arrange for both cancellation paths to transfer Running to a named shutdown owner that awaits Running::join, while preserving background teardown when start() is dropped. Add a lifecycle test that verifies the shutdown owner joins the task.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

@MaxHeimbrock

Copy link
Copy Markdown
Contributor Author

Some feedback after also wrapping session in call for threads:

"Call::room() is the wrong fix for avatar tracking. I only needed the room to watch for track events and then re-check agent_avatar_video_track(). That's a raw room-event subscription just to learn that the avatar appeared or went away. The room() docs also warn that the call has already taken the chat and transcription stream readers, which is a trap. If the avatar and agent audio tracks came as events or state, most apps would never need the room. I'd keep room() for advanced use, but nothing in the normal path should need it."

"Avatar tracking through room():
The normal path should not need the room. The session actor already reacts to track subscribed and unsubscribed events by calling refresh(), which recomputes the agent state from the participants. Adding avatar presence there is cheap. The catch is that SessionState is Copy and the remote video track type is neither Copy nor PartialEq, so the track itself cannot live in the state or in an Eq event. Put flags in the session state instead, such as avatar_video and agent_audio, computed in the refresh. The thread gets them for free through its call state. The app re-reads the track accessor when a flag flips. Keep room() as the escape hatch."

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants