Skip to content

Commit 0d74834

Browse files
authored
feat(desktop): show background desktop chats and keep the chat view display-only for them (#8670)
* feat(desktop): show which chats a desktop runs in the background Behind mothership-desktop-background-executor (resolved server-side and passed through the workspace feature flags): - GET /api/desktop/activity lists the caller's chats in a workspace whose turn runs on one of their desktops, as running, needs input (a call waits for approval) or blocked (that desktop is offline), with the device name. - The sidebar marks those chats with a status dot whose tooltip reads "Running on <device>", "Needs input" or "Blocked: <device> is offline"; unread replies keep the existing Ready dot. - A chat view showing a turn a desktop runs in the background only displays its desktop calls and never executes them. * fix(desktop): learn a turn's executor from its stream and announce background chats Builds on the background activity commit, against the final binding contract. - Display only, deterministically. Sim marks a turn its desktop's executor runs on the stream that serves it: a response header on the send and on every reconnect, and a flag on the replay batch a reload attaches with. The chat view only shows that turn's desktop calls and never starts or reports them. This replaces reading the cached activity list, which a new turn's first call could beat. - Needs input comes from the durable record: a call Sim asked the user about and they have not answered, or a running terminal handoff or browser takeover. An auto-allowed command no longer reads as needing input. Presence Sim cannot read is not reported as offline. - Turn-complete notifications for chats in the background: a chat that finishes while the user is elsewhere is announced and opens on click; the chat on screen announces itself as before. "Notify only for chats in the background" (the existing default) now means exactly that, so a background chat is announced even while Sim is focused. - The activity list refreshes when a turn starts or ends. * fix(desktop): keep background chat signals behind the flag and to workspace chats - Turn-complete notifications for chats in the background follow the background executor flag, so a flag-off user sees exactly what they saw before. - Only a workspace chat's turn binds to a desktop. Its sidebar shows the status and an approval notification links back to it; an organization chat stays with its chat view. - A desktop's status reads as running when Sim cannot track presence at all, not as blocked. - "Running on <device>" shows in the chat row's tooltip. The status dot sits in the row's indicator slot, which takes no pointer and gives way to the row's actions on hover, so a tooltip on the dot itself could never open. * test(desktop): observe what the chat view starts and announces instead of asserting mock calls The background-chat tests now record the calls a view starts and the notifications it asks the desktop app to show, and read whether the desktop activity query went stale from a real query client, instead of asserting that mocks were called. * fix(desktop): keep the old notification rule with the flag off, and announce only the user's own chats - Every existing notification is held back whenever the window is focused, as before. Only a background chat's completion, now marked as such, is shown when the focused window is somewhere else. With the flag off, a flag-off user watching the workflow panel is no longer told about the chat they are looking at. The settings label is back to its original wording. - Workspace chat events carry every member's chats. A completion is now announced only for a chat this user's own desktop was running, so a teammate's finished chat raises nothing. - The sidebar's activity reads get their own rate-limit bucket, so polling tabs can never spend the tokens a device needs to renew its leases. - An offline desktop's chat now says it is waiting for the device to come back, not that it is blocked. - The rule for which turns bind to a desktop is a named function, tested on its outcome. * fix(desktop): announce a background completion only for the very turn the desktop ran Desktop activity now names each bound turn's stream, and a completion is announced only when that turn was seen running on the user's own desktop. An earlier desktop turn no longer makes a later chat-view turn in the same chat look eligible. * style(desktop): color the background activity dot with Tailwind classes
1 parent ca11667 commit 0d74834

31 files changed

Lines changed: 1250 additions & 24 deletions

File tree

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
import { beforeEach, describe, expect, it, vi } from 'vitest'
2+
3+
vi.mock('electron', () => import('@/test/electron-mock'))
4+
5+
import type { BrowserWindow } from 'electron'
6+
import { createDesktopSettingsService } from '@/main/desktop-settings'
7+
import { Notification } from '@/test/electron-mock'
8+
9+
function service(window: { url: string; focused: boolean } | null) {
10+
const config = new Map<string, unknown>()
11+
return createDesktopSettingsService({
12+
config: {
13+
get: (key: string) => config.get(key),
14+
set: (key: string, value: unknown) => config.set(key, value),
15+
flush: () => true,
16+
} as never,
17+
getMainWindow: () =>
18+
window
19+
? ({
20+
isFocused: () => window.focused,
21+
webContents: { getURL: () => window.url },
22+
} as unknown as BrowserWindow)
23+
: null,
24+
openMainWindowAt: vi.fn(),
25+
setAutoDownloadUpdates: vi.fn(),
26+
setTrayEnabled: vi.fn(),
27+
setBrowserEnabled: vi.fn(),
28+
setTerminalEnabled: vi.fn(),
29+
setBrowserTheme: vi.fn(),
30+
setBrowserDefaultZoom: vi.fn(),
31+
setTerminalDefaultZoom: vi.fn(),
32+
getDefaultBrowserDownloadDirectory: () => '/tmp',
33+
chooseBrowserDownloadDirectory: async () => null,
34+
})
35+
}
36+
37+
const BACKGROUND_CHAT = '/workspace/ws-1/chat/chat-b'
38+
39+
describe('desktop notifications in the default "background only" mode', () => {
40+
beforeEach(() => {
41+
Notification.instances.length = 0
42+
})
43+
44+
it('announces a chat in the background while the user works in another one', () => {
45+
const shown = service({
46+
url: 'https://sim.ai/workspace/ws-1/chat/chat-c',
47+
focused: true,
48+
}).notify({
49+
title: 'Fix CI',
50+
body: 'Sim finished responding.',
51+
route: BACKGROUND_CHAT,
52+
background: true,
53+
})
54+
55+
expect(shown).toBe(true)
56+
expect(Notification.instances).toHaveLength(1)
57+
})
58+
59+
it('stays quiet about the chat the focused window shows', () => {
60+
const shown = service({ url: `https://sim.ai${BACKGROUND_CHAT}`, focused: true }).notify({
61+
title: 'Fix CI',
62+
body: 'Sim finished responding.',
63+
route: BACKGROUND_CHAT,
64+
background: true,
65+
})
66+
67+
expect(shown).toBe(false)
68+
expect(Notification.instances).toHaveLength(0)
69+
})
70+
71+
it('stays quiet about the app as a whole while it is focused', () => {
72+
const shown = service({ url: 'https://sim.ai/workspace/ws-1/home', focused: true }).notify({
73+
title: 'Update ready',
74+
body: 'Restart to update.',
75+
})
76+
77+
expect(shown).toBe(false)
78+
})
79+
80+
it('holds back an ordinary notification whenever the window is focused, wherever it is', () => {
81+
// The workflow panel's chat completes while the user watches it on the workflow page.
82+
const shown = service({ url: 'https://sim.ai/workspace/ws-1/w/wf-1', focused: true }).notify({
83+
title: 'Task complete',
84+
body: 'Sim finished responding.',
85+
route: BACKGROUND_CHAT,
86+
})
87+
88+
expect(shown).toBe(false)
89+
expect(Notification.instances).toHaveLength(0)
90+
})
91+
})

‎apps/desktop/src/main/desktop-settings.ts‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,21 @@ function readPreferences(
105105
}
106106
}
107107

108+
/**
109+
* Whether a focused window already shows what a notification is about. One without a route is
110+
* about the app as a whole, which a focused window always shows; one about a chat is news only
111+
* while that chat is in the background.
112+
*/
113+
function showsRoute(window: BrowserWindow, route: string | undefined): boolean {
114+
if (!route) return true
115+
try {
116+
const shown = new URL(window.webContents.getURL()).pathname
117+
return shown === new URL(route, 'https://sim.invalid').pathname
118+
} catch {
119+
return false
120+
}
121+
}
122+
108123
/**
109124
* Owns device preferences and their native side effects. Renderer code can
110125
* request a change, but only this main-process service touches login items,
@@ -203,7 +218,13 @@ export function createDesktopSettingsService(
203218
return false
204219
}
205220
const window = deps.getMainWindow()
206-
if (preferences.notificationsOnlyWhenUnfocused && window?.isFocused()) {
221+
// A focused window holds back every notification, as it always has. Only a background
222+
// chat's completion is shown anyway when the focused window is somewhere else.
223+
if (
224+
preferences.notificationsOnlyWhenUnfocused &&
225+
window?.isFocused() &&
226+
(!payload.background || showsRoute(window, payload.route))
227+
) {
207228
return false
208229
}
209230

‎apps/desktop/src/main/ipc.ts‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -305,10 +305,11 @@ export function parseDesktopNotificationPayload(raw: unknown): DesktopNotificati
305305
if (typeof raw !== 'object' || raw === null) {
306306
return null
307307
}
308-
const { title, body, route } = raw as {
308+
const { title, body, route, background } = raw as {
309309
title?: unknown
310310
body?: unknown
311311
route?: unknown
312+
background?: unknown
312313
}
313314
if (
314315
typeof title !== 'string' ||
@@ -323,7 +324,12 @@ export function parseDesktopNotificationPayload(raw: unknown): DesktopNotificati
323324
if (route !== undefined && (typeof route !== 'string' || !isSafeInternalPath(route))) {
324325
return null
325326
}
326-
return { title, body, ...(route !== undefined ? { route } : {}) }
327+
return {
328+
title,
329+
body,
330+
...(route !== undefined ? { route } : {}),
331+
...(background === true ? { background: true } : {}),
332+
}
327333
}
328334

329335
export interface IpcDeps {
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
/**
2+
* A chat view learns from the stream it reads whether the desktop app's background executor runs
3+
* the turn's desktop tools, so it only shows those calls. Exercised against real PostgreSQL and
4+
* Redis through the production reconnect route, for the live tail and the replay batch a view
5+
* attaches with after a reload.
6+
*/
7+
import { authMock, authMockFns } from '@sim/testing/mocks/auth.mock'
8+
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'
9+
10+
const { redisUrl } = await vi.hoisted(async () => {
11+
const { readTestRedisUrl } = await import('@sim/db/testing/test-infrastructure')
12+
const url = readTestRedisUrl()
13+
if (url) process.env.REDIS_URL = url
14+
return { redisUrl: url }
15+
})
16+
17+
vi.mock('@/lib/auth', () => authMock)
18+
19+
import { db } from '@sim/db'
20+
import {
21+
copilotChats,
22+
copilotRuns,
23+
desktopDevices,
24+
permissions,
25+
user,
26+
workspace,
27+
} from '@sim/db/schema'
28+
import { generateId } from '@sim/utils/id'
29+
import { eq } from 'drizzle-orm'
30+
import { NextRequest } from 'next/server'
31+
import { closeRedisConnection } from '@/lib/core/config/redis'
32+
import { MOTHERSHIP_DESKTOP_EXECUTOR_HEADER } from '@/lib/mothership/constants'
33+
import { GET as streamGET } from '@/app/api/copilot/chat/stream/route'
34+
35+
const userId = generateId()
36+
const workspaceId = generateId()
37+
const chatId = generateId()
38+
const deviceId = generateId()
39+
40+
async function finishedRun(desktopDeviceId: string | null): Promise<string> {
41+
const streamId = generateId()
42+
await db.insert(copilotRuns).values({
43+
id: generateId(),
44+
executionId: generateId(),
45+
chatId,
46+
userId,
47+
workspaceId,
48+
streamId,
49+
status: 'complete',
50+
desktopDeviceId,
51+
})
52+
return streamId
53+
}
54+
55+
function reconnect(streamId: string, batch: boolean) {
56+
return streamGET(
57+
new NextRequest(
58+
`http://localhost:3000/api/copilot/chat/stream?streamId=${streamId}&after=0${batch ? '&batch=true' : ''}`
59+
),
60+
{ params: Promise.resolve({}) }
61+
)
62+
}
63+
64+
describe.runIf(Boolean(redisUrl))('desktop executor binding on a chat stream', () => {
65+
beforeAll(async () => {
66+
const now = new Date()
67+
await db.insert(user).values({
68+
id: userId,
69+
name: 'Desktop stream fixture',
70+
email: `${userId}@desktop-stream.test`,
71+
emailVerified: true,
72+
createdAt: now,
73+
updatedAt: now,
74+
})
75+
await db.insert(workspace).values({
76+
id: workspaceId,
77+
name: 'Desktop stream fixture',
78+
ownerId: userId,
79+
billedAccountUserId: userId,
80+
})
81+
await db.insert(permissions).values({
82+
id: generateId(),
83+
userId,
84+
entityType: 'workspace',
85+
entityId: workspaceId,
86+
permissionType: 'admin',
87+
})
88+
await db.insert(copilotChats).values({ id: chatId, userId, workspaceId, type: 'mothership' })
89+
await db.insert(desktopDevices).values({
90+
id: deviceId,
91+
userId,
92+
name: 'Studio Mac',
93+
appVersion: '0.9.0',
94+
platform: 'darwin-arm64',
95+
capabilities: { executor: 1, browser: true, terminal: true, localFiles: true },
96+
})
97+
authMockFns.mockGetSession.mockResolvedValue({
98+
user: { id: userId },
99+
session: { id: generateId() },
100+
})
101+
})
102+
103+
afterAll(async () => {
104+
await db.delete(copilotRuns).where(eq(copilotRuns.chatId, chatId))
105+
await db.delete(copilotChats).where(eq(copilotChats.id, chatId))
106+
await db.delete(desktopDevices).where(eq(desktopDevices.id, deviceId))
107+
await db.delete(permissions).where(eq(permissions.userId, userId))
108+
await db.delete(workspace).where(eq(workspace.id, workspaceId))
109+
await db.delete(user).where(eq(user.id, userId))
110+
await closeRedisConnection()
111+
})
112+
113+
it('marks the live tail and the replay batch of a turn a desktop runs', async () => {
114+
const streamId = await finishedRun(deviceId)
115+
116+
const tail = await reconnect(streamId, false)
117+
await tail.text()
118+
const batch = await (await reconnect(streamId, true)).json()
119+
120+
expect(tail.headers.get(MOTHERSHIP_DESKTOP_EXECUTOR_HEADER)).toBe('device')
121+
expect(batch.desktopToolsOnDevice).toBe(true)
122+
})
123+
124+
it('leaves a turn the chat view runs unmarked', async () => {
125+
const streamId = await finishedRun(null)
126+
127+
const tail = await reconnect(streamId, false)
128+
await tail.text()
129+
const batch = await (await reconnect(streamId, true)).json()
130+
131+
expect(tail.headers.get(MOTHERSHIP_DESKTOP_EXECUTOR_HEADER)).toBeNull()
132+
expect(batch.desktopToolsOnDevice).toBeUndefined()
133+
})
134+
})

‎apps/sim/app/api/copilot/chat/stream/route.ts‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,10 @@ import {
1313
} from '@/lib/api/server/routes'
1414
import { encodeSSEComment } from '@/lib/core/utils/sse'
1515
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
16-
import { MOTHERSHIP_STREAM_REPLAY_HEADER } from '@/lib/mothership/constants'
16+
import {
17+
MOTHERSHIP_DESKTOP_EXECUTOR_HEADER,
18+
MOTHERSHIP_STREAM_REPLAY_HEADER,
19+
} from '@/lib/mothership/constants'
1720
import {
1821
MothershipStreamV1CompletionStatus,
1922
MothershipStreamV1EventType,
@@ -285,6 +288,7 @@ async function handleResumeRequestBody({
285288
previewSessions,
286289
status: run.status,
287290
...(run.chatId ? { chatId: run.chatId } : {}),
291+
...(run.desktopDeviceId ? { desktopToolsOnDevice: true } : {}),
288292
})
289293
}
290294

@@ -606,8 +610,10 @@ async function handleResumeRequestBody({
606610
}
607611

608612
return new Response(stream, {
609-
headers: replayBody
610-
? { ...SSE_RESPONSE_HEADERS, [MOTHERSHIP_STREAM_REPLAY_HEADER]: 'log' }
611-
: SSE_RESPONSE_HEADERS,
613+
headers: {
614+
...SSE_RESPONSE_HEADERS,
615+
...(replayBody ? { [MOTHERSHIP_STREAM_REPLAY_HEADER]: 'log' } : {}),
616+
...(run.desktopDeviceId ? { [MOTHERSHIP_DESKTOP_EXECUTOR_HEADER]: 'device' } : {}),
617+
},
612618
})
613619
}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import { listDesktopActivityContract } from '@/lib/api/contracts/desktop-executor'
2+
import { defineInternalJsonRoute, internalSessionAuth } from '@/lib/api/server/routes'
3+
import {
4+
desktopActivityRateLimit,
5+
desktopExecutorErrorPolicy,
6+
} from '@/lib/api/server/routes/desktop-executor'
7+
import { listDesktopActivity } from '@/lib/desktop/application/activity'
8+
9+
export const dynamic = 'force-dynamic'
10+
11+
export const GET = defineInternalJsonRoute({
12+
contract: listDesktopActivityContract,
13+
auth: internalSessionAuth,
14+
operation: listDesktopActivity.operation,
15+
rateLimit: desktopActivityRateLimit,
16+
errorPolicy: desktopExecutorErrorPolicy,
17+
mapInput: ({ query }) => ({ workspaceId: query.workspaceId }),
18+
useCase: listDesktopActivity,
19+
staticResponseHeaders: { 'Cache-Control': 'no-store' },
20+
})

‎apps/sim/app/o/[organizationId]/home/components/composer/composer.test.tsx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,7 @@ async function render(
235235
'table-row-ttl': false,
236236
'mothership-model-selector': mocks.advanced,
237237
'mothership-plan-mode': mocks.plan,
238+
'mothership-desktop-background-executor': false,
238239
}}
239240
>
240241
<Harness />
@@ -313,6 +314,7 @@ it.each([
313314
'table-row-ttl': false,
314315
'mothership-model-selector': false,
315316
'mothership-plan-mode': planEnabled,
317+
'mothership-desktop-background-executor': false,
316318
}}
317319
>
318320
<Harness />
@@ -408,6 +410,7 @@ it('keeps restored queued skills scoped when replacing a draft', async () => {
408410
'table-row-ttl': false,
409411
'mothership-model-selector': mocks.advanced,
410412
'mothership-plan-mode': mocks.plan,
413+
'mothership-desktop-background-executor': false,
411414
}}
412415
>
413416
<Harness />

‎apps/sim/app/o/[organizationId]/layout.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ export default async function OrganizationLayout({
7979
'table-row-ttl': tableRowTtlEnabled,
8080
'mothership-model-selector': modelSelectorEnabled,
8181
'mothership-plan-mode': planModeEnabled,
82+
'mothership-desktop-background-executor': false,
8283
}}
8384
>
8485
<OrganizationProvider context={context}>

‎apps/sim/app/workspace/[workspaceId]/home/hooks/stream-protocol.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ export type StreamBatchResponse = {
2525
previewSessions?: FilePreviewSession[]
2626
status: string
2727
chatId?: string
28+
/** The desktop app's background executor runs this turn's desktop tools. */
29+
desktopToolsOnDevice?: true
2830
}
2931

3032
const STREAM_SCHEMA_ENFORCEMENT_PREFIX = 'Client stream schema enforcement failed.'
@@ -108,6 +110,7 @@ export function parseStreamBatchResponse(value: unknown): StreamBatchResponse {
108110
...(previewSessions ? { previewSessions } : {}),
109111
status: typeof value.status === 'string' ? value.status : 'unknown',
110112
...(typeof value.chatId === 'string' && value.chatId ? { chatId: value.chatId } : {}),
113+
...(value.desktopToolsOnDevice === true ? { desktopToolsOnDevice: true as const } : {}),
111114
}
112115
}
113116

0 commit comments

Comments
 (0)