Add in-process (FFI) transport for the Node.js SDK by SteveSandersonMS · Pull Request #1953 · github/copilot-sdk

SteveSandersonMS · 2026-07-09T12:11:57Z

Summary

Adds an in-process (FFI) transport for the Node.js SDK. Instead of spawning the Copilot runtime as a child process, the SDK can host it in-process by loading the native runtime library and speaking JSON-RPC over its C ABI via koffi.

This is the Node-only split of the combined Node+Rust in-process FFI work (stevesa/ffi-inproc-rust-ts); the Rust SDK changes will follow in a separate PR.

What's included

RuntimeConnection.forInProcess() + InProcessRuntimeConnection (experimental)
FfiRuntimeHost: koffi-based C ABI host with deferred, non-reentrant inbound frame delivery and an async broker pump so foreign-thread outbound callbacks are serviced while the loop is idle
client.ts wiring for start/stop/forceStop over the FFI host
E2E harness support (isInProcessTransport, per-test proxy/env mirroring), a dedicated inprocess_ffi E2E test, and an inprocess CI matrix cell
skipIf(isInProcessTransport) for tests that require per-client env or a child process (telemetry, request-cancel, checkpoint u32, process-kill), documented against In-process (FFI) transport: options lowered to env vars (auth token, COPILOT_HOME, keytar, telemetry) are not honored in-process #1934
Graceful stop() (not forceStop) for inproc resume clients so the runtime closes session-store.db and Windows temp-dir cleanup can unlink it
Bump @github/copilot to ^1.0.70-0; add koffi ^3.1.0

Excluded (temporary scaffolding)

All env-gated diagnostic tracing (COPILOT_FFI_TRACE, COPILOT_EVENT_TRACE) and the koffi-callback repro test + its CI job have been stripped out — this branch contains only the production feature.

Testing

typecheck ✅ · eslint ✅
Inproc E2E smoke: streaming_fidelity + inprocess_ffi → 6/6 ✅
Full default + inprocess transport matrix runs in CI

Host the Copilot runtime in-process by loading the native runtime library and speaking JSON-RPC over its C ABI via koffi, instead of spawning a runtime child process. Adds: - RuntimeConnection.forInProcess() + InProcessRuntimeConnection (experimental) - FfiRuntimeHost: koffi-based C ABI host with deferred, non-reentrant inbound frame delivery and an async broker pump so foreign-thread outbound callbacks are delivered while the loop is idle - client.ts wiring for start/stop/forceStop over the FFI host - E2E harness support (isInProcessTransport, per-test proxy/env mirroring), a dedicated inprocess_ffi E2E test, and an "inprocess" CI matrix cell - skipIf(isInProcessTransport) for tests that rely on per-client env or a child process (telemetry, request-cancel, checkpoint u32, process-kill), documented against #1934 - Graceful stop() (not forceStop) for inproc resume clients so the runtime closes session-store.db and Windows temp-dir cleanup can unlink it - Bump @github/copilot to ^1.0.70-0; add koffi ^3.1.0 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…rence - rpc_workspace_checkpoints: replace the MAX_SAFE_INTEGER "unknown checkpoint" sentinel with a 32-bit-safe value (4294967294) so the test runs on the in-process transport too, and remove the skipIf(isInProcessTransport). The runtime narrows the checkpoint number to u32 only at the napi wire arg and immediately widens it back to u64, so any real (small) checkpoint number works; only the pathological sentinel exceeded u32. - inprocess_ffi: describe the Node smoke test on its own terms instead of referencing the .NET test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…no longer needed With the runtime worker-starvation/threading fault fixed (runtime 1.0.70-0), the in-process FFI host reduces to standard koffi. Revalidated on Windows (inprocess permissions/streaming/session/checkpoint suites, plus a bare-node event-loop probe) and removed the compensating workarounds: - Broker pump: the continuous log_dropped_count async FFI call is gone. koffi delivers the foreign-thread outbound callback on the event loop via a threadsafe function; idle-turn model responses arrive without pumping. - Deferred inbound delivery: feedInbound writes straight to receiveStream (the callback already runs on a loop tick, not nested on the runtime's stack, so there is no reentrancy to defer around). - Deferred callback unregister: koffi.unregister runs synchronously in dispose; no DEP0168 at teardown. Kept, because it is inherent to in-process hosting rather than a workaround: a referenced keep-alive timer. The FFI transport has no socket/pipe handle, and koffi's callback TSFN does not reference the loop, so a bare-node app would otherwise exit mid-turn (probe exited with code 13 without it). Also sharpen the copilot_request_cancel_error inproc-skip comment: confirmed the runtime enforces a single process-wide LLM inference provider (a second requestHandler client's llmInference.setProvider is rejected), so that skip is legitimate and stays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The shared e2e harness blanked GH_TOKEN/GITHUB_TOKEN to "" and relied on an ambient COPILOT_HMAC_KEY (a CI-only secret) to supply the credential. Locally there is no HMAC key, so tests that create their own client without passing gitHubToken (the stdio/tcp "works without onPermissionRequest" cases) had no credential and failed with a 401. Use the proxy-recognized token instead so these authenticate against the replay proxy in both CI and local runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The three "works without onPermissionRequest" tests force-stopped their clients in onTestFinished. Force-stopping kills the runtime worker abruptly, so on Windows it can still hold the session.db file lock (EBUSY) after the test, stalling the suite's afterAll teardown. Shut the clients down gracefully instead, matching the rest of the file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…andle A turn still running when the runtime disposes the session can leave that session's SQLite session.db handle open. Over the in-process transport the runtime shares the test process, so the handle is not reclaimed and the file stays locked on Windows, causing EBUSY/ENOTEMPTY when removing the session-state directory during teardown. Abort each session before disconnect so the turn cancels and releases the handle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions

Generated by SDK Consistency Review Agent for issue #1953 · sonnet46 2.2M

github-actions · 2026-07-09T17:31:50Z

+        // removal of the session-state directory. Aborting lets the turn cancel
+        // and release the handle. Best-effort and idempotent: a session with no
+        // active turn is a no-op.
+        await Promise.allSettled(activeSessions.map((session) => session.abort()));


Cross-SDK parity note — .NET StopAsync() may need the same fix

This abort-before-teardown step is specifically needed for the in-process transport so that in-flight SQLite handles are released before the session is destroyed (otherwise Windows file locking can prevent temp-directory cleanup).

The .NET SDK already ships RuntimeConnection.ForInProcess(), but its StopAsync() calls session.DisposeAsync() directly without first calling session.AbortAsync():

// dotnet/src/Client.cs — StopAsync() foreach (var session in _sessions.Values.ToArray()) { try { await session.DisposeAsync(); // ← no AbortAsync() call first } ... }

DisposeAsync() sends session.destroy but doesn't send session.abort, so the same SQLite-handle-open-on-Windows issue this commit fixes for Node.js may already be present for .NET in-process clients. Suggesting a follow-up to add the equivalent AbortAsync() step in .NET StopAsync() for in-process connections.

Only abort in-flight session turns on teardown for SDK-owned runtimes, so a connecting client against an external server leaves its pending work intact for resume. Gate the two disconnect-driven capability tests (multi-client, ui_elicitation) with it.skipIf(isInProcessTransport) since the in-process transport shares the host process. For 'should delete session by id', abort the session before disconnect/delete so the delete-triggered reload leaves no unauthenticated background promise to reject host-side after the harness env teardown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Switch client unit tests and e2e teardowns from forceStop() to graceful stop() so in-process (FFI) workers can release their native/SQLite handles and exit cleanly. In client.test.ts this goes through a new stopClient() helper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The abort-before-teardown only exists to release the runtime's SQLite session.db handle over the in-process (FFI) transport, where the runtime shares this process. stdio/tcp runtimes run in a child process we kill on shutdown, which already frees the handle, so don't abort there. Gate on connectionConfig.kind === 'inprocess' and note it's temporary until the runtime cleans up fully on shutdown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Keep graceful stop() for normal teardown (needed so in-process workers release handles and exit), but revert to forceStop() where the test intent requires it: - client.e2e 'should forceStop without cleanup' body - connection_token unauthenticated clients whose start() rejects - client.test.ts stdin-error test using a mock connection - pending_work_resume, which forceStop()s to simulate a crash leaving pending work for resume (restored to base) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds an experimental in-process (FFI) transport to the Node.js SDK so the runtime can be hosted in the same process (via koffi + the runtime C ABI) instead of spawning a child process, and updates CI + E2E harness/tests to exercise and accommodate the new transport.

Changes:

Introduces RuntimeConnection.forInProcess() and FfiRuntimeHost to host/connect to the runtime over FFI streams.
Wires CopilotClient.start/stop/forceStop to support the in-process host lifecycle and adjusts shutdown behavior (notably for resume/Windows cleanup).
Extends the E2E harness and CI matrix with an inprocess cell, skipping or adapting tests where in-process limitations apply.

Show a summary per file

File	Description
nodejs/src/ffiRuntimeHost.ts	New koffi-based FFI host that loads `runtime.node`, starts the embedded worker, and bridges JSON-RPC frames via streams.
nodejs/src/types.ts	Adds `InProcessRuntimeConnection` type + docs and `RuntimeConnection.forInProcess()` factory.
nodejs/src/index.ts	Re-exports `InProcessRuntimeConnection` from the public entrypoint.
nodejs/src/client.ts	Adds in-process start/connect plumbing, FFI host lifecycle management, and refactors runtime env building for stdio/TCP.
nodejs/package.json	Adds `koffi` dependency and bumps `@github/copilot` version range.
nodejs/package-lock.json	Locks `koffi` (+ platform optional deps) and `@github/copilot` version bump.
nodejs/test/e2e/inprocess_ffi.e2e.test.ts	New smoke test validating start + ping round-trip over in-process FFI transport.
nodejs/test/e2e/harness/sdkTestContext.ts	Adds `isInProcessTransport`, mirrors env into the host process for inproc, neutralizes HMAC, and resolves default connection based on env.
nodejs/test/e2e/telemetry.e2e.test.ts	Skips telemetry export validation under in-process transport (per #1934 limitations).
nodejs/test/e2e/ui_elicitation.e2e.test.ts	Skips a multi-client elicitation capability test under in-process transport.
nodejs/test/e2e/copilot_request_cancel_error.e2e.test.ts	Skips cancellation test under in-process due to process-wide provider restriction.
nodejs/test/e2e/multi-client.e2e.test.ts	Skips a disconnect/tool-removal scenario under in-process transport.
nodejs/test/e2e/client.e2e.test.ts	Switches many cleanups to `stop()` and skips forceStop test under in-process; adds inproc skip guard for child-kill scenario.
nodejs/test/e2e/client_options.e2e.test.ts	Uses `stop()` instead of `forceStop()` in cleanup paths.
nodejs/test/e2e/client_api.e2e.test.ts	Adds `session.abort()` before disconnect/delete in a session management test.
nodejs/test/e2e/rpc.e2e.test.ts	Uses `stop()` instead of `forceStop()` for RPC test cleanup.
nodejs/test/e2e/rpc_mcp_and_skills.e2e.test.ts	Uses `stop()` instead of `forceStop()` for cleanup.
nodejs/test/e2e/rpc_mcp_config.e2e.test.ts	Uses `stop()` instead of `forceStop()` for cleanup.
nodejs/test/e2e/rpc_server.e2e.test.ts	Uses `stop()` instead of `forceStop()` for cleanup.
nodejs/test/e2e/rpc_server_misc.e2e.test.ts	Uses `stop()` instead of `forceStop()` in helper teardown.
nodejs/test/e2e/rpc_server_plugins.e2e.test.ts	Uses `stop()` instead of `forceStop()` in helper teardown.
nodejs/test/e2e/rpc_server_remote_control.e2e.test.ts	Uses `stop()` instead of `forceStop()` in remote control helper.
nodejs/test/e2e/rpc_session_state_extras.e2e.test.ts	Uses `stop()` instead of `forceStop()` for auth client cleanup.
nodejs/test/e2e/rpc_workspace_checkpoints.e2e.test.ts	Adjusts checkpoint number to a 32-bit-safe high value for “missing checkpoint” scenario.
nodejs/test/e2e/session.e2e.test.ts	Uses `stop()` instead of `forceStop()` for multiple client cleanups.
nodejs/test/e2e/session_fs.e2e.test.ts	Uses `stop()` instead of `forceStop()` for session-fs tests cleanup.
nodejs/test/e2e/streaming_fidelity.e2e.test.ts	Uses `stop()` instead of `forceStop()` for resume-client cleanup.
nodejs/test/e2e/suspend.e2e.test.ts	Renames forceStop cleanup helper to stop-based cleanup for suspend tests.
nodejs/test/client.test.ts	Uses a stop helper in place of `forceStop()` for unit-test cleanup.
.github/workflows/nodejs-sdk-tests.yml	Adds `transport` matrix including `inprocess` and sets `COPILOT_SDK_DEFAULT_CONNECTION=inprocess` for that cell.

Review details

Files not reviewed (1)

nodejs/package-lock.json: Generated file

Files reviewed: 29/30 changed files
Comments generated: 4
Review effort level: Low

…c, guard late FFI frames - resolveDefaultConnection() reads COPILOT_SDK_DEFAULT_CONNECTION from process.env only (not the deprecated options.env), so the host-process transport switch can't be silently disabled by a per-client env block. - Reject a workingDirectory with RuntimeConnection.forInProcess() instead of mutating the shared process-global cwd across the ~30s host_start; removed the chdir dance from FfiRuntimeHost. The e2e harness now changes cwd itself around in-process worker startup to preserve the working-directory behavior. - Guard feedInbound() against a native outbound callback delivered after dispose() ended receiveStream (ERR_STREAM_WRITE_AFTER_END): drop late frames. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-07-09T22:30:16Z

Cross-SDK Consistency Review

Summary

This PR primarily adds the in-process FFI transport to the Node.js SDK, bringing it to parity with .NET, which already has this feature. The changes are internally consistent across the two implementations.

✅ Consistent across Node.js and .NET

Feature	Node.js (this PR)	.NET (pre-existing)
`InProcessRuntimeConnection` type	`InProcessRuntimeConnection` interface	`InProcessRuntimeConnection` class
Factory method	`RuntimeConnection.forInProcess()`	`RuntimeConnection.ForInProcess()`
`COPILOT_SDK_DEFAULT_CONNECTION` env var	✅ Added	✅ Already present
`--no-auto-update` for FFI worker	✅ Present from the start	✅ Added in this PR (bug fix)
Experimental caveat about per-client env vars	✅ Documented	✅ Documented

The --no-auto-update flag (which pins the bundled CLI to prevent ABI skew) is now consistently applied in both SDKs.

⚠️ Known gaps in other SDKs

The following SDKs do not yet have an in-process FFI transport. This is expected given the incremental rollout described in the PR:

Rust — explicitly deferred to a follow-up PR per the description. No action needed here.
Python, Go, Java — no FFI transport. These are not addressed in this PR or its stated scope.

If cross-SDK parity is a goal for the FFI transport, it may be worth opening tracking issues for Python, Go, and Java once the Node.js + Rust rollout stabilizes.

No blocking issues

This PR maintains consistency between the two SDKs that currently support in-process FFI (Node.js and .NET). No changes are required for merge.

Generated by SDK Consistency Review Agent for issue #1953 · sonnet46 1M · ◷

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/src/ffiRuntimeHost.ts Outdated

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/src/ffiRuntimeHost.ts Outdated

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/src/ffiRuntimeHost.ts Outdated

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/src/ffiRuntimeHost.ts Outdated

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/src/ffiRuntimeHost.ts Outdated

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/test/e2e/copilot_request_cancel_error.e2e.test.ts Outdated

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/test/e2e/inprocess_ffi.e2e.test.ts Outdated

SteveSandersonMS commented Jul 9, 2026

View reviewed changes

Comment thread nodejs/test/e2e/rpc_workspace_checkpoints.e2e.test.ts Outdated

This comment has been minimized.

Sign in to view

SteveSandersonMS and others added 6 commits July 9, 2026 16:40

SteveSandersonMS force-pushed the stevesa/ffi-inproc-node branch from 1b75f56 to 4f8f8cb Compare July 9, 2026 15:41

This comment has been minimized.

Sign in to view

github-actions Bot reviewed Jul 9, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

SteveSandersonMS and others added 2 commits July 9, 2026 19:57

SteveSandersonMS force-pushed the stevesa/ffi-inproc-node branch from fa0dd5c to d88a121 Compare July 9, 2026 18:58

This comment has been minimized.

Sign in to view

SteveSandersonMS force-pushed the stevesa/ffi-inproc-node branch from d88a121 to d2c1f6c Compare July 9, 2026 20:09

This comment has been minimized.

Sign in to view

test: skip 'should forceStop without cleanup' on inproc

67d1120

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This comment has been minimized.

Sign in to view

SteveSandersonMS added 2 commits July 9, 2026 21:55

Explanation

5867a22

Cleanup

b2a021b

This comment has been minimized.

Sign in to view

Cleanup

ce6b96f

SteveSandersonMS force-pushed the stevesa/ffi-inproc-node branch from 90205fc to ce6b96f Compare July 9, 2026 21:13

Simplify comment

97fe96c

This comment has been minimized.

Sign in to view

Lazy-load koffi

bb8b064

SteveSandersonMS marked this pull request as ready for review July 9, 2026 21:49

SteveSandersonMS requested a review from a team as a code owner July 9, 2026 21:49

Copilot AI review requested due to automatic review settings July 9, 2026 21:49

Copilot started reviewing on behalf of SteveSandersonMS July 9, 2026 21:49 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread nodejs/src/ffiRuntimeHost.ts Outdated

Comment thread nodejs/test/e2e/harness/sdkTestContext.ts

Comment thread nodejs/src/ffiRuntimeHost.ts

Comment thread nodejs/src/client.ts Outdated

This comment has been minimized.

Sign in to view

Use --no-auto-update

a67adf8

SteveSandersonMS mentioned this pull request Jul 9, 2026

[.NET 1.0.6][FFI][Windows] In-process host loads global user-profile package and can hit ABI mismatch #1959

Open

SteveSandersonMS merged commit 35673cb into main Jul 9, 2026
43 of 44 checks passed

SteveSandersonMS deleted the stevesa/ffi-inproc-node branch July 9, 2026 22:35

Uh oh!

Conversation

SteveSandersonMS commented Jul 9, 2026

Summary

What's included

Excluded (temporary scaffolding)

Testing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

github-actions Bot commented Jul 9, 2026

Cross-SDK Consistency Review

Summary

✅ Consistent across Node.js and .NET

⚠️ Known gaps in other SDKs

No blocking issues

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants