Skip to content

src: experimental green threads implementation - #67288

Open
jasnell wants to merge 34 commits into
nodejs:mainfrom
jasnell:green-threads
Open

jasnell wants to merge 34 commits into
nodejs:mainfrom
jasnell:green-threads

Conversation

@jasnell

@jasnell jasnell commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

This is an experiment... opening for feedback. This is ready for review.

This implements green threads for Node.js. A green thread is a lightweight kind of worker. It has its own V8 isolate with no Node.js APIs. They run on a small pool of physical threads that they share, instead of each getting an OS thread and event loop like a worker thread. Each one gets work from the host through messages, RPC calls, or its module's exports, and every piece of work runs in an invocation context of its own. Each green thread can have limits on CPU time and memory, and the host can terminate it. This makes them cheap to start and to keep around in large numbers, and suited to running many small, untrusted pieces of code, such as plugins, per-tenant handlers, or user-supplied scripts.

const { GreenThread } = require('node:green_threads');

const greenThread = new GreenThread(`
  onmessage = ({ data }) => postMessage(data * 2);
`);
greenThread.on('message', (value) => {
  console.log(value);  // Prints: 42
  greenThread.terminate();
});
greenThread.postMessage(21);

For later, the plan is to add the following APIs (but not in this PR)

  • TextEncoder/TextDecoder
  • URL/URLSearchParams
  • structuredClone()
Details

Compared with worker threads

Green threads have real advantages over workers, in specific areas:

  • Memory: 2–6× less per instance, depending on the build and the
    isolate-group setting. A thousand tenants take about 1 GB instead of 6.7 GB.
  • Startup: up to about 80× faster with a warm pool (about 20 µs each).
  • Many instances: 1000 instances share 8 OS threads instead of using
    1000, and serve about 15–20% more requests per second.
  • Sandboxing: no Node.js APIs, plus per-instance CPU-time and memory
    limits.

They don't win on the speed of a single message, and they can't spread heavy
computation across all cores, since the pool is capped at 8 threads.

Setup for this performance comparison

  • Machine: 88 CPUs, 250 GB RAM, Linux x64.
  • Two builds:
    • Release: the default build, without pointer compression. It doesn't
      support isolate groups, so every green thread is in V8's default group.
    • Pointer compression: supports isolate groups. By default, each green
      thread is in an isolate group of its own.
  • Green thread pool: 8 threads (kMaxPoolThreads).

Each measurement runs in a fresh child process, so that memory numbers don't
include what earlier measurements left behind.

Kinds of green threads measured:

Kind Meaning
green Default options: an isolate group of its own where supported
green-default isolateGroup: 'default', V8's default isolate group
green-shared One IsolateGroup shared by all instances
green-warm Default options, taken from a warm pool filled beforehand

Memory and startup

1000 idle instances, started at once, measured once each has sent a message.

RSS per instance Starts per second OS threads
Worker 6.7–7.9 MB ~720 1000
Green, Release build 5.2 MB ~750 8
Green, pointer compression (own isolate group) 2.8 MB ~1,220 8
Green, pointer compression (isolateGroup: 'default') 2.6 MB ~810 8
Green, shared IsolateGroup 1.03 MB ~1,590 8
Green, from a warm pool same as cold 44,000–66,000 8
  • Memory: a green thread uses 1.5× to 6.5× less memory than a worker. A
    worker carries a whole Node.js environment, an event loop and an OS thread
    with its stack. A green thread is a V8 isolate with a small runtime.
  • Threads: green threads share a pool of 8 threads, however many there
    are.
  • Warm pool: setWarmPoolSize(n) prepares isolates in advance, so
    starting a green thread takes about 20 µs. Workers have no equivalent.
  • Cold start is only slightly faster than a worker's, except with a
    shared isolate group.

Speed

Message round trips

Sequential round trips with one instance.

Round trips per second Latency
Green ~57,000 ~17.5 µs
Worker ~61,000 ~16.5 µs

Workers are about 5% faster per message.

CPU-bound tasks

2000 tasks of fib(27), each executor taking the next task once it's done.
Release build.

Tasks per second Time
Green ×8 2,466 811 ms
Green ×2000 (one per task) 2,226 898 ms
Worker ×8 2,478 807 ms
Worker ×88 9,759 205 ms

At equal parallelism, CPU-bound code runs at the same speed. Green threads
can't use more than 8 cores, though, so for heavy parallel computation
workers win clearly. The cap is a choice in this implementation, not
inherent, and could be made configurable.

Many tenants

1000 tenants, each with code and state of its own, serving 50,000 requests
spread over all of them, 64 at a time.

All started Requests per second RSS total
Worker (Release) 1,653 ms ~129,000 7,974 MB
Green (Release) 1,332 ms ~150,000 5,267 MB
Worker (pointer compression) 1,387 ms ~126,000 6,725 MB
Green (pointer compression) 836 ms ~149,000 2,883 MB
Green, shared IsolateGroup 632 ms ~149,000 1,112 MB

Green threads serve about 15–20% more requests, since 8 pool threads don't
compete for the CPU the way 1000 OS threads do.

Runaway code

One of 100 tenants runs into an endless loop, while the others keep serving.

Stopped by Stopped after Others served in 200 ms
Green ERR_GREEN_THREAD_CPU_LIMIT (maxTaskCpuTimeMs: 50) ~60 ms ~35,000
Worker A host timer, then terminate() ~70 ms ~31,500

Both stop it. Green threads do it with a limit on CPU time, which the runtime
enforces. Workers need a timer by wall time on the host, which also stops
code that's only waiting.

What's easier to build: a plugin host

The same plugin host, built with each. It runs five plugins, which are ES
modules exporting transform(text). Plugins can import a shared module, and
call back into the host for configuration and logging. Three of them
misbehave.

Plugin Green threads Workers
upper, reverse (well-behaved) Work Work
spin (endless loop) ERR_GREEN_THREAD_CPU_LIMIT after 100 ms of CPU time The host's own wall-clock timeout, which also stops plugins that are only waiting
hog (allocates ArrayBuffers) Allocation fails at the 32 MB maxMemoryMb limit Not limited, since resourceLimits only cover the JS heap. Reached 91 MB within the 100 ms timeout
fail (throws with a code) Error arrives with code: 'ERR_PLUGIN' Only with hand-written error forwarding

The green thread version is 43 lines:

const plugin = new GreenThread(source, {
  type: 'module',
  filename: `file:///plugins/${name}/index.js`,
  modules: plugins.shared,
  env: {
    async config(key) { return plugins.config[key]; },
    log(...args) { console.log(`[${name}]`, ...args); },
  },
  resourceLimits: { maxTaskCpuTimeMs: 100, maxMemoryMb: 32 },
});
const { transform } = await plugin.exports;
console.log(await transform('hello world'));

The worker version is 155 lines, since it has to build what green threads
provide:

  • Module loading: workers load modules from files or URLs, so the
    plugins are written to a temporary directory. Green threads take a
    modules bundle and resolve() / load() hooks.
  • Calls in both directions: workers need call IDs, a table of pending
    calls, and error forwarding, roughly what Comlink does. Green threads get
    exports and env as RPC stubs.
  • Limits: workers have no CPU-time limit and no ArrayBuffer limit.
    Green threads have both.
  • Isolation: worker plugins have all of Node.js' APIs, including
    require('fs'). The permission model applies to the whole process, so it
    can't restrict a single worker. Green threads have no Node.js APIs at all:
    they can only do what the host passes them. This is the biggest
    qualitative difference.

When to use which

Use green threads for:

  • Multi-tenant or serverless-style hosting of many small handlers.
  • Plugin and extension systems that run untrusted code.
  • User-supplied scripts, rules, or transforms that need CPU and memory
    limits.
  • Many short-lived or mostly idle units of code, where startup time and
    memory per instance matter.

Use workers for:

  • Code that needs Node.js' APIs (fs, net, child_process), or npm
    packages that use them.
  • Heavy parallel computation across all cores, while the green thread pool
    is capped at 8 threads.
  • Blocking code: Atomics.wait() and synchronous waits are disabled in green
    threads, since a blocked isolate would hold up a shared pool thread.
  • A few long-lived instances, where the cost per instance doesn't matter.

Caveats

  • The pointer-compression numbers come from the build at f14fd2d7548,
    which doesn't have the ES modules and errors commits. These scenarios don't
    use either.
  • The warm-pool numbers for the pointer-compression build come from a later
    run than the rest of its table. The first run used a setting that bypassed
    the pool.
  • Worker startup throughput depends on the number of cores. On smaller
    machines, the gap is likely larger.
  • Some JIT-compiled loops ignore termination, a V8 bug. Green threads rely on
    a watchdog that abandons such isolates. Workers are affected by the same
    bug when terminated.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/config
  • @nodejs/gyp
  • @nodejs/loaders
  • @nodejs/performance
  • @nodejs/startup

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Nov 6, 2026
@jasnell
jasnell requested review from addaleax and mcollina November 9, 2026 18:01
@glenjamin

Copy link
Copy Markdown
Contributor

I'm not clear what the etiquette is for feedback on things like this from random Node.js users, so if this isn't the place for it please let me know / delete/edit this as appropriate.

This seems really neat, but from reading the comparisons it got me wondering:

How many of the benefits are inherent to what green-threads can do, and how many of them are things that could conceivably be added to the existing workers implementation?

@ljharb

ljharb commented Nov 12, 2026

Copy link
Copy Markdown
Member

@jasnell does each green thread still have its own JS environment, like workers and worker threads?

@jasnell

jasnell commented Nov 12, 2026 •

Copy link
Copy Markdown
Member Author

@ljharb:

does each green thread still have its own JS environment, like workers and worker threads?
Not a full Node.js environment, no. That's intentional. You won't get the full set of globals and imports go to the parent rather than to the module system.

@jasnell

jasnell commented Nov 12, 2026

Copy link
Copy Markdown
Member Author

@glenjamin:

How many of the benefits are inherent to what green-threads can do, and how many of them are things that could conceivably be added to the existing workers implementation?

The benefits realized here are because they are not implemented the way existing workers are. Specifically, a green thread is not bound to a single physical thread the way a worker thread is. A green thread does not have an entire Node.js Environment associated with it. It doesn't have the full set of globals. And so forth. That is, we get the benefits here by subtracting from existing workers implementation and not adding to it.

@ljharb

ljharb commented Nov 12, 2026

Copy link
Copy Markdown
Member

Is it possible to directly share objects between green threads?

@jasnell

jasnell commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

Share-ish. You can postMessage values to it that are structured cloned. There's also an rpc mechanism for accessing objects across the boundary. Otherwise, however, in this impl, no truly shared objects.

@ljharb

ljharb commented Nov 13, 2026

Copy link
Copy Markdown
Member

OK - last question, I think :-) the RPC mechanism; does that allow two threads to mutate the same object at the same time?

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rlgtm, this is a good first step and addition to Node

@glenjamin

Copy link
Copy Markdown
Contributor

@glenjamin:

How many of the benefits are inherent to what green-threads can do, and how many of them are things that could conceivably be added to the existing workers implementation?

The benefits realized here are because they are not implemented the way existing workers are. Specifically, a green thread is not bound to a single physical thread the way a worker thread is. A green thread does not have an entire Node.js Environment associated with it. It doesn't have the full set of globals. And so forth. That is, we get the benefits here by subtracting from existing workers implementation and not adding to it.

What I meant is: is it feasible to allow choosing to subtract those features from workers piecemeal?

ie. Could the RPC thing be made available to workers? Could workers have a flag to get a limited environment? or the CPU/memory limits etc etc

I appreciate the M:N aspect is specific to green threads, but naively the other features appear to be choices made while implementing this.

@jasnell

jasnell commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

...is it feasible to allow choosing to subtract those features from workers piecemeal?

Not without essentially reproducing what I have here already. It would end up being about the same but with more effort required to ensure we don't break anything.

@jasnell

jasnell commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

@ljharb:

does that allow two threads to mutate the same object at the same time

It's still serialized safely in the receiving side. There won't be thread safety issues

@devsnek

devsnek commented Nov 13, 2026

Copy link
Copy Markdown
Member

This is really neat. I would be interested to see a comparison to building a pool of n worker threads and having them use a vm api to manage the isolates.

@jasnell

jasnell commented Nov 13, 2026 •

Copy link
Copy Markdown
Member Author

@devsnek .... Here's the comparision for a pool of 8 workerthreads (the same size as the green thread pool), with tenants spread across them as vm contexts and called through script.runInContext(), with no pointer compression:

tl;dr As one should expect, the pool of vm contexts wins on raw performance but loses the isolation between them; in particular, since they all share an Isolate, the IsolateGroup under pointer compression means every context is within the same 4GB cage. So it's a deliberate trade off. Green threads aren't meant to replace the vm contexts.

Details

Setup

  • 88 CPUs, Linux x64. Two builds: the default Release build, and one with pointer compression, which supports isolate groups.
  • Green threads: one per tenant, on the shared pool of 8 threads. By default each is in its own isolate group (pointer-compression build only). "Shared group" means all of them in one IsolateGroup.
  • vm pool: 8 workers (the same size as the green thread pool), or 88 (one per CPU), with tenants spread across them. Each tenant is a vm context, and requests run with script.runInContext(). "timeout" means runInContext(context, { timeout: 50 }), which a host needs to stop tenants that loop.
  • Warm: for green threads, setWarmPoolSize(n), which creates isolates in advance. For the vm pool, each worker creates spare contexts in advance. Preparation isn't timed, but its memory is counted.
  • Each scenario runs in a fresh process. Single runs, so treat differences of a few percent as noise.
  • Benchmark: green-vs-vm-pool.js

Startup and memory (1000 tenants)

Release build

Tenants per second RSS per tenant Preparation (not timed)
Green threads, cold 747 5.2 MB –
Green threads, warm 48,245 5.2 MB 2,649 ms
vm pool, 8 workers, cold 10,122 0.25 MB –
vm pool, 8 workers, warm 96,239 0.25 MB 92 ms

Update: The memory measurement here in for green threads is off (too high) by about 1.3mb each. It's only about 3.86 MB per.... essentially the cost of an Isolate.

Pointer-compression build

Tenants per second RSS per tenant Preparation (not timed)
Green threads, cold 1,141 2.9 MB –
Green threads, warm 50,095 2.9 MB 1,610 ms
Green threads, shared group, cold 1,452 1.1 MB –
vm pool, 8 workers, cold 9,972 0.16 MB –
vm pool, 8 workers, warm 98,729 0.16 MB 90 ms

The vm pool wins clearly on density and startup, cold or warm. A context costs 7–20× less memory than a green thread, depending on the build and isolate-group setting. Warm vm contexts start about 2× faster than warm green threads, and take about 90 ms to prepare against 1.6–2.6 s. The warm green thread pool only applies to the default options, so it can't be combined with a shared group.

Throughput (1000 tenants, 50k requests, 64 at a time)

Release: requests/s Release: p99 Pointer compression: requests/s Pointer compression: p99
Green threads 139k 1.8 ms 140k 1.8 ms
Green threads, shared group – – 142k 1.9 ms
vm pool, 8 workers 151k 0.9 ms 136k 0.9 ms
vm pool, 88 workers 136k 1.0 ms 128k 1.2 ms
vm pool, 8 workers, timeout 81k 2.3 ms 82k 3.5 ms
vm pool, 88 workers, timeout 60k 2.8 ms 58k 3.1 ms

Without timeout, throughput is about equal: the vm pool is slightly ahead in the Release build and slightly behind with pointer compression. The vm pool has a lower p99. A host running untrusted tenants needs timeout, though, which costs 40–55% of throughput, because each call starts a watchdog. With it, green threads are about 1.7× faster.

When one of 100 tenants misbehaves

The vm pool uses 8 workers and timeout: 50; green threads use maxTaskCpuTimeMs: 50. "Others" are the other 99 tenants, which keep sending requests for 300 ms. Release build, with pointer-compression results in parentheses where they differ.

Problem Stopped after Tenants lost Others' requests in 300 ms Others' slowest request
Synchronous loop Green threads 51 ms 1 45k (44k) 3 ms (8 ms)
vm pool 50 ms 0 17k 51 ms
Microtask loop Green threads 52 ms (55 ms) 1 45k (44k) 3 ms (6 ms)
vm pool 102 ms, by killing the worker 13 13k 16 ms (14 ms)
vm pool, microtaskMode: 'afterEvaluate' 50 ms 0 18k (17k) 51 ms
  • Synchronous loop: both stop it after 50 ms. In the vm pool, the other tenants on its worker wait for the whole 50 ms, and the pool serves about 40% as many requests in the meantime.
  • Microtask loop: timeout doesn't cover microtasks that run after the call returned. The only way to stop the loop is to kill the worker, along with the 12 other tenants on it. microtaskMode: 'afterEvaluate' brings microtasks under timeout, but has pitfalls of its own: promises created outside the context aren't drained with it.
One tenant fills its heap Stopped after Tenants lost Others still answering
Green threads (maxMemoryMb: 64) 479 ms (1,413 ms) 1 99
vm pool (workers with maxOldGenerationSizeMb: 64) 434 ms (1,046 ms) 13 87

vm has no per-context memory limits, so the worker runs out of memory and dies, taking all its tenants with it.

Summary

It's a density versus isolation trade-off:

  • A vm pool wins on density and startup, cold or warm, by 7–20× on memory and about 2× on warm startup. Throughput is about equal without timeout.
  • Green threads win when one tenant can affect the others.
    • Each tenant has its own heap and its own CPU and memory limits.
    • Tenants can't stall each other, and nothing needs a timeout on every call. With timeout, green threads are about 1.7× faster.
    • A failure takes down one tenant, not a whole worker's worth.
  • vm isn't a security boundary, as its docs say. All the tenants on a worker share an isolate, so an escape from one context exposes the others and the worker's Node.js APIs. Each green thread is a separate isolate with no Node.js APIs, in its own isolate group by default.

For many trusted, well-behaved tenants, a vm pool is the cheaper choice. For untrusted ones, green threads give per-tenant isolation without paying a worker per tenant.

@jasnell

jasnell commented Nov 13, 2026 •

Copy link
Copy Markdown
Member Author

Just more on the approach here... looking at the isolation/sandboxing characterstics here... Assuming the V8 isolate boundary holds, so no V8 bug lets code escape its isolate... (most of this AI written with tweaks)

Details
Worker thread vm context (in a worker or the main thread) Green thread
Boundary None for code: same process, full Node.js A separate global object in a shared isolate. Explicitly not an isolation boundary A separate isolate, no Node.js or system access via API
What untrusted code can reach Everything the process can: fs, network, child processes, env vars, native addons Whatever leaks from the host realm, and through the usual leaks the host's full Node.js Only what the host passes: env, RPC stubs, modules, ports (Capability-based model)
Escape without a V8 bug? Not needed, it already has everything Yes, through well-known JS techniques. vm isn't a security mechanism, by its own docs No: no object graph is shared
CPU limit per tenant No, only wall-clock terminate() timeout, synchronous code only by default Yes: maxTaskCpuTimeMs, maxCpuTimeMs
Memory limit per tenant JS heap only, not ArrayBuffers None, shares the worker's heap Yes, heap and ArrayBuffers together
What a crash or out-of-memory takes down That worker Every tenant on the worker (or the main thread) That tenant
Can one tenant stall others? No (own thread) Yes, everyone on the same worker Not for longer than the CPU limit

Workers aren't a sandbox

A worker runs with the whole process's authority. The permission model applies to the whole process, so it can't restrict one worker. With fs, a worker can even read /proc/self/mem, which exposes every other thread's memory. The isolate boundary only helps against code that doesn't use Node.js's APIs, and a worker always has them.

vm contexts can be escaped without a V8 bug

All contexts on a worker share one isolate, so objects cross between them, and any host-realm object reachable from the sandbox is a way out. For example, this.constructor.constructor('return process')() reaches the host's Function constructor through the sandbox object. The same applies to host functions or objects passed in, and to errors and promises the host creates. These are JavaScript semantics, not V8 bugs. Hardening it means wrapping everything that crosses over, which vm2 tried and repeatedly got wrong. A successful escape lands in the worker's full Node.js environment, along with the other tenants on that worker.

Green threads only have what the host passes in

No JS object is ever shared:

  • Messages, env values and RPC arguments are structured clones, built in the receiver's own realm. There's no host prototype chain to climb.
  • Functions and RpcTargets become stubs. Calls are checked against an allowlist of method names on the side that owns the object. constructor, and anything from Object.prototype, Function.prototype or RpcTarget.prototype, is excluded.
  • Errors that reach the host are rebuilt from data properties only. No getters run, and no green-side code runs on the host.

The attack surface is the host's own env functions and RpcTargets. They receive untrusted arguments, so they must validate them like any server handler, and must not act as a confused deputy.

Isolate-group settings

  • Own group (the default where supported): each tenant has its own 4 GB pointer-compression cage, and can't use up heap space that others or the main thread need.
  • Shared IsolateGroup: tenants share a cage. Without maxMemoryMb, one tenant can exhaust it for the others. That's a denial-of-service risk, not a data leak.
  • isolateGroup: 'default': the cage is shared with the main thread and workers, so a tenant without a memory limit can starve the host.

Against V8 bugs, which are outside this comparison, separate groups also limit how far a heap-corruption bug can reach.

What remains for every option

  • Side channels. Everything shares one process and address space, and isolates, contexts and workers aren't a Spectre boundary. Passing tenants a SharedArrayBuffer gives them a high-resolution timer. Hostile code that might attempt timing attacks needs process isolation, or mitigations like coarse timers and no shared memory.
  • Denial of service against the host. Green threads limit CPU and memory per tenant, but don't rate-limit RPC calls or messages. A JIT-compiled loop that ignores termination (a known V8 bug) gets its isolate abandoned, and abandoned isolates are never freed, so a tenant that can trigger it repeatedly can leak memory. Workers hit the same bug, and there terminate() doesn't complete.
  • Host capabilities. Anything passed in, such as a database handle as an RpcTarget, or a port connected to a worker, is exactly as powerful as its host-side implementation.

In short

Workers give no sandboxing. vm contexts give isolation by convention, which ordinary JavaScript can break. Green threads give isolation by construction following a capabilities model: the only way out is through the capabilities the host passes in, or a V8 bug. In every option, defending against timing side channels takes process-level isolation.

The internal NewIsolate() always registers the isolate with the
platform using a libuv loop, which ties V8's foreground tasks to that
loop. Add an overload that registers an IsolatePlatformDelegate
instead, so that embedder-level schedulers can run an isolate's
foreground tasks themselves, and share the rest of the setup between
both overloads.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Add GreenThread, which runs JavaScript in its own V8 isolate without an
OS thread, event loop or Node.js environment of its own. Green threads
are scheduled onto a process-wide pool of threads, each with a libuv
loop. An isolate stays on the pool thread it was assigned to, and runs
one task at a time with the isolate locked, followed by a microtask
checkpoint.

A green thread only gets the ECMAScript built-ins, the default context
of the startup snapshot, plus postMessage(), onmessage, close(),
queueMicrotask(), timers and a console that is forwarded to the host.
The host can send messages, evaluate scripts and terminate the green
thread. Uncaught exceptions, unhandled rejections and reaching the heap
limit emit 'error' and stop the green thread.

V8 foreground tasks for green isolates go through their own task
runner, so they run on the right pool thread with the isolate locked.
Terminations requested while an isolate is unlocked are tracked
separately, because acquiring the lock discards them. Green threads are
stopped when their host Environment is torn down, and before the
platform is disposed when the process exits.

The module is only available with --experimental-green-threads.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Green isolates were pinned to the pool thread they were created on, so
one that never yielded blocked every other green thread assigned to the
same thread.

Queue the work of each isolate in FIFO order instead, and let whichever
pool thread holds the isolate's lock run a slice of it. Work is either
unbound, like messages, evaluations and V8 tasks, or bound to the pool
thread that owns the I/O it continues. Isolates with unbound work wait
in a process-wide ready queue, from which idle threads take them,
preferring the thread they last ran on. No thread ever blocks waiting
for an isolate.

Every message and evaluation now runs in its own invocation context,
bound to the thread that runs it. Timers belong to the current context,
which is carried through promise reactions as continuation-preserved
embedder data, and their handles live, fire and are closed on that
context's thread.

Also destroy V8 tasks that never ran before the isolate is freed.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Closing a GreenThread handle terminated the green thread and then
waited for a pool thread to dispose of it. When every pool thread was
busy with green threads that never yield, and those were only going to
be terminated after it, tearing down the Environment hung. If no pool
thread is running the isolate, shut it down on the calling thread
instead.

A delayed V8 task could be destroyed after its isolate was freed when
its timer fired while the isolate shut down, because the timer's
callback shared ownership of the task. Only let the scheduler own it.

After an uncaught error or reaching the heap limit, the microtask
checkpoint could still run, since IsExecutionTerminating() is false
again once the termination has unwound the stack.

Fire an interval at most once while a previous fire is still queued,
so an isolate that is busy on another thread doesn't run a backlog of
callbacks once it is free.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Add the maxCpuTimeMs and maxTaskCpuTimeMs resource limits. A
process-wide watchdog thread interrupts a green thread at the earliest
point at which its task could have used up its CPU time budget. The
interrupt then checks the CPU time of the thread that runs the task,
and either terminates the green thread or waits for the rest of the
budget. This preempts code that never yields, such as busy loops,
endless microtask chains and long-running built-ins.

Add the maxMemoryMb resource limit, which covers the heap and
ArrayBuffers together. ArrayBuffer allocations that would exceed it
throw a RangeError, and the green thread is terminated when the memory
it keeps alive exceeds it after a garbage collection.

Most of the time it takes to start a green thread is spent creating its
isolate, which deserializes the isolate part of the startup snapshot.
Add setWarmPoolSize(), which keeps isolates with the default isolate
options ready, created in the background when pool threads have nothing
else to do. Green threads now get an isolate in two steps: preparing
it, which a warm isolate already is, and activating it.

Green threads that no Environment owns, like warm ones, are now also
stopped in TearDownOncePerProcess().

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Compare how fast green threads, warm green threads, workers and vm
contexts start, and how fast green threads and workers echo messages.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Return a promise from setWarmPoolSize() that is fulfilled once the warm
pool holds the requested number of initialized isolates, so that
callers, like the startup benchmark, no longer have to guess how long
filling it takes.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
The ArrayBuffer allocator checked allocations against the size of the
heap as of the last garbage collection, which was 0 until the first
one. ArrayBuffers could then take up the whole memory limit, and the
next garbage collection terminated the green thread for being over the
limit, instead of the allocation throwing a RangeError. Even with an
accurate heap size, the heap grows a little between garbage collections,
for example by the ArrayBuffer objects themselves.

Record the size of the heap once the context is created, and leave a
sixteenth of the limit for the heap to grow into.

Also stop using `new Array(n)` in the test for heap and ArrayBuffer
memory together. V8 may compile such a loop with on-stack replacement
into code that no longer handles interrupts, so that the green thread
is never terminated.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
With pointer compression, all isolates in a V8 isolate group share one
4 GB pointer compression cage. Where the build supports more than one
group, give each green thread a group of its own by default, like
Node.js does for its own isolates, so that green threads can't use up
the heap space of the main thread or of each other. Add the
isolateGroup option to select V8's default group, or an IsolateGroup
that green threads share, instead.

Isolates in groups that only green threads use start from V8's own
startup snapshot rather than from Node.js', since all isolates in a
group need the same snapshot and green threads don't need any of
Node.js' per-isolate state. With pointer compression, this makes
starting a green thread about twice as fast, and its memory use about
40% lower, or 65% lower when green threads share a group.

The ArrayBuffer allocator of a green thread now allocates from the
sandbox of its isolate group in sandbox builds.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
V8 can compile code, such as some loops that it optimizes with on-stack
replacement, in a way that no longer handles interrupts. Such code
can't be terminated, so a green thread running it never reached its CPU
or memory limit, terminate() never completed, and a host that waited
for it, such as one that exits, waited forever.

Add a backup watchdog. When a green thread was asked to stop, or a CPU
time check that was requested never ran, it gets a grace period of one
second. After that, the green thread is abandoned: its host is told why
it stopped, it exits with code 1, a GreenThreadWarning is emitted, and
its isolate is never used again or freed.

The watchdog then signals the pool thread that runs it until the signal
arrives while the thread runs code that V8 generated, which holds no
locks. The signal handler then jumps back to the pool thread's loop,
leaving the frames of the green thread behind, and the pool thread is
cleaned up and runs other green threads again. Where that's not
supported, or doesn't succeed within 10 seconds, the process is aborted.

V8 and the platform are not disposed at exit if green threads were
abandoned, since their isolates still use them.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
The pool that green threads run on has as many threads as the
available parallelism, but at most 8. That limits how many green
threads can run JavaScript at the same time: 256 green threads that
each run for 20 ms per request handle 400 requests per second on a
machine with 88 CPUs, and 4,205 with 88 pool threads.

Add an environment variable that sets the number of pool threads, up
to 1024, like UV_THREADPOOL_SIZE does for libuv's threadpool.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
SetIsolateCreateParamsForNode() queried the total and the constrained
memory, which reads files on Linux, even when the isolate's heap already
had a limit, and the result wasn't used. Workers call it twice, so every
worker read them at least once for nothing.

Only query them when the heap has no limit, and expose the query as
GetMemoryForHeapDefaults().

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads without a limit for the old generation got V8's defaults
for the physical memory from NewIsolate(), which replaced the limit for
the young generation, so maxYoungGenerationSizeMb on its own had no
effect. Creating them also read /proc/meminfo and cgroup files each
time, about 2% of the time it took to start them concurrently.

Configure the defaults before setting the young generation limit, the
way workers do, with the memory queried once per process. Report the
heap spaces in __greenThreadInfo(), for tests.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Only green threads without memory limits, in the default isolate
group, could use the warm pool, since those are fixed when an isolate
is created. Hosts that run untrusted code usually set limits, so their
green threads always started cold.

Let setWarmPoolSize() take the isolateGroup and resourceLimits options
of new GreenThread(), and keep a pool for each isolate group and set of
memory limits it was called with. Green threads only use the pool for
theirs, so they get exactly the limits they would get without it. CPU
time limits are applied when green threads start, so they don't
matter.

With maxMemoryMb set, 1000 green threads, 64 at a time, start at about
60,000/s from a full pool rather than 4,100/s in a release build.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
V8 serializes external strings as ordinary strings unless their
resources are external references. The snapshots of green threads only
had the runtime's callbacks as external references, so they contained
copies of the sources of its builtins, about 65 KB, which every green
thread deserialized into its heap.

Add Node.js' external references, which include the sources of all
builtins, so that the snapshots refer to the sources in the binary.

This makes each green thread use 60 KB less memory (1.34 MB rather than
1.40 in a release build) and the startup snapshots 65 KB smaller.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
jasnell force-pushed the green-threads branch 2 times, most recently from 6f21b2c to 4b6fe8b Compare November 14, 2026 17:19
@jasnell

jasnell commented Nov 14, 2026 •

Copy link
Copy Markdown
Member Author

I pushed a few more commits here to optimize performance and memory further.

Details are an AI generated summary... (slightly edited to remove slop)

tl;dr -- Compared with one worker per tenant, green threads use about 4.5× less memory (1.36 MB vs 6.17 MB PSS each), start about 4× faster cold (~3,700/s vs ~980/s), and reach 65,000–100,000/s from a warm pool. Message round trips are the same, and so is CPU-bound throughput at equal parallelism (with NODE_GREEN_THREADS_POOL_SIZE raised to match). Compared with a pool of workers running vm contexts, vm contexts are cheaper per tenant (0.25 MB and 15,000 cold starts/s on 8 workers, against 1.36 MB and 3,800/s), and raw request throughput is about the same (190,000 requests/s). But a host has to pass timeout to stop runaway tenants, which halves the pool's throughput, and timeout misses microtask loops unless the context uses microtaskMode: 'afterEvaluate'. A tenant that loops also delays its neighbours (52 ms vs 3 ms for the slowest other request), and one that fills its heap takes down every tenant on its worker. Green threads stop such a tenant in 50 ms with per-tenant CPU and memory limits and no per-call cost, without affecting anyone else. On top of that, they only have the capabilities the host passes in, while vm contexts aren't a security boundary.

Details

Updated comparison: green threads vs workers vs a pool of workers running vm contexts

Since the last update, green threads in V8's default isolate group start from a
small snapshot of their own instead of Node.js' snapshot. That brought one cold
start from ~3.6 ms to ~1.1 ms and memory from ~3.7 MB to ~1.3 MB per green
thread. The warm pool now also serves green threads with memory limits and
isolate groups.

Green threads vs one worker per tenant

Startup and memory, 1000 idle instances

starts/s PSS per instance PSS total OS threads
green thread 3,739 1.36 MB 1,364 MB 8 (shared pool)
green thread, from warm pool 80,778 1.38 MB 1,377 MB 8
worker 977 6.17 MB 6,171 MB 1,000

Pointer compression build:

starts/s PSS per instance
green thread, own isolate group (default) 2,556 1.35 MB
green thread, V8's default group 3,926 0.95 MB
green thread, one shared IsolateGroup 4,170 0.95 MB
green thread, from warm pool 101,977 1.36 MB
worker 960 4.85 MB

1000 isolated tenants, 50,000 requests

all started requests/s PSS total
green threads 271 ms 201,834 1,415 MB
workers 1,025 ms 157,475 6,216 MB

Message round trips, one instance: the same, 49,187/s vs 49,221/s (20.3 µs each).

CPU-bound tasks, 2000 × fib(27)

tasks/s
8 green threads 5,023
8 workers 5,052
88 green threads, NODE_GREEN_THREADS_POOL_SIZE=88 21,516
88 workers 22,230
one green thread per task (2000), pool of 8 4,151

At equal parallelism, CPU-bound throughput is the same. The pool size caps how
much JavaScript runs in parallel. It defaults to availableParallelism(), up
to 8, and NODE_GREEN_THREADS_POOL_SIZE raises it.

One of 100 tenants loops forever

stopped by stopped after others served in 200 ms
green threads maxTaskCpuTimeMs: 50 52 ms 46,332
workers terminate() after a 50 ms timer 65 ms 36,135

Green threads vs a pool of workers running vm contexts

(@devsnek's suggestion: N workers, tenants spread over them, one vm context
per tenant, requests run with script.runInContext().)

Startup and memory, 1000 tenants

tenants/s PSS per tenant prepared beforehand
green threads 3,781 1.36 MB –
green threads, warm pool 64,951 1.39 MB 394 ms
vm pool, 8 workers 15,360 0.25 MB –
vm pool, 8 workers, contexts created beforehand 121,545 0.25 MB 57 ms

Pointer compression build: green threads 2,439/s at 1.36 MB, in a shared
IsolateGroup 3,851/s at 0.96 MB, warm pool 69,346/s; vm pool 16,242/s at
0.16 MB.

A vm context is much cheaper than an isolate. It shares the worker's heap,
GC and V8 per-isolate structures, so cold starts are ~4× faster and memory per
tenant is ~5–8× lower. That's the price of the isolation below. About 360 KB of
each green thread (Release) is V8's per-isolate stub caches alone, which a
vm context doesn't pay.

1000 tenants, 50,000 requests, 64 at a time

requests/s p50 p99
green threads 186,481 0.32 ms 1.27 ms
vm pool, 8 workers 190,804 0.33 ms 0.72 ms
vm pool, 88 workers 169,638 0.36 ms 0.82 ms
vm pool, 8 workers, with timeout 80,415 0.63 ms 2.59 ms
vm pool, 88 workers, with timeout 58,469 1.00 ms 2.90 ms

Without a timeout, throughput is the same. But a host has to pass
runInContext()'s timeout to stop tenants that loop, and that more than
halves the pool's throughput (it starts a watchdog per call). Green threads
enforce CPU limits without a per-call cost.

One of 100 tenants runs a synchronous infinite loop

stopped after tenants lost others served in 300 ms slowest other
green threads 52 ms 1 (the bad one) 59,499 3 ms
vm pool, 8 workers, timeout: 50 51 ms 0 18,414 52 ms

One of 100 tenants runs an infinite microtask loop

stopped after tenants lost others served in 300 ms slowest other
green threads 54 ms 1 (the bad one) 61,776 3 ms
vm pool, timeout: 50 101 ms (worker terminated) 13 13,659 16 ms
vm pool, timeout: 50, microtaskMode: 'afterEvaluate' 51 ms 0 18,216 52 ms

timeout doesn't cover microtasks unless the context uses
microtaskMode: 'afterEvaluate'. Without it, the only way out is terminating
the worker, which takes every tenant on it along.

One of 100 tenants fills its heap

stopped after tenants lost others still answering
green threads 312 ms 1 (the bad one) 99
vm pool, 8 workers 284 ms 13 (whole worker) 87

In a vm pool, a tenant that loops blocks the other tenants on its worker until
it's stopped (slowest other request: 52 ms vs 3 ms). A tenant that exhausts the
heap takes down every tenant on its worker, since they share one heap.

Which benefits are inherent to green threads (@glenjamin)

Inherent, from one isolate per tenant on a shared pool:

  • A heap and GC per tenant: memory limits and out-of-memory only affect that tenant. A vm context can't have its own heap limit.
  • CPU limits per tenant without a per-call cost: they cover microtasks, and other tenants keep running on other pool threads.
  • One tenant can be terminated without affecting the others. In a vm pool, that requires terminating the worker.
  • No Node.js API surface by default: a green thread only has what the host passes in. vm contexts are not a security boundary (escapable without V8 bugs), and workers have full Node.js.
  • Cheaper than workers: about 4.5× less memory (1.36 vs 6.17 MB) and 4× faster cold starts, because there's no OS thread, event loop or Node.js environment per instance.

Not inherent, or where the alternatives win:

  • Memory and cold start per tenant: a vm context is cheaper, because it shares its worker's isolate. Most of a green thread's remaining 1–1.4 MB is V8's per-isolate overhead, which only V8 changes could reduce.
  • Raw request throughput: the same as a vm pool without timeouts, and the same as workers for CPU-bound work at equal parallelism.
  • Warm pools: they hide cold-start cost for green threads (65,000–100,000 starts/s), much like pre-creating vm contexts does for a pool.

@jasnell
jasnell marked this pull request as ready for review November 14, 2026 17:44
@jasnell jasnell added experimental Issues and PRs related to experimental features. large-pr PRs subject to the large-PR policy. labels Nov 14, 2026

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jasnell jasnell added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Nov 14, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Nov 14, 2026
@nodejs-github-bot

This comment was marked as outdated.

Format the code with clang-format, and declare the V8 names that
src/node_green_thread.cc uses with using-declarations at the top of the
file rather than qualifying them with v8::. Remove an unused declaration
and keep all of the file's using-declarations sorted, as
tools/checkimports.py requires.

Regenerate doc/node.1, which was missing the updated description of
NODE_GREEN_THREADS_POOL_SIZE.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

SnapshotBuilder::CreateSnapshot() built the startup snapshot of green
threads in the default isolate group for every snapshot without a
builder script, including the ones that embedders create through
CommonEnvironmentSetup::CreateSnapshot(). Embedders have platforms of
their own, so per_process::v8_platform has none and creating the
snapshot crashed.

Build it only for GenerateAsSource(), which is its only user.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

Green threads could only send messages to one receiver at a time.

Add BroadcastChannel to green threads, for the names in the new
broadcastScopes option of new GreenThread(). The channels are Node.js'
own, process-wide, so messages reach every BroadcastChannel with the
same name, in the main thread, in workers and in other green threads.
Other names throw ERR_GREEN_THREAD_BROADCAST_SCOPE, so green threads
can only use the channels that their host allows.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads that ignored termination and were abandoned kept their
ports and BroadcastChannels open. The ports they were entangled with
were never closed, and messages for them queued up until the process
exited.

Disentangle the ports of green threads when they are abandoned, and drop
their messages. Since the green thread may still be running, the ports
are only entangled, disentangled and used to post messages while a lock
is held, which is never held while JavaScript runs.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads could only be terminated, which interrupts whatever they
are doing.

Add greenThread.close([options]), which calls the green thread's
onclose function, and makes it exit with exit code 0 once nothing that
it waits for is pending: timers, RPC calls in either direction,
evaluations, its main module, and tasks queued before. Messages posted
after close() are dropped, and evaluate() rejects with the new
ERR_GREEN_THREAD_CLOSING. If the green thread hasn't exited within
`timeoutMs`, 5000 by default, it is terminated.

Assisted-by: Claude Opus 5.5
Signed-off-by: James M Snell <jasnell@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@ronag ronag left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little unclear when one should use this. I have an idea but it doesn't become super clear from the documentation.

I assume this is for compute tasks (i.e. no IO) and in general should be preferred over Workers for those kind of tasks. Might be good to have a paragraph in the beginning of the docs explaining the use case.

@ronag

ronag commented Nov 15, 2026

Copy link
Copy Markdown
Member

Why ceil it at 8 threads? In practical terms I would suggest a default like:

availableParallelism() / (SMT ? 2 : 1) - uvthreads / 4

Can the green thread worker pool dynamically size based on load? e.g. start with one thread and grow up to max threads on load?

@jasnell

jasnell commented Nov 15, 2026

Copy link
Copy Markdown
Member Author

Why ceil it at 8 threads

Had to start somewhere :-)

It's actually min(8, availableParallellism) and it's configurable with an env var. We can explore making it dynamic.

@devsnek

devsnek commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

@jasnell sorry for the confusion, i meant: what if an api for managing isolates was added to the vm module? i would assume the scheduling aspect of v8 lockers would be the largest challenge there, compared to your hardcoded "green thread" approach. though with prior art like web locks, maybe a similar pattern is possible?

@jasnell

jasnell commented Nov 15, 2026

Copy link
Copy Markdown
Member Author

The scheduling and interactivity are the most complicated bits. Overall this could be added to vm nevertheless but I don't know if it makes it any simpler/better. Can be convinced tho

@Eric24

Eric24 commented Nov 15, 2026

Copy link
Copy Markdown

It's a little unclear when one should use this.

And some examples, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

experimental Issues and PRs related to experimental features. large-pr PRs subject to the large-PR policy. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants