Repository navigation
Conversation
|
Review requested:
|
|
I'm not clear what the etiquette is for feedback on things like this from random Node.js users, so if this isn't the place for it please let me know / delete/edit this as appropriate. This seems really neat, but from reading the comparisons it got me wondering: How many of the benefits are inherent to what green-threads can do, and how many of them are things that could conceivably be added to the existing workers implementation? |
|
@jasnell does each green thread still have its own JS environment, like workers and worker threads? |
|
The benefits realized here are because they are not implemented the way existing workers are. Specifically, a green thread is not bound to a single physical thread the way a worker thread is. A green thread does not have an entire Node.js |
|
Is it possible to directly share objects between green threads? |
|
Share-ish. You can postMessage values to it that are structured cloned. There's also an rpc mechanism for accessing objects across the boundary. Otherwise, however, in this impl, no truly shared objects. |
|
OK - last question, I think :-) the RPC mechanism; does that allow two threads to mutate the same object at the same time? |
mcollina
left a comment
There was a problem hiding this comment.
rlgtm, this is a good first step and addition to Node
What I meant is: is it feasible to allow choosing to subtract those features from workers piecemeal? ie. Could the RPC thing be made available to workers? Could workers have a flag to get a limited environment? or the CPU/memory limits etc etc I appreciate the M:N aspect is specific to green threads, but naively the other features appear to be choices made while implementing this. |
Not without essentially reproducing what I have here already. It would end up being about the same but with more effort required to ensure we don't break anything. |
It's still serialized safely in the receiving side. There won't be thread safety issues |
|
This is really neat. I would be interested to see a comparison to building a pool of n worker threads and having them use a vm api to manage the isolates. |
|
@devsnek .... Here's the comparision for a pool of 8 workerthreads (the same size as the green thread pool), with tenants spread across them as tl;dr As one should expect, the pool of vm contexts wins on raw performance but loses the isolation between them; in particular, since they all share an Isolate, the DetailsSetup
Startup and memory (1000 tenants)Release build
Update: The memory measurement here in for green threads is off (too high) by about 1.3mb each. It's only about 3.86 MB per.... essentially the cost of an Isolate. Pointer-compression build
The Throughput (1000 tenants, 50k requests, 64 at a time)
Without When one of 100 tenants misbehavesThe
SummaryIt's a density versus isolation trade-off:
For many trusted, well-behaved tenants, a |
|
Just more on the approach here... looking at the isolation/sandboxing characterstics here... Assuming the V8 isolate boundary holds, so no V8 bug lets code escape its isolate... (most of this AI written with tweaks) Details
Workers aren't a sandboxA worker runs with the whole process's authority. The permission model applies to the whole process, so it can't restrict one worker. With
|
6685824 to
8b7df9b
Compare
The internal NewIsolate() always registers the isolate with the platform using a libuv loop, which ties V8's foreground tasks to that loop. Add an overload that registers an IsolatePlatformDelegate instead, so that embedder-level schedulers can run an isolate's foreground tasks themselves, and share the rest of the setup between both overloads. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Add GreenThread, which runs JavaScript in its own V8 isolate without an OS thread, event loop or Node.js environment of its own. Green threads are scheduled onto a process-wide pool of threads, each with a libuv loop. An isolate stays on the pool thread it was assigned to, and runs one task at a time with the isolate locked, followed by a microtask checkpoint. A green thread only gets the ECMAScript built-ins, the default context of the startup snapshot, plus postMessage(), onmessage, close(), queueMicrotask(), timers and a console that is forwarded to the host. The host can send messages, evaluate scripts and terminate the green thread. Uncaught exceptions, unhandled rejections and reaching the heap limit emit 'error' and stop the green thread. V8 foreground tasks for green isolates go through their own task runner, so they run on the right pool thread with the isolate locked. Terminations requested while an isolate is unlocked are tracked separately, because acquiring the lock discards them. Green threads are stopped when their host Environment is torn down, and before the platform is disposed when the process exits. The module is only available with --experimental-green-threads. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green isolates were pinned to the pool thread they were created on, so one that never yielded blocked every other green thread assigned to the same thread. Queue the work of each isolate in FIFO order instead, and let whichever pool thread holds the isolate's lock run a slice of it. Work is either unbound, like messages, evaluations and V8 tasks, or bound to the pool thread that owns the I/O it continues. Isolates with unbound work wait in a process-wide ready queue, from which idle threads take them, preferring the thread they last ran on. No thread ever blocks waiting for an isolate. Every message and evaluation now runs in its own invocation context, bound to the thread that runs it. Timers belong to the current context, which is carried through promise reactions as continuation-preserved embedder data, and their handles live, fire and are closed on that context's thread. Also destroy V8 tasks that never ran before the isolate is freed. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Closing a GreenThread handle terminated the green thread and then waited for a pool thread to dispose of it. When every pool thread was busy with green threads that never yield, and those were only going to be terminated after it, tearing down the Environment hung. If no pool thread is running the isolate, shut it down on the calling thread instead. A delayed V8 task could be destroyed after its isolate was freed when its timer fired while the isolate shut down, because the timer's callback shared ownership of the task. Only let the scheduler own it. After an uncaught error or reaching the heap limit, the microtask checkpoint could still run, since IsExecutionTerminating() is false again once the termination has unwound the stack. Fire an interval at most once while a previous fire is still queued, so an isolate that is busy on another thread doesn't run a backlog of callbacks once it is free. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Add the maxCpuTimeMs and maxTaskCpuTimeMs resource limits. A process-wide watchdog thread interrupts a green thread at the earliest point at which its task could have used up its CPU time budget. The interrupt then checks the CPU time of the thread that runs the task, and either terminates the green thread or waits for the rest of the budget. This preempts code that never yields, such as busy loops, endless microtask chains and long-running built-ins. Add the maxMemoryMb resource limit, which covers the heap and ArrayBuffers together. ArrayBuffer allocations that would exceed it throw a RangeError, and the green thread is terminated when the memory it keeps alive exceeds it after a garbage collection. Most of the time it takes to start a green thread is spent creating its isolate, which deserializes the isolate part of the startup snapshot. Add setWarmPoolSize(), which keeps isolates with the default isolate options ready, created in the background when pool threads have nothing else to do. Green threads now get an isolate in two steps: preparing it, which a warm isolate already is, and activating it. Green threads that no Environment owns, like warm ones, are now also stopped in TearDownOncePerProcess(). Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Compare how fast green threads, warm green threads, workers and vm contexts start, and how fast green threads and workers echo messages. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Return a promise from setWarmPoolSize() that is fulfilled once the warm pool holds the requested number of initialized isolates, so that callers, like the startup benchmark, no longer have to guess how long filling it takes. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
The ArrayBuffer allocator checked allocations against the size of the heap as of the last garbage collection, which was 0 until the first one. ArrayBuffers could then take up the whole memory limit, and the next garbage collection terminated the green thread for being over the limit, instead of the allocation throwing a RangeError. Even with an accurate heap size, the heap grows a little between garbage collections, for example by the ArrayBuffer objects themselves. Record the size of the heap once the context is created, and leave a sixteenth of the limit for the heap to grow into. Also stop using `new Array(n)` in the test for heap and ArrayBuffer memory together. V8 may compile such a loop with on-stack replacement into code that no longer handles interrupts, so that the green thread is never terminated. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
With pointer compression, all isolates in a V8 isolate group share one 4 GB pointer compression cage. Where the build supports more than one group, give each green thread a group of its own by default, like Node.js does for its own isolates, so that green threads can't use up the heap space of the main thread or of each other. Add the isolateGroup option to select V8's default group, or an IsolateGroup that green threads share, instead. Isolates in groups that only green threads use start from V8's own startup snapshot rather than from Node.js', since all isolates in a group need the same snapshot and green threads don't need any of Node.js' per-isolate state. With pointer compression, this makes starting a green thread about twice as fast, and its memory use about 40% lower, or 65% lower when green threads share a group. The ArrayBuffer allocator of a green thread now allocates from the sandbox of its isolate group in sandbox builds. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
V8 can compile code, such as some loops that it optimizes with on-stack replacement, in a way that no longer handles interrupts. Such code can't be terminated, so a green thread running it never reached its CPU or memory limit, terminate() never completed, and a host that waited for it, such as one that exits, waited forever. Add a backup watchdog. When a green thread was asked to stop, or a CPU time check that was requested never ran, it gets a grace period of one second. After that, the green thread is abandoned: its host is told why it stopped, it exits with code 1, a GreenThreadWarning is emitted, and its isolate is never used again or freed. The watchdog then signals the pool thread that runs it until the signal arrives while the thread runs code that V8 generated, which holds no locks. The signal handler then jumps back to the pool thread's loop, leaving the frames of the green thread behind, and the pool thread is cleaned up and runs other green threads again. Where that's not supported, or doesn't succeed within 10 seconds, the process is aborted. V8 and the platform are not disposed at exit if green threads were abandoned, since their isolates still use them. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
The pool that green threads run on has as many threads as the available parallelism, but at most 8. That limits how many green threads can run JavaScript at the same time: 256 green threads that each run for 20 ms per request handle 400 requests per second on a machine with 88 CPUs, and 4,205 with 88 pool threads. Add an environment variable that sets the number of pool threads, up to 1024, like UV_THREADPOOL_SIZE does for libuv's threadpool. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
SetIsolateCreateParamsForNode() queried the total and the constrained memory, which reads files on Linux, even when the isolate's heap already had a limit, and the result wasn't used. Workers call it twice, so every worker read them at least once for nothing. Only query them when the heap has no limit, and expose the query as GetMemoryForHeapDefaults(). Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads without a limit for the old generation got V8's defaults for the physical memory from NewIsolate(), which replaced the limit for the young generation, so maxYoungGenerationSizeMb on its own had no effect. Creating them also read /proc/meminfo and cgroup files each time, about 2% of the time it took to start them concurrently. Configure the defaults before setting the young generation limit, the way workers do, with the memory queried once per process. Report the heap spaces in __greenThreadInfo(), for tests. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Only green threads without memory limits, in the default isolate group, could use the warm pool, since those are fixed when an isolate is created. Hosts that run untrusted code usually set limits, so their green threads always started cold. Let setWarmPoolSize() take the isolateGroup and resourceLimits options of new GreenThread(), and keep a pool for each isolate group and set of memory limits it was called with. Green threads only use the pool for theirs, so they get exactly the limits they would get without it. CPU time limits are applied when green threads start, so they don't matter. With maxMemoryMb set, 1000 green threads, 64 at a time, start at about 60,000/s from a full pool rather than 4,100/s in a release build. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
V8 serializes external strings as ordinary strings unless their resources are external references. The snapshots of green threads only had the runtime's callbacks as external references, so they contained copies of the sources of its builtins, about 65 KB, which every green thread deserialized into its heap. Add Node.js' external references, which include the sources of all builtins, so that the snapshots refer to the sources in the binary. This makes each green thread use 60 KB less memory (1.34 MB rather than 1.40 in a release build) and the startup snapshots 65 KB smaller. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
6f21b2c to
4b6fe8b
Compare
|
I pushed a few more commits here to optimize performance and memory further. Details are an AI generated summary... (slightly edited to remove slop) tl;dr -- Compared with one worker per tenant, green threads use about 4.5× less memory (1.36 MB vs 6.17 MB PSS each), start about 4× faster cold (~3,700/s vs ~980/s), and reach 65,000–100,000/s from a warm pool. Message round trips are the same, and so is CPU-bound throughput at equal parallelism (with DetailsUpdated comparison: green threads vs workers vs a pool of workers running
|
| starts/s | PSS per instance | PSS total | OS threads | |
|---|---|---|---|---|
| green thread | 3,739 | 1.36 MB | 1,364 MB | 8 (shared pool) |
| green thread, from warm pool | 80,778 | 1.38 MB | 1,377 MB | 8 |
| worker | 977 | 6.17 MB | 6,171 MB | 1,000 |
Pointer compression build:
| starts/s | PSS per instance | |
|---|---|---|
| green thread, own isolate group (default) | 2,556 | 1.35 MB |
| green thread, V8's default group | 3,926 | 0.95 MB |
green thread, one shared IsolateGroup |
4,170 | 0.95 MB |
| green thread, from warm pool | 101,977 | 1.36 MB |
| worker | 960 | 4.85 MB |
1000 isolated tenants, 50,000 requests
| all started | requests/s | PSS total | |
|---|---|---|---|
| green threads | 271 ms | 201,834 | 1,415 MB |
| workers | 1,025 ms | 157,475 | 6,216 MB |
Message round trips, one instance: the same, 49,187/s vs 49,221/s (20.3 µs each).
CPU-bound tasks, 2000 × fib(27)
| tasks/s | |
|---|---|
| 8 green threads | 5,023 |
| 8 workers | 5,052 |
88 green threads, NODE_GREEN_THREADS_POOL_SIZE=88 |
21,516 |
| 88 workers | 22,230 |
| one green thread per task (2000), pool of 8 | 4,151 |
At equal parallelism, CPU-bound throughput is the same. The pool size caps how
much JavaScript runs in parallel. It defaults to availableParallelism(), up
to 8, and NODE_GREEN_THREADS_POOL_SIZE raises it.
One of 100 tenants loops forever
| stopped by | stopped after | others served in 200 ms | |
|---|---|---|---|
| green threads | maxTaskCpuTimeMs: 50 |
52 ms | 46,332 |
| workers | terminate() after a 50 ms timer |
65 ms | 36,135 |
Green threads vs a pool of workers running vm contexts
(@devsnek's suggestion: N workers, tenants spread over them, one vm context
per tenant, requests run with script.runInContext().)
Startup and memory, 1000 tenants
| tenants/s | PSS per tenant | prepared beforehand | |
|---|---|---|---|
| green threads | 3,781 | 1.36 MB | – |
| green threads, warm pool | 64,951 | 1.39 MB | 394 ms |
| vm pool, 8 workers | 15,360 | 0.25 MB | – |
| vm pool, 8 workers, contexts created beforehand | 121,545 | 0.25 MB | 57 ms |
Pointer compression build: green threads 2,439/s at 1.36 MB, in a shared
IsolateGroup 3,851/s at 0.96 MB, warm pool 69,346/s; vm pool 16,242/s at
0.16 MB.
A vm context is much cheaper than an isolate. It shares the worker's heap,
GC and V8 per-isolate structures, so cold starts are ~4× faster and memory per
tenant is ~5–8× lower. That's the price of the isolation below. About 360 KB of
each green thread (Release) is V8's per-isolate stub caches alone, which a
vm context doesn't pay.
1000 tenants, 50,000 requests, 64 at a time
| requests/s | p50 | p99 | |
|---|---|---|---|
| green threads | 186,481 | 0.32 ms | 1.27 ms |
| vm pool, 8 workers | 190,804 | 0.33 ms | 0.72 ms |
| vm pool, 88 workers | 169,638 | 0.36 ms | 0.82 ms |
vm pool, 8 workers, with timeout |
80,415 | 0.63 ms | 2.59 ms |
vm pool, 88 workers, with timeout |
58,469 | 1.00 ms | 2.90 ms |
Without a timeout, throughput is the same. But a host has to pass
runInContext()'s timeout to stop tenants that loop, and that more than
halves the pool's throughput (it starts a watchdog per call). Green threads
enforce CPU limits without a per-call cost.
One of 100 tenants runs a synchronous infinite loop
| stopped after | tenants lost | others served in 300 ms | slowest other | |
|---|---|---|---|---|
| green threads | 52 ms | 1 (the bad one) | 59,499 | 3 ms |
vm pool, 8 workers, timeout: 50 |
51 ms | 0 | 18,414 | 52 ms |
One of 100 tenants runs an infinite microtask loop
| stopped after | tenants lost | others served in 300 ms | slowest other | |
|---|---|---|---|---|
| green threads | 54 ms | 1 (the bad one) | 61,776 | 3 ms |
vm pool, timeout: 50 |
101 ms (worker terminated) | 13 | 13,659 | 16 ms |
vm pool, timeout: 50, microtaskMode: 'afterEvaluate' |
51 ms | 0 | 18,216 | 52 ms |
timeout doesn't cover microtasks unless the context uses
microtaskMode: 'afterEvaluate'. Without it, the only way out is terminating
the worker, which takes every tenant on it along.
One of 100 tenants fills its heap
| stopped after | tenants lost | others still answering | |
|---|---|---|---|
| green threads | 312 ms | 1 (the bad one) | 99 |
| vm pool, 8 workers | 284 ms | 13 (whole worker) | 87 |
In a vm pool, a tenant that loops blocks the other tenants on its worker until
it's stopped (slowest other request: 52 ms vs 3 ms). A tenant that exhausts the
heap takes down every tenant on its worker, since they share one heap.
Which benefits are inherent to green threads (@glenjamin)
Inherent, from one isolate per tenant on a shared pool:
- A heap and GC per tenant: memory limits and out-of-memory only affect that tenant. A
vmcontext can't have its own heap limit. - CPU limits per tenant without a per-call cost: they cover microtasks, and other tenants keep running on other pool threads.
- One tenant can be terminated without affecting the others. In a vm pool, that requires terminating the worker.
- No Node.js API surface by default: a green thread only has what the host passes in.
vmcontexts are not a security boundary (escapable without V8 bugs), and workers have full Node.js. - Cheaper than workers: about 4.5× less memory (1.36 vs 6.17 MB) and 4× faster cold starts, because there's no OS thread, event loop or Node.js environment per instance.
Not inherent, or where the alternatives win:
- Memory and cold start per tenant: a
vmcontext is cheaper, because it shares its worker's isolate. Most of a green thread's remaining 1–1.4 MB is V8's per-isolate overhead, which only V8 changes could reduce. - Raw request throughput: the same as a vm pool without timeouts, and the same as workers for CPU-bound work at equal parallelism.
- Warm pools: they hide cold-start cost for green threads (65,000–100,000 starts/s), much like pre-creating
vmcontexts does for a pool.
This comment was marked as outdated.
This comment was marked as outdated.
Format the code with clang-format, and declare the V8 names that src/node_green_thread.cc uses with using-declarations at the top of the file rather than qualifying them with v8::. Remove an unused declaration and keep all of the file's using-declarations sorted, as tools/checkimports.py requires. Regenerate doc/node.1, which was missing the updated description of NODE_GREEN_THREADS_POOL_SIZE. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
This comment was marked as outdated.
This comment was marked as outdated.
SnapshotBuilder::CreateSnapshot() built the startup snapshot of green threads in the default isolate group for every snapshot without a builder script, including the ones that embedders create through CommonEnvironmentSetup::CreateSnapshot(). Embedders have platforms of their own, so per_process::v8_platform has none and creating the snapshot crashed. Build it only for GenerateAsSource(), which is its only user. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
This comment was marked as outdated.
This comment was marked as outdated.
Green threads could only send messages to one receiver at a time. Add BroadcastChannel to green threads, for the names in the new broadcastScopes option of new GreenThread(). The channels are Node.js' own, process-wide, so messages reach every BroadcastChannel with the same name, in the main thread, in workers and in other green threads. Other names throw ERR_GREEN_THREAD_BROADCAST_SCOPE, so green threads can only use the channels that their host allows. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads that ignored termination and were abandoned kept their ports and BroadcastChannels open. The ports they were entangled with were never closed, and messages for them queued up until the process exited. Disentangle the ports of green threads when they are abandoned, and drop their messages. Since the green thread may still be running, the ports are only entangled, disentangled and used to post messages while a lock is held, which is never held while JavaScript runs. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads could only be terminated, which interrupts whatever they are doing. Add greenThread.close([options]), which calls the green thread's onclose function, and makes it exit with exit code 0 once nothing that it waits for is pending: timers, RPC calls in either direction, evaluations, its main module, and tasks queued before. Messages posted after close() are dropped, and evaluate() rejects with the new ERR_GREEN_THREAD_CLOSING. If the green thread hasn't exited within `timeoutMs`, 5000 by default, it is terminated. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
ronag
left a comment
There was a problem hiding this comment.
It's a little unclear when one should use this. I have an idea but it doesn't become super clear from the documentation.
I assume this is for compute tasks (i.e. no IO) and in general should be preferred over Workers for those kind of tasks. Might be good to have a paragraph in the beginning of the docs explaining the use case.
|
Why ceil it at 8 threads? In practical terms I would suggest a default like: availableParallelism() / (SMT ? 2 : 1) - uvthreads / 4 Can the green thread worker pool dynamically size based on load? e.g. start with one thread and grow up to max threads on load? |
Had to start somewhere :-) It's actually min(8, availableParallellism) and it's configurable with an env var. We can explore making it dynamic. |
|
@jasnell sorry for the confusion, i meant: what if an api for managing isolates was added to the vm module? i would assume the scheduling aspect of v8 lockers would be the largest challenge there, compared to your hardcoded "green thread" approach. though with prior art like web locks, maybe a similar pattern is possible? |
|
The scheduling and interactivity are the most complicated bits. Overall this could be added to vm nevertheless but I don't know if it makes it any simpler/better. Can be convinced tho |
And some examples, too. |
This is an experiment... opening for feedback.This is ready for review.This implements green threads for Node.js. A green thread is a lightweight kind of worker. It has its own V8 isolate with no Node.js APIs. They run on a small pool of physical threads that they share, instead of each getting an OS thread and event loop like a worker thread. Each one gets work from the host through messages, RPC calls, or its module's exports, and every piece of work runs in an invocation context of its own. Each green thread can have limits on CPU time and memory, and the host can terminate it. This makes them cheap to start and to keep around in large numbers, and suited to running many small, untrusted pieces of code, such as plugins, per-tenant handlers, or user-supplied scripts.
For later, the plan is to add the following APIs (but not in this PR)
TextEncoder/TextDecoderURL/URLSearchParamsstructuredClone()Details
Compared with worker threads
Green threads have real advantages over workers, in specific areas:
isolate-group setting. A thousand tenants take about 1 GB instead of 6.7 GB.
1000, and serve about 15–20% more requests per second.
limits.
They don't win on the speed of a single message, and they can't spread heavy
computation across all cores, since the pool is capped at 8 threads.
Setup for this performance comparison
support isolate groups, so every green thread is in V8's default group.
thread is in an isolate group of its own.
kMaxPoolThreads).Each measurement runs in a fresh child process, so that memory numbers don't
include what earlier measurements left behind.
Kinds of green threads measured:
greengreen-defaultisolateGroup: 'default', V8's default isolate groupgreen-sharedIsolateGroupshared by all instancesgreen-warmMemory and startup
1000 idle instances, started at once, measured once each has sent a message.
isolateGroup: 'default')IsolateGroupworker carries a whole Node.js environment, an event loop and an OS thread
with its stack. A green thread is a V8 isolate with a small runtime.
are.
setWarmPoolSize(n)prepares isolates in advance, sostarting a green thread takes about 20 µs. Workers have no equivalent.
shared isolate group.
Speed
Message round trips
Sequential round trips with one instance.
Workers are about 5% faster per message.
CPU-bound tasks
2000 tasks of
fib(27), each executor taking the next task once it's done.Release build.
At equal parallelism, CPU-bound code runs at the same speed. Green threads
can't use more than 8 cores, though, so for heavy parallel computation
workers win clearly. The cap is a choice in this implementation, not
inherent, and could be made configurable.
Many tenants
1000 tenants, each with code and state of its own, serving 50,000 requests
spread over all of them, 64 at a time.
IsolateGroupGreen threads serve about 15–20% more requests, since 8 pool threads don't
compete for the CPU the way 1000 OS threads do.
Runaway code
One of 100 tenants runs into an endless loop, while the others keep serving.
ERR_GREEN_THREAD_CPU_LIMIT(maxTaskCpuTimeMs: 50)terminate()Both stop it. Green threads do it with a limit on CPU time, which the runtime
enforces. Workers need a timer by wall time on the host, which also stops
code that's only waiting.
What's easier to build: a plugin host
The same plugin host, built with each. It runs five plugins, which are ES
modules exporting
transform(text). Plugins can import a shared module, andcall back into the host for configuration and logging. Three of them
misbehave.
upper,reverse(well-behaved)spin(endless loop)ERR_GREEN_THREAD_CPU_LIMITafter 100 ms of CPU timehog(allocatesArrayBuffers)maxMemoryMblimitresourceLimitsonly cover the JS heap. Reached 91 MB within the 100 ms timeoutfail(throws with a code)code: 'ERR_PLUGIN'The green thread version is 43 lines:
The worker version is 155 lines, since it has to build what green threads
provide:
plugins are written to a temporary directory. Green threads take a
modulesbundle andresolve()/load()hooks.calls, and error forwarding, roughly what Comlink does. Green threads get
exportsandenvas RPC stubs.ArrayBufferlimit.Green threads have both.
require('fs'). The permission model applies to the whole process, so itcan't restrict a single worker. Green threads have no Node.js APIs at all:
they can only do what the host passes them. This is the biggest
qualitative difference.
When to use which
Use green threads for:
limits.
memory per instance matter.
Use workers for:
fs,net,child_process), or npmpackages that use them.
is capped at 8 threads.
Atomics.wait()and synchronous waits are disabled in greenthreads, since a blocked isolate would hold up a shared pool thread.
Caveats
f14fd2d7548,which doesn't have the ES modules and errors commits. These scenarios don't
use either.
run than the rest of its table. The first run used a setting that bypassed
the pool.
machines, the gap is likely larger.
a watchdog that abandons such isolates. Workers are affected by the same
bug when terminated.