[Djuffin]

Avoid accessing or mutating JS objects from background parallel steps in VideoFrame.copyTo():

• Check if destination is detached before starting the copy.
• Capture [[resource reference]], options, and PlaneLayout results synchronously on the event loop thread (fixing PlaneLayout.stride to use destinationStride).
• Perform RGB pixel format and color space conversion on the media resource in parallel instead of synchronously constructing a temporary VideoFrame and recursing.
• Write directly to shared memory in parallel when destination is backed by a SharedArrayBuffer; otherwise, copy into destination on a queued event loop task after re-verifying detachment and byteLength (while allowing UAs to copy synchronously when the resource is already in CPU memory).
• Set the default value of VideoFrameCopyToOptions.colorSpace to "srgb" in IDL.

Fixes #955

---

Preview | Diff

[youennf]

It seems it should also be happening if |sharedMemory| is not null.

[youennf]

|sharedMemory| definition is somehow ambiguous, it is not clear whether keeping a ref to it is protecting destination's memory from being GCed.
Maybe destination.[[ArrayBufferData]] is what we need.

[youennf]

This note will likely trigger web compatibility troubles, for instance:

let buffer = new Uint8Array(...);
videoFrame.copyTo(buffer); // do not await here.
channel.postMessage(buffer, [buffer]);

As per the above algorithm, no data would be copied to buffer in copyTo.
If implementing the note, copyTo will copy data to buffer before buffer is transferred.

It seems we should either remove that note or make it mandatory.
I am also not sure why doing things synchronously avoids an extra copy.

[padenot]

Looks find. I checked Firefox and we're already doing the right thing.

[padenot]

I'm adding a wpt in https://bugzilla.mozilla.org/show_bug.cgi?id=2089555 though for this.

[Djuffin]

Gist of the change

1. Captures state synchronously: copyTo() grabs the internal media resource reference, options, and PlaneLayout results on the event loop thread before starting background work.
2. Splits buffer writing by type:
   • SharedArrayBuffer: Pixel data is written directly in parallel from the background thread.
   • Regular ArrayBuffer: Background steps only wait for the frame to reach CPU memory (e.g., GPU readback); the actual write into destination happens in a queued task on the event loop thread after re-verifying that the buffer wasn't detached or shrunk.
3. Moves RGB conversion off the main thread: YUV-to-RGB and color space conversion now run in parallel on the media resource instead of synchronously creating a temporary VideoFrame on the JS thread.
4. Spec bug fixes: Returned PlaneLayout.stride now correctly reports destinationStride (instead of unpadded rowBytes), ArrayBufferView.byteOffset is properly respected, and colorSpace defaults to "srgb" in WebIDL.

---

What It Means for Web Developers

• frame.close() immediately after copyTo() is safe: You don't need to await the copy before closing the VideoFrame:

  const p = frame.copyTo(buffer);
  frame.close(); // Safe: copyTo already holds an internal reference to the resource
  const layouts = await p;

• Always await copyTo() before using or transferring an ArrayBuffer: Data is never written synchronously during the copyTo() call. If you transfer (postMessage) or shrink a resizable ArrayBuffer before await copyTo() resolves, the promise will reject with a TypeError.
• Use SharedArrayBuffer to avoid main-thread copy overhead: Passing a SharedArrayBuffer (e.g., Wasm shared memory) lets the browser write pixels directly from a background thread without bouncing through the JS event loop thread.
• Non-blocking RGB conversion: Using { format: "RGBA" } (and colorSpace) runs off the main thread and works properly with custom rect crops.