From 0ad268835c0acc7f0209750244dbf0fb159972b3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Tue, 15 Sep 2026 18:36:39 +0200 Subject: [PATCH 1/5] Hand a quiescent transport's descriptor back to the host MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A turnloop socket owned its descriptor with no way to get it back, so the class of socket that might later need a mid-stream TLS upgrade — Node's socket.upgradeToTLS, PostgreSQL's SSLRequest — could not start on turnloop at all. Ownership now leaves through the existing detach: Detached::into_fd on Unix, Detached::into_socket/into_handle on Windows. detach already proves quiescence, so the host's guarantee is total, and conversion restores what the backend changed on adoption before releasing ownership. Driver::raw_transport reports a live transport's native identity for Node's socket._handle.fd. It is borrowed and reporting-only; ownership still leaves only through detach. WASI 0.2/0.3 and web report Unsupported: their sockets are component-model resource handles or host objects, not descriptors. --- DESIGN.md | 13 +- crates/turnloop-contract/Cargo.toml | 2 +- crates/turnloop-contract/src/handoff.rs | 235 +++++++ crates/turnloop-contract/src/lib.rs | 1 + crates/turnloop-contract/tests/allocations.rs | 67 ++ crates/turnloop-contract/tests/handoff.rs | 622 ++++++++++++++++++ crates/turnloop-contract/tests/wasi.rs | 6 + .../tests/web/web_contract.rs | 5 + crates/turnloop/src/backend/iocp/mod.rs | 97 ++- crates/turnloop/src/backend/mod.rs | 20 +- crates/turnloop/src/backend/unix.rs | 59 +- crates/turnloop/src/driver.rs | 45 ++ crates/turnloop/src/types.rs | 18 + docs/BACKEND_REVISION_2.md | 27 + protocols/turnloop-tls/Cargo.toml | 6 +- protocols/turnloop-tls/tests/upgrade.rs | 154 +++++ 16 files changed, 1358 insertions(+), 19 deletions(-) create mode 100644 crates/turnloop-contract/src/handoff.rs create mode 100644 crates/turnloop-contract/tests/handoff.rs create mode 100644 protocols/turnloop-tls/tests/upgrade.rs diff --git a/DESIGN.md b/DESIGN.md index e1ee007..907f2c4 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -258,6 +258,10 @@ There are two ways a host drives a loop: 5. **Handle transfer:** - **Between loops:** `Loop::detach(h) -> Detached` (`Send`) and `Loop::attach(Detached, token) -> Handle`. For sockets, pipes and servers across threads or workers, it cancels in-flight ops with the usual exactly-once completions before detaching. - **Between processes:** fd passing via `SCM_RIGHTS` on Unix and `WSADuplicateSocketW` / `DuplicateHandle` on Windows, exposed on pipe handles so `child.send(msg, handle)` and cluster round-robin can move sockets (today `emitter.rs:430` drops the handle). + - **Out of turnloop entirely (host handoff):** the same `detach` followed by `Detached::into_fd()` on Unix, or `into_socket()` / `into_handle()` on Windows. This is Node's mid-stream `socket.upgradeToTLS` — PostgreSQL's `SSLRequest` hands a live, already-connected socket to a TLS layer — so the class of socket that *might* later be upgraded no longer has to choose its transport at creation. `detach` already proves quiescence, so the guarantee the host gets is total: no operation, no buffer, no registration, no completion, ever again, for that transport. Conversion restores what the backend changed on adoption (Unix status flags and termios, Windows console mode) and hands the descriptor over in the mode the loop held it: non-blocking for a loop-created socket. + - **Windows:** a handle's IOCP association is permanent — Windows cannot dissociate one, and rejects a second `CreateIoCompletionPort` with `ERROR_INVALID_PARAMETER`. Quiescence makes it inert (no packet can ever arrive for it), and the receiving host has three ways to work: synchronous/non-blocking Winsock calls, overlapped calls with `OVERLAPPED.hEvent`'s low-order bit set (which suppresses the completion packet), or — sockets only — `WSADuplicateSocketW` + `WSASocketW` to obtain a fresh, unassociated socket for the same connection. A named-pipe instance keeps `FILE_FLAG_OVERLAPPED` and cannot duplicate out of its association (`DuplicateHandle` shares the file object), so the tagged-`hEvent` rule is how a host drives one. + - **WASI 0.2/0.3 and web:** `Unsupported`. A WASI socket is a component-model resource handle in the component's own table, not a descriptor, and there is no interface that hands one to the embedder; a browser resource is a host JS object. Neither has an identity a host could act on. + - **Reporting only:** `Loop::raw_transport(h) -> RawTransport` reports a live transport's native identity for Node's `socket._handle.fd`. The loop keeps ownership; the value is valid until the handle is closed or detached, and is for reporting and read-only queries, never for I/O, closing, mode changes or registration elsewhere. 6. **Multi-threaded accept:** - **Kernel-balanced:** where the kernel balances load (`SO_REUSEPORT` on Linux/FreeBSD), each loop gets its own listener with `ListenOpts::reuse_port`. - **Everywhere else** (macOS doesn't balance, and on Windows a socket can join only one completion port): one accepting loop hands connections to other loops with `detach`/`attach`. The policy (round-robin, least-loaded) belongs to the host. @@ -311,10 +315,15 @@ impl Loop { pub fn poster(&self) -> Poster; // Send + Sync + Clone: post (token, payload) to THIS loop pub fn integration(&mut self) -> io::Result; // Fd | Event | HostCallback | RuntimeOwned - // multithreading (§5a) + // multithreading and host handoff (§5a) pub fn detach(&mut self, h: Handle) -> io::Result; // Detached: Send pub fn attach(&mut self, d: Detached, tok: Token) -> io::Result; pub fn send_handle(&mut self, pipe: Handle, h: Handle, tok: Token) -> io::Result; // SCM_RIGHTS / DuplicateHandle + pub fn raw_transport(&self, h: Handle) -> io::Result; // borrowed, reporting only: socket._handle.fd + // and, on the transport a detach returned, ownership leaves for good: + // unix: Detached::into_fd(self) -> OwnedFd + // windows: Detached::into_socket(self) -> io::Result + // Detached::into_handle(self) -> io::Result // timers pub fn timer(&mut self, at: Instant, repeat: Option, tok: Token) -> Handle; @@ -441,6 +450,8 @@ Two backends, because both versions matter now: | Outbound HTTP | protocol crate | protocol crate | protocol crate | protocol crate or `wasi:http` | protocol crate or `wasi:http` | host `fetch` | | WebSocket | protocol crate | protocol crate | protocol crate | protocol crate | protocol crate | host `WebSocket` | | Local IPC | AF_UNIX | AF_UNIX | named pipes (overlapped) | unsupported | unsupported | `postMessage` | +| Descriptor handoff (§5a) | `Detached::into_fd` | `Detached::into_fd` | `into_socket`/`into_handle`; IOCP association is permanent, so tagged `hEvent` or `WSADuplicateSocketW` | unsupported (resource handle, not a descriptor) | unsupported (resource handle, not a descriptor) | unsupported (host object) | +| Native identity reporting (`_handle.fd`) | `RawTransport::Fd` | `RawTransport::Fd` | `RawTransport::Socket`/`Handle` | unsupported | unsupported | unsupported | | Stdio pipes | readiness | readiness | overlapped, or reader thread | `wasi:cli` streams | `wasi:cli` streams | unsupported | | TTY | termios + readiness | termios + readiness | console API reader thread, VT modes | size only | size only | unsupported | | Child processes | pidfd / SIGCHLD | EVFILT_PROC | RegisterWaitForSingleObject + Job Object | unsupported | unsupported | unsupported | diff --git a/crates/turnloop-contract/Cargo.toml b/crates/turnloop-contract/Cargo.toml index 87fae1e..5c25219 100644 --- a/crates/turnloop-contract/Cargo.toml +++ b/crates/turnloop-contract/Cargo.toml @@ -23,7 +23,7 @@ workspace = true libc.workspace = true [target.'cfg(windows)'.dependencies] -windows-sys = { version = "=0.61.2", features = ["Wdk_Foundation", "Win32_Foundation", "Win32_UI_WindowsAndMessaging", "Win32_System_Threading", "Win32_System_Console", "Win32_Storage_FileSystem", "Win32_Security", "Win32_System_IO", "Win32_System_Pipes"] } +windows-sys = { version = "=0.61.2", features = ["Wdk_Foundation", "Win32_Foundation", "Win32_Networking_WinSock", "Win32_UI_WindowsAndMessaging", "Win32_System_Threading", "Win32_System_Console", "Win32_Storage_FileSystem", "Win32_Security", "Win32_System_IO", "Win32_System_Pipes"] } [package.metadata.turnloop-ci] role = "contract" diff --git a/crates/turnloop-contract/src/handoff.rs b/crates/turnloop-contract/src/handoff.rs new file mode 100644 index 0000000..f172ec9 --- /dev/null +++ b/crates/turnloop-contract/src/handoff.rs @@ -0,0 +1,235 @@ +//! Handing a transport back to the host (DESIGN §5a; issue #35). +//! +//! These are the loop-side halves of the contract: which requests are refused, +//! and what the loop still knows about a transport once the host owns it. The +//! halves that need the descriptor itself — bytes flowing over it after the +//! handoff, a TLS handshake on it, the Windows completion-port rules — are +//! platform code and live in `tests/handoff.rs`, because a descriptor cannot be +//! used through a generic `B::Detached`. +use super::*; + +/// The error kind of a refused handoff. `expect_err` needs `Debug`, which a +/// backend's transport type is not required to implement. +fn refused(result: Result, what: &str) -> ErrorKind { + match result { + Ok(_) => panic!("{what} was handed out"), + Err(e) => e.kind, + } +} + +/// Drive the loop until every handle is gone, so a fixture cannot leak one. +fn close_all(l: &mut Driver, handles: &[Handle]) { + let mut out = Completions::default(); + for (i, h) in handles.iter().enumerate() { + if l.close(*h, Token(900 + i as u64)).is_err() { + continue; + } + } + let until = l.now() + Duration::from_secs(5); + while l.alive() { + assert!(l.now() < until, "close never completed"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + out.drain(); + } +} + +/// A handle with work in flight is never handed out, and cancellation is never +/// silent: the refusal is `WouldBlock`, the host turns, collects the terminal +/// completion it is owed, and only then owns the transport. +pub fn pending_operations_refuse_handoff() { + let mut l = Driver::::new(Config::default()).expect("loop"); + let (server, client, conn) = pair(&mut l); + let mut bytes = [0x3c; 24]; + // SAFETY: the region stays fixed and untouched until Cancelled is delivered. + let provided = unsafe { IoBufMut::from_raw_parts(bytes.as_mut_ptr(), bytes.len()) }; + let read = l + .read(conn, ReadBuf::Provided(provided), Token(1)) + .expect("pending read"); + assert_eq!( + refused::(l.detach(conn), "a busy handle"), + ErrorKind::WouldBlock, + "a handle with an outstanding operation must not be handed out" + ); + // The refusal must not have cancelled anything behind the host's back either: + // the identity is still live and still refuses, turn after turn. + let mut out = Completions::default(); + assert_eq!( + refused::(l.detach(conn), "a busy handle"), + ErrorKind::WouldBlock + ); + l.turn(Timeout::Now, &mut out).expect("cancel turn"); + assert_eq!(out.len(), 1, "exactly one terminal completion is owed"); + assert_eq!(out[0].op, Some(read)); + assert!(matches!(out[0].result, OpResult::Cancelled)); + assert_eq!(bytes, [0x3c; 24], "the loop wrote into a cancelled buffer"); + out.drain(); + + let transport = l.detach(conn).expect("quiescent handoff"); + assert_eq!( + refused::(l.detach(conn), "an already handed-off transport"), + ErrorKind::NotFound, + "a transport cannot be handed out twice" + ); + assert_eq!( + l.raw_transport(conn).expect_err("gone").kind, + ErrorKind::NotFound + ); + assert!( + l.read(conn, ReadBuf::Pooled, Token(2)).is_err(), + "the loop still accepts operations on a transport it gave away" + ); + // Nothing for that handle can arrive any more, however long the host turns. + let until = l.now() + Duration::from_millis(60); + while l.now() < until { + l.turn(Timeout::After(Duration::from_millis(10)), &mut out) + .expect("idle turn"); + assert_eq!(out.len(), 0, "a handed-off transport produced {out:?}"); + } + drop(transport); + close_all(&mut l, &[client, server]); +} + +/// Close is the other owner of a transport's end of life; the two never overlap. +pub fn closing_and_closed_handles_refuse_handoff() { + let mut l = Driver::::new(Config::default()).expect("loop"); + let (server, client, conn) = pair(&mut l); + l.read_start(conn, Token(1)).expect("multishot read"); + l.close(conn, Token(2)).expect("close"); + assert!(l.is_closing(conn), "close has not finished yet"); + assert_eq!( + refused::(l.detach(conn), "a closing transport"), + ErrorKind::InvalidInput, + "a closing transport must not be handed out" + ); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut closed = false; + while !closed { + assert!(l.now() < until, "close never completed"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + closed |= matches!(c.result, OpResult::Closed); + } + } + assert_eq!( + refused::(l.detach(conn), "a closed transport"), + ErrorKind::NotFound + ); + assert_eq!( + l.raw_transport(conn).expect_err("closed handle").kind, + ErrorKind::NotFound + ); + close_all(&mut l, &[client, server]); +} + +/// Both ends of an accept are ordinary transports: the listener and the +/// connection it produced are each handed over, and the loop keeps nothing. +pub fn listeners_and_accepted_sockets_are_handed_off() { + let mut l = Driver::::new(Config::default()).expect("loop"); + let (server, client, conn) = pair(&mut l); + let accepted = l.detach(conn).expect("accepted socket"); + let listener = l.detach(server).expect("listener"); + assert!( + l.local_addr(server).is_err() && l.local_addr(conn).is_err(), + "the loop still answers for transports it gave away" + ); + // The client is the only handle left, so liveness proves the other two are + // gone from the core's accounting and not merely unregistered natively. + let mut out = Completions::default(); + l.turn(Timeout::Now, &mut out).expect("idle turn"); + assert_eq!(out.len(), 0); + assert!(l.alive(), "the remaining client still counts"); + close_all(&mut l, &[client]); + assert!(!l.alive()); + drop(accepted); + drop(listener); +} + +/// Reporting a descriptor is not owning it: the value is stable while the loop +/// holds the transport, and every non-transport handle says `Unsupported`. +pub fn raw_transport_reports_live_transports() { + let mut l = Driver::::new(Config::default()).expect("loop"); + let (server, client, conn) = pair(&mut l); + let first = l.raw_transport(conn).expect("live transport"); + assert_eq!(first, l.raw_transport(conn).expect("stable")); + assert_ne!( + first, + l.raw_transport(client).expect("client"), + "two live transports reported the same identity" + ); + assert_eq!( + l.raw_transport(server).expect("listener"), + l.raw_transport(server).expect("stable listener") + ); + // Reporting is read-only: the socket is still fully usable afterwards. + l.write(conn, WriteBuf::Owned(b"report".to_vec()), Token(1)) + .expect("write"); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut wrote = 0; + while wrote == 0 { + assert!(l.now() < until, "write after reporting never completed"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + if let OpResult::Wrote(n) = c.result { + assert_eq!(n, 6); + wrote += 1; + } + } + } + assert_eq!(l.raw_transport(conn).expect("still live"), first); + + let timer = l + .timer(l.now() + Duration::from_secs(30), None, Token(2)) + .expect("timer"); + assert_eq!( + l.raw_transport(timer).expect_err("timer").kind, + ErrorKind::Unsupported, + "a timer is not a transport and has no descriptor" + ); + assert_eq!( + refused::(l.detach(timer), "a timer"), + ErrorKind::InvalidInput + ); + close_all(&mut l, &[timer, client, conn, server]); +} + +/// A platform with no descriptor to give says so, rather than inventing one. +/// WASI 0.2 and 0.3 sockets are component-model resource handles and web +/// resources are host objects: neither has an identity a host could act on. +pub fn handoff_is_unsupported() { + let mut l = Driver::::new(Config::default()).expect("loop"); + let (server, client, conn) = pair(&mut l); + for h in [server, client, conn] { + assert_eq!( + refused::(l.detach(h), "a transport on a platform without descriptors"), + ErrorKind::Unsupported + ); + assert_eq!( + l.raw_transport(h).expect_err("identity").kind, + ErrorKind::Unsupported + ); + } + // The refusals changed nothing: the connection still works. + l.write(conn, WriteBuf::Owned(b"still here".to_vec()), Token(1)) + .expect("write"); + l.read(client, ReadBuf::Pooled, Token(2)).expect("read"); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut read = 0; + while read == 0 { + assert!(l.now() < until, "refused handoff broke the connection"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + if let OpResult::Read { + n, + lease: Some(data), + } = c.result + { + assert_eq!(&data.as_slice()[..n], b"still here"); + read += 1; + } + } + } + close_all(&mut l, &[client, conn, server]); +} diff --git a/crates/turnloop-contract/src/lib.rs b/crates/turnloop-contract/src/lib.rs index eb3e0dc..cb03fa2 100644 --- a/crates/turnloop-contract/src/lib.rs +++ b/crates/turnloop-contract/src/lib.rs @@ -1608,6 +1608,7 @@ pub fn no_spin() { #[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))] pub mod filesystem; +pub mod handoff; pub mod native_surface; pub mod sockopts; diff --git a/crates/turnloop-contract/tests/allocations.rs b/crates/turnloop-contract/tests/allocations.rs index 47b2768..567c957 100644 --- a/crates/turnloop-contract/tests/allocations.rs +++ b/crates/turnloop-contract/tests/allocations.rs @@ -519,6 +519,73 @@ fn steady_socket_options_allocate_nothing() { turnloop_contract::sockopts::close_all(&mut l, &[client, conn, server, udp]); } +/// Handing a transport to the host and taking it back allocates nothing: the +/// identity is read out of the backend's own table, `detach` only unregisters, +/// and the conversion to an owned descriptor moves the resource it already held. +#[test] +fn steady_handoff_allocates_nothing() { + let mut l = Loop::new(Config::default()).expect("loop"); + let (server, client, conn) = + turnloop_contract::sockopts::plain_pair(&mut l, &ListenOpts::default()); + let mut h = conn; + let mut total = 0; + let mut cycles = 0; + for i in 0..101u32 { + ALLOCS.with(|n| n.set(0)); + ACTIVE.with(|v| v.set(i != 0)); + let reported = l.raw_transport(h).expect("identity"); + let transport = l.detach(h).expect("handoff"); + #[cfg(unix)] + let (owned, back) = { + use std::os::fd::AsRawFd; + let fd = transport.into_fd(); + let owned = RawTransport::Fd(fd.as_raw_fd()); + (owned, Detached::from_fd(fd).expect("re-adopt")) + }; + #[cfg(windows)] + let (owned, back) = { + use std::os::windows::io::AsRawSocket; + let socket = transport.into_socket().expect("socket transport"); + let owned = RawTransport::Socket(socket.as_raw_socket() as usize); + (owned, Detached::from_socket(socket).expect("re-adopt")) + }; + h = l.attach(back, Token(0)).expect("attach"); + ACTIVE.with(|v| v.set(false)); + assert_eq!( + owned, reported, + "the host received a different transport from the reported one" + ); + if i != 0 { + cycles += 1; + total += ALLOCS.with(|n| n.get()); + } + } + assert_eq!(cycles, 100, "the handoff subject ran"); + assert_eq!(total, 0, "steady handoff allocations"); + // The socket survived a hundred round trips: the cycle was not vacuous. + let mut out = Completions::default(); + l.write(h, WriteBuf::Owned(b"round trip".to_vec()), Token(1)) + .expect("write"); + l.read(client, ReadBuf::Pooled, Token(2)).expect("read"); + let until = l.now() + Duration::from_secs(5); + let mut read = 0; + while read == 0 { + assert!(l.now() < until, "the re-attached socket stopped working"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + if let OpResult::Read { + n, + lease: Some(data), + } = c.result + { + assert_eq!(&data.as_slice()[..n], b"round trip"); + read += 1; + } + } + } + turnloop_contract::sockopts::close_all(&mut l, &[client, h, server]); +} + #[test] fn steady_deadline_poll_allocate_nothing() { let mut l = Loop::new(Config::default()).expect("loop"); diff --git a/crates/turnloop-contract/tests/handoff.rs b/crates/turnloop-contract/tests/handoff.rs new file mode 100644 index 0000000..1197adc --- /dev/null +++ b/crates/turnloop-contract/tests/handoff.rs @@ -0,0 +1,622 @@ +//! Handing a transport's descriptor back to the host (issue #35). +//! +//! The contract functions cover the loop side — what is refused, and what the +//! loop still knows afterwards. Everything here goes around turnloop entirely: +//! it takes the descriptor the loop handed over and drives it from the test +//! process with `libc`/Winsock or `std::net`, so a backend that kept a claim on +//! the transport, or handed back the wrong one, could not pass. +#![deny(unsafe_op_in_unsafe_fn)] +#![cfg(all( + not(loom), + any( + target_vendor = "apple", + target_os = "linux", + target_os = "android", + target_os = "freebsd", + windows + ) +))] +use std::{ + io::{Read, Write}, + net::{TcpListener, TcpStream}, + time::Duration, +}; +use turnloop::{backend::Platform, *}; + +macro_rules! contract { + ($($name:ident),+ $(,)?) => { + $(#[test] fn $name() { turnloop_contract::handoff::$name::(); })+ + }; +} +contract!( + pending_operations_refuse_handoff, + closing_and_closed_handles_refuse_handoff, + listeners_and_accepted_sockets_are_handed_off, + raw_transport_reports_live_transports, +); + +/// A loop-driven connection whose peer is an ordinary blocking `TcpStream` this +/// test owns, so the peer outlives the loop and can prove bytes still flow. +fn connected(l: &mut Loop) -> (Handle, TcpStream) { + let listener = TcpListener::bind("127.0.0.1:0").expect("peer listener"); + let address = listener.local_addr().expect("peer address"); + let h = l + .tcp_connect(address, &TcpOpts::default(), Token(1)) + .expect("connect"); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut connected = false; + while !connected { + assert!(l.now() < until, "connect timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + assert!(matches!(c.result, OpResult::Connected), "{c:?}"); + connected = true; + } + } + let (peer, _) = listener.accept().expect("accept"); + peer.set_read_timeout(Some(Duration::from_secs(5))) + .expect("read timeout"); + (h, peer) +} + +/// Write from the loop and read the bytes on the peer, so the connection is +/// provably live and mid-stream before it is handed over. +fn exchange_through_the_loop(l: &mut Loop, h: Handle, peer: &mut TcpStream) { + l.write(h, WriteBuf::Owned(b"mid-stream".to_vec()), Token(2)) + .expect("write"); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut wrote = 0; + while wrote == 0 { + assert!(l.now() < until, "loop write timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + match c.result { + OpResult::Wrote(n) => { + assert_eq!(n, 10); + wrote += 1; + } + other => panic!("unexpected {other:?}"), + } + } + } + let mut bytes = [0; 10]; + peer.read_exact(&mut bytes).expect("peer read"); + assert_eq!(&bytes, b"mid-stream"); +} + +/// Turn an idle loop for a while; nothing may arrive for a transport it gave away. +fn assert_quiet(l: &mut Loop) { + let mut out = Completions::default(); + let until = l.now() + Duration::from_millis(60); + while l.now() < until { + l.turn(Timeout::After(Duration::from_millis(10)), &mut out) + .expect("idle turn"); + assert_eq!(out.len(), 0, "a handed-off transport produced {out:?}"); + } +} + +#[test] +fn a_handed_off_socket_carries_bytes_after_its_loop_is_dropped() { + let mut l = Loop::new(Config::default()).expect("loop"); + let (h, mut peer) = connected(&mut l); + exchange_through_the_loop(&mut l, h, &mut peer); + + let reported = l.raw_transport(h).expect("live identity"); + let transport = l.detach(h).expect("quiescent handoff"); + assert_eq!( + l.raw_transport(h).expect_err("gone").kind, + ErrorKind::NotFound + ); + assert_quiet(&mut l); + assert!(!l.alive(), "the loop still counts a transport it gave away"); + + let mut stream = platform::into_stream(transport, reported); + // The loop is gone entirely: nothing it owned can be keeping this alive. + drop(l); + + stream.write_all(b"after handoff").expect("host write"); + let mut bytes = [0; 13]; + peer.read_exact(&mut bytes).expect("peer read"); + assert_eq!(&bytes, b"after handoff"); + + peer.write_all(b"and back again").expect("peer write"); + let mut bytes = [0; 14]; + stream.read_exact(&mut bytes).expect("host read"); + assert_eq!(&bytes, b"and back again"); +} + +#[test] +fn a_handed_off_socket_is_driven_by_the_bare_descriptor() { + let mut l = Loop::new(Config::default()).expect("loop"); + let (h, mut peer) = connected(&mut l); + exchange_through_the_loop(&mut l, h, &mut peer); + let reported = l.raw_transport(h).expect("live identity"); + let transport = l.detach(h).expect("quiescent handoff"); + + // No std wrapper, no turnloop: the descriptor itself, as the host received it. + let raw = platform::into_raw(transport, reported); + assert_eq!(platform::send(raw, b"raw descriptor"), 14); + let mut bytes = [0; 14]; + peer.read_exact(&mut bytes).expect("peer read"); + assert_eq!(&bytes, b"raw descriptor"); + + peer.write_all(b"raw reply").expect("peer write"); + let mut bytes = [0; 9]; + assert_eq!(platform::recv(raw, &mut bytes), 9); + assert_eq!(&bytes, b"raw reply"); + assert_quiet(&mut l); + platform::close(raw); +} + +#[test] +fn a_handed_off_listener_accepts_in_the_host() { + let mut l = Loop::new(Config::default()).expect("loop"); + let server = l + .tcp_listen( + "127.0.0.1:0".parse().expect("address"), + &ListenOpts::default(), + ) + .expect("listen"); + let address = l.local_addr(server).expect("address"); + let reported = l.raw_transport(server).expect("live identity"); + let transport = l.detach(server).expect("listener handoff"); + let listener: TcpListener = platform::into_listener(transport, reported); + listener.set_nonblocking(false).expect("blocking listener"); + + // The loop is now a client of a listener the host owns. + let client = l + .tcp_connect(address, &TcpOpts::default(), Token(1)) + .expect("connect"); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut connected = false; + while !connected { + assert!( + l.now() < until, + "connect to a handed-off listener timed out" + ); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + assert!(matches!(c.result, OpResult::Connected), "{c:?}"); + connected = true; + } + } + let (mut accepted, _) = listener.accept().expect("host accept"); + accepted + .set_read_timeout(Some(Duration::from_secs(5))) + .expect("read timeout"); + l.write(client, WriteBuf::Owned(b"host accepted".to_vec()), Token(2)) + .expect("write"); + let until = l.now() + Duration::from_secs(5); + let mut wrote = 0; + while wrote == 0 { + assert!(l.now() < until, "write timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + match c.result { + OpResult::Wrote(n) => { + assert_eq!(n, 13); + wrote += 1; + } + other => panic!("unexpected {other:?}"), + } + } + } + let mut bytes = [0; 13]; + accepted.read_exact(&mut bytes).expect("read"); + assert_eq!(&bytes, b"host accepted"); + l.close(client, Token(3)).expect("close"); + let until = l.now() + Duration::from_secs(5); + while l.alive() { + assert!(l.now() < until, "close timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + out.drain(); + } +} + +#[cfg(unix)] +mod platform { + use super::*; + use std::os::fd::{AsRawFd, FromRawFd, IntoRawFd, OwnedFd, RawFd}; + + fn owned(transport: Detached, reported: RawTransport) -> OwnedFd { + let fd = transport.into_fd(); + assert_eq!( + RawTransport::Fd(fd.as_raw_fd()), + reported, + "the host received a different descriptor from the reported one" + ); + fd + } + pub fn into_stream(transport: Detached, reported: RawTransport) -> TcpStream { + let stream = TcpStream::from(owned(transport, reported)); + // Loop-created sockets are handed over non-blocking, as documented. + stream.set_nonblocking(false).expect("blocking"); + stream + .set_read_timeout(Some(Duration::from_secs(5))) + .expect("read timeout"); + stream + } + pub fn into_listener(transport: Detached, reported: RawTransport) -> TcpListener { + TcpListener::from(owned(transport, reported)) + } + pub fn into_raw(transport: Detached, reported: RawTransport) -> RawFd { + owned(transport, reported).into_raw_fd() + } + pub fn send(fd: RawFd, bytes: &[u8]) -> usize { + // SAFETY: a live descriptor this test owns, and a valid readable slice. + let n = unsafe { libc::send(fd, bytes.as_ptr().cast(), bytes.len(), 0) }; + assert!(n >= 0, "send: {}", std::io::Error::last_os_error()); + n as usize + } + pub fn recv(fd: RawFd, bytes: &mut [u8]) -> usize { + // The descriptor is still non-blocking; poll it rather than spinning. + let mut poll = libc::pollfd { + fd, + events: libc::POLLIN, + revents: 0, + }; + // SAFETY: one initialized pollfd describing a descriptor this test owns. + let ready = unsafe { libc::poll(&mut poll, 1, 5_000) }; + assert_eq!(ready, 1, "poll: {}", std::io::Error::last_os_error()); + // SAFETY: a live descriptor this test owns, and a valid writable slice. + let n = unsafe { libc::recv(fd, bytes.as_mut_ptr().cast(), bytes.len(), 0) }; + assert!(n >= 0, "recv: {}", std::io::Error::last_os_error()); + n as usize + } + pub fn close(fd: RawFd) { + // SAFETY: this test owns the descriptor and uses it no further. + drop(unsafe { OwnedFd::from_raw_fd(fd) }); + } +} + +#[cfg(windows)] +mod platform { + use super::*; + use std::os::windows::io::{AsRawSocket, FromRawSocket, IntoRawSocket, OwnedSocket, RawSocket}; + use windows_sys::Win32::Networking::WinSock as ws; + + fn owned(transport: Detached, reported: RawTransport) -> OwnedSocket { + let socket = transport.into_socket().expect("socket transport"); + assert_eq!( + RawTransport::Socket(socket.as_raw_socket() as usize), + reported, + "the host received a different socket from the reported one" + ); + socket + } + pub fn into_stream(transport: Detached, reported: RawTransport) -> TcpStream { + let stream = TcpStream::from(owned(transport, reported)); + // Loop-created sockets are handed over non-blocking, as documented. + stream.set_nonblocking(false).expect("blocking"); + stream + .set_read_timeout(Some(Duration::from_secs(5))) + .expect("read timeout"); + stream + } + pub fn into_listener(transport: Detached, reported: RawTransport) -> TcpListener { + TcpListener::from(owned(transport, reported)) + } + pub fn into_raw(transport: Detached, reported: RawTransport) -> RawSocket { + owned(transport, reported).into_raw_socket() + } + pub fn send(socket: RawSocket, bytes: &[u8]) -> usize { + // SAFETY: a live socket this test owns, and a valid readable slice. + let n = unsafe { ws::send(socket as usize, bytes.as_ptr(), bytes.len() as i32, 0) }; + assert_ne!( + n, + ws::SOCKET_ERROR, + "send: {}", + std::io::Error::last_os_error() + ); + n as usize + } + pub fn recv(socket: RawSocket, bytes: &mut [u8]) -> usize { + // The socket is still non-blocking (FIONBIO), as documented. + let until = std::time::Instant::now() + Duration::from_secs(5); + loop { + // SAFETY: a live socket this test owns, and a valid writable slice. + let n = unsafe { ws::recv(socket as usize, bytes.as_mut_ptr(), bytes.len() as i32, 0) }; + if n != ws::SOCKET_ERROR { + return n as usize; + } + let error = std::io::Error::last_os_error(); + assert_eq!( + error.kind(), + std::io::ErrorKind::WouldBlock, + "recv: {error}" + ); + assert!(std::time::Instant::now() < until, "recv timed out"); + std::thread::yield_now(); + } + } + pub fn close(socket: RawSocket) { + // SAFETY: this test owns the socket and uses it no further. + drop(unsafe { OwnedSocket::from_raw_socket(socket) }); + } +} + +/// The Windows half of the answer: a handle's completion-port association is +/// permanent, so what the receiving host may do with it is the whole contract. +#[cfg(windows)] +mod iocp { + use super::*; + use std::{ + os::windows::io::{AsRawHandle, AsRawSocket, FromRawHandle, OwnedHandle, RawSocket}, + ptr, + }; + use windows_sys::Win32::{ + Foundation::{ERROR_INVALID_PARAMETER, HANDLE, INVALID_HANDLE_VALUE, WAIT_OBJECT_0}, + Networking::WinSock as ws, + Storage::FileSystem::{ReadFile, WriteFile}, + System::{ + IO::{CreateIoCompletionPort, GetOverlappedResult, OVERLAPPED}, + Threading::{CreateEventW, GetCurrentProcessId, WaitForSingleObject}, + }, + }; + + /// A completion port of this test's own, to try the association against. + fn port() -> OwnedHandle { + // SAFETY: INVALID_HANDLE_VALUE with a null port creates a new bare port. + let raw = unsafe { CreateIoCompletionPort(INVALID_HANDLE_VALUE, ptr::null_mut(), 0, 1) }; + assert!(!raw.is_null(), "{}", std::io::Error::last_os_error()); + // SAFETY: a newly created, uniquely owned kernel handle. + unsafe { OwnedHandle::from_raw_handle(raw) } + } + fn associate(handle: HANDLE, port: &OwnedHandle, key: usize) -> std::io::Result<()> { + // SAFETY: a live handle this test owns and a live port; association + // starts no I/O and retains no pointer. + let result = unsafe { CreateIoCompletionPort(handle, port.as_raw_handle(), key, 0) }; + if result.is_null() { + return Err(std::io::Error::last_os_error()); + } + Ok(()) + } + /// `WSADuplicateSocketW` + `WSASocketW`: the documented escape hatch for a + /// host that needs a socket on a completion port of its own. + fn duplicate(socket: RawSocket) -> ws::SOCKET { + // SAFETY: valid zeroed C output storage for the protocol information. + let mut info: ws::WSAPROTOCOL_INFOW = unsafe { std::mem::zeroed() }; + // SAFETY: a live socket this test owns and writable protocol storage. + let code = + unsafe { ws::WSADuplicateSocketW(socket as usize, GetCurrentProcessId(), &mut info) }; + assert_eq!(code, 0, "duplicate: {}", std::io::Error::last_os_error()); + // SAFETY: `info` was just filled by WSADuplicateSocketW and stays live. + let duplicate = unsafe { + ws::WSASocketW( + ws::FROM_PROTOCOL_INFO, + ws::FROM_PROTOCOL_INFO, + ws::FROM_PROTOCOL_INFO, + &info, + 0, + ws::WSA_FLAG_OVERLAPPED, + ) + }; + assert_ne!( + duplicate, + ws::INVALID_SOCKET, + "socket from protocol info: {}", + std::io::Error::last_os_error() + ); + duplicate + } + + #[test] + fn a_handed_off_socket_keeps_its_association_and_duplicates_out_of_it() { + let mut l = Loop::new(Config::default()).expect("loop"); + let (h, mut peer) = connected(&mut l); + exchange_through_the_loop(&mut l, h, &mut peer); + let reported = l.raw_transport(h).expect("live identity"); + let transport = l.detach(h).expect("quiescent handoff"); + let socket = transport.into_socket().expect("socket transport"); + assert_eq!( + RawTransport::Socket(socket.as_raw_socket() as usize), + reported + ); + + let port = port(); + let refused = associate(socket.as_raw_socket() as HANDLE, &port, 1) + .expect_err("a socket already on a port cannot join another"); + assert_eq!( + refused.raw_os_error(), + Some(ERROR_INVALID_PARAMETER as i32), + "unexpected association error: {refused}" + ); + + // Synchronous Winsock calls never touch a completion port, so the + // association being permanent costs the receiving host nothing. + let raw = socket.as_raw_socket(); + assert_eq!(platform::send(raw, b"still ours"), 10); + let mut bytes = [0; 10]; + peer.read_exact(&mut bytes).expect("peer read"); + assert_eq!(&bytes, b"still ours"); + peer.write_all(b"and back").expect("peer write"); + let mut bytes = [0; 8]; + assert_eq!(platform::recv(raw, &mut bytes), 8); + assert_eq!(&bytes, b"and back"); + assert_quiet(&mut l); + + // A host that wants its own overlapped I/O duplicates first; the + // duplicate is a new socket for the same connection, with no association. + let duplicate = duplicate(raw); + associate(duplicate as HANDLE, &port, 2).expect("a duplicate joins a port"); + drop(socket); + // SAFETY: the duplicate is this test's and is used no further. + assert_eq!(unsafe { ws::closesocket(duplicate) }, 0); + } + + #[test] + fn a_handed_off_named_pipe_is_driven_with_a_tagged_event() { + let name = PipeName(format!(r"\\.\pipe\tl-handoff-{}", std::process::id()).into()); + let mut l = Loop::new(Config::default()).expect("loop"); + let (listener, client, conn) = + turnloop_contract::native_surface::pipe_pair::(&mut l, &name); + turnloop_contract::native_surface::transfer(&mut l, client, conn, b"mid-stream pipe"); + + let reported = l.raw_transport(client).expect("live identity"); + let transport = l.detach(client).expect("pipe handoff"); + let handle = transport.into_handle().expect("pipe handle"); + assert_eq!( + RawTransport::Handle(handle.as_raw_handle() as usize), + reported + ); + let port = port(); + let refused = associate(handle.as_raw_handle(), &port, 1) + .expect_err("the association travels with the pipe instance"); + assert_eq!( + refused.raw_os_error(), + Some(ERROR_INVALID_PARAMETER as i32), + "unexpected association error: {refused}" + ); + + // FILE_FLAG_OVERLAPPED survives the handoff, so every call needs an + // OVERLAPPED. Tagging hEvent's low-order bit suppresses the completion + // packet, which is what keeps the source loop's port clean. + // SAFETY: a manual-reset, initially unsignalled, unnamed event. + let event = unsafe { CreateEventW(ptr::null(), 1, 0, ptr::null()) }; + assert!(!event.is_null(), "{}", std::io::Error::last_os_error()); + // SAFETY: a newly created, uniquely owned kernel handle. + let event = unsafe { OwnedHandle::from_raw_handle(event) }; + let tagged = (event.as_raw_handle() as usize | 1) as HANDLE; + + let raw = handle.as_raw_handle(); + let bytes = b"from the host"; + // SAFETY: all-zero OVERLAPPED is valid; only hEvent is set. + let mut overlapped: OVERLAPPED = unsafe { std::mem::zeroed() }; + overlapped.hEvent = tagged; + // SAFETY: a live overlapped handle this test owns; the buffer and the + // OVERLAPPED stay alive and unmoved until GetOverlappedResult returns. + let started = unsafe { + WriteFile( + raw, + bytes.as_ptr(), + bytes.len() as u32, + ptr::null_mut(), + &mut overlapped, + ) + }; + let mut written = 0; + if started == 0 { + let error = std::io::Error::last_os_error(); + assert_eq!( + error.raw_os_error(), + Some(windows_sys::Win32::Foundation::ERROR_IO_PENDING as i32), + "host write: {error}" + ); + // SAFETY: the event is live and owned by this test. + let waited = unsafe { WaitForSingleObject(event.as_raw_handle(), 5_000) }; + assert_eq!( + waited, WAIT_OBJECT_0, + "the host's own event never signalled" + ); + } + // SAFETY: the same live handle and pinned OVERLAPPED as the write. + let finished = unsafe { GetOverlappedResult(raw, &overlapped, &mut written, 1) }; + assert_ne!( + finished, + 0, + "host write result: {}", + std::io::Error::last_os_error() + ); + assert_eq!(written as usize, bytes.len()); + + // The loop still owns the other end and reads what the host wrote. + l.read(conn, ReadBuf::Pooled, Token(30)).expect("read"); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut seen = Vec::new(); + while seen.len() < bytes.len() { + assert!(l.now() < until, "the loop never saw the host's write"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + match c.result { + OpResult::Read { + n, + lease: Some(data), + } => { + seen.extend_from_slice(&data.as_slice()[..n]); + if seen.len() < bytes.len() { + l.read(conn, ReadBuf::Pooled, Token(30)).expect("continue"); + } + } + other => panic!("unexpected {other:?}"), + } + } + } + assert_eq!(seen, bytes); + + // And back: the loop writes, the host reads through its own event. + l.write(conn, WriteBuf::Owned(b"to the host".to_vec()), Token(31)) + .expect("write"); + let until = l.now() + Duration::from_secs(5); + let mut wrote = 0; + while wrote == 0 { + assert!(l.now() < until, "loop write timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + match c.result { + OpResult::Wrote(n) => { + assert_eq!(n, 11); + wrote += 1; + } + other => panic!("unexpected {other:?}"), + } + } + } + let mut buffer = [0u8; 11]; + // SAFETY: all-zero OVERLAPPED is valid; only hEvent is set. + let mut overlapped: OVERLAPPED = unsafe { std::mem::zeroed() }; + overlapped.hEvent = tagged; + // SAFETY: a live overlapped handle this test owns; the buffer and the + // OVERLAPPED stay alive and unmoved until GetOverlappedResult returns. + let started = unsafe { + ReadFile( + raw, + buffer.as_mut_ptr(), + buffer.len() as u32, + ptr::null_mut(), + &mut overlapped, + ) + }; + let mut read = 0; + if started == 0 { + let error = std::io::Error::last_os_error(); + assert_eq!( + error.raw_os_error(), + Some(windows_sys::Win32::Foundation::ERROR_IO_PENDING as i32), + "host read: {error}" + ); + // SAFETY: the event is live and owned by this test. + let waited = unsafe { WaitForSingleObject(event.as_raw_handle(), 5_000) }; + assert_eq!( + waited, WAIT_OBJECT_0, + "the host's own event never signalled" + ); + } + // SAFETY: the same live handle and pinned OVERLAPPED as the read. + let finished = unsafe { GetOverlappedResult(raw, &overlapped, &mut read, 1) }; + assert_ne!( + finished, + 0, + "host read result: {}", + std::io::Error::last_os_error() + ); + assert_eq!(&buffer[..read as usize], b"to the host"); + + // The source loop's port never saw a packet for the handle it gave away. + assert_quiet(&mut l); + drop(handle); + let mut out = Completions::default(); + for (i, h) in [conn, listener].into_iter().enumerate() { + l.close(h, Token(40 + i as u64)).expect("close"); + } + let until = l.now() + Duration::from_secs(5); + while l.alive() { + assert!(l.now() < until, "close timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + out.drain(); + } + } +} diff --git a/crates/turnloop-contract/tests/wasi.rs b/crates/turnloop-contract/tests/wasi.rs index 535f5aa..a43f38c 100644 --- a/crates/turnloop-contract/tests/wasi.rs +++ b/crates/turnloop-contract/tests/wasi.rs @@ -366,6 +366,12 @@ fn wasi_random_fills_both_getrandom_generations_and_bson() { fn revision_two_unsupported_native_capabilities() { contract::single_agent::unsupported_native::(); } +/// A WASI socket is a component-model resource handle, not a descriptor: there +/// is nothing to hand to a host, and nothing to report as `_handle.fd` (#35). +#[test] +fn transports_have_no_descriptor_to_hand_out() { + contract::handoff::handoff_is_unsupported::(); +} #[test] fn external_wait_routing_cancellation_and_capacity() { contract::single_agent::waits::(); diff --git a/crates/turnloop-contract/tests/web/web_contract.rs b/crates/turnloop-contract/tests/web/web_contract.rs index 2b81573..251a5fb 100644 --- a/crates/turnloop-contract/tests/web/web_contract.rs +++ b/crates/turnloop-contract/tests/web/web_contract.rs @@ -504,6 +504,11 @@ async fn capability_errors_and_oversize_response_are_terminal() { l.detach(h).expect_err("transfer").kind, ErrorKind::Unsupported ); + // A browser resource is a host object with no descriptor identity (#35). + assert_eq!( + l.raw_transport(h).expect_err("identity").kind, + ErrorKind::Unsupported + ); let read = l.read(h, ReadBuf::Pooled, Token(3)).expect("read"); assert_eq!( l.read(h, ReadBuf::Pooled, Token(4)) diff --git a/crates/turnloop/src/backend/iocp/mod.rs b/crates/turnloop/src/backend/iocp/mod.rs index 53550d1..cae0350 100644 --- a/crates/turnloop/src/backend/iocp/mod.rs +++ b/crates/turnloop/src/backend/iocp/mod.rs @@ -105,9 +105,89 @@ impl Detached { accept_defaults: AcceptDefaults::EMPTY, } } -} -impl Drop for Detached { - fn drop(&mut self) { + /// The socket or handle this transport owns, for host reporting only. + /// A pipe listener owns no instance of its own and reports Unsupported. + pub fn raw_transport(&self) -> Result { + match &self.native { + Native::Socket(s) => Ok(crate::RawTransport::Socket(s.as_raw_socket() as usize)), + Native::Handle(h) => Ok(crate::RawTransport::Handle(h.as_raw_handle() as usize)), + Native::PipeListener => Err(unsupported()), + } + } + /// Give the socket to the caller; turnloop never touches it again. + /// + /// `Driver::detach` already proved quiescence, so no operation, kernel + /// storage or worker of this loop refers to the socket. It is handed over + /// exactly as the loop held it: non-blocking (`FIONBIO`), and with + /// `FILE_SKIP_COMPLETION_PORT_ON_SUCCESS` still set if the provider is an + /// IFS provider. Call `ioctlsocket(FIONBIO, 0)` (or + /// `TcpStream::set_nonblocking(false)`) for blocking I/O, which is what a + /// synchronous TLS handshake on the socket needs. + /// + /// **The IOCP association is permanent and travels with the socket.** Windows + /// has no way to dissociate a handle from a completion port, and rejects a + /// second `CreateIoCompletionPort` for one with `ERROR_INVALID_PARAMETER`, so + /// the receiving host cannot put this socket on a port of its own. What it + /// can do: + /// + /// * **Synchronous or non-blocking Winsock calls** — `recv`/`send`/`select` + /// and `WSARecv`/`WSASend` without an `OVERLAPPED`. These never touch a + /// completion port and are the supported way to use a handed-over socket. + /// * **Overlapped calls with `hEvent` tagged** — set the low-order bit of + /// `OVERLAPPED.hEvent` (`hEvent | 1`). Windows then skips queueing the + /// completion packet, and the host waits on its own event. + /// * **Its own completion port** — duplicate first: + /// `WSADuplicateSocketW` into `WSAPROTOCOL_INFOW`, then `WSASocketW` with + /// `FROM_PROTOCOL_INFO`. The duplicate is a new, unassociated socket for the + /// same underlying connection; drop this one once it exists. + /// + /// Issuing an untagged overlapped call is the one thing that is not allowed: + /// its completion packet would arrive on the source loop's port carrying an + /// `OVERLAPPED` that loop does not own, and that loop's next `turn` reports + /// `InvalidInput` rather than dereferencing it. + pub fn into_socket(self) -> Result { + if !matches!(self.native, Native::Socket(_)) { + return Err(invalid()); + } + match self.take_native() { + Native::Socket(socket) => Ok(socket), + _ => Err(invalid()), + } + } + /// Give the named-pipe instance, console or adopted stream handle to the + /// caller; turnloop never touches it again. A socket is `InvalidInput` + /// (use [`into_socket`](Self::into_socket)) and a pipe listener, which owns + /// no instance of its own, is `Unsupported`. + /// + /// A captured console mode is restored first, exactly as on close. Every + /// other property is handed over unchanged, including + /// `FILE_FLAG_OVERLAPPED` on a pipe instance: the receiving host must supply + /// an `OVERLAPPED` for every `ReadFile`/`WriteFile`, and the IOCP rules in + /// [`into_socket`](Self::into_socket) apply unchanged. For a pipe the + /// duplication escape hatch does **not** exist — `DuplicateHandle` shares the + /// same file object and therefore the same association — so tagging + /// `OVERLAPPED.hEvent` with its low-order bit (`hEvent | 1`) is the only way + /// to drive it, and it is enough: the host waits on its own event and the + /// source loop's port never sees a packet. + pub fn into_handle(self) -> Result { + match self.native { + Native::Handle(_) => {} + Native::Socket(_) => return Err(invalid()), + Native::PipeListener => return Err(unsupported()), + } + match self.take_native() { + Native::Handle(handle) => Ok(handle), + _ => Err(invalid()), + } + } + /// Restore adopted settings, then move the native resource out. What is left + /// behind owns nothing, so the usual Drop releases the rest of the transport. + fn take_native(mut self) -> Native { + self.restore(); + self.mode = None; + std::mem::replace(&mut self.native, Native::PipeListener) + } + fn restore(&self) { if let Some(mode) = self.mode { // SAFETY: owned console handle is still live; mode was captured on adoption. unsafe { @@ -116,6 +196,11 @@ impl Drop for Detached { } } } +impl Drop for Detached { + fn drop(&mut self) { + self.restore(); + } +} struct Resource { handle: Handle, transport: Detached, @@ -1489,6 +1574,12 @@ unsafe impl Backend for Iocp { self.resources[h.index()] = None; } } + fn raw_transport(&self, h: Handle) -> Result { + if self.services.contains(h) || self.watches.contains(h) { + return Err(unsupported()); + } + self.get(h)?.transport.raw_transport() + } fn detach(&mut self, h: Handle) -> Result { let r = self.get(h)?; if matches!(r.transport.kind, Kind::PipeListener | Kind::PipeConnecting) { diff --git a/crates/turnloop/src/backend/mod.rs b/crates/turnloop/src/backend/mod.rs index 0424721..4983d54 100644 --- a/crates/turnloop/src/backend/mod.rs +++ b/crates/turnloop/src/backend/mod.rs @@ -2,7 +2,10 @@ //! //! Public only so the contract runner and independently developed backends can use //! it; not a stable end-user extension API. All identifiers and buffers are core -//! types. No raw fd, OVERLAPPED pointer, pollable or browser object crosses here. +//! types. No raw fd, OVERLAPPED pointer, pollable or browser object crosses here, +//! with one deliberate exception: `raw_transport` reports a `RawTransport` the core +//! passes straight through to the host and never acts on (§5a handle transfer). +//! Ownership still leaves through `detach`/`Detached`, never through that value. //! //! # Ownership and completion rules (D1–D4) //! @@ -75,6 +78,11 @@ //! * `Detached` may be an enum, not an integer: IOCP socket migration may require //! duplication; WASI/browser objects need not support transfer. `Resource` is //! intentionally absent: each backend owns its native tables keyed by Handle. +//! * A `Detached` is also how a descriptor leaves turnloop for good: the platform +//! types expose `into_fd`/`into_socket`/`into_handle`, which restore whatever +//! the backend changed on adoption (status flags, terminal or console mode) and +//! then release ownership. Because `detach` already proved quiescence, the loop +//! has no operation, buffer or registration left for that transport. //! * `Wake` is Send + Sync on every target, but single-thread WASI/web may implement //! it as a flag/host scheduler token. No backend creates a loop-driving thread. //! Windows' explicitly requested Integration::Event helper is the D7 exception. @@ -374,6 +382,16 @@ pub unsafe trait Backend: Sized + 'static { ) -> Result { Err(Error::new(crate::ErrorKind::Unsupported)) } + /// Report the native identity of a live transport, for the host only. + /// + /// The core never acts on the value and never stores it: it reads it out of + /// the backend's own table and returns it, so a host can implement Node's + /// `socket._handle.fd`. Backends return Unsupported for a resource with no + /// descriptor identity of its own (timers, processes, signals, watches, and + /// every WASI/web resource). See `Driver::raw_transport` for the host rules. + fn raw_transport(&self, _handle: Handle) -> Result { + Err(Error::new(crate::ErrorKind::Unsupported)) + } /// Query current terminal dimensions. fn tty_window_size(&self, _handle: Handle) -> Result { Err(Error::new(crate::ErrorKind::Unsupported)) diff --git a/crates/turnloop/src/backend/unix.rs b/crates/turnloop/src/backend/unix.rs index 12e0d12..b2627f0 100644 --- a/crates/turnloop/src/backend/unix.rs +++ b/crates/turnloop/src/backend/unix.rs @@ -50,6 +50,46 @@ impl std::fmt::Debug for Detached { } } impl Detached { + /// The descriptor this transport owns, for host reporting only. + pub fn raw_transport(&self) -> crate::RawTransport { + crate::RawTransport::Fd(self.fd.as_raw_fd()) + } + /// Give the descriptor to the caller; turnloop never touches it again. + /// + /// The transport is already unregistered and quiescent: `Driver::detach` + /// refuses a handle with an outstanding operation, so no loop, poller or + /// buffer refers to this descriptor any more. Status flags and terminal + /// settings captured when the descriptor was adopted are restored first, + /// exactly as they would be on close, and then the close is *not* performed. + /// + /// A descriptor turnloop created itself was created non-blocking with + /// `FD_CLOEXEC`, and it is handed over that way: nothing is restored, + /// because nothing was changed. Call `fcntl(F_SETFL)` (or + /// `TcpStream::set_nonblocking(false)`) if the receiving code wants blocking + /// I/O, which is what a synchronous TLS handshake on the descriptor needs. + pub fn into_fd(self) -> OwnedFd { + self.restore(); + let this = std::mem::ManuallyDrop::new(self); + // SAFETY: `this` is never dropped, so this move of the single owning + // field cannot be observed twice; the remaining fields are Copy/plain + // data whose Drop is a no-op. `restore` already ran, and it is the only + // thing this type's Drop does besides releasing `fd`. + unsafe { std::ptr::read(&this.fd) } + } + fn restore(&self) { + if let Some(mode) = &self.original_mode { + // SAFETY: descriptor is still owned; restore before OwnedFd drops. + unsafe { + libc::tcsetattr(self.fd.as_raw_fd(), libc::TCSANOW, mode); + } + } + if let Some(flags) = self.original_flags { + // SAFETY: descriptor is still owned and flags came from F_GETFL. + unsafe { + libc::fcntl(self.fd.as_raw_fd(), libc::F_SETFL, flags); + } + } + } pub(super) fn new(fd: OwnedFd, kind: Kind) -> Self { // SAFETY: termios is plain C storage, filled by tcgetattr on a terminal. let mut mode: libc::termios = unsafe { std::mem::zeroed() }; @@ -73,18 +113,7 @@ impl Detached { } impl Drop for Detached { fn drop(&mut self) { - if let Some(mode) = &self.original_mode { - // SAFETY: descriptor is still owned; restore before OwnedFd drops. - unsafe { - libc::tcsetattr(self.fd.as_raw_fd(), libc::TCSANOW, mode); - } - } - if let Some(flags) = self.original_flags { - // SAFETY: descriptor is still owned and flags came from F_GETFL. - unsafe { - libc::fcntl(self.fd.as_raw_fd(), libc::F_SETFL, flags); - } - } + self.restore(); } } struct Resource { @@ -705,6 +734,12 @@ unsafe impl Backend for Unix { } // Closing the final descriptor removes its registration from epoll/kqueue. } + fn raw_transport(&self, h: Handle) -> Result { + if self.services.contains(h) || self.watches.contains(h) { + return Err(Error::new(ErrorKind::Unsupported)); + } + Ok(self.get(h)?.transport.raw_transport()) + } fn detach(&mut self, h: Handle) -> Result { let r = self.get(h)?; if r.heads.iter().any(Option::is_some) { diff --git a/crates/turnloop/src/driver.rs b/crates/turnloop/src/driver.rs index 1b3edf4..1a1f997 100644 --- a/crates/turnloop/src/driver.rs +++ b/crates/turnloop/src/driver.rs @@ -692,6 +692,31 @@ impl Driver { } self.backend.get_option(h, kind) } + /// Report the native identity of a live transport handle, for the host's own + /// bookkeeping. This is Node's `socket._handle.fd`. + /// + /// The loop keeps ownership. The value is **borrowed and reporting-only**, + /// valid until this handle is closed, detached or its loop is dropped, after + /// which the OS may reuse the number for something else. A host may print it, + /// expose it to script, compare it, or pass it to a read-only query such as + /// `getsockname`. A host must not do I/O on it, close or shut it down, change + /// its blocking mode, register it with another poller or completion port, or + /// give it to anything that takes ownership: every one of those breaks the + /// exactly-once completion and buffer-ownership contracts, and turnloop cannot + /// detect it. To take ownership, [`detach`](Self::detach) it and convert the + /// resulting transport (`into_fd` on Unix, `into_socket`/`into_handle` on + /// Windows), which is the only supported way for a descriptor to leave a loop. + /// + /// Timers, and any resource whose platform has no descriptor for it (WASI 0.2 + /// and 0.3 sockets are component-model resource handles; web resources are + /// host objects), report `Unsupported`. + pub fn raw_transport(&self, h: Handle) -> Result { + let r = self.resource(h)?; + if !matches!(r.kind, Kind::Socket) { + return Err(Error::new(ErrorKind::Unsupported)); + } + self.backend.raw_transport(h) + } fn submit(&mut self, h: Handle, operation: Operation, token: Token) -> Result { let r = self.resource(h)?; if r.closing.is_some() || !matches!(r.kind, Kind::Socket) { @@ -860,6 +885,26 @@ impl Driver { Ok(()) } /// Cancel pending operations and detach once acknowledgements are delivered; WouldBlock means turn and retry. + /// + /// Detaching is how a transport leaves a loop, whether it is going to another + /// loop ([`attach`](Self::attach)) or to the host for good. It never cancels + /// silently: operations still outstanding are cancelled, `WouldBlock` is + /// returned while their terminal completions drain, and the host turns this + /// loop and retries. On success this loop owns nothing of that transport — + /// the handle is gone (a later call reports `NotFound`), it is unregistered + /// from the poller or worker, no buffer is retained, and no completion for it + /// will ever be produced again. + /// + /// To hand the descriptor itself to the host — Node's mid-stream + /// `socket.upgradeToTLS`, which gives a live connected socket to a TLS layer + /// — convert the returned transport: `into_fd` on Unix, + /// `into_socket`/`into_handle` on Windows (where the completion-port rules in + /// those methods' documentation apply). WASI 0.2, WASI 0.3 and web report + /// `Unsupported`: their sockets are component-model resource handles or host + /// objects, not descriptors, and neither platform can pass one out. + /// + /// [`raw_transport`](Self::raw_transport) is the borrowed, reporting-only + /// counterpart for a socket the loop keeps. pub fn detach(&mut self, h: Handle) -> Result { let r = self.resource(h)?; if r.closing.is_some() || !matches!(r.kind, Kind::Socket) { diff --git a/crates/turnloop/src/types.rs b/crates/turnloop/src/types.rs index adfa2ae..7ffe411 100644 --- a/crates/turnloop/src/types.rs +++ b/crates/turnloop/src/types.rs @@ -34,6 +34,24 @@ macro_rules! id { id!(Handle); id!(OpId); +#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)] +/// The native identity of a transport, for host reporting and for a descriptor +/// the host has taken ownership of (DESIGN §5a). +/// +/// [`Driver::raw_transport`](crate::Driver::raw_transport) reports this for a +/// live, loop-owned handle; it is Node's `socket._handle.fd`. The value is +/// **reporting only**: see that method for the rules. Taking ownership is a +/// separate, owning step (`detach` then `Detached::into_fd`/`into_socket`/ +/// `into_handle`). +pub enum RawTransport { + /// A Unix file descriptor. + Fd(i32), + /// A Windows `SOCKET`. + Socket(usize), + /// A Windows `HANDLE`: a named-pipe instance, console or adopted stream. + Handle(usize), +} + #[derive(Clone, Copy, Debug, Eq, PartialEq)] /// Portable error categories independent of native numeric error codes. pub enum ErrorKind { diff --git a/docs/BACKEND_REVISION_2.md b/docs/BACKEND_REVISION_2.md index 8dfaff4..b610867 100644 --- a/docs/BACKEND_REVISION_2.md +++ b/docs/BACKEND_REVISION_2.md @@ -30,6 +30,7 @@ tl-i01b amendment below refines the single-wait rule. | `Backend::kill(handle, Signal, group)` | Signal the owned child or its explicitly created process group/tree. Never signal a reused process identity. | | `Backend::signal(handle, Signal)` | Install a per-loop subscription to the process-wide dispatcher, then accept `WatchSignal`. | | `Backend::tty_set_mode` / `tty_window_size` | Set/restore terminal mode and query character dimensions. Save original state for final release/drop. | +| `Backend::raw_transport(handle) -> Result` | Report a live transport's native identity (Unix fd, Windows SOCKET/HANDLE) for the host's own bookkeeping, Node's `socket._handle.fd`. Read it out of the backend's existing table; touch no operation storage and allocate nothing. Resources with no descriptor identity of their own — timers, processes, signals, watches, and every WASI/web resource — return `Unsupported`. Default is `Unsupported`. | The five optional native capability methods default to `Unsupported`; no native success is synthesized. `set_notifier` and `prepare_close` have separate no-op defaults. Backend @@ -253,6 +254,32 @@ and executor contracts run through the required platform runners. Windows IOCP runs in all three required native CI modes. Cross-checking is not runtime proof; browser and Windows runtime status remains explicit in the root lane report. +## Descriptor handoff (§5a, issue #35) + +`raw_transport` is the only place a native identity crosses the backend boundary, +and it crosses *outwards only*: the core returns the value to the host unchanged +and never acts on it. Ownership does not travel that way. It travels through the +existing `detach`, whose returned `Detached` now also converts into an owned +descriptor — `into_fd` on Unix, `into_socket`/`into_handle` on Windows — instead +of only into another loop's `attach`. That is what lets a host perform Node's +mid-stream `socket.upgradeToTLS` without choosing the transport at creation time. + +Nothing in the trait changes for that conversion: `detach` already guarantees a +quiescent, unregistered resource, so the backend has nothing left to release and +the conversion is a move plus whatever the backend restores on adoption (Unix +status flags and termios, Windows console mode). A backend that cannot hand a +resource out keeps refusing in `detach`, as the IOCP backend does for pipe +listeners and connecting pipes. + +Windows is the one platform where the handoff has a standing consequence. An IOCP +association cannot be undone, so it travels with the handle; quiescence makes it +inert, and the receiving host uses synchronous/non-blocking calls, tags +`OVERLAPPED.hEvent` with its low-order bit to suppress the completion packet, or +(sockets only) duplicates out of the association with `WSADuplicateSocketW`. An +untagged overlapped call would deliver a packet to the source loop's port with a +foreign `OVERLAPPED`; that loop reports `InvalidInput` from `turn` rather than +dereferencing it, which is the existing `entry` guard, not a new rule. + ## Specification clarifications proposed for review DESIGN §7.3 now reflects the NT timer decision already recorded in §15 question 3. diff --git a/protocols/turnloop-tls/Cargo.toml b/protocols/turnloop-tls/Cargo.toml index 7c0856f..17da587 100644 --- a/protocols/turnloop-tls/Cargo.toml +++ b/protocols/turnloop-tls/Cargo.toml @@ -34,7 +34,7 @@ workspace = true [package.metadata.turnloop-ci] role = "protocol" service-group = "http" -integration-tests = ["tls", "asynchronous", "async_allocations"] +integration-tests = ["tls", "upgrade", "asynchronous", "async_allocations"] wasi-tests = ["asynchronous", "async_allocations", "portable", "channel_binding"] [[test]] @@ -45,6 +45,10 @@ harness = false name = "asynchronous" required-features = ["turnloop"] +[[test]] +name = "upgrade" +required-features = ["turnloop"] + [target.'cfg(all(target_os = "wasi", target_env = "p3"))'.dependencies] turnloop-io = { workspace = true, optional = true, features = ["wasi-p3-experimental"] } diff --git a/protocols/turnloop-tls/tests/upgrade.rs b/protocols/turnloop-tls/tests/upgrade.rs new file mode 100644 index 0000000..fdabd8e --- /dev/null +++ b/protocols/turnloop-tls/tests/upgrade.rs @@ -0,0 +1,154 @@ +//! The mid-stream TLS upgrade, end to end (issue #35). +//! +//! This is Perry's `node:net` `socket.upgradeToTLS` shape, and PostgreSQL's +//! `SSLRequest` exactly: a plaintext connection is established and used, the +//! server agrees to upgrade, and only then does TLS start — on the *same* +//! connection. The socket starts on a turnloop `Loop`, exchanges plaintext +//! through it, is handed to this test as an owned descriptor, and finishes a +//! real rustls handshake there with the loop already dropped. Nothing in the +//! handshake goes through turnloop, so a loop that kept any claim on the +//! descriptor would show up as a failed or hung handshake. +#![cfg(all(feature = "turnloop", not(target_arch = "wasm32")))] +mod support; +use std::{ + io::{Read, Write}, + net::{TcpListener, TcpStream}, + thread, + time::Duration, +}; +use support::*; +use turnloop_io::turnloop::{ + self, Completions, Config, Loop, OpResult, ReadBuf, TcpOpts, Timeout, Token, WriteBuf, +}; +use turnloop_tls::{ClientConfig, ClientOptions, rustls::pki_types::ServerName}; + +/// PostgreSQL's `SSLRequest`: length 8, request code 80877103. +const SSL_REQUEST: [u8; 8] = [0, 0, 0, 8, 0x04, 0xd2, 0x16, 0x2f]; + +#[cfg(unix)] +fn into_stream(transport: turnloop::Detached) -> TcpStream { + TcpStream::from(transport.into_fd()) +} +#[cfg(windows)] +fn into_stream(transport: turnloop::Detached) -> TcpStream { + TcpStream::from(transport.into_socket().expect("socket transport")) +} + +#[test] +fn a_plaintext_socket_is_handed_off_mid_stream_for_a_real_tls_handshake() { + let cert = certificate(); + let server = server_config(&cert); + let client = ClientConfig::new( + ClientOptions { + extra_ca_pem: cert.cert.pem().into_bytes(), + alpn: vec![b"h2".to_vec()], + ..Default::default() + }, + NOW, + ) + .expect("client config"); + + let listener = TcpListener::bind("127.0.0.1:0").expect("listener"); + let address = listener.local_addr().expect("address"); + let peer = thread::spawn(move || { + let (mut socket, _) = listener.accept().expect("accept"); + socket + .set_read_timeout(Some(Duration::from_secs(5))) + .expect("read timeout"); + let mut request = [0; SSL_REQUEST.len()]; + socket.read_exact(&mut request).expect("SSLRequest"); + assert_eq!(request, SSL_REQUEST); + // 'S' is PostgreSQL's "yes, start TLS now, on this connection". + socket.write_all(b"S").expect("agreement"); + let mut stream = Stream::new(server.accept().expect("server"), socket); + let mut ping = [0; 4]; + stream.read_exact(&mut ping).expect("encrypted read"); + assert_eq!(&ping, b"ping"); + assert_eq!(stream.engine.alpn_protocol(), Some(b"h2".as_slice())); + stream.write_all(b"pong").expect("encrypted write"); + }); + + // Phase 1: plaintext, entirely on the loop. + let mut l = Loop::new(Config::default()).expect("loop"); + let h = l + .tcp_connect(address, &TcpOpts::default(), Token(1)) + .expect("connect"); + let mut out = Completions::default(); + let until = l.now() + Duration::from_secs(5); + let mut connected = false; + while !connected { + assert!(l.now() < until, "connect timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + assert!(matches!(c.result, OpResult::Connected), "{c:?}"); + connected = true; + } + } + l.write(h, WriteBuf::Owned(SSL_REQUEST.to_vec()), Token(2)) + .expect("write SSLRequest"); + l.read(h, ReadBuf::Pooled, Token(3)).expect("read"); + let until = l.now() + Duration::from_secs(5); + let (mut wrote, mut agreed) = (false, false); + while !wrote || !agreed { + assert!(l.now() < until, "SSLRequest exchange timed out"); + l.turn(Timeout::Until(until), &mut out).expect("turn"); + for c in out.drain() { + match c.result { + OpResult::Wrote(n) => { + assert_eq!(n, SSL_REQUEST.len()); + wrote = true; + } + OpResult::Read { + n, + lease: Some(data), + } => { + assert_eq!(&data.as_slice()[..n], b"S", "the server refused TLS"); + agreed = true; + } + other => panic!("unexpected {other:?}"), + } + } + } + + // Phase 2: the upgrade. The loop hands over the connected socket, keeps + // nothing, and is dropped before a single TLS byte is written. + let reported = l.raw_transport(h).expect("live identity"); + let transport = l.detach(h).expect("quiescent handoff"); + assert!(!l.alive(), "the loop still counts the upgraded socket"); + let stream = into_stream(transport); + #[cfg(unix)] + assert_eq!( + turnloop::RawTransport::Fd(std::os::fd::AsRawFd::as_raw_fd(&stream)), + reported + ); + #[cfg(windows)] + assert_eq!( + turnloop::RawTransport::Socket( + std::os::windows::io::AsRawSocket::as_raw_socket(&stream) as usize + ), + reported + ); + // Loop-created sockets are handed over non-blocking; a synchronous + // handshake wants blocking I/O, which is one call away. + stream.set_nonblocking(false).expect("blocking"); + drop(l); + + // Phase 3: a real rustls handshake and application data on that descriptor. + let mut tls = Stream::new( + client + .connect(ServerName::try_from("localhost").expect("name")) + .expect("client connection"), + stream, + ); + tls.write_all(b"ping").expect("encrypted write"); + let mut pong = [0; 4]; + tls.read_exact(&mut pong).expect("encrypted read"); + assert_eq!(&pong, b"pong"); + assert_eq!(tls.engine.alpn_protocol(), Some(b"h2".as_slice())); + assert_eq!( + tls.engine.handshake_kind(), + Some(turnloop_tls::rustls::HandshakeKind::Full), + "the upgrade must be a real handshake, not a resumption" + ); + peer.join().expect("peer"); +} From 95f5e1cb1e257154a3f90cfb43dc1a87a608c6fe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Tue, 15 Sep 2026 18:50:35 +0200 Subject: [PATCH 2/5] Record the handle-transfer lane report and gate the allocation probe to native --- crates/turnloop-contract/tests/allocations.rs | 7 + docs/lanes/handle-transfer.md | 309 ++++++++++++++++++ 2 files changed, 316 insertions(+) create mode 100644 docs/lanes/handle-transfer.md diff --git a/crates/turnloop-contract/tests/allocations.rs b/crates/turnloop-contract/tests/allocations.rs index 567c957..56d726d 100644 --- a/crates/turnloop-contract/tests/allocations.rs +++ b/crates/turnloop-contract/tests/allocations.rs @@ -522,6 +522,13 @@ fn steady_socket_options_allocate_nothing() { /// Handing a transport to the host and taking it back allocates nothing: the /// identity is read out of the backend's own table, `detach` only unregisters, /// and the conversion to an owned descriptor moves the resource it already held. +#[cfg(any( + target_vendor = "apple", + target_os = "linux", + target_os = "android", + target_os = "freebsd", + windows +))] #[test] fn steady_handoff_allocates_nothing() { let mut l = Loop::new(Config::default()).expect("loop"); diff --git a/docs/lanes/handle-transfer.md b/docs/lanes/handle-transfer.md new file mode 100644 index 0000000..4be7e3f --- /dev/null +++ b/docs/lanes/handle-transfer.md @@ -0,0 +1,309 @@ +# handle-transfer — giving a transport's descriptor back to the host (issue #35) + +Base: `909e92f`. Branch `lane/handle-transfer`, clone +`/Users/amlug/projects/perry/windlass-lanes/handle-transfer`. Linux (epoll) +coverage ran in `/root/claude-handle-transfer` on the shared build box, under an +`OWNER.txt` naming this lane. + +A turnloop socket owned its descriptor and exposed no way to get it back. Perry's +`node:net` migration therefore had to leave outbound TCP clients on tokio, because +`socket.upgradeToTLS` hands a live, **already-connected** socket to a TLS layer +mid-stream — PostgreSQL's `SSLRequest` is exactly that, and Perry has a gap test +for it. Transport was effectively fixed at creation: any socket that *might* be +upgraded later could not start on turnloop at all. `socket._handle.fd`, which Node +exposes, was also unimplementable. + +## API + +```rust +impl Loop { + // existing, now also the way out of turnloop entirely + pub fn detach(&mut self, h: Handle) -> Result; + + // new: borrowed, reporting only — Node's socket._handle.fd + pub fn raw_transport(&self, h: Handle) -> Result; +} + +pub enum RawTransport { + Fd(i32), // Unix file descriptor + Socket(usize), // Windows SOCKET + Handle(usize), // Windows HANDLE: named-pipe instance, console, adopted stream +} + +#[cfg(unix)] +impl Detached { + pub fn into_fd(self) -> OwnedFd; // infallible + pub fn raw_transport(&self) -> RawTransport; +} +#[cfg(windows)] +impl Detached { + pub fn into_socket(self) -> Result; // InvalidInput if not a socket + pub fn into_handle(self) -> Result; // InvalidInput for a socket, + // Unsupported for a pipe listener + pub fn raw_transport(&self) -> Result; +} + +// internal contract (turnloop::backend::Backend), default Unsupported +fn raw_transport(&self, handle: Handle) -> Result; +``` + +Decisions, all written into the rustdoc, DESIGN §5a/§7.6 and +`docs/BACKEND_REVISION_2.md` so they cannot drift: + +- **Ownership leaves through `detach`, and only through `detach`.** The issue + offered `Detached::into_fd()` as option 1 and a borrowed `as_raw_fd` as option 2; + both are implemented, but they are deliberately *different* things. `detach` + already cancels in-flight work, waits for the terminal completions the host is + owed, unregisters the resource and removes the handle. Converting the transport + it returns therefore needs no new guarantee: there is nothing left to cancel, no + buffer to release, no registration to remove and no completion that can still be + produced. `raw_transport` adds no guarantee at all — it is an integer the core + hands through and never acts on. +- **A refusal, never a silent cancellation.** A handle with an outstanding + operation is `WouldBlock` (the existing `detach` semantics), a closing handle is + `InvalidInput`, and a handle that is closed, already handed off, or never existed + is `NotFound`. Nothing is cancelled behind the host's back: the refusal is + repeatable, the terminal completion still arrives, and only then does `detach` + succeed. +- **The descriptor is handed over in the mode the loop held it.** Loop-created + sockets were created non-blocking (with `FD_CLOEXEC` on Unix), so that is what + the host receives; a blocking TLS handshake is one `set_nonblocking(false)` away. + What the backend *changed on adoption* is restored first, exactly as on close: + Unix status flags and termios, Windows console mode. Restoring the mode a + loop-created socket never had would be an invention. +- **`raw_transport` is narrow on purpose.** It reports; it does not lend. The + documented rules are: valid until the handle is closed or detached, usable for + printing, comparing and read-only queries (`getsockname`-class), and not for + I/O, closing, mode changes, registration with another poller or completion port, + or anything taking ownership. Those rules are what makes it sound: the function + is safe, returns a `Copy` integer, and every way to break the loop's contracts + with it requires the host to make an OS call turnloop cannot see. The narrow + door for actually taking the descriptor is `detach` + `into_fd`. +- **Non-transport handles say `Unsupported`, not a number.** Timers have no + descriptor. Process, signal and filesystem-watch handles have internal ones + (pidfd, signalfd, inotify, `RegisterWaitForSingleObject`) that are turnloop's + implementation and not the host's resource; reporting them would invite exactly + the misuse the rules forbid. +- **One documented exception to "no raw fd crosses the backend boundary."** The + `Backend` rustdoc said no raw fd, OVERLAPPED pointer, pollable or browser object + crosses the trait. `raw_transport` is now the single exception, and it is stated + as one: it crosses outwards only, the core never acts on the value, and ownership + still travels through `detach`/`Detached`. It is an optional method defaulting to + `Unsupported`, so WASI 0.2, WASI 0.3, web and the driver's own test backend get + the right answer without implementing anything. + +## Per-backend behaviour + +| Backend | Owning handoff | Reporting (`raw_transport`) | Notes | +| --- | --- | --- | --- | +| Linux (epoll) | `Detached::into_fd` | `RawTransport::Fd` | `detach` deregisters from epoll first; closing the last descriptor would too, but the loop no longer owns it | +| macOS/BSD (kqueue) | `Detached::into_fd` | `RawTransport::Fd` | identical; termios restored for an adopted terminal | +| Windows (IOCP), socket | `Detached::into_socket` | `RawTransport::Socket` | IOCP association is permanent — see below | +| Windows (IOCP), named-pipe instance | `Detached::into_handle` | `RawTransport::Handle` | keeps `FILE_FLAG_OVERLAPPED`; association is permanent and cannot be duplicated away | +| Windows (IOCP), pipe listener / connecting pipe | refused by `detach` (`Unsupported`) | `Unsupported` (no instance of its own) | unchanged from before this lane | +| WASI 0.2 | `Unsupported` | `Unsupported` | a `wasi:sockets` socket is a component-model resource handle in the component's own table, not a descriptor; no interface hands one to the embedder | +| WASI 0.3 | `Unsupported` | `Unsupported` | same reason | +| Web | `Unsupported` | `Unsupported` | a `WebSocket`/`fetch` resource is a host JS object with no descriptor identity | + +### The IOCP association answer + +Windows cannot dissociate a handle from a completion port. `CreateIoCompletionPort` +on an already-associated handle fails with `ERROR_INVALID_PARAMETER`, and the +association lives on the *file object*, so `DuplicateHandle` shares it. The +association therefore travels with every socket and pipe instance turnloop hands +over, for the life of that handle. + +What saves this is quiescence: `detach` refuses until every operation has +terminated, so **no completion packet can ever be posted to the source loop's port +for that handle by turnloop**. The association is inert. The receiving host has +three ways to work with it: + +1. **Synchronous or non-blocking Winsock calls** — `recv`/`send`/`select`, or + `WSARecv`/`WSASend` with no `OVERLAPPED`. These never involve a completion port. + This is the supported mode, and it is what a mid-stream TLS upgrade needs. +2. **Overlapped calls with a tagged event** — set the low-order bit of + `OVERLAPPED.hEvent` (`hEvent | 1`). Windows then does not queue the completion + packet, and the host waits on its own event and calls `GetOverlappedResult`. + This is the *only* way to drive a handed-over **named pipe**, because a pipe + instance keeps `FILE_FLAG_OVERLAPPED` (every `ReadFile`/`WriteFile` needs an + `OVERLAPPED`) and cannot duplicate out of its association. turnloop's own + `pipes::Connect` already relies on the inverse of this rule. +3. **Duplicating out of it — sockets only** — `WSADuplicateSocketW` into a + `WSAPROTOCOL_INFOW`, then `WSASocketW` with `FROM_PROTOCOL_INFO`. The result is + a *new*, unassociated socket for the same connection, which the host may put on + its own completion port; the original is then closed. + +The one thing a host must not do is an **untagged overlapped call**. Its completion +packet would arrive on the source loop's port carrying an `OVERLAPPED` that loop +does not own. That is not memory-unsafe — `Iocp::entry` already refuses to +dereference a pointer outside its own kernel slab and returns `InvalidInput` — but +the source loop's next `turn` fails, which is not something the host wants. +`into_socket`/`into_handle` say all of this in their rustdoc. + +## Files + +- `crates/turnloop/src/types.rs` — `RawTransport`. +- `crates/turnloop/src/driver.rs` — `Driver::raw_transport`, and the expanded + `detach` rustdoc (what the loop guarantees; where ownership goes; WASI/web). +- `crates/turnloop/src/backend/mod.rs` — `Backend::raw_transport` (default + `Unsupported`), the amended "no raw fd crosses here" statement, and the transfer + section's new paragraph on ownership leaving through `Detached`. +- `crates/turnloop/src/backend/unix.rs` — `Detached::into_fd`, `raw_transport`, + `restore()` factored out of `Drop`, and the backend's `raw_transport`. +- `crates/turnloop/src/backend/iocp/mod.rs` — `Detached::into_socket`, + `into_handle`, `raw_transport`, `take_native`/`restore`, and the backend's + `raw_transport`. +- `crates/turnloop-contract/src/handoff.rs` — the loop-side contract functions. +- `crates/turnloop-contract/tests/handoff.rs` — the native scenarios, including the + Windows IOCP and named-pipe probes. +- `crates/turnloop-contract/tests/allocations.rs` — `steady_handoff_allocates_nothing`. +- `crates/turnloop-contract/tests/wasi.rs`, `tests/web/web_contract.rs` — the + `Unsupported` assertions on the platforms that have no descriptor. +- `protocols/turnloop-tls/tests/upgrade.rs` (+ `Cargo.toml` test target and + `integration-tests` metadata) — the upgrade scenario end to end. +- `DESIGN.md` §5a (handle transfer), §6 (API sketch), §7.6 (two new matrix rows); + `docs/BACKEND_REVISION_2.md` (boundary table row and a "Descriptor handoff" + section). + +Only one dependency line changed: `turnloop-contract`'s existing `windows-sys` +gained the `Win32_Networking_WinSock` feature, for the Winsock probes in the test. +No new crates. + +## Tests — and how each one proves its subject + +Every one of these goes around turnloop for the part that matters: it uses the +descriptor from the test process with `libc`/Winsock or `std::net`, so a backend +that kept any claim on the transport could not pass. + +**`a_handed_off_socket_carries_bytes_after_its_loop_is_dropped`** — the headline. +The peer is a plain `TcpStream` the test owns, so it outlives the loop. Bytes are +exchanged *through the loop* first (the connection is provably live and +mid-stream), the socket is handed over, the loop is asserted quiet and `!alive()`, +**the whole `Loop` is dropped**, and only then do bytes flow both ways over the +returned descriptor. Nothing turnloop owned can be keeping that connection open. + +**`a_handed_off_socket_is_driven_by_the_bare_descriptor`** — the same, with no std +wrapper at all: `libc::send`/`libc::poll`/`libc::recv` (Winsock `send`/`recv` on +Windows) straight on the number the loop returned. It also asserts the returned +descriptor equals what `raw_transport` reported before the handoff, so a backend +handing back a *different* (for example duplicated) descriptor would fail. + +**`a_handed_off_listener_accepts_in_the_host`** — a listener is handed over and +becomes a `std::net::TcpListener`; the loop then connects *to it as a client*, the +test accepts, and a loop-side write is read on the host-accepted socket. + +**`listeners_and_accepted_sockets_are_handed_off`** — both ends of an accept, and +the liveness assertion that the loop's accounting really dropped them. + +**`pending_operations_refuse_handoff`** — a provided-buffer read is outstanding; +`detach` is `WouldBlock`, twice (the refusal does not cancel behind the host's +back); the turn delivers exactly one `Cancelled`; the caller's buffer is +byte-for-byte untouched; `detach` then succeeds; a second `detach` and +`raw_transport` are `NotFound`; submitting is refused; and 60 ms of turning +produces **zero** completions. + +**`closing_and_closed_handles_refuse_handoff`** — `InvalidInput` while closing, +`NotFound` after `Closed`. + +**`raw_transport_reports_live_transports`** — the value is stable across calls, +different for two live transports, unchanged after the socket is used, and the +socket still works after being reported (reporting is read-only). A timer reports +`Unsupported` from `raw_transport` and `InvalidInput` from `detach`. + +**`a_handed_off_socket_keeps_its_association_and_duplicates_out_of_it`** +(Windows) — `CreateIoCompletionPort` on the handed-over socket fails with +`ERROR_INVALID_PARAMETER`; synchronous Winsock I/O then carries bytes both ways; +the loop stays quiet; and `WSADuplicateSocketW` + `WSASocketW` produces a socket +that **does** associate with the test's own port. The documented escape hatch is +executed, not asserted in prose. + +**`a_handed_off_named_pipe_is_driven_with_a_tagged_event`** (Windows) — a +connected named-pipe instance is handed over; association is proved permanent the +same way; then `WriteFile`/`ReadFile` with `OVERLAPPED.hEvent | 1` carry bytes both +ways while the loop still owns the other end, and the loop is asserted quiet +afterwards — i.e. the tagged event really did keep the packet off the loop's port. + +**`a_plaintext_socket_is_handed_off_mid_stream_for_a_real_tls_handshake`** +(`turnloop-tls`) — Perry's actual pattern. The client sends PostgreSQL's 8-byte +`SSLRequest` through the loop and reads the server's `S` through the loop; the +socket is detached, converted, the loop dropped; and a **real rustls handshake** +runs on that descriptor, with ALPN `h2`, `HandshakeKind::Full` asserted (so it +cannot be a resumption of some other connection) and encrypted `ping`/`pong` +exchanged. No part of the handshake goes through turnloop. + +**`transports_have_no_descriptor_to_hand_out`** (WASI 0.2 and 0.3) — `detach` and +`raw_transport` are `Unsupported` for a listener, a client and an accepted socket, +and the connection still works afterwards, so the refusal is inert. +`capability_errors_and_oversize_response_are_terminal` (web) gained the same +`raw_transport` assertion next to the existing `detach` one. + +**Allocation gate**: `steady_handoff_allocates_nothing` runs 100 measured cycles of +`raw_transport` → `detach` → `into_fd`/`into_socket` → re-adopt → `attach`, +asserting `0` allocations, that the identity handed over matches the one reported +on every cycle, and — after the hundred round trips — that the socket still carries +bytes. It cannot pass having done nothing. + +## Verification + +macOS (kqueue) is this machine, `aarch64-apple-darwin`, pinned +`nightly-2026-08-20`. Linux (epoll) is `x86_64-unknown-linux-gnu` on the shared +build box. + +| Command | Result | +| --- | --- | +| `cargo +nightly-2026-08-20 fmt --all --check` | PASS | +| `python3 scripts/ci/check-paths.py` | PASS — 1505 tracked files, 266 references | +| `python3 scripts/ci/feature_modes.py` | PASS — 6 public features, 18 required arms | +| `cargo +nightly-2026-08-20 clippy --locked --workspace --all-targets --all-features -- -D warnings -D clippy::undocumented_unsafe_blocks` | PASS (macOS) | +| same, `--target x86_64-unknown-linux-gnu`, `-p turnloop -p turnloop-contract -p turnloop-io` | PASS | +| same, `--target x86_64-pc-windows-msvc` | PASS | +| same, `--target wasm32-wasip2` | PASS | +| same, `--target wasm32-unknown-unknown` | PASS | +| `cargo +nightly-2026-09-07 clippy … --target wasm32-wasip3 --all-features` | PASS | +| `RUSTDOCFLAGS='-D warnings' cargo +nightly-2026-08-20 doc --locked --workspace --all-features --no-deps` | PASS | +| `cargo +stable check --locked --workspace --all-targets --all-features` | PASS | +| `cargo test --locked --workspace --no-fail-fast -- --test-threads=1` (macOS) | PASS — exit 0, 66 `test result: ok` groups, 0 FAILED | +| `cargo test -p turnloop-contract --test handoff -- --test-threads=1` (macOS) | PASS — 7/7 | +| `cargo test -p turnloop-tls --features turnloop --test upgrade -- --test-threads=1` (macOS) | PASS — 1/1 | +| Linux six required modes (`default`, `executor`, `epoll-timerfd`, `process-sigchld`, `fallbacks`, `all-features`), each `cargo +nightly-2026-08-20 test --locked --workspace --no-fail-fast … -- --test-threads=1 --skip permission_denied_is_reported` | PASS — all six exit 0, 413 `test result: ok` lines, 0 FAILED; the handoff suite and the allocation gate each ran 6×, the TLS upgrade test once (its `turnloop` feature is only on in `all-features`) | +| `python3 scripts/ci/run-tests.py wasi --target wasm32-wasip2` (Wasmtime 46.0.0) | PASS — 43 contract (incl. `transports_have_no_descriptor_to_hand_out`) + 13 allocation tests | +| `python3 scripts/ci/run-tests.py wasi --target wasm32-wasip3` (nightly-2026-09-07) | PASS — 43 contract + 14 allocation tests | +| `python3 scripts/ci/run-tests.py node` (web backend under Node 26.5.1) | PASS — 16/16 | +| `bash scripts/ci/no-tokio.sh` | PASS — 5 graphs, zero runtime crates | +| `python3 scripts/ci/soak.py` | PASS — 251 locked versions, 1 pre-existing rustls exception | +| `cargo +nightly-2026-08-20 deny --locked check` | PASS — advisories, bans, licenses, sources | +| `python3 scripts/ci/lint-workflows.py` | PASS — no workflow files were changed | +| `python3 -m unittest discover -s scripts/ci -p 'test_*.py'` | PASS — 98 tests | +| `python3 scripts/ci/run-tests.py web` (headless Chromium/Firefox) | UNRUN locally (no browsers on this machine); covered by CI | +| Windows `cargo test` (the IOCP and named-pipe probes) | UNRUN locally; cross-compiled clean and covered by the three `windows-2025` CI arms | + +### Pre-existing failure seen on the build box (not this lane) + +`turnloop-contract --test filesystem permission_denied_is_reported` fails there +because the box runs as **root**, and root ignores a read-only file's mode. It is +the same failure the sockopts lane recorded on its own base commit, in filesystem +code this lane does not touch; CI runs unprivileged. The first Linux run +(`run-tests.py native --mode default`) reproduced exactly that one failure and +nothing else, which is why the six-mode matrix skips it by name. + +### CI + +| Run | SHA | Result | +| --- | --- | --- | +| _(filled in below)_ | | | + +## Follow-ups this lane deliberately did not take + +- **TLS on turnloop as the answer for the upgrade case** (the issue's option 3). + `turnloop-tls` exists and Perry P5 uses it; this lane makes the *general* handoff + work, which is what unblocks P1 now and what `socket._handle.fd` needs anyway. +- **No automatic duplication on Windows.** `into_socket` could have duplicated out + of the IOCP association for the host, but a pipe cannot (the association is on + the file object), so it would be an inconsistency dressed as a convenience — and + it would silently change the socket the host asked for. The recipe is documented + and tested instead. +- **No `Detached::into_fd` for WASI.** A `wasi:sockets` resource could in principle + be handed to another component, but there is no descriptor and no interface for + it; inventing one would be a `wasi:sockets` proposal, not a turnloop change. +- **No cross-process handoff surface.** `send_handle`/`recv_handle` already move + transports between processes; this lane is about leaving turnloop entirely. From 58b4a8f7f0c7ee2dc1ef6a2c432c364e4014648e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Tue, 15 Sep 2026 19:02:15 +0200 Subject: [PATCH 3/5] A Windows socket duplicate inherits its IOCP association Windows CI refused CreateIoCompletionPort for a WSADuplicateSocketW duplicate with ERROR_INVALID_PARAMETER on all three arms. The association belongs to the underlying socket, not the descriptor, so duplication is no escape from it, exactly as DuplicateHandle is none for a pipe. The test now asserts what Windows does, and the rustdoc, DESIGN and the backend revision record say that a host wanting completion-port-driven I/O again hands the transport back through attach, whose imported-association routing exists for this. --- DESIGN.md | 4 +- crates/turnloop-contract/tests/handoff.rs | 25 ++++++++--- crates/turnloop/src/backend/iocp/mod.rs | 30 +++++++------ docs/BACKEND_REVISION_2.md | 18 +++++--- docs/lanes/handle-transfer.md | 55 ++++++++++++++--------- 5 files changed, 81 insertions(+), 51 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 907f2c4..3fc2182 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -259,7 +259,7 @@ There are two ways a host drives a loop: - **Between loops:** `Loop::detach(h) -> Detached` (`Send`) and `Loop::attach(Detached, token) -> Handle`. For sockets, pipes and servers across threads or workers, it cancels in-flight ops with the usual exactly-once completions before detaching. - **Between processes:** fd passing via `SCM_RIGHTS` on Unix and `WSADuplicateSocketW` / `DuplicateHandle` on Windows, exposed on pipe handles so `child.send(msg, handle)` and cluster round-robin can move sockets (today `emitter.rs:430` drops the handle). - **Out of turnloop entirely (host handoff):** the same `detach` followed by `Detached::into_fd()` on Unix, or `into_socket()` / `into_handle()` on Windows. This is Node's mid-stream `socket.upgradeToTLS` — PostgreSQL's `SSLRequest` hands a live, already-connected socket to a TLS layer — so the class of socket that *might* later be upgraded no longer has to choose its transport at creation. `detach` already proves quiescence, so the guarantee the host gets is total: no operation, no buffer, no registration, no completion, ever again, for that transport. Conversion restores what the backend changed on adoption (Unix status flags and termios, Windows console mode) and hands the descriptor over in the mode the loop held it: non-blocking for a loop-created socket. - - **Windows:** a handle's IOCP association is permanent — Windows cannot dissociate one, and rejects a second `CreateIoCompletionPort` with `ERROR_INVALID_PARAMETER`. Quiescence makes it inert (no packet can ever arrive for it), and the receiving host has three ways to work: synchronous/non-blocking Winsock calls, overlapped calls with `OVERLAPPED.hEvent`'s low-order bit set (which suppresses the completion packet), or — sockets only — `WSADuplicateSocketW` + `WSASocketW` to obtain a fresh, unassociated socket for the same connection. A named-pipe instance keeps `FILE_FLAG_OVERLAPPED` and cannot duplicate out of its association (`DuplicateHandle` shares the file object), so the tagged-`hEvent` rule is how a host drives one. + - **Windows:** a handle's IOCP association is permanent and inescapable. Windows cannot dissociate one, rejects a second `CreateIoCompletionPort` with `ERROR_INVALID_PARAMETER`, and neither `WSADuplicateSocketW` nor `DuplicateHandle` escapes it — both produce another descriptor for the *same* socket or file object, which is where the association lives. Quiescence makes it inert (no packet can ever arrive for it), and the receiving host drives the transport with synchronous/non-blocking Winsock calls or with overlapped calls whose `OVERLAPPED.hEvent` has its low-order bit set, which suppresses the completion packet. A named-pipe instance keeps `FILE_FLAG_OVERLAPPED`, so for a pipe the tagged-`hEvent` form is the only one. The route back to completion-port-driven I/O is `attach`: an imported association is exactly what the IOCP backend's overlapped-event routing exists for. - **WASI 0.2/0.3 and web:** `Unsupported`. A WASI socket is a component-model resource handle in the component's own table, not a descriptor, and there is no interface that hands one to the embedder; a browser resource is a host JS object. Neither has an identity a host could act on. - **Reporting only:** `Loop::raw_transport(h) -> RawTransport` reports a live transport's native identity for Node's `socket._handle.fd`. The loop keeps ownership; the value is valid until the handle is closed or detached, and is for reporting and read-only queries, never for I/O, closing, mode changes or registration elsewhere. 6. **Multi-threaded accept:** @@ -450,7 +450,7 @@ Two backends, because both versions matter now: | Outbound HTTP | protocol crate | protocol crate | protocol crate | protocol crate or `wasi:http` | protocol crate or `wasi:http` | host `fetch` | | WebSocket | protocol crate | protocol crate | protocol crate | protocol crate | protocol crate | host `WebSocket` | | Local IPC | AF_UNIX | AF_UNIX | named pipes (overlapped) | unsupported | unsupported | `postMessage` | -| Descriptor handoff (§5a) | `Detached::into_fd` | `Detached::into_fd` | `into_socket`/`into_handle`; IOCP association is permanent, so tagged `hEvent` or `WSADuplicateSocketW` | unsupported (resource handle, not a descriptor) | unsupported (resource handle, not a descriptor) | unsupported (host object) | +| Descriptor handoff (§5a) | `Detached::into_fd` | `Detached::into_fd` | `into_socket`/`into_handle`; the IOCP association is permanent and inescapable, so synchronous calls or a tagged `hEvent` | unsupported (resource handle, not a descriptor) | unsupported (resource handle, not a descriptor) | unsupported (host object) | | Native identity reporting (`_handle.fd`) | `RawTransport::Fd` | `RawTransport::Fd` | `RawTransport::Socket`/`Handle` | unsupported | unsupported | unsupported | | Stdio pipes | readiness | readiness | overlapped, or reader thread | `wasi:cli` streams | `wasi:cli` streams | unsupported | | TTY | termios + readiness | termios + readiness | console API reader thread, VT modes | size only | size only | unsupported | diff --git a/crates/turnloop-contract/tests/handoff.rs b/crates/turnloop-contract/tests/handoff.rs index 1197adc..e4615d3 100644 --- a/crates/turnloop-contract/tests/handoff.rs +++ b/crates/turnloop-contract/tests/handoff.rs @@ -374,8 +374,9 @@ mod iocp { } Ok(()) } - /// `WSADuplicateSocketW` + `WSASocketW`: the documented escape hatch for a - /// host that needs a socket on a completion port of its own. + /// `WSADuplicateSocketW` + `WSASocketW`. The new descriptor references the + /// *same underlying socket*, which is where the completion-port association + /// lives, so this does not escape it — the test below proves that. fn duplicate(socket: RawSocket) -> ws::SOCKET { // SAFETY: valid zeroed C output storage for the protocol information. let mut info: ws::WSAPROTOCOL_INFOW = unsafe { std::mem::zeroed() }; @@ -404,7 +405,7 @@ mod iocp { } #[test] - fn a_handed_off_socket_keeps_its_association_and_duplicates_out_of_it() { + fn a_handed_off_socket_keeps_its_association_even_through_a_duplicate() { let mut l = Loop::new(Config::default()).expect("loop"); let (h, mut peer) = connected(&mut l); exchange_through_the_loop(&mut l, h, &mut peer); @@ -438,13 +439,23 @@ mod iocp { assert_eq!(&bytes, b"and back"); assert_quiet(&mut l); - // A host that wants its own overlapped I/O duplicates first; the - // duplicate is a new socket for the same connection, with no association. + // There is no way out of the association. `WSADuplicateSocketW` gives a + // new *descriptor* for the same underlying socket, and the association + // belongs to that socket, so the duplicate inherits it. (Windows CI + // caught this: the first version of this test asserted the opposite.) + // The route back to completion-port-driven I/O is `Loop::attach`, which + // detects an imported association and routes through overlapped events. let duplicate = duplicate(raw); - associate(duplicate as HANDLE, &port, 2).expect("a duplicate joins a port"); - drop(socket); + let refused = associate(duplicate as HANDLE, &port, 2) + .expect_err("a duplicate shares the original's association"); + assert_eq!( + refused.raw_os_error(), + Some(ERROR_INVALID_PARAMETER as i32), + "unexpected duplicate association error: {refused}" + ); // SAFETY: the duplicate is this test's and is used no further. assert_eq!(unsafe { ws::closesocket(duplicate) }, 0); + drop(socket); } #[test] diff --git a/crates/turnloop/src/backend/iocp/mod.rs b/crates/turnloop/src/backend/iocp/mod.rs index cae0350..1eaae2b 100644 --- a/crates/turnloop/src/backend/iocp/mod.rs +++ b/crates/turnloop/src/backend/iocp/mod.rs @@ -126,9 +126,14 @@ impl Detached { /// /// **The IOCP association is permanent and travels with the socket.** Windows /// has no way to dissociate a handle from a completion port, and rejects a - /// second `CreateIoCompletionPort` for one with `ERROR_INVALID_PARAMETER`, so - /// the receiving host cannot put this socket on a port of its own. What it - /// can do: + /// second `CreateIoCompletionPort` with `ERROR_INVALID_PARAMETER`. Nor is + /// there a way around it: the association belongs to the underlying socket, + /// and `WSADuplicateSocketW` only produces another descriptor *for that same + /// socket*, so a duplicate is refused too (this is asserted, not assumed). + /// + /// It is inert, though. `Driver::detach` proved the transport quiescent, so + /// no completion packet will ever be posted for it. The receiving host has + /// two ways to drive it, and one route back: /// /// * **Synchronous or non-blocking Winsock calls** — `recv`/`send`/`select` /// and `WSARecv`/`WSASend` without an `OVERLAPPED`. These never touch a @@ -136,10 +141,9 @@ impl Detached { /// * **Overlapped calls with `hEvent` tagged** — set the low-order bit of /// `OVERLAPPED.hEvent` (`hEvent | 1`). Windows then skips queueing the /// completion packet, and the host waits on its own event. - /// * **Its own completion port** — duplicate first: - /// `WSADuplicateSocketW` into `WSAPROTOCOL_INFOW`, then `WSASocketW` with - /// `FROM_PROTOCOL_INFO`. The duplicate is a new, unassociated socket for the - /// same underlying connection; drop this one once it exists. + /// * **Back onto a loop** — `Detached::from_socket` and `Driver::attach`. + /// That is what an imported association is for: the backend detects it and + /// routes completions through overlapped events instead of the port. /// /// Issuing an untagged overlapped call is the one thing that is not allowed: /// its completion packet would arrive on the source loop's port carrying an @@ -163,12 +167,12 @@ impl Detached { /// other property is handed over unchanged, including /// `FILE_FLAG_OVERLAPPED` on a pipe instance: the receiving host must supply /// an `OVERLAPPED` for every `ReadFile`/`WriteFile`, and the IOCP rules in - /// [`into_socket`](Self::into_socket) apply unchanged. For a pipe the - /// duplication escape hatch does **not** exist — `DuplicateHandle` shares the - /// same file object and therefore the same association — so tagging - /// `OVERLAPPED.hEvent` with its low-order bit (`hEvent | 1`) is the only way - /// to drive it, and it is enough: the host waits on its own event and the - /// source loop's port never sees a packet. + /// [`into_socket`](Self::into_socket) apply unchanged — `DuplicateHandle` + /// shares the file object and therefore the association, exactly as + /// `WSADuplicateSocketW` does for a socket. So tagging `OVERLAPPED.hEvent` + /// with its low-order bit (`hEvent | 1`) is how a host drives one, and it is + /// enough: the host waits on its own event and the source loop's port never + /// sees a packet. pub fn into_handle(self) -> Result { match self.native { Native::Handle(_) => {} diff --git a/docs/BACKEND_REVISION_2.md b/docs/BACKEND_REVISION_2.md index b610867..2bc44b2 100644 --- a/docs/BACKEND_REVISION_2.md +++ b/docs/BACKEND_REVISION_2.md @@ -272,13 +272,17 @@ resource out keeps refusing in `detach`, as the IOCP backend does for pipe listeners and connecting pipes. Windows is the one platform where the handoff has a standing consequence. An IOCP -association cannot be undone, so it travels with the handle; quiescence makes it -inert, and the receiving host uses synchronous/non-blocking calls, tags -`OVERLAPPED.hEvent` with its low-order bit to suppress the completion packet, or -(sockets only) duplicates out of the association with `WSADuplicateSocketW`. An -untagged overlapped call would deliver a packet to the source loop's port with a -foreign `OVERLAPPED`; that loop reports `InvalidInput` from `turn` rather than -dereferencing it, which is the existing `entry` guard, not a new rule. +association cannot be undone and cannot be duplicated away — it belongs to the +underlying socket or file object, so `WSADuplicateSocketW` and `DuplicateHandle` +both inherit it — so it travels with the handle. Quiescence makes it inert, and +the receiving host uses synchronous/non-blocking calls or tags +`OVERLAPPED.hEvent` with its low-order bit to suppress the completion packet; a +host that wants completion-port-driven I/O again hands the transport back through +`Detached::from_socket`/`from_handle` and `attach`, whose imported-association +routing already exists for exactly this. An untagged overlapped call would deliver +a packet to the source loop's port with a foreign `OVERLAPPED`; that loop reports +`InvalidInput` from `turn` rather than dereferencing it, which is the existing +`entry` guard, not a new rule. ## Specification clarifications proposed for review diff --git a/docs/lanes/handle-transfer.md b/docs/lanes/handle-transfer.md index 4be7e3f..b8aa64b 100644 --- a/docs/lanes/handle-transfer.md +++ b/docs/lanes/handle-transfer.md @@ -98,8 +98,8 @@ Decisions, all written into the rustdoc, DESIGN §5a/§7.6 and | --- | --- | --- | --- | | Linux (epoll) | `Detached::into_fd` | `RawTransport::Fd` | `detach` deregisters from epoll first; closing the last descriptor would too, but the loop no longer owns it | | macOS/BSD (kqueue) | `Detached::into_fd` | `RawTransport::Fd` | identical; termios restored for an adopted terminal | -| Windows (IOCP), socket | `Detached::into_socket` | `RawTransport::Socket` | IOCP association is permanent — see below | -| Windows (IOCP), named-pipe instance | `Detached::into_handle` | `RawTransport::Handle` | keeps `FILE_FLAG_OVERLAPPED`; association is permanent and cannot be duplicated away | +| Windows (IOCP), socket | `Detached::into_socket` | `RawTransport::Socket` | IOCP association is permanent and inescapable — see below | +| Windows (IOCP), named-pipe instance | `Detached::into_handle` | `RawTransport::Handle` | keeps `FILE_FLAG_OVERLAPPED`; same permanent association | | Windows (IOCP), pipe listener / connecting pipe | refused by `detach` (`Unsupported`) | `Unsupported` (no instance of its own) | unchanged from before this lane | | WASI 0.2 | `Unsupported` | `Unsupported` | a `wasi:sockets` socket is a component-model resource handle in the component's own table, not a descriptor; no interface hands one to the embedder | | WASI 0.3 | `Unsupported` | `Unsupported` | same reason | @@ -109,14 +109,26 @@ Decisions, all written into the rustdoc, DESIGN §5a/§7.6 and Windows cannot dissociate a handle from a completion port. `CreateIoCompletionPort` on an already-associated handle fails with `ERROR_INVALID_PARAMETER`, and the -association lives on the *file object*, so `DuplicateHandle` shares it. The -association therefore travels with every socket and pipe instance turnloop hands -over, for the life of that handle. +association lives on the underlying socket / file object rather than on the +descriptor — so **duplication does not escape it either**: `DuplicateHandle` for a +pipe and `WSADuplicateSocketW` + `WSASocketW` for a socket both produce another +descriptor for the same object, and the duplicate is refused too. The association +therefore travels with every socket and pipe instance turnloop hands over, for the +life of that object. + +That last point is a correction this lane's own test forced. The first version of +`a_handed_off_socket_keeps_its_association_and_duplicates_out_of_it` asserted that +a `WSADuplicateSocketW` duplicate *could* join a fresh port — the reading of MSDN +that seemed obvious — and Windows CI failed it with `ERROR_INVALID_PARAMETER` on +all three arms. The test is now +`a_handed_off_socket_keeps_its_association_even_through_a_duplicate` and asserts +what Windows actually does, and the rustdoc, DESIGN §5a and +`docs/BACKEND_REVISION_2.md` were corrected with it. What saves this is quiescence: `detach` refuses until every operation has terminated, so **no completion packet can ever be posted to the source loop's port for that handle by turnloop**. The association is inert. The receiving host has -three ways to work with it: +two ways to drive the transport, and one route back: 1. **Synchronous or non-blocking Winsock calls** — `recv`/`send`/`select`, or `WSARecv`/`WSASend` with no `OVERLAPPED`. These never involve a completion port. @@ -125,13 +137,14 @@ three ways to work with it: `OVERLAPPED.hEvent` (`hEvent | 1`). Windows then does not queue the completion packet, and the host waits on its own event and calls `GetOverlappedResult`. This is the *only* way to drive a handed-over **named pipe**, because a pipe - instance keeps `FILE_FLAG_OVERLAPPED` (every `ReadFile`/`WriteFile` needs an - `OVERLAPPED`) and cannot duplicate out of its association. turnloop's own - `pipes::Connect` already relies on the inverse of this rule. -3. **Duplicating out of it — sockets only** — `WSADuplicateSocketW` into a - `WSAPROTOCOL_INFOW`, then `WSASocketW` with `FROM_PROTOCOL_INFO`. The result is - a *new*, unassociated socket for the same connection, which the host may put on - its own completion port; the original is then closed. + instance keeps `FILE_FLAG_OVERLAPPED`: every `ReadFile`/`WriteFile` needs an + `OVERLAPPED`. turnloop's own `pipes::Connect` already relies on the inverse of + this rule ("the event's low bit is clear for IOCP delivery"). +3. **Back onto a loop** — `Detached::from_socket`/`from_handle` + `Loop::attach`. + An imported association is precisely what the IOCP backend's overlapped-event + routing exists for, so a host that wants completion-driven async I/O again asks + turnloop for it rather than fighting Windows. The allocation gate exercises this + round trip 100 times on Windows. The one thing a host must not do is an **untagged overlapped call**. Its completion packet would arrive on the source loop's port carrying an `OVERLAPPED` that loop @@ -210,12 +223,12 @@ different for two live transports, unchanged after the socket is used, and the socket still works after being reported (reporting is read-only). A timer reports `Unsupported` from `raw_transport` and `InvalidInput` from `detach`. -**`a_handed_off_socket_keeps_its_association_and_duplicates_out_of_it`** +**`a_handed_off_socket_keeps_its_association_even_through_a_duplicate`** (Windows) — `CreateIoCompletionPort` on the handed-over socket fails with `ERROR_INVALID_PARAMETER`; synchronous Winsock I/O then carries bytes both ways; -the loop stays quiet; and `WSADuplicateSocketW` + `WSASocketW` produces a socket -that **does** associate with the test's own port. The documented escape hatch is -executed, not asserted in prose. +the loop stays quiet; and a `WSADuplicateSocketW` + `WSASocketW` duplicate is +refused by the same port for the same reason, because it is another descriptor for +the same socket. This is the test that corrected the documentation. **`a_handed_off_named_pipe_is_driven_with_a_tagged_event`** (Windows) — a connected named-pipe instance is handed over; association is proved permanent the @@ -297,11 +310,9 @@ nothing else, which is why the six-mode matrix skips it by name. - **TLS on turnloop as the answer for the upgrade case** (the issue's option 3). `turnloop-tls` exists and Perry P5 uses it; this lane makes the *general* handoff work, which is what unblocks P1 now and what `socket._handle.fd` needs anyway. -- **No automatic duplication on Windows.** `into_socket` could have duplicated out - of the IOCP association for the host, but a pipe cannot (the association is on - the file object), so it would be an inconsistency dressed as a convenience — and - it would silently change the socket the host asked for. The recipe is documented - and tested instead. +- **No automatic duplication on Windows.** It was considered and is now known to + be impossible: a duplicate inherits the association (see above), so there is + nothing `into_socket` could have done for the host that the host cannot do. - **No `Detached::into_fd` for WASI.** A `wasi:sockets` resource could in principle be handed to another component, but there is no descriptor and no interface for it; inventing one would be a `wasi:sockets` proposal, not a turnloop change. From 2aba6a0e9547d6857fc6fb0226eae9f6b14542d5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Tue, 15 Sep 2026 19:11:39 +0200 Subject: [PATCH 4/5] Record the green CI run in the lane report --- docs/lanes/handle-transfer.md | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/docs/lanes/handle-transfer.md b/docs/lanes/handle-transfer.md index b8aa64b..e690a96 100644 --- a/docs/lanes/handle-transfer.md +++ b/docs/lanes/handle-transfer.md @@ -287,8 +287,9 @@ build box. | `cargo +nightly-2026-08-20 deny --locked check` | PASS — advisories, bans, licenses, sources | | `python3 scripts/ci/lint-workflows.py` | PASS — no workflow files were changed | | `python3 -m unittest discover -s scripts/ci -p 'test_*.py'` | PASS — 98 tests | -| `python3 scripts/ci/run-tests.py web` (headless Chromium/Firefox) | UNRUN locally (no browsers on this machine); covered by CI | -| Windows `cargo test` (the IOCP and named-pipe probes) | UNRUN locally; cross-compiled clean and covered by the three `windows-2025` CI arms | +| `python3 scripts/ci/run-tests.py web` (headless Chromium/Firefox) | UNRUN locally (no browsers on this machine); PASS **in CI** (run 34998733403) | +| Windows `cargo test` (the IOCP and named-pipe probes) | UNRUN locally; cross-compiled clean and PASS **in CI** on all three `windows-2025` arms (run 34998733403) | +| `python3 scripts/ci/run-tests.py protocol` (real-server fixtures, incl. `turnloop-tls/upgrade`) | UNRUN locally (needs the fixture servers); PASS **in CI** (run 34998733403) | ### Pre-existing failure seen on the build box (not this lane) @@ -301,9 +302,21 @@ nothing else, which is why the six-mode matrix skips it by name. ### CI +CI runs on `pull_request` and pushes to `main`, so this branch's runs are +`workflow_dispatch` on `lane/handle-transfer`. No PR was opened. + | Run | SHA | Result | | --- | --- | --- | -| _(filled in below)_ | | | +| [34997592013](https://github.com/PerryTS/turnloop/actions/runs/34997592013) | `95f5e1c` | FAIL — every job green except the three `windows-2025` arms, each failing only `a_handed_off_socket_keeps_its_association_and_duplicates_out_of_it` at the *duplicate* assertion: `a duplicate joins a port: Os { code: 87, … "The parameter is incorrect." }`. Everything else on Windows passed first time, including the named-pipe tagged-event test, the eight other handoff tests and the allocation gate | +| [34998733403](https://github.com/PerryTS/turnloop/actions/runs/34998733403) | `58b4a8f` | **PASS — every job, `ci-gate` green.** 36 jobs: Linux x86_64 and arm64 (six modes each), macOS (three), Windows (three), WASI 0.2 and 0.3, headless-browser `web`, `protocol`, `protocol-wasi`, `h2spec`, `loom`, `miri`, `instructions`, `dependencies`, the four `lint-native`/`lint-wasm` arms and `workflow-lint` | + +Subjects confirmed to have executed in the green run, not merely to have not +thrown: `iocp::a_handed_off_named_pipe_is_driven_with_a_tagged_event`, +`iocp::a_handed_off_socket_keeps_its_association_even_through_a_duplicate`, +`steady_handoff_allocates_nothing` and the rest of `tests/handoff.rs` in every +`windows-2025` arm (workspace and per-member runs), and +`a_plaintext_socket_is_handed_off_mid_stream_for_a_real_tls_handshake` in +`protocol` (`turnloop-tls/upgrade | PASS | 1 tests passed`). ## Follow-ups this lane deliberately did not take From af3ea6245c319e118458fce36f4979be965f692c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Tue, 15 Sep 2026 19:20:20 +0200 Subject: [PATCH 5/5] Record the final green CI run --- docs/lanes/handle-transfer.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/lanes/handle-transfer.md b/docs/lanes/handle-transfer.md index e690a96..51b76ae 100644 --- a/docs/lanes/handle-transfer.md +++ b/docs/lanes/handle-transfer.md @@ -310,6 +310,8 @@ CI runs on `pull_request` and pushes to `main`, so this branch's runs are | [34997592013](https://github.com/PerryTS/turnloop/actions/runs/34997592013) | `95f5e1c` | FAIL — every job green except the three `windows-2025` arms, each failing only `a_handed_off_socket_keeps_its_association_and_duplicates_out_of_it` at the *duplicate* assertion: `a duplicate joins a port: Os { code: 87, … "The parameter is incorrect." }`. Everything else on Windows passed first time, including the named-pipe tagged-event test, the eight other handoff tests and the allocation gate | | [34998733403](https://github.com/PerryTS/turnloop/actions/runs/34998733403) | `58b4a8f` | **PASS — every job, `ci-gate` green.** 36 jobs: Linux x86_64 and arm64 (six modes each), macOS (three), Windows (three), WASI 0.2 and 0.3, headless-browser `web`, `protocol`, `protocol-wasi`, `h2spec`, `loom`, `miri`, `instructions`, `dependencies`, the four `lint-native`/`lint-wasm` arms and `workflow-lint` | +| [34999737266](https://github.com/PerryTS/turnloop/actions/runs/34999737266) | `2aba6a0` | **PASS — every job, `ci-gate` green.** Re-run on the final tree, which adds only this report | + Subjects confirmed to have executed in the green run, not merely to have not thrown: `iocp::a_handed_off_named_pipe_is_driven_with_a_tagged_event`, `iocp::a_handed_off_socket_keeps_its_association_even_through_a_duplicate`,