net: add a default host netdev backed by raw Linux syscalls - #59
Conversation
|
Also see: https://github.com/niemeyer/muslnet |
OK that looks very interesting indeed cc @soypat |
| // | ||
| // TINYGO: Trimmed copy of the standard library's net.DNSError so that name | ||
| // resolution errors keep a familiar, type-assertable shape. | ||
| type DNSError struct { |
There was a problem hiding this comment.
DNSError and a lot of other logic has been added. Please rebase branch
Line 10 in d5da3dd
On the native linux target the net package defaulted to a NOP netdev that returns "Netdev not set" for every operation, so net.Dial/Listen, DNS and anything built on them (net/http, tls) failed at runtime. Native linux does not override the syscall package, and the TinyGo compiler lowers syscall.Syscall/RawSyscall into real system calls, so the standard library socket functions work directly (musl's omitted network module is not needed). Register a default netdev implementing the netdever interface on top of syscall.Socket/Connect/Bind/Listen/Accept/Send/Recv/SetSockOpt, plus a small UDP DNS resolver (/etc/hosts, /etc/resolv.conf) for GetHostByName. Also restore the net.DNSError type used by resolution errors. Blocking sockets under the threads scheduler; IPv4 only. This avoids the internal/poll netpoller dependency that blocked the upstream-net approach. Refs tinygo-org#28
|
Rebased on main. You were right that #63 landed on top of this: it added On Worth noting these sources are not a standalone Go module, so I could not build them outside TinyGo. The rebase was mechanical (dropping a duplicate type), and I checked for duplicate top-level declarations across the package and for the identifiers the dropped block used. |
|
This landed already: #60 was stacked on this branch, so merging it brought Rebasing confirms it — the commit is detected as already applied and the branch comes out empty against main. Closing rather than leaving a conflicting duplicate open. Thanks for the reviews. |
Summary
On the native
linuxtarget thenetpackage defaults to a NOP netdev, so everyoperation returns
Netdev not setandnet.Dial/Listen, DNS, and anything built onthem (
net/http,tls) fail at runtime. This adds a default netdev for native Linux thatimplements the
netdeverinterface directly on top of raw kernel sockets, so thenetpackage works on a regular Linux host with no network driver.
Addresses #28.
Why this approach
The earlier attempt in #28 tried to drop TinyGo's
netpackage and pull in the upstreamstdlib
net, which stalled on the runtime netpoller:This PR keeps the existing lightweight netdev-based
netpackage and instead supplies theone missing piece — a netdev for the host — which sidesteps that blocker entirely. It works
because:
syscallpackage (loader/goroot.go), and theTinyGo compiler lowers
syscall.Syscall/RawSyscallinto real system calls(
compiler/syscall.go), so the stdlib socket functions(
syscall.Socket/Connect/Bind/Listen/Accept/...) are available and work directly. musl'somitted
src/networkmodule is not needed.threads, so blocking socket syscalls per goroutine aresafe. This deliberately avoids the
internal/pollnetpoller dependency above.What's included
netdev_native.go—hostNetdevimplementing the fullnetdeverinterface(
Socket/Bind/Connect/Listen/Accept/Send/Recv/Close/SetSockOpt/Addr) over thesyscallpackage, registered as the default via
init(). Deadlines map toSO_RCVTIMEO/SO_SNDTIMEO(re-armed per write iteration so a deadline bounds the wholeSend);Sendloops over short writes; EOF and timeout errors are surfaced(
timeoutErrorsatisfiesnet.Error);Connectretries onEINTR.GetHostByNameresolves IP literals, then/etc/hosts, then a small UDP DNS resolverdriven by
/etc/resolv.conf(the reply is validated against the query ID and theresponse/RCODE bits). Build-tagged
linux && !baremetal && !nintendoswitch && !wasm_unknown && !tinygo.wasm, so only thenative Linux target is affected — embedded/WASM netdevs are untouched.
lookup.go— restores the standardnet.DNSErrortype used by resolution errors.Verification
Built and run with
tinygoonlinux/amd64:net.Listen+Accept+net.Dialon127.0.0.1round-trips (the case fromTinyGo build of the default binary: ~3.5× smaller, offline commands work; blockers documented skycoin/skycoin#2902).
example.com:80with deadlines —200 OK.http.Get("http://example.com/")—200 OK.[::1]) returns a clean "only IPv4 is supported" error rather thanpanicking.
... failed: Netdev not set.Scope / caveats
fine for modest connection counts; not intended to match upstream Go's scalability at very
high connection counts.
netpackage. IPv6 would require broadeningthe package's address handling (e.g.
DialTCP'slen(IP) != 4guard) and is left as afollow-up.
search-domain handling);sufficient for typical client/daemon use, not a full resolver.
Happy to add tests or adjust the default-registration approach / build constraints to taste.