Skip to content

net: add a default host netdev backed by raw Linux syscalls - #59

Closed
0pcom wants to merge 1 commit into
tinygo-org:mainfrom
0magnet:native-netdev
Closed

net: add a default host netdev backed by raw Linux syscalls#59
0pcom wants to merge 1 commit into
tinygo-org:mainfrom
0magnet:native-netdev

Conversation

@0pcom

@0pcom 0pcom commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

On the native linux target the net package defaults to a NOP netdev, so every
operation returns Netdev not set and net.Dial/Listen, DNS, and anything built on
them (net/http, tls) fail at runtime. This adds a default netdev for native Linux that
implements the netdever interface directly on top of raw kernel sockets, so the net
package works on a regular Linux host with no network driver.

Addresses #28.

Why this approach

The earlier attempt in #28 tried to drop TinyGo's net package and pull in the upstream
stdlib net, which stalled on the runtime netpoller:

ld.lld: undefined symbol: internal/poll.runtime_pollReset
ld.lld: undefined symbol: internal/poll.runtime_pollWait
ld.lld: undefined symbol: internal/poll.runtime_pollSetDeadline

This PR keeps the existing lightweight netdev-based net package and instead supplies the
one missing piece — a netdev for the host — which sidesteps that blocker entirely. It works
because:

  • Native Linux does not override the syscall package (loader/goroot.go), and the
    TinyGo compiler lowers syscall.Syscall/RawSyscall into real system calls
    (compiler/syscall.go), so the stdlib socket functions
    (syscall.Socket/Connect/Bind/Listen/Accept/...) are available and work directly. musl's
    omitted src/network module is not needed.
  • The native Linux scheduler is threads, so blocking socket syscalls per goroutine are
    safe. This deliberately avoids the internal/poll netpoller dependency above.

What's included

  • netdev_native.gohostNetdev implementing the full netdever interface
    (Socket/Bind/Connect/Listen/Accept/Send/Recv/Close/SetSockOpt/Addr) over the syscall
    package, registered as the default via init(). Deadlines map to
    SO_RCVTIMEO/SO_SNDTIMEO (re-armed per write iteration so a deadline bounds the whole
    Send); Send loops over short writes; EOF and timeout errors are surfaced
    (timeoutError satisfies net.Error); Connect retries on EINTR.
    GetHostByName resolves IP literals, then /etc/hosts, then a small UDP DNS resolver
    driven by /etc/resolv.conf (the reply is validated against the query ID and the
    response/RCODE bits). Build-tagged
    linux && !baremetal && !nintendoswitch && !wasm_unknown && !tinygo.wasm, so only the
    native Linux target is affected — embedded/WASM netdevs are untouched.
  • lookup.go — restores the standard net.DNSError type used by resolution errors.

Verification

Built and run with tinygo on linux/amd64:

Scope / caveats

  • Blocking sockets, one OS thread per blocked connection (no epoll netpoller). Correct, and
    fine for modest connection counts; not intended to match upstream Go's scalability at very
    high connection counts.
  • IPv4 only, matching the rest of the TinyGo net package. IPv6 would require broadening
    the package's address handling (e.g. DialTCP's len(IP) != 4 guard) and is left as a
    follow-up.
  • The DNS resolver is a minimal stub (A records, first answer, no search-domain handling);
    sufficient for typical client/daemon use, not a full resolver.

Happy to add tests or adjust the default-registration approach / build constraints to taste.

@b0ch3nski

Copy link
Copy Markdown

Also see: https://github.com/niemeyer/muslnet

@deadprogram

Copy link
Copy Markdown
Member

Also see: https://github.com/niemeyer/muslnet

OK that looks very interesting indeed cc @soypat

Comment thread lookup.go Outdated
//
// TINYGO: Trimmed copy of the standard library's net.DNSError so that name
// resolution errors keep a familiar, type-assertable shape.
type DNSError struct {

@soypat soypat Jul 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DNSError and a lot of other logic has been added. Please rebase branch

type DNSError struct {

On the native linux target the net package defaulted to a NOP netdev that
returns "Netdev not set" for every operation, so net.Dial/Listen, DNS and
anything built on them (net/http, tls) failed at runtime.

Native linux does not override the syscall package, and the TinyGo compiler
lowers syscall.Syscall/RawSyscall into real system calls, so the standard
library socket functions work directly (musl's omitted network module is not
needed). Register a default netdev implementing the netdever interface on top
of syscall.Socket/Connect/Bind/Listen/Accept/Send/Recv/SetSockOpt, plus a
small UDP DNS resolver (/etc/hosts, /etc/resolv.conf) for GetHostByName. Also
restore the net.DNSError type used by resolution errors.

Blocking sockets under the threads scheduler; IPv4 only. This avoids the
internal/poll netpoller dependency that blocked the upstream-net approach.

Refs tinygo-org#28
@0pcom

0pcom commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on main. You were right that #63 landed on top of this: it added DNSError in dnserror.go, so the copy this branch was carrying in lookup.go is now redundant and has been dropped in favour of yours — the netdev only constructs *DNSError values, and the field set it uses is identical. errNoSuchHost also comes from main now rather than from here. The diff is back to two files: lookup.go and the new netdev_native.go.

On niemeyer/muslnet — thanks, that is genuinely relevant prior art and I had not seen it. It reaches the same place from the other direction: linking musl to get a real libc resolver and socket layer, where this uses raw syscalls plus a small /etc/hosts and /etc/resolv.conf resolver, and so needs no C at all. Happy to be told the musl route is the one you would rather carry — it would mean deleting most of this — but the two do not conflict, so I have left this as-is pending your call.

Worth noting these sources are not a standalone Go module, so I could not build them outside TinyGo. The rebase was mechanical (dropping a duplicate type), and I checked for duplicate top-level declarations across the package and for the identifiers the dropped block used.

@0pcom

0pcom commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

This landed already: #60 was stacked on this branch, so merging it brought 7bf471a ("net: add default host netdev backed by raw Linux syscalls") onto main along with the IPv6 commit on 2026-08-20.

Rebasing confirms it — the commit is detected as already applied and the branch comes out empty against main. Closing rather than leaving a conflicting duplicate open. Thanks for the reviews.

@0pcom 0pcom closed this Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants