nostr: never advertise a rendezvous we are not registered at - #26
Conversation
BtfActiveRelay() fell back to vBtfMeetingRelays[0] whenever registration had not completed yet. ThreadNostrSeed and ThreadBtfAccept start in parallel (net.cpp), so at boot that fallback was exactly what PublishDescriptor wrote into the descriptor -- including the one-time self-test, which publishes to every relay in the list. Peers reading it dialed a rendezvous this node was never registered at, and whenever the first seed happened to be down (92.246.128.180 was, for part of today) every one of those dials was wasted. BtfActiveRelay() now returns only the relay ThreadBtfAccept confirmed, and PublishDescriptor skips publishing until there is one. The seed loop republishes on every pass, so the only cost is the delay until registration succeeds; the self-test waits up to 60s for it so it still measures the publish/resolve chain instead of losing the race against it. Also fix the WebSocket handshake reader. On headers past the cap it did a bare break, leaving the socket parked in the middle of the header block -- and since the status line had already been read, the " 101 " check still passed, so the caller got a "connected" socket whose next bytes were leftover HTTP that the frame parser then read as garbage frames. It now errors out instead, and the cap is 8192 because relays behind Cloudflare send large header blocks.
|
Field evidence for this, from trying to test something else. Two fresh nodes on one machine, node B dialing node A by A's descriptor advertised B burned 199 rediscoveries and a continuous dial loop on a node that could not There is a second effect worth folding in. iRelay = (size_t)GetRand(relays.size());while the fallback always hands out Separately, and not addressed here: a node whose |
PR #25 was opened, reviewed and approved as a single-file change to net.cpp: the .btf peer cache, +169/-0. I then pushed peer exchange and a /port option onto that same branch while the PR was open, which silently widened it to six files. The merge carried all of it in. Nothing about the peer exchange had been validated. Its two-node test never completed -- node B could not dial node A at all -- so what landed on main is unproven network protocol code that no one agreed to take. This restores main to what #25 actually described. The peer cache stays, with its runtime numbers (36.3s to 2.1s to first peer). #26 stays. What comes out: - the "btfpeers" message, both directions - BtfPexCollect / BtfPexAccept and the per-message caps - CNode::nLastPexRecv - the `desc` field on the cached peer record - BtfLocalDescriptor / BtfSecpContext - /port Peer exchange returns as its own pull request once its test passes. The /port option is worth having on its own -- two nodes cannot share a machine without it, which is what blocked the test -- and should go up separately rather than ride along again. Release v1.2.2 is unaffected: it was built from 3caee27, before either merge.
A node could not become reachable. RvServiceRegister connected, sent its 'S'+pubkey header, and then blocked reading the relay's 0x01 byte -- which the relay only writes when a client pairs with the service. So the call did not return when the node was registered; it returned when somebody had already dialled it. ThreadBtfAccept called BtfSetActiveRelay after that. The result is a deadlock: - to be dialled, a node must publish a descriptor naming its meeting node - to publish, BtfActiveRelay() must be set - it is only set once somebody has dialled Before #26 this was masked. BtfActiveRelay() fell back to vBtfMeetingRelays[0], so a node always published something, and ThreadBtfAccept picks its relay at random -- so it worked whenever the two happened to match, roughly one time in the number of relays. That is the flakiness this project has been chasing: not a bad relay, an advertisement that usually named the wrong one. #26 removed the fallback, which was right on its own terms and turned an unreliable heuristic into a reliable deadlock. This is the other half of that change and they belong together. RvServiceRegister now returns as soon as the relay has the service listed. RvServiceWaitPaired does the blocking read. ThreadBtfAccept registers, advertises, then waits. RvClientConnect is untouched -- a dialer blocking until it is paired is correct. Registration failure also had no log line at all, which is why this stayed invisible: a node that never registered looked identical to one waiting quietly for a peer. There are now lines for registering, failing over, and being dropped. Verified: a fresh node reports `registered at 90.156.222.107:8434, waiting for a dial` and then self-resolves with `meeting_node=90.156.222.107:8434` -- the relay it is actually on. Two nodes on one machine reach each other.
Everything since 1.2.2 is about nodes being able to find and reach each other, which is the problem this project actually had: #30 rendezvous registration and waiting split apart. Registering used to block until somebody dialled, so a node could not advertise a meeting node until it had already been reached at one. This is the root cause of the flaky connectivity, not dead relays or stale descriptors. #26 a node no longer advertises a rendezvous it is not registered at #25 .btf peers that answered are remembered and dialled first (36.3s -> 2.1s) #32 peers exchange signed descriptors, so discovery survives a relay outage #29 debug.log stopped dropping lines under concurrency #28 /port, so two nodes can share a machine
The bug
BtfActiveRelay()fell back tovBtfMeetingRelays[0]whenever registration had not completed yet:strBtfActiveRelayis only set byBtfSetActiveRelay(), called fromThreadBtfAcceptafter registration actually succeeds (net.cpp). ButThreadNostrSeedandThreadBtfAcceptare started in parallel:So at boot the fallback is what
PublishDescriptor()writes intomeeting_node. Worse,ThreadNostrSeedopens with a one-time self-test that publishes the descriptor to every relay in the list before the seed loop starts — so the bad value gets fanned out network-wide.Every peer that resolves such a descriptor dials a rendezvous the node was never registered at. When the first seed happens to be down, that is a guaranteed wasted dial for every peer that reads it.
92.246.128.180— first in the seed list — was down for part of today, which is how this surfaced.The
"rendezvous-pending"guard already inPublishDescriptorwas dead code:BtfActiveRelay()could not return empty while the seed list was non-empty.The fix
BtfActiveRelay()returns only the relayThreadBtfAcceptconfirmed, empty until then.PublishDescriptor()skips publishing while it is empty. The seed loop republishes every pass, so the only cost is the delay until registration succeeds.Second fix, same file
The WebSocket handshake reader did a bare
breakon oversized headers:The status line is read first, so
" 101 "still matched and the function returned success — with the socket parked in the middle of the header block. The caller then handed that socket to the frame parser, which read leftover HTTP as WebSocket frames. Now it errors out, and the cap is 8192 because relays behind Cloudflare send large header blocks.Not done here, on purpose
I was going to drop
relay.damus.ioandrelay.nostr.bandfrom the public relay list, since both spam the log. Checking first:relay.nostr.band— TCP connect fails from two different networks. Genuinely down.relay.damus.io— works. Sending byte-for-byte the same handshake this code builds returnsHTTP/1.1 101 Switching Protocols.So the damus failures are ours to explain, not a reason to delete the relay. Left the list alone.
Testing
Full Windows build green (MSYS2 UCRT64). Touches only Nostr discovery — no consensus, wallet, or P2P message handling.