Skip to content

nostr: never advertise a rendezvous we are not registered at - #26

Merged
Bitflash-sh merged 1 commit into
mainfrom
fix/rendezvous-descriptor
Jul 29, 2026
Merged

nostr: never advertise a rendezvous we are not registered at#26
Bitflash-sh merged 1 commit into
mainfrom
fix/rendezvous-descriptor

Conversation

@Bitflash-sh

Copy link
Copy Markdown
Owner

The bug

BtfActiveRelay() fell back to vBtfMeetingRelays[0] whenever registration had not completed yet:

CRITICAL_BLOCK(cs_activeRelay)
    if (!strBtfActiveRelay.empty())
        return strBtfActiveRelay;
if (!vBtfMeetingRelays.empty())
    return vBtfMeetingRelays[0];   // a relay we are NOT registered at

strBtfActiveRelay is only set by BtfSetActiveRelay(), called from ThreadBtfAccept after registration actually succeeds (net.cpp). But ThreadNostrSeed and ThreadBtfAccept are started in parallel:

if (_beginthread(ThreadNostrSeed, 0, NULL) == -1)          // publishes descriptors
...
if (_beginthread(ThreadBtfAccept, 0, NULL) == -1)          // performs registration

So at boot the fallback is what PublishDescriptor() writes into meeting_node. Worse, ThreadNostrSeed opens with a one-time self-test that publishes the descriptor to every relay in the list before the seed loop starts — so the bad value gets fanned out network-wide.

Every peer that resolves such a descriptor dials a rendezvous the node was never registered at. When the first seed happens to be down, that is a guaranteed wasted dial for every peer that reads it. 92.246.128.180 — first in the seed list — was down for part of today, which is how this surfaced.

The "rendezvous-pending" guard already in PublishDescriptor was dead code: BtfActiveRelay() could not return empty while the seed list was non-empty.

The fix

  • BtfActiveRelay() returns only the relay ThreadBtfAccept confirmed, empty until then.
  • PublishDescriptor() skips publishing while it is empty. The seed loop republishes every pass, so the only cost is the delay until registration succeeds.
  • The self-test waits up to 60s for registration, so it still measures the publish/resolve chain instead of losing the race against it.

Second fix, same file

The WebSocket handshake reader did a bare break on oversized headers:

resp += c;
if (resp.size() > 4096) break;
}
if (resp.find(" 101 ") == string::npos)
    return error("Nostr: handshake refused by %s", host.c_str());

The status line is read first, so " 101 " still matched and the function returned success — with the socket parked in the middle of the header block. The caller then handed that socket to the frame parser, which read leftover HTTP as WebSocket frames. Now it errors out, and the cap is 8192 because relays behind Cloudflare send large header blocks.

Not done here, on purpose

I was going to drop relay.damus.io and relay.nostr.band from the public relay list, since both spam the log. Checking first:

  • relay.nostr.band — TCP connect fails from two different networks. Genuinely down.
  • relay.damus.ioworks. Sending byte-for-byte the same handshake this code builds returns HTTP/1.1 101 Switching Protocols.

So the damus failures are ours to explain, not a reason to delete the relay. Left the list alone.

Testing

Full Windows build green (MSYS2 UCRT64). Touches only Nostr discovery — no consensus, wallet, or P2P message handling.

BtfActiveRelay() fell back to vBtfMeetingRelays[0] whenever registration had
not completed yet. ThreadNostrSeed and ThreadBtfAccept start in parallel
(net.cpp), so at boot that fallback was exactly what PublishDescriptor wrote
into the descriptor -- including the one-time self-test, which publishes to
every relay in the list. Peers reading it dialed a rendezvous this node was
never registered at, and whenever the first seed happened to be down
(92.246.128.180 was, for part of today) every one of those dials was wasted.

BtfActiveRelay() now returns only the relay ThreadBtfAccept confirmed, and
PublishDescriptor skips publishing until there is one. The seed loop
republishes on every pass, so the only cost is the delay until registration
succeeds; the self-test waits up to 60s for it so it still measures the
publish/resolve chain instead of losing the race against it.

Also fix the WebSocket handshake reader. On headers past the cap it did a
bare break, leaving the socket parked in the middle of the header block --
and since the status line had already been read, the " 101 " check still
passed, so the caller got a "connected" socket whose next bytes were
leftover HTTP that the frame parser then read as garbage frames. It now
errors out instead, and the cap is 8192 because relays behind Cloudflare
send large header blocks.
@Bitflash-sh

Copy link
Copy Markdown
Owner Author

Field evidence for this, from trying to test something else.

Two fresh nodes on one machine, node B dialing node A by .btf. B resolved A
fine and then failed every single dial:

[net] trying m2lgnyqunvfmzg3z4w4nl7aaywdpmksbsmmdb5nfye4hyl6pm4wuiua.btf
[net] ConnectNodeBtf: tunnel to m2lgny...btf via 92.246.128.180:8434 failed
[nostr] (199 similar in 3s, last: Nostr: discovered .btf peer m2lgny...btf)

A's descriptor advertised 92.246.128.180:8434vBtfMeetingRelays[0], the
fallback. A's own log contains no ThreadBtfAccept started and no rendezvous
registration at all, so it was never reachable there. Its self-test still
reported success, because publish/resolve/verify all work fine on a descriptor
that points nowhere:

[nostr] Nostr: .btf self-resolve OK via wss://nos.lol (meeting_node=92.246.128.180:8434, ...)

B burned 199 rediscoveries and a continuous dial loop on a node that could not
be reached. That is the cost this PR removes, measured.

There is a second effect worth folding in. ThreadBtfAccept picks its relay
at random:

iRelay = (size_t)GetRand(relays.size());

while the fallback always hands out vBtfMeetingRelays[0]. So even when
registration does succeed, the published meeting node is likely the wrong one
until the seed loop republishes — with three seeds plus discovered relays, most
of the time. The window this PR closes is therefore much wider than "briefly at
boot": it is "until the next republish, usually pointing at the wrong relay".

Separately, and not addressed here: a node whose ThreadBtfAccept never runs
still publishes descriptors and still looks healthy to itself. Worth its own
look — I have not established why A's thread did not start.

@Bitflash-sh
Bitflash-sh merged commit 37562ac into main Jul 29, 2026
Bitflash-sh added a commit that referenced this pull request Jul 29, 2026
PR #25 was opened, reviewed and approved as a single-file change to net.cpp:
the .btf peer cache, +169/-0. I then pushed peer exchange and a /port option
onto that same branch while the PR was open, which silently widened it to six
files. The merge carried all of it in.

Nothing about the peer exchange had been validated. Its two-node test never
completed -- node B could not dial node A at all -- so what landed on main is
unproven network protocol code that no one agreed to take.

This restores main to what #25 actually described. The peer cache stays, with
its runtime numbers (36.3s to 2.1s to first peer). #26 stays. What comes out:

- the "btfpeers" message, both directions
- BtfPexCollect / BtfPexAccept and the per-message caps
- CNode::nLastPexRecv
- the `desc` field on the cached peer record
- BtfLocalDescriptor / BtfSecpContext
- /port

Peer exchange returns as its own pull request once its test passes. The /port
option is worth having on its own -- two nodes cannot share a machine without
it, which is what blocked the test -- and should go up separately rather than
ride along again.

Release v1.2.2 is unaffected: it was built from 3caee27, before either merge.
Bitflash-sh added a commit that referenced this pull request Jul 29, 2026
A node could not become reachable. RvServiceRegister connected, sent its
'S'+pubkey header, and then blocked reading the relay's 0x01 byte -- which the
relay only writes when a client pairs with the service. So the call did not
return when the node was registered; it returned when somebody had already
dialled it.

ThreadBtfAccept called BtfSetActiveRelay after that. The result is a deadlock:

  - to be dialled, a node must publish a descriptor naming its meeting node
  - to publish, BtfActiveRelay() must be set
  - it is only set once somebody has dialled

Before #26 this was masked. BtfActiveRelay() fell back to
vBtfMeetingRelays[0], so a node always published something, and ThreadBtfAccept
picks its relay at random -- so it worked whenever the two happened to match,
roughly one time in the number of relays. That is the flakiness this project has
been chasing: not a bad relay, an advertisement that usually named the wrong one.

#26 removed the fallback, which was right on its own terms and turned an
unreliable heuristic into a reliable deadlock. This is the other half of that
change and they belong together.

RvServiceRegister now returns as soon as the relay has the service listed.
RvServiceWaitPaired does the blocking read. ThreadBtfAccept registers, advertises,
then waits. RvClientConnect is untouched -- a dialer blocking until it is paired
is correct.

Registration failure also had no log line at all, which is why this stayed
invisible: a node that never registered looked identical to one waiting quietly
for a peer. There are now lines for registering, failing over, and being dropped.

Verified: a fresh node reports `registered at 90.156.222.107:8434, waiting for a
dial` and then self-resolves with `meeting_node=90.156.222.107:8434` -- the relay
it is actually on. Two nodes on one machine reach each other.
Bitflash-sh added a commit that referenced this pull request Jul 29, 2026
Everything since 1.2.2 is about nodes being able to find and reach each other,
which is the problem this project actually had:

  #30  rendezvous registration and waiting split apart. Registering used to
       block until somebody dialled, so a node could not advertise a meeting
       node until it had already been reached at one. This is the root cause
       of the flaky connectivity, not dead relays or stale descriptors.
  #26  a node no longer advertises a rendezvous it is not registered at
  #25  .btf peers that answered are remembered and dialled first (36.3s -> 2.1s)
  #32  peers exchange signed descriptors, so discovery survives a relay outage
  #29  debug.log stopped dropping lines under concurrency
  #28  /port, so two nodes can share a machine
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant