Skip to content

Remember .btf peers that answered, and dial them first on start - #25

Merged
Bitflash-sh merged 3 commits into
mainfrom
feat/btf-peer-cache
Jul 29, 2026
Merged

Remember .btf peers that answered, and dial them first on start#25
Bitflash-sh merged 3 commits into
mainfrom
feat/btf-peer-cache

Conversation

@Bitflash-sh

@Bitflash-sh Bitflash-sh commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Attacks the seven-minute wait before a node finds its first peer, which we measured repeatedly today.

Why it takes so long

Discovery is entirely dependent on the Nostr relays, and the P2P protocol carries no peer information at all — version, block, tx, inv, getblocks, getdata and nothing else. Two nodes can be connected and syncing without ever telling each other that anyone else exists.

So every start repeats the same sequence: reach the relays, pull down ~270 descriptors, dial them blindly, and wait while roughly 96% fail because the peers behind them are long gone.

A node that has been running for hours knows exactly which addresses answer. It throws all of it away on exit.

What this does

Every peer that completes a tunnel is written to btfpeers.json along with its meeting node and encryption key — everything ConnectNodeBtfResolved needs to redial it without touching a relay. On start, a thread tries those first, stopping once eight are connected.

Entries expire after a week and the file holds fifty at most, so a node that has been off for a long time still falls back to discovery rather than hammering addresses that have certainly moved.

Scope

Cache only. A first-ever run is unchanged, and a missing or unparseable file just falls through to the relays. Nothing here is consensus.

What it does not fix

The dependency itself. If the relays are unreachable — and today's logs show relay.damus.io refusing handshakes and relay.nostr.band failing outright — a node with an empty cache still cannot find anyone.

The real answer is peer exchange over the existing P2P link. A .btf address reveals no IP, being a hash of a public key, so gossiping them is safer than Bitcoin's addr messages ever were. After first contact the network would sustain itself, with the relays as bootstrap rather than lifeline. That is a bigger change and wants its own PR.

Not built yet — needs make windows.

Discovery depends entirely on the Nostr relays. A node that has been up
for hours knows exactly which peers respond, and throws all of it away on
exit; the next start reaches for the relays again and walks the same
descriptor list, of which roughly 96% is dead. That is what makes a
restart take minutes before the first tunnel opens -- measured today at
about seven.

Every peer that completes a tunnel is now written to btfpeers.json with
its meeting node and encryption key, which is everything needed to redial
it. On start a thread tries those before the relays have answered
anything, stopping once eight are connected. Entries older than a week
are dropped, and the file holds at most fifty.

Cache only: nothing here changes how a first-ever run bootstraps, and a
missing or corrupt file just falls through to relay discovery.
@Bitflash-sh

Copy link
Copy Markdown
Owner Author

Runtime test done — this was the piece missing before merge.

Two rounds on a clean isolated datadir, same binary, same machine, network in its
current (healthy) state. Timing is wall-clock from process start to the first
Added time data line, which only appears once a peer completes the version
handshake — i.e. the first moment a real peer exists.

Round Cache First peer First block
1 empty datadir, no btfpeers.json 36.3s 38.3s
2 same datadir, cache present 2.1s 4.1s

~17x faster to first peer. Round 1 wrote a cache with 2 entries; round 2 dialed
them directly and skipped the relays entirely.

Cache file after round 1, well-formed:

[
  {
    "btf": "varrjzrzlzlc63h2l7hmssdinrjr75t3jz2gx4vurpjlcmcl4564gra.btf",
    "enc": "5cd605d288e654fd15983f00e11b53e0c3e41761ab758856b5ee0b87d03e983b",
    "meeting": "92.246.128.180:8434",
    "seen": 1785272843
  },
  ...
]

One thing worth knowing, not a blocker: both entries pin the same meeting relay.
That relay was down for part of today, and a cached entry whose meeting node has died
is undialable until the node falls back to Nostr re-resolution. The cache is a
fast path, not a replacement for discovery — which is how it is written, so this is
an observation rather than a defect.

Unrelated finding while setting the test up: net.cpp says nListenPort is
"tunable via /port", but nothing in main_gui.cpp parses /port or -port. The
option does not exist, so two nodes cannot share a machine. Either the comment or
the option should go; separate change.

Discovery has depended entirely on the Nostr relays. Two connected nodes never
told each other who else existed, so a relay outage left a running network
unable to grow, and a node that found one stale peer had no way to learn there
was anything better. The old `addr` gossip went out with the IRC seed.

Peers now trade signed descriptors over a new "btfpeers" message: our own
first, then peers that actually answered us. The receiver checks each Schnorr
signature against the key the `.btf` address decodes to, so the sender is
trusted for nothing -- forging an entry needs somebody else's secret key. That
leaves flooding with self-generated keys, which the per-message caps and the
"peers that answered" ordering are there to blunt.

Unknown commands are ignored (main.cpp), so nodes without this just drop the
message; nothing here is a protocol break.

The cache from the peer-cache change now keeps each peer's descriptor verbatim
alongside the parsed fields, because that signature is what makes a peer
relayable. Entries without one -- resolved through Nostr, or written before
this -- stay dialable, just not passed on. A dial never clears a descriptor
already on file.
net.cpp has said nListenPort is "tunable via /port" since it was written, but
nothing ever parsed the option. Two nodes could not share a machine, which is
exactly what testing peer exchange needs. Sets addrLocalHost.port too, so we
do not advertise a port we never bound.
@Bitflash-sh

Copy link
Copy Markdown
Owner Author

For the record, since this PR's description does not match what it merged.

It was reviewed as one file (net.cpp, +169/-0, the peer cache). I later pushed peer exchange and a /port option onto the same branch while the PR was open, which widened it to six files without updating the description. The merge carried all of it.

The peer cache itself is what the numbers above measured and is unaffected. The peer exchange had not been validated — its two-node test never completed. #27 removes it from main; it will come back as its own PR once it passes.

Release v1.2.2 was built from 3caee27, before this merge, so nothing published is affected.

Bitflash-sh added a commit that referenced this pull request Jul 29, 2026
PR #25 was opened, reviewed and approved as a single-file change to net.cpp:
the .btf peer cache, +169/-0. I then pushed peer exchange and a /port option
onto that same branch while the PR was open, which silently widened it to six
files. The merge carried all of it in.

Nothing about the peer exchange had been validated. Its two-node test never
completed -- node B could not dial node A at all -- so what landed on main is
unproven network protocol code that no one agreed to take.

This restores main to what #25 actually described. The peer cache stays, with
its runtime numbers (36.3s to 2.1s to first peer). #26 stays. What comes out:

- the "btfpeers" message, both directions
- BtfPexCollect / BtfPexAccept and the per-message caps
- CNode::nLastPexRecv
- the `desc` field on the cached peer record
- BtfLocalDescriptor / BtfSecpContext
- /port

Peer exchange returns as its own pull request once its test passes. The /port
option is worth having on its own -- two nodes cannot share a machine without
it, which is what blocked the test -- and should go up separately rather than
ride along again.

Release v1.2.2 is unaffected: it was built from 3caee27, before either merge.
Vic-Nas added a commit to Vic-Nas/bitflash-mod that referenced this pull request Jul 29, 2026
…sh-sh#25)

Discovery otherwise depends entirely on the Nostr relays -- a node
that's been up for hours and knows exactly which peers respond throws
all of it away on exit, and the next start walks the same mostly-dead
descriptor list again. Now persists (address, meeting node, encryption
key) for peers that actually connected, and a new thread dials up to 8
of them before relay discovery has said anything. Upstream measured
this as 36.3s to 2.1s to first peer.

Ported only the part of Bitflash-sh#25 that's still actually on their
main: the peer-exchange protocol and desc-field additions that rode
along on the same PR were reverted by them shortly after (Bitflash-sh#27,
merged unreviewed, its own two-node test never completed) -- not
carrying that part over.

main_gui: implement the port command-line option, which net.cpp already documented (Bitflash-sh#28)

nListenPort was described as tunable via a command-line option since
it was written; nothing ever parsed it, so a second node couldn't
start on a machine already running one. Sets addrLocalHost.port too,
or the node announces a port it never bound.

util: stop debug.log dropping lines under concurrency (Bitflash-sh#29)

strDebugFile's path resolution was outside any lock while the write
itself (plus our rotation logic, added independently) was already
inside one -- so one thread could read the path while another was
still assigning it. On a race, fopen got a torn path, returned NULL,
and the line was dropped silently: a missing line then reads as
"this thread never ran," which is unsound. Path resolution now shares
the same lock as the write/rotate.

nostr: cap reassembled WebSocket message size (rest of Bitflash-sh#17)

Only fixed the j[0]-type-check half of this earlier and dismissed the
frame-size half as moot post-libwebsockets -- that was wrong. lws
bounds a single fragment, but m_curFrame reassembly across fragments
had no ceiling of its own, so a relay sending one message in many
small fragments could still make us allocate unbounded memory. Capped
at 4 MiB, matching the number used in the original hand-rolled-reader
fix this project shipped once already.
Bitflash-sh added a commit that referenced this pull request Jul 29, 2026
Everything since 1.2.2 is about nodes being able to find and reach each other,
which is the problem this project actually had:

  #30  rendezvous registration and waiting split apart. Registering used to
       block until somebody dialled, so a node could not advertise a meeting
       node until it had already been reached at one. This is the root cause
       of the flaky connectivity, not dead relays or stale descriptors.
  #26  a node no longer advertises a rendezvous it is not registered at
  #25  .btf peers that answered are remembered and dialled first (36.3s -> 2.1s)
  #32  peers exchange signed descriptors, so discovery survives a relay outage
  #29  debug.log stopped dropping lines under concurrency
  #28  /port, so two nodes can share a machine
Vic-Nas added a commit to Vic-Nas/bitflash-mod that referenced this pull request Jul 29, 2026
mapAddressBook and CWalletDB::WriteName/ReadName/EraseName already
existed -- leftover from the Bitcoin 0.1 codebase this forked from --
but nothing exposed a way to actually use it. Added AddressBookName()/
SetAddressBookName() as a plain API, and a "More" window (View menu)
with three sections:

- Connected Peers: the live vNodes list, with each entry's .btf
  address (outbound only -- an inbound connection doesn't tell us its
  own .btf address as part of the handshake, nothing to show there
  yet) and an inline rename control.
- Cached Peers: everything from the persistent peer-cache file (Bitflash-sh#25),
  same rename control, so a peer you've dealt with before but aren't
  currently connected to can still be named.
- Address Book: the full mapAddressBook listing, plus a manual
  add-any-address-with-a-name form -- for payout addresses, worker
  addresses, or anything else that isn't a live peer at all.

One shared DrawBookNameRow control across all three, and also wired
into the operator's existing "Current Round Workers" table (both the
payout-address and worker-name columns), so a name saved anywhere
shows up everywhere that address appears -- exactly the "rename
workers, save friend addresses, cached peers use the book name if
there is one" behavior asked for.

net.h/net.cpp: added CNode::strBtfAddr (populated on outbound connect,
empty for inbound) so the peers panel has something to show besides a
raw transport address. Also exposed CachedBtfPeer/LoadCachedBtfPeers
via net.h instead of file-local in net.cpp, since the GUI needs them
now too.
Vic-Nas added a commit to Vic-Nas/bitflash-mod that referenced this pull request Jul 29, 2026
…-sh#32)

Discovery has depended entirely on the Nostr relays. Two connected
nodes never told each other who else existed, so a relay outage left
a running network unable to grow, and a node that found one stale
peer had no way to learn there was anything better.

Peers now trade signed descriptors over a new "btfpeers" message: our
own first, then peers that actually answered us. The receiver checks
each Schnorr signature against the key the .btf address decodes to,
so the sender is trusted for nothing -- forging an entry needs
somebody else's secret key. Per-message caps (MAX_PEX_DESCRIPTORS,
MAX_PEX_DESCRIPTOR_BYTES) and a per-connection rate limit
(PEX_MIN_INTERVAL via CNode::nLastPexRecv) bound flooding with
self-generated keys, which signatures alone don't prevent.

Unknown commands are already ignored, so a node without this just
drops the message -- not a protocol break.

Adapted rather than cherry-picked, same as the other Bitflash-sh#25/Bitflash-sh#28/Bitflash-sh#29/Bitflash-sh#30
ports: our net.cpp diverged (CachedBtfPeer/RememberBtfPeer already
exist here from the peer-cache port, and use our own
BtfPeerCacheHexToBytes/BytesToHex naming rather than generic
HexToBytes/BytesToHex to avoid symbol collisions elsewhere in this
codebase). CachedBtfPeer gains a `desc` field -- the peer's own
descriptor kept verbatim, since that signature is what makes a peer
relayable to someone else. RememberBtfPeer takes it as an optional
arg and never lets a plain dial (empty desc) clear one already on
file, so learning a peer through exchange doesn't get undone the next
time we happen to also reach them directly.

nostr.cpp gains BtfLocalDescriptor() (reads strBtfActiveRelay
directly, not through BtfActiveRelay()'s fallback -- only a
rendezvous we've actually confirmed registration at is worth handing
to a peer) and BtfSecpContext() (so net.cpp can verify without
pulling secp256k1 into net.h).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant