Remember .btf peers that answered, and dial them first on start - #25
Conversation
Discovery depends entirely on the Nostr relays. A node that has been up for hours knows exactly which peers respond, and throws all of it away on exit; the next start reaches for the relays again and walks the same descriptor list, of which roughly 96% is dead. That is what makes a restart take minutes before the first tunnel opens -- measured today at about seven. Every peer that completes a tunnel is now written to btfpeers.json with its meeting node and encryption key, which is everything needed to redial it. On start a thread tries those before the relays have answered anything, stopping once eight are connected. Entries older than a week are dropped, and the file holds at most fifty. Cache only: nothing here changes how a first-ever run bootstraps, and a missing or corrupt file just falls through to relay discovery.
|
Runtime test done — this was the piece missing before merge. Two rounds on a clean isolated datadir, same binary, same machine, network in its
~17x faster to first peer. Round 1 wrote a cache with 2 entries; round 2 dialed Cache file after round 1, well-formed: [
{
"btf": "varrjzrzlzlc63h2l7hmssdinrjr75t3jz2gx4vurpjlcmcl4564gra.btf",
"enc": "5cd605d288e654fd15983f00e11b53e0c3e41761ab758856b5ee0b87d03e983b",
"meeting": "92.246.128.180:8434",
"seen": 1785272843
},
...
]One thing worth knowing, not a blocker: both entries pin the same Unrelated finding while setting the test up: |
Discovery has depended entirely on the Nostr relays. Two connected nodes never told each other who else existed, so a relay outage left a running network unable to grow, and a node that found one stale peer had no way to learn there was anything better. The old `addr` gossip went out with the IRC seed. Peers now trade signed descriptors over a new "btfpeers" message: our own first, then peers that actually answered us. The receiver checks each Schnorr signature against the key the `.btf` address decodes to, so the sender is trusted for nothing -- forging an entry needs somebody else's secret key. That leaves flooding with self-generated keys, which the per-message caps and the "peers that answered" ordering are there to blunt. Unknown commands are ignored (main.cpp), so nodes without this just drop the message; nothing here is a protocol break. The cache from the peer-cache change now keeps each peer's descriptor verbatim alongside the parsed fields, because that signature is what makes a peer relayable. Entries without one -- resolved through Nostr, or written before this -- stay dialable, just not passed on. A dial never clears a descriptor already on file.
net.cpp has said nListenPort is "tunable via /port" since it was written, but nothing ever parsed the option. Two nodes could not share a machine, which is exactly what testing peer exchange needs. Sets addrLocalHost.port too, so we do not advertise a port we never bound.
356cd57 to
316c5fb
Compare
|
For the record, since this PR's description does not match what it merged. It was reviewed as one file ( The peer cache itself is what the numbers above measured and is unaffected. The peer exchange had not been validated — its two-node test never completed. #27 removes it from main; it will come back as its own PR once it passes. Release v1.2.2 was built from 3caee27, before this merge, so nothing published is affected. |
PR #25 was opened, reviewed and approved as a single-file change to net.cpp: the .btf peer cache, +169/-0. I then pushed peer exchange and a /port option onto that same branch while the PR was open, which silently widened it to six files. The merge carried all of it in. Nothing about the peer exchange had been validated. Its two-node test never completed -- node B could not dial node A at all -- so what landed on main is unproven network protocol code that no one agreed to take. This restores main to what #25 actually described. The peer cache stays, with its runtime numbers (36.3s to 2.1s to first peer). #26 stays. What comes out: - the "btfpeers" message, both directions - BtfPexCollect / BtfPexAccept and the per-message caps - CNode::nLastPexRecv - the `desc` field on the cached peer record - BtfLocalDescriptor / BtfSecpContext - /port Peer exchange returns as its own pull request once its test passes. The /port option is worth having on its own -- two nodes cannot share a machine without it, which is what blocked the test -- and should go up separately rather than ride along again. Release v1.2.2 is unaffected: it was built from 3caee27, before either merge.
…sh-sh#25) Discovery otherwise depends entirely on the Nostr relays -- a node that's been up for hours and knows exactly which peers respond throws all of it away on exit, and the next start walks the same mostly-dead descriptor list again. Now persists (address, meeting node, encryption key) for peers that actually connected, and a new thread dials up to 8 of them before relay discovery has said anything. Upstream measured this as 36.3s to 2.1s to first peer. Ported only the part of Bitflash-sh#25 that's still actually on their main: the peer-exchange protocol and desc-field additions that rode along on the same PR were reverted by them shortly after (Bitflash-sh#27, merged unreviewed, its own two-node test never completed) -- not carrying that part over. main_gui: implement the port command-line option, which net.cpp already documented (Bitflash-sh#28) nListenPort was described as tunable via a command-line option since it was written; nothing ever parsed it, so a second node couldn't start on a machine already running one. Sets addrLocalHost.port too, or the node announces a port it never bound. util: stop debug.log dropping lines under concurrency (Bitflash-sh#29) strDebugFile's path resolution was outside any lock while the write itself (plus our rotation logic, added independently) was already inside one -- so one thread could read the path while another was still assigning it. On a race, fopen got a torn path, returned NULL, and the line was dropped silently: a missing line then reads as "this thread never ran," which is unsound. Path resolution now shares the same lock as the write/rotate. nostr: cap reassembled WebSocket message size (rest of Bitflash-sh#17) Only fixed the j[0]-type-check half of this earlier and dismissed the frame-size half as moot post-libwebsockets -- that was wrong. lws bounds a single fragment, but m_curFrame reassembly across fragments had no ceiling of its own, so a relay sending one message in many small fragments could still make us allocate unbounded memory. Capped at 4 MiB, matching the number used in the original hand-rolled-reader fix this project shipped once already.
Everything since 1.2.2 is about nodes being able to find and reach each other, which is the problem this project actually had: #30 rendezvous registration and waiting split apart. Registering used to block until somebody dialled, so a node could not advertise a meeting node until it had already been reached at one. This is the root cause of the flaky connectivity, not dead relays or stale descriptors. #26 a node no longer advertises a rendezvous it is not registered at #25 .btf peers that answered are remembered and dialled first (36.3s -> 2.1s) #32 peers exchange signed descriptors, so discovery survives a relay outage #29 debug.log stopped dropping lines under concurrency #28 /port, so two nodes can share a machine
mapAddressBook and CWalletDB::WriteName/ReadName/EraseName already existed -- leftover from the Bitcoin 0.1 codebase this forked from -- but nothing exposed a way to actually use it. Added AddressBookName()/ SetAddressBookName() as a plain API, and a "More" window (View menu) with three sections: - Connected Peers: the live vNodes list, with each entry's .btf address (outbound only -- an inbound connection doesn't tell us its own .btf address as part of the handshake, nothing to show there yet) and an inline rename control. - Cached Peers: everything from the persistent peer-cache file (Bitflash-sh#25), same rename control, so a peer you've dealt with before but aren't currently connected to can still be named. - Address Book: the full mapAddressBook listing, plus a manual add-any-address-with-a-name form -- for payout addresses, worker addresses, or anything else that isn't a live peer at all. One shared DrawBookNameRow control across all three, and also wired into the operator's existing "Current Round Workers" table (both the payout-address and worker-name columns), so a name saved anywhere shows up everywhere that address appears -- exactly the "rename workers, save friend addresses, cached peers use the book name if there is one" behavior asked for. net.h/net.cpp: added CNode::strBtfAddr (populated on outbound connect, empty for inbound) so the peers panel has something to show besides a raw transport address. Also exposed CachedBtfPeer/LoadCachedBtfPeers via net.h instead of file-local in net.cpp, since the GUI needs them now too.
…-sh#32) Discovery has depended entirely on the Nostr relays. Two connected nodes never told each other who else existed, so a relay outage left a running network unable to grow, and a node that found one stale peer had no way to learn there was anything better. Peers now trade signed descriptors over a new "btfpeers" message: our own first, then peers that actually answered us. The receiver checks each Schnorr signature against the key the .btf address decodes to, so the sender is trusted for nothing -- forging an entry needs somebody else's secret key. Per-message caps (MAX_PEX_DESCRIPTORS, MAX_PEX_DESCRIPTOR_BYTES) and a per-connection rate limit (PEX_MIN_INTERVAL via CNode::nLastPexRecv) bound flooding with self-generated keys, which signatures alone don't prevent. Unknown commands are already ignored, so a node without this just drops the message -- not a protocol break. Adapted rather than cherry-picked, same as the other Bitflash-sh#25/Bitflash-sh#28/Bitflash-sh#29/Bitflash-sh#30 ports: our net.cpp diverged (CachedBtfPeer/RememberBtfPeer already exist here from the peer-cache port, and use our own BtfPeerCacheHexToBytes/BytesToHex naming rather than generic HexToBytes/BytesToHex to avoid symbol collisions elsewhere in this codebase). CachedBtfPeer gains a `desc` field -- the peer's own descriptor kept verbatim, since that signature is what makes a peer relayable to someone else. RememberBtfPeer takes it as an optional arg and never lets a plain dial (empty desc) clear one already on file, so learning a peer through exchange doesn't get undone the next time we happen to also reach them directly. nostr.cpp gains BtfLocalDescriptor() (reads strBtfActiveRelay directly, not through BtfActiveRelay()'s fallback -- only a rendezvous we've actually confirmed registration at is worth handing to a peer) and BtfSecpContext() (so net.cpp can verify without pulling secp256k1 into net.h).
Attacks the seven-minute wait before a node finds its first peer, which we measured repeatedly today.
Why it takes so long
Discovery is entirely dependent on the Nostr relays, and the P2P protocol carries no peer information at all —
version,block,tx,inv,getblocks,getdataand nothing else. Two nodes can be connected and syncing without ever telling each other that anyone else exists.So every start repeats the same sequence: reach the relays, pull down ~270 descriptors, dial them blindly, and wait while roughly 96% fail because the peers behind them are long gone.
A node that has been running for hours knows exactly which addresses answer. It throws all of it away on exit.
What this does
Every peer that completes a tunnel is written to
btfpeers.jsonalong with its meeting node and encryption key — everythingConnectNodeBtfResolvedneeds to redial it without touching a relay. On start, a thread tries those first, stopping once eight are connected.Entries expire after a week and the file holds fifty at most, so a node that has been off for a long time still falls back to discovery rather than hammering addresses that have certainly moved.
Scope
Cache only. A first-ever run is unchanged, and a missing or unparseable file just falls through to the relays. Nothing here is consensus.
What it does not fix
The dependency itself. If the relays are unreachable — and today's logs show
relay.damus.iorefusing handshakes andrelay.nostr.bandfailing outright — a node with an empty cache still cannot find anyone.The real answer is peer exchange over the existing P2P link. A
.btfaddress reveals no IP, being a hash of a public key, so gossiping them is safer than Bitcoin'saddrmessages ever were. After first contact the network would sustain itself, with the relays as bootstrap rather than lifeline. That is a bigger change and wants its own PR.Not built yet — needs
make windows.