Pool fixed - #21
Conversation
…ort forwarding needed
…icipant Stratum client
Wrong target, previous commit: 'make windows' never calls build.sh at all -- it goes deps-windows (root Makefile) -> cd src && make -f makefile.mingw directly. build.sh is an orphaned script with a hardcoded personal path (/c/Users/ti/Bitflash/src) that isn't part of the real build chain; deleted it along with the pacman step I'd added to it, since that step was in the wrong file. Added libwebsockets to both real dependency lists instead: - deps-windows (root Makefile): mingw-w64-ucrt-x86_64-libwebsockets - deps-apt (root Makefile, Linux path): libwebsockets-dev Fixed src/readme.txt, which documented the dead build.sh as the Windows build instructions -- now points at 'make windows' from the repo root, the path that actually runs. secp256k1 and RandomX remain untouched in both dep lists -- deps-windows and deps-linux already build those from source themselves (into ../deps and ../RandomX respectively), so nothing needed there. Unverified: still no MSYS2/Windows environment available here to run 'make windows' end-to-end. Package names confirmed against the official MSYS2 index; the full dependency chain (pacman install -> secp256k1 build -> RandomX build -> make -f makefile.mingw -> zip) has not been exercised.
…ay code Hardening (src/btfrv.cpp), addressing all five original TODO items: - 10s read timeout on the 33-byte pairing header. Previously unbounded -- a peer that connects and never sends anything tied up a thread forever. - Registration expiry: tuned TCP keepalive (60s idle / 10s interval / 3 probes -- Linux's 2-hour default is useless here) on parked registrations, plus a 15-minute TTL reaper as a backstop for whatever keepalive still misses. Long/infrequent by design: a legitimately idle registration (e.g. a pool between miners) should be able to wait far longer than this without getting reaped. - Connection cap (4096 concurrent) via an atomic counter checked at accept() time, closing the socket immediately rather than spawning a thread once at capacity. - Backoff on accept() errors (100ms, 1s after 10 consecutive) instead of a tight busy-loop -- important specifically because fd exhaustion (the most likely cause of repeated accept() failures) gets worse, not better, if the response is spinning CPU at 100% retrying immediately. - Logging added throughout: connects, pairs, rejects, cap hits, reaper sweeps. Plus two related findings from auditing the client-side protocol (btfrv.cpp/btftunnel.h) alongside this: - Per-pubkey cap (8) on queued registrations. Registration has zero authentication -- anyone can register as 'S' for any pubkey -- so without this, flooding registrations for a target's pubkey can starve their real registrations out of the FIFO queue indefinitely. Doesn't fix the missing auth (that's a bigger design question -- the echan layer already prevents an imposter from reading real traffic, per the existing header comment), just bounds the blast radius of the griefing vector it enables. - btftunnel.cpp: BtfServiceWrap()'s read of the client's handshake ephemeral pubkey had the same unbounded-wait bug as the header read above -- a peer that pairs (trivial, given no registration auth) and then never sends its handshake tied up that thread forever too. Same 10s-timeout-then-clear pattern applied. Verified: full node build clean; test_btfrv.cpp (register -> pair -> encrypted round-trip -> reject-unknown-pubkey) passes repeatedly against the hardened code. Confirmed a pre-existing shutdown-hang quirk in this sandbox reproduces identically on unmodified code, ruling it out as a regression. test_btftunnel.cpp couldn't be built here (unconditional winsock2.h include, pre-existing Windows-only limitation in that test file, unrelated to this change) -- validated the btftunnel.cpp fix by review and clean compilation instead. Deduplication: deleted relay/btfrv.cpp and relay/btfrv.h. Per git log, both were forked from src/btfrv.cpp exactly once, in the initial fair-launch release commit, and never touched again -- meanwhile src/btfrv.cpp received four real fixes since (372530e multi-registration FIFO fix, 0a67bba socket timeout, b80cb56 idle-disconnect fix, c9aed6b bind fix) that never made it into the relay/ copy. Whatever's actually deployed on the VPS via relay/ has been running unpatched code missing those four fixes on top of never getting this hardening pass. relay/build.sh now builds directly against ../src/btfrv.cpp -- single source of truth, verified this compiles and runs. relay/install-bitflash-relay.sh (the one-shot curl-installer) no longer embeds a static heredoc copy of the source, since embedding a static copy is exactly what caused the staleness bug above -- it now curls the canonical src/btfrv.h, src/btfrv.cpp, and relay/bitflashd.cpp from the repo at install time, so it structurally can't go stale the same way again. Added a curl availability check matching the existing g++ one. Both scripts pass bash -n.
error() was printing unconditionally on every single call, completely bypassing the burst-collapse dedup LogPrint already had (and was specifically widened to 5 minutes for -- see 8ce24a4). This is the actual root cause of the debug.log noise reported: relay connect failures go through 'return error(...)', so every retry in every discovery cycle printed its own near-identical line with zero collapsing. A short real run here went from dozens of near-duplicate 'ERROR: Nostr: TLS handshake with X failed' lines down to one, with a periodic '(N similar in 300s, last: ...)' summary covering the rest -- same trade-off already accepted for LogPrint on this exact problem shape, now applied consistently to error() too. error() is now a macro (error(...) -> ErrorImpl(__FILE__, __LINE__, ...)), the same pattern as the LogPrint macro, since ErrorImpl needs the caller's call site to key the dedup and a ~100-call-site plain function can't see that on its own. Checked every call site in src/ for macro-collision risk first (bare 'error' identifiers, .error() member calls) -- found one, a lambda parameter named 'error' in rpc.cpp, which is safe since it's never followed by '(' the way the macro requires to expand. Unlike LogPrint, error() is never gated by fDebug/LogAcceptsCategory -- an error is always worth seeing at least once; only true repeats from the same call site collapse. Extracted the dedup bookkeeping (window/suppressed-count/last-message tracking) into a shared DedupDecide() so LogPrintDedup and ErrorImpl share the exact same collapse logic and only differ in output formatting (category bracket vs 'ERROR:' prefix). Verified: full build clean; a live run reproducing the original all-relays-unreachable scenario collapsed 'ERROR: Nostr: TLS handshake' spam from dozens of lines to one, confirmed via direct log inspection.
… nodes Instead of sending a peer the full set of transactions for a block -- almost all of which they already have in their mempool from normal tx relay -- send a 48-bit keyed short ID per transaction instead. A peer that already has every referenced transaction reconstructs the full block locally without any tx bytes crossing the wire; only genuinely missing ones (in this implementation, only ever the coinbase is prefilled, everything else is a short ID) need a follow-up round trip. Backward compatibility is the main design constraint, not an afterthought: - Old, unupgraded nodes are completely unaffected. ProcessMessage already ignores unknown commands (confirmed by reading it, not assumed), so they simply never see sendcmpct/cmpctblock/ getblocktxn/blocktxn. They keep receiving today's full block message exactly as before -- that handler is untouched. - A new node still fully understands and accepts full block messages from anyone, old or new -- nothing about receiving is gated behind compact-block support. - The compact form is only ever sent to a peer that has itself first sent sendcmpct (right after the version handshake) -- capability is announced per-peer, not assumed. Old nodes never send it, so they never trigger the new code path on the sending side either. - No coordinated network upgrade needed: this rolls out node-by-node, and any two upgraded nodes automatically get the bandwidth win the moment they connect to each other. Scoped down from full BIP152 for this chain's scale/age: no high-bandwidth (unsolicited push) mode, only the current chain tip is ever offered in compact form (an older block is unlikely to still overlap much with any peer's present-day mempool, so full block is sent for those, same as today), and light/fClient peers are excluded entirely (they keep no mempool to resolve short IDs against). New files: compactblock.h/.cpp -- CCmpctBlock, short-ID computation (SHA256-derived per-block key + libsodium's SipHash-2-4, so IDs can't be precomputed before a block exists and differ block-to-block for the same tx), BuildCompactBlock, and mempool-based reconstruction (TryReconstructBlock/FillMissingAndFinish) with explicit 48-bit short-ID collision handling (falls back to requesting both candidates rather than risking a wrong match -- correct either way, only occasionally not optimal). net.h/net.cpp: CNode gets fWantsCmpctBlock plus pending-reconstruction state for an in-flight cmpctblock awaiting getblocktxn. The pending block is a raw CBlock* (not a member) because net.h is included before main.h in this codebase's header chain, so CBlock isn't a complete type there -- deletion happens in an out-of-line ~CNode() in net.cpp instead, where it is. main.cpp: sendcmpct sent after processing a peer's version (skipped for fClient peers); getdata's MSG_BLOCK branch sends cmpctblock instead of block only when the peer opted in and the block is the current tip; three new handlers (cmpctblock/getblocktxn/blocktxn) mirroring the existing inv/getdata/block request-response shape already in this file. Verified: full node build clean (Linux); node starts and runs normally. Wrote test_compactblock.cpp, a standalone unit test (not wired into the P2P layer, which would need the rendezvous relay in the loop to test end-to-end now that the direct-IP listener is gone) -- 17 checks, all passing: full-mempool reconstruction produces a byte-identical block hash to the original; partial-mempool reconstruction correctly reports missing indices and completes via the getblocktxn/blocktxn round trip; a mismatched blocktxn response is correctly rejected rather than silently corrupting the block; and short IDs for the same transaction differ across different simulated blocks and nonces, confirming the per-block keying actually works rather than degenerating to a txid-only hash.
nostr.cpp requires cacert.pem next to the exe for TLS on Windows (no system trust store access via OpenSSL there). The zip target built the exe but never copied the CA bundle alongside it.
…rsion - Root VERSION file replaces the hardcoded 'VERSION := 1.2.2' in the Makefile - Both linux and windows sub-makes now receive VERSION and define BITFLASH_APP_VERSION at compile time, so the running binary knows its own version (needed for the upcoming auto-update check).
- update.h/update.cpp: checks releases/latest on Vic-Nas/bitflash-mod (UPDATE_REPO, fixed at build time), compares dotted version numbers, downloads platform asset via libcurl, replaces the running binary (rename() on Linux AppImage, detached cmd helper + move on Windows since it can't self-overwrite), relaunches. - Makefiles: link libcurl, add -DUPDATE_REPO, add curl to apt/pacman deps. - .github/workflows/release.yml: on push to main touching VERSION (or manual dispatch), builds Linux AppImage + Windows zip in parallel, generates a short changelog (top commits by diff size since last tag), publishes one GitHub Release with both assets.
… in nostr.cpp last time)
… 'Bitflash'; fix missing pillow dep for AppImage icon step
|
Thanks for this, and sorry for the slow review — there is a lot of real work in The headline number is misleading: 84 files and +73k lines, but most of that is Two things block it as it stands. Neither is a criticism of the code you wrote. 1. It would revert the 1.2.2 hardeningYou mentioned merging the security fixes, and the 1.2.1 ones are all here — I The 1.2.2 set is not, and the timing explains why — you opened this at 18:09 and
The last one is the problem. 2.
|
A transaction naming a parent the node does not have is never validated, just stored in mapOrphanTransactions -- and there was no ceiling. Sending them costs a peer nothing: no proof of work, no valid signature, no fee, and the node keeps every one. Any peer could grow that map until the node died of memory exhaustion. Cap at 100, evicting at random rather than in map order so an attacker cannot keep its own orphans resident by choosing hashes that sort low. Local policy, not consensus -- no effect on block or transaction validity.
…itflash-sh#18) The vector unserializer grows in 5 MB steps so a bogus length cannot make it allocate more than has actually been received. The string one resized straight to whatever ReadCompactSize returned, and ReadCompactSize has no ceiling -- nine bytes on the wire can name four gigabytes. Not reachable from the network today: the only string on a serialized type is CWalletTx::mapValue, and RelayMessage casts to CTransaction before sending, so it never leaves disk. This is a guard against the next network message that carries a string -- Bitcoin's own version message carries strSubVer -- rather than a fix for a live hole.
Everything an unauthenticated socket can reach in the pool server, in one
pass.
A line that never ends. RecvLine appended to the per-miner buffer and
looked for a newline; a peer that sends bytes and no newline grew that
buffer for as long as it kept writing, before authorizing or subscribing.
Capped at 64 KB, which drops the miner.
No limit on connections. Every accepted socket became a Miner with its
own buffer and nothing bounded the list. Capped at 1000.
Malformed requests crashed the node, not just the pool. ThreadRPCServer
has no exception handler, so anything thrown from line handling reached
the top of the thread and terminated the process. Three ways in, all
valid JSON that parses cleanly:
[1,2,3] -- value() on a non-object throws
{"method": 1} -- value() with the wrong type throws
submit with params [1,2,3] -- get<std::string>() on a number throws
mining.authorize had the subtler shape of the same thing: it tested
p.size() without testing p.is_array(), and indexing a json object with a
number throws while size() happily returns its key count.
Each is now rejected on its own terms, and the call is wrapped so a throw
drops that miner instead of the node.
MAX_SIZE bounds a block's bytes, not what it costs to check. Every OP_CHECKSIG is an elliptic-curve verification, and blocks may be 32 MB here, so one packed with signature operations takes every node minutes of CPU to reject -- while at this chain's difficulty it stays cheap to mine. There was no counter at all. CheckBlock now sums signature operations across the block and rejects past 20000, the figure Bitcoin settled on. OP_CHECKMULTISIG counts as 20 since it may verify that many. Consensus change. Real blocks here carry one or two operations each -- coinbase pay-to-pubkey and standard pay-to-pubkey-hash spends -- so the ceiling is four orders of magnitude above anything in the chain, but it still needs verifying against the full history before merge.
…ling; sign release assets in CI Addresses the gap flagged in the bitflash-sh PR review: the updater previously only checked that the download was non-empty. It now refuses to install anything not signed by the release key. Also fixes the same unchecked-j[0] JSON crash pattern in nostr.cpp that was fixed upstream in Bitflash-sh#17 (the frame-size-cap part of that fix doesn't apply here since our nostr.cpp uses libwebsockets, not a hand-rolled reader).
|
That was fast, thank you. I reviewed the code rather than the commit messages, Consensus: The updater: this is a real fix, not a gesture. Compiled-in 32-byte Ed25519 Three things left. 1. The signing key has to be this project's
Nothing about that is aimed at you — it is simply the wrong place for the key to So: I will generate a keypair for the project, hold the private half as a secret 2. When it gets wired up, it must ask every timeThe updater is not called from anywhere yet, so how it surfaces is still open. A signed auto-updater on a consensus system is a channel for changing the rules on 3. Two smaller things
The offer to split this into separate PRs still stands and I still think it would |
CURLOPT_CAINFO was a bare relative path at all three call sites in update.cpp. Fine when launched from its own folder, silently broken (fails closed, TLS just never connects) from anywhere else.
|
Here is the project's update signing key. Replacing the constant in // Ed25519 public key that release signatures are checked against. The
// matching private key never touches this repo; it lives only as a CI
// secret and is used to sign each release asset (see release.yml).
const unsigned char kUpdatePubKey[32] = {
0x73, 0x59, 0x32, 0xc7, 0x06, 0x69, 0x3b, 0xb4, 0x0f, 0x3b, 0x3c, 0x3a,
0xd7, 0x41, 0x3e, 0x3f, 0x69, 0x73, 0xb8, 0x6d, 0x6a, 0xc5, 0x63, 0xd1,
0x60, 0xff, 0x6a, 0x6d, 0x43, 0x28, 0xb2, 0xb2
};The private half is an Your comment above it can stay exactly as written; it describes the arrangement Still open from my last review, all minor next to what you have already done:
Two things landed on main meanwhile that will touch your rebase, both small:
|
|
Correction to my comment above — you fixed the |
Checks on launch and via File > Check for Updates, both in the background. Never applies automatically -- Update_ApplyAndRelaunch is only ever called from the dialog's 'Update Now' button, once per prompt, exactly the check/apply split the header already documented. Skip just dismisses; the check/apply functions stay decoupled so nothing can wire them together silently later.
|
Saw you closed this — no problem, and no need to explain. Just so it is said plainly: the invitation stands, with no deadline and no Either way, today was good work. You closed a consensus revert and built proper |
The pool system is fragile.
I made a few fixes but there r still some connexions problems.
I merged your security fixes