fix(wallet): bound a reservation deadline by the observing clock, not only by the first push's - #528
Conversation
Salvage anchor for the dig-node#502 lane. Version assigned 0.251.0 (origin/main is 0.247.0; 0.248-0.250 are held by sibling lanes). Co-Authored-By: Claude <noreply@anthropic.com>
Four failing db-layer tests for dig-node#502, plus the constant they measure against. `MAX_RESERVATION_HOLD_MS` is defined as a multiple of `RESERVATION_TTL_MS` in one place so the two cannot drift; the TTL itself is unchanged. The acceptance test steps by less than a TTL past the cap and asserts its own iteration count: a one- or two-push fixture is satisfied by the unfixed code, because the first hold has not lapsed yet. Co-Authored-By: Claude <noreply@anthropic.com>
…nputs `reserve_spend` re-armed `expires_at` to `now + RESERVATION_TTL_MS` on every push of a given transaction id, so a caller re-pushing the same signed bundle more often than the TTL renewed its hold forever and the inputs never returned. That is the lockout failure the TTL's own doc names as the worse of the two, reachable without a single dishonest answer. Two composed bounds, neither of which shortens the TTL: * A TOTAL cap anchored on the FIRST push. `MAX_RESERVATION_HOLD_MS` is defined as `6 * RESERVATION_TTL_MS` in one place, so lengthening the TTL scales the cap and the two cannot drift. `submitted_at` is not in the upsert's `DO UPDATE SET` list, so the stored value is a stable anchor; a test pins it. * A reason-conditional re-arm. `chain::refusal_forecloses_a_later_push` names the four CLVM-execution / cost refusals that complain about the bundle's own contents and that no better-synced node can turn into an acceptance. Those still HOLD -- the verdict is height-dependent, so this crate declines to trust one node's view of it -- but they may not RENEW the hold. Everything else extends, including an unrecognised reason, an `Err`, and a bare verdict. The two compose into the gate's "every observed refusal was foreclosing" case without a per-attempt history, because a re-push may never move the deadline EARLIER: if every attempt is non-extending the deadline never leaves the first `submitted_at + TTL`, and an extending attempt's grant survives every later one. The clamp lives in the SQL so it is atomic against the stored anchor; a read-then-write above this layer would race two concurrent pushes. `coin_reservations`' `ON CONFLICT(coin_id) DO NOTHING` first-claim-wins rule is untouched and pinned by a test. Closes #502 Co-Authored-By: Claude <noreply@anthropic.com>
…clamp The adversarial gate on #505 refuted the reason-conditional half and it is removed in full: VIEW_DEPENDENT_BUNDLE_CONTENT_REFUSALS, refusal_forecloses_a_later_push, attempt_may_extend_the_hold, the PendingTransactionRow::may_extend_expiry field and the CASE arm in reserve_spend. The four names it listed -- GENERATOR_RUNTIME_ERROR, BLOCK_COST_EXCEEDS_MAX, INVALID_BLOCK_COST, INVALID_SPEND_BUNDLE -- do not identify a bundle no destination is holding. push_tx relays to up to three destinations and only the LAST answer returns, so such a refusal from the last says nothing about the first, which may have admitted and gossiped the bundle. The removed code freed inputs up to 550s earlier than main for a bundle that lands, with no attacker: the double-spend direction #497 exists to close. What ships is the clamp alone: expires_at is bounded by submitted_at + 6 * RESERVATION_TTL_MS. The outer MAX is retained because a non-monotonic clock is the one case that can still drive an incoming deadline below a live one, and shortening a live hold is the dangerous direction. SPEC.md 18.9a gains the total-hold bound as a normative clause: without it a reimplementation built from the spec as written reproduces the unbounded re-arm this change fixes. Three limitations are now stated in MAX_RESERVATION_HOLD_MS' doc and two are pinned by tests: the bound is on CONTINUOUS hold and a re-push after the prune gets a fresh anchor; a bundle whose timelock matures past the cap has its inputs freed while the network genuinely still holds it; and inside the last TTL before the cap a re-push buys strictly less than a full TTL. The clamp also fails OPEN under an absurd clock, since SQLite promotes integer overflow to REAL. Refs #502 Co-Authored-By: Claude <noreply@anthropic.com>
…total-hold # Conflicts: # Cargo.lock # Cargo.toml
…total-hold # Conflicts: # Cargo.lock # Cargo.toml
cargo fmt wanted the multi-line call form at both boundary tests. Formatted those two files only; the workspace-wide check is now clean at zero diffs. Co-Authored-By: Claude <noreply@anthropic.com>
The comment sat on `reserved_coin_ids`, which IS assembled from a stored table and has no `true`. It described `may_extend_expiry`, the bool this branch removed in fce2358, and pointed at a field doc that no longer exists. Also tighten SPEC 18.9a: a re-push does not unconditionally 'update the deadline' -- at or past the cap, and under a backwards clock, it correctly leaves the deadline unchanged. State the re-arm as subject to the bound. Co-Authored-By: Claude <noreply@anthropic.com>
…total-hold # Conflicts: # Cargo.lock # Cargo.toml
…total-hold # Conflicts: # Cargo.lock # Cargo.toml
…tion freeze) Version anchor only. The fix follows: a far-forward clock at a bundle's FIRST push writes a `submitted_at` far in the future, and #505's outer `MAX` then pins `expires_at` there permanently, so no later correct-clock push and no prune can ever release the coins. Refs #525 Co-Authored-By: Claude <noreply@anthropic.com>
#525) `reserve_pushed_bundle` reads the clock once and writes both `submitted_at` and `expires_at` from that reading, so a single reading far in the future stores a deadline decades out. Nothing could retire it: `prune_reservations` deletes on `expires_at <= now`, which never arrives, and #502's upsert clamp is `MAX(stored, ...)`, so a later push under a corrected clock leaves the stored deadline alone. The coin was withheld from selection for ever and `reset_chain_cache` refused while the row existed. `prune_reservations` now repairs at OBSERVATION: a row whose deadline exceeds `now + MAX_RESERVATION_HOLD_MS` contradicts its own columns against the clock (an honest row satisfies `expires_at <= submitted_at + CAP` and `submitted_at <= now`), so it is re-anchored to `now` and granted one fresh `RESERVATION_TTL_MS`. `submitted_at` moves too, or #502's cap clause would stop binding on that row for ever. The client hold table gets the same repair, keyed on the SAME threshold so a five-minute backwards step cannot re-clamp a healthy hold, and granted its own ceiling since its requested TTL is unrecoverable. All four statements now share one write-first transaction. `reset_coin_db` prunes first, like every other reservation-sensitive entry point, so the refusal message telling a user to wait becomes true. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
loop-security interim — IN PROGRESS, not the verdictAudited head: Q1 — clock provenance on every
|
| site | how now_ms is obtained |
verdict |
|---|---|---|
sage/rpc.rs:875 (reservations_held) |
let now_ms = custody::now_ms() as i64 local |
node clock |
sage/rpc.rs:893 (reserve_coins) |
let now_ms = custody::now_ms() as i64 local |
node clock |
sage/rpc.rs:1205 (reset_coin_db, NEW) |
parameter now_ms: i64 — see below |
node clock at the only caller |
sage/rpc.rs:2969 (get_pending_transactions) |
custody::now_ms() inline |
node clock |
sage/rpc.rs:3236 (spendable_coins) |
custody::now_ms() inline |
node clock |
sage/rpc.rs:3703 (CAT send selection) |
custody::now_ms() inline |
node clock |
The new site is the only one worth argument, because reset_coin_db(&self, now_ms: i64) is pub and takes the instant as an argument. Its sole production caller is dig-node-service/src/control.rs:2612, and that handler computes the instant itself at control.rs:2603-2609 from SystemTime::now().duration_since(UNIX_EPOCH) — never from params. The reasoning the brief asked me to check for is present and explicit at control.rs:2600-2602:
The node's own clock. A caller-supplied instant would be a lapse oracle: a far-future value makes every live spend reservation read as expired, which is exactly the guard being asked to stand down.
So the lapse-oracle reasoning documented on held_reservations was applied on this path, including the new one. custody::now_ms() (sage/custody.rs:441) is a plain SystemTime::now() read with no caller input.
No remote reservation-release primitive exists via the clock. A caller cannot pass an instant that deletes holds, and cannot pass one that trips the repair.
Still working: repair weaponisation without the clock (Q2), DoS/index/row-bounds (Q3), transaction+SQLITE_BUSY→Unavailable (Q4), saturation (Q5), the anchor-move vs #502 (Q6), reset_chain_cache ordering (Q7), and the submitted_at truthfulness question (Q8) — which is the one I expect to take longest.
MichaelTaylor3d
left a comment
There was a problem hiding this comment.
Correctness gate — PASS
Head reviewed: 2edd6241ca170aa2382d75f5a03e6610646cd502 (resolved from gh pr view 528 --json headRefOid; matches the brief).
Diff reviewed: git diff 15341ee..HEAD — SPEC.md, crates/dig-wallet/src/sage/db.rs, crates/dig-wallet/src/sage/rpc.rs.
Measured myself: cargo test -p dig-wallet --lib -> 800 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out, exit 0. Matches the brief's head figure. Non-zero test count, so this is a real run and not a filtered-to-nothing green.
1. The soundness argument is TRUE
Every production writer of pending_transactions.submitted_at / .expires_at was enumerated:
- INSERT arm,
db.rs:2801— the only production caller isreserve_pushed_bundle(rpc.rs:2350, viareserve_spend; every otherreserve_spend(hit in the crate is a test). It writessubmitted_at = now_candexpires_at = now_c + RESERVATION_TTL_MS, soexpires_at = submitted_at + TTL <= submitted_at + CAP. DO UPDATEarm,db.rs:2805-2808—MAX(stored.expires_at, MIN(excluded.expires_at, stored.submitted_at + CAP)).submitted_atis not inSET, and bothMAXoperands are<= submitted_at + CAPby induction. Invariant preserved.- The new repair
UPDATE,db.rs:2924— writessubmitted_at = now,expires_at = now + TTL. Invariant preserved, and idempotent: a repaired row can never re-trip the detector.
No other statement in the crate writes either column. So expires_at <= submitted_at + CAP holds for every reachable row, and expires_at > now + CAP implies submitted_at > now. The doc comment's claim is correct. No false-positive class from a healthy writer.
2. The 50-minute floor is CONFIRMED (the shorthand formula is conservative, not wrong)
With E = expires_at - submitted_at = min(e_p + TTL, CAP), where e_p is the row age at its last push, the detector fires iff now < submitted_at + E - CAP. For a backwards step d on a row of true age e, that is exactly
d > e + max(CAP - e_p - TTL, 0)
- Never-re-pushed row (
e_p = 0,E = TTL): thresholde + 50 min. - At-cap row (
E = CAP): thresholde, ande >= e_p >= CAP - TTL = 50 min.
The infimum over all reachable rows is CAP - TTL = 50 minutes, attained as e -> 0. The load-bearing number in the money argument is correct, and the earlier draft's "1 ms step trips an at-cap row" is refuted: an at-cap row needs d > e >= 50 min.
The comment's shorthand d > max(CAP - TTL, e) is a lower bound on the true threshold (exact in the at-cap case, understated for a young never-re-pushed row) — conservative in the safe direction. Non-gating; note (a) below.
3. The client-hold asymmetry is correct and correctly guarded
Healthy client row: expires_at_ms = now_w + ttl, with ttl clamped to <= CLIENT_RESERVATION_MAX_TTL_MS in reserve_client_coins. A detector at now + CAP therefore needs now_w > now + CAP - ttl >= now + 50 min — the same 50-minute floor as the bundle table, which is exactly the justification given. Keying on the table's own 10-minute ceiling would indeed have produced a ~5-minute false-fire floor.
The const _: () = assert!(CLIENT_RESERVATION_MAX_TTL_MS <= MAX_RESERVATION_HOLD_MS) at db.rs:4440 prevents precisely the unsoundness it names: raise the client ceiling above CAP and healthy client rows become detectable. The guard is load-bearing and correctly stated.
Granting the ceiling rather than the (unrecoverable) requested TTL errs long; for a build-window hold that is the safe direction, as argued. Post-repair expires = now + CLIENT_MAX <= now + CAP, so the repair is idempotent here too.
4. The transaction change is sound
- Return value is still
Ok(n)wherenisrows_affected()of the firstDELETEonly (db.rs:2958-2969,:2991). The clientDELETEresult is discarded, unchanged. Read, not assumed. - A repaired row cannot be deleted by the
DELETEbelow it: after repairexpires_at = now + TTL > now, soexpires_at <= ?nowis false. It can still be deleted by the spent-coin cascade arm, which is correct. - Write-first ordering:
self.pool.begin()is DEFERRED, and the first statement in the transaction is now anUPDATE, so the write lock is taken before anything else.prune_reservationscontains no reads at all, so the read-then-write shapereserve_client_coinswarns about atdb.rs:4615-4619is unreachable here. - The adjacent risk the brief did not raise:
reserve_coins(rpc.rs:893) propagates prune'ssqlx::ErrorthroughFrom<sqlx::Error> for ReserveClientCoinsError(db.rs:4567) intoUnavailable. So prune's lock window does feed that mapping — but it did before this PR too (two autocommit writes), sqlx's defaultbusy_timeoutis 5 s, and four statements in one transaction is one lock acquisition instead of two. No regression.
5. The tests assert at the decision, and the corrected one is right
a_reservation_anchored_to_a_far_future_clock_stops_freezing_its_coin_for_everasserts onunreserved_unspent_coins— the selection surface, not a raw row count. With the repair reverted,expires_atstays decades out, theDELETEnever fires, and the second assertion goes red. Not vacuous.- Its start state (
submitted_at = F,expires_at = F + TTL) is exactly whatreserve_pushed_bundlewrites from one clock reading. Reachable. - The boundary test's
at-boundrow (expires_at = submitted_at + CAP) is reachable via theDO UPDATEclamp, which leavessubmitted_atalone and pinsexpires_atatsubmitted_at + CAP.past-boundis deliberately unreachable and is the negative probe. Correct pairing. - The #505 revert-proof correction (
a_repush_under_a_clock_that_stepped_backwards_never_shortens_a_live_hold) is right: moving both columns bystep_backis the only production-reachable shape, sincereserve_pushed_bundlereads the clock once.step_back = 60_000is far under the 50-minute floor, so the repair provably cannot fire and the test still exercises the clamp alone — its original assertion (expires_atunchanged) still holds for the now-reachable state.
6. Mutation claim — analytically confirmed, NOT executed
I did not run the mutation, and say so rather than implying otherwise: the tree is shared read-only and cutting a worktree for one compile was not worth the spend. Analytically, dropping SET submitted_at = ?now leaves submitted_at = F, so MIN(late + TTL, F + CAP) = late + TTL and MAX(corrected + TTL, late + TTL) = late + TTL, which is not corrected + MAX_RESERVATION_HOLD_MS — so ..._re_anchors_submitted_at_so_the_cap_still_binds goes red. The other five are unaffected: tests 1 and 2 depend only on expires_at, and tests 4-6 never touch that column. The unique-localisation claim holds.
7. SPEC.md
The insert lands at SPEC.md:5796-5808, inside the 18.9a In-flight coin reservation section opening at :5711. The second 18.9a at :5869 (offer suite) is pre-existing duplication and is untouched — nothing renumbered.
The normative text is true of the shipped code: repair before expiry evaluation (db.rs:2924 precedes :2958), client hold re-granted its own maximum (:2949), and the "never hold beyond MAX_RESERVATION_HOLD_MS from an observed instant" clause is discharged because the repaired row is anchored at now and the DO UPDATE cap binds from that anchor. No overclaim found.
8. rpc.rs:1198
reset_chain_cache's refusal counts pending_transactions WHERE expires_at > ?now (db.rs:3086). Pruning first repairs the poisoned row to now + TTL, which still refuses — but now refuses for a bounded time, so the message's "wait for them to confirm or expire" becomes true. No re-entrancy: prune commits its own transaction before reset_chain_cache opens one; no nesting, no lock held across the call. No ordering problem.
9. Beautiful-code gate
Comments are long but every paragraph carries a WHY, and I found no claim the code does not do. Names are intent-revealing; the test names state properties rather than outcomes.
Non-gating notes (recorded here, deliberately NOT as inline threads, so they cannot bar the merge)
(a) db.rs around 2900 — the false-fire shorthand d > max(CAP - TTL, e) is a lower bound, not the exact threshold; the exact one is d > e + max(CAP - e_p - TTL, 0). The stated 50-minute minimum is correct either way. Worth a word if the file is touched again; not worth a round trip.
(b) db.rs:2865-2869 — "A deferred transaction that read first would reintroduce exactly that" describes a shape prune_reservations cannot have, since it performs no reads. The ordering property asserted is true and the code satisfies it; the rationale is borrowed from reserve_client_coins, where it does bite.
(c) test ..._re_anchors_submitted_at_so_the_cap_still_binds passes far_forward as the re-push's submitted_at. It is inert (the DO UPDATE arm ignores that column) and the assertion is unaffected, but late would be the production-faithful value.
None of the three changes a behaviour, a bound, or a money outcome.
Verdict: PASS. No gating findings, so this gate leaves no open threads. The orchestrator owns the undraft and the merge; this review does not undraft, and the tree was not modified.
loop-security interim 2 — IN PROGRESS, not the verdictHead Q2 — can the repair be trip'd without touching the clock? NO. Clean.Neither table lets a caller push a stored deadline past
The repair is also idempotent, which I checked because a re-firing repair would be a hold-renewal primitive. It grants "Evade repair by staying just under" gains nothing: staying under the threshold is the healthy state, and the hold is still bounded by the FINDING 1 — the repair DESTROYS a live hold on any backwards clock step over ~50 min, and the release is at clock RECOVERY, not "one TTL later". Severity MEDIUM, defense-in-depth-with-a-caveat — see the ask at the end.
The comment says:
Measured, that is backwards. The repair writes
So the real behaviour is "released on clock recovery", which is the phrase the comment uses for the safe alternative. A reader sizing this risk will read "one TTL" as ten minutes of remaining protection that does not exist. Concrete scenario, no attacker required. Before this diff the same excursion was harmless: the row kept its true deadline through the excursion and expired on time. Cheap mitigation that keeps the whole benefit. The detector's soundness argument only needs the threshold to be an upper bound no healthy row can exceed — I am not gating on this by itself — the decision to prefer a bounded release over a permanent freeze is defensible and is the ticket's whole point. What I do want before merge is the comment and Still open: Q3 DoS quantification, Q4 |
Adversarial gate: NOT-REFUTED, with one finding that must be fixed before mergeIndependent Opus context, prompted to refute. It could not construct a permanent freeze, and could FINDING (gating, doc-only): the normative sentence this PR adds to SPEC.md is FALSE
A forward clock glitch at first push evades the detector by construction, and the true bound is To evade the detector at the first corrected observation a row needs
Released at observed The code is not wrong; the sentence is. This 110-minute ceiling is #505's cap behaviour under a Remedy: state the true bound and name the forward-glitch residue explicitly, the same way the Two secondary findings, both accepted
Attacks that failed, for the recordThe induction survives an upgrade (both deadline columns are |
…te forward-glitch residue The adversarial gate on #525 found the normative sentence this PR added to SPEC.md was false: it claimed a reservation is never held beyond MAX_RESERVATION_HOLD_MS (60 min) from an observed instant. A forward clock glitch of up to CAP - TTL (50 min) at the first push evades the clock-contradiction detector by construction, and the true worst case is 2*CAP - TTL = 110 minutes, tight. - SPEC.md 18.9a now states the 110-minute bound explicitly, and names the CAP-TTL constant as both the forward-glitch evasion window and the backwards-step false-fire floor -- one constant, two sides. - db.rs: doc comments on the repair explain the residue instead of overclaiming past it. - A new compile-time assert pins RESERVATION_TTL_MS <= MAX_RESERVATION_HOLD_MS -- unreachable today (CAP = 6*TTL) but load-bearing if that ratio is ever narrowed, since a TTL above the cap would make every repaired row re-trigger the detector forever. - The existing boundary test is renamed and its doc comment states plainly that its past-bound row is synthetic and unreachable by any writer -- it pins the SQL predicate's `>` only, not production behaviour. - A new regression test, built entirely from real reserve_spend/prune_reservations calls (no hand-placed rows), measures the actual 110-minute residue under a real retry loop. No behaviour change: the repair itself is unchanged. This corrects a normative claim born false in the commit that wrote it, and pins the honest bound in its place. Co-Authored-By: Claude <noreply@anthropic.com>
…-bound # Conflicts: # Cargo.lock # Cargo.toml # SPEC.md # crates/dig-wallet/src/sage/db.rs
…-bound # Conflicts: # Cargo.lock # Cargo.toml
…-bound # Conflicts: # Cargo.lock # Cargo.toml
Co-Authored-By: Claude <noreply@anthropic.com>
…53.1) Co-Authored-By: Claude <noreply@anthropic.com>
…53.1) Co-Authored-By: Claude <noreply@anthropic.com>
…53.4) Co-Authored-By: Claude <noreply@anthropic.com>
…se PRs Co-Authored-By: Claude <noreply@anthropic.com>
…53.7) # Conflicts: # Cargo.lock # Cargo.toml
…54.1) Co-Authored-By: Claude <noreply@anthropic.com>
…l genesis (#240) (#533) * test(peer): stub for genesis bring-up e2e (#240) Co-Authored-By: Claude <noreply@anthropic.com> * test(peer): prove bring-up installs the downstream engines on the real genesis (#240) The acceptance test dig-node#240 has been open for: with the default DIG mainnet genesis and no DIG_NETWORK_GENESIS override, the peer-network bring-up gets past gossip-config validation and installs the pool handle, the P2P content engine and the DHT inventory-refresh hook, then binds the mTLS peer-RPC listener. Asserts the DOWNSTREAM post-conditions rather than peerStatus.running, which is set before GossipService::new and so holds even when the pool, DHT, content engine and PEX all fail. Also corrects a stale doc-comment that still described the genesis as a pre-launch placeholder that invalidates the gossip config -- the reason the #213 test sees no P2P convergence is environmental (relay off, loopback only), not a rejected config. Co-Authored-By: Claude <noreply@anthropic.com> * style(peer): rustfmt the #240 bring-up test Co-Authored-By: Claude <noreply@anthropic.com> * chore(release): v0.253.5 Merge origin/main into loop/240-genesis-bringup and take the next available patch version (0.253.0-0.253.2 are already claimed by concurrent open PRs #524/#528/#518). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
…54.3) # Conflicts: # Cargo.lock # Cargo.toml
…54.13) Co-Authored-By: Claude <noreply@anthropic.com>
#539) * fix(wallet): discipline reservation liveness against a monotonic clock Reservation deadlines (#502/#525/#528) are anchored entirely on wall-clock readings. #528 closes the case where the clock is already wrong at the moment a reservation is FIRST written. It does not close the general form (#532): a wall clock stepped FORWARD while a reservation is already live, mid-hold -- an NTP step, a VM pause/resume, an operator setting the clock -- produces no self-contradiction for #528's check to catch, yet the very next prune reads the jump as elapsed time and can retire a bundle's hold while it is still genuinely in flight, with no bound on how far forward the step goes (the #348/#497 double-spend direction). Add ClockGovernor: it disciplines every reservation-lifecycle "now" reading so it cannot advance, between two observations, faster than a monotonic clock says real time has actually elapsed. A forward wall-clock jump is absorbed rather than trusted and the disciplined clock simply runs behind until real time catches up, at which point it resumes tracking the wall clock with no special unfreeze step. A backward step is passed straight through unclamped, since it can only lengthen a hold, never shorten one -- the safe direction #502/#528 already accept elsewhere. The governor lives for the process's lifetime and is not persisted: a restart re-seeds it from the wall clock at that moment, so a clock already wrong at boot remains #528's write-time contradiction check's problem, not this one's. Closes #532 Co-Authored-By: Claude <noreply@anthropic.com> * chore(release): bump workspace version to 0.254.20 Root workspace version, per the main lane -- the minor field's version scheme is being fixed separately under #521/#522; this is the interim number to carry PR #539 (dig-node#532) through the version-increment gate. Cargo.lock refreshed in the same commit (cargo update -w --offline) so dig-node-service's locked entry matches -- every CI job runs --locked, and a manifest-only bump here fails Clippy/Test+coverage/all three package builds together on a change that cannot otherwise break a build. Co-Authored-By: Claude <noreply@anthropic.com> * chore(release): bump workspace version to 0.254.43 Per the main lane: main advanced to exactly 0.254.41 after #535's rebase, tying this branch's version. 0.254.43 clears main and every sibling PR in the version-bump queue (#542=0.254.42, #543=0.254.44, #536=0.254.50, #544=0.254.51). Cargo.lock re-synced with `git checkout origin/main -- Cargo.lock` followed by `cargo update -w --offline` (never hand-editing lock conflict markers), confirmed clean with `--dry-run` -> `Locking 0 packages`. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
…543) * fix(wallet): discipline reservation liveness against a monotonic clock Reservation deadlines (#502/#525/#528) are anchored entirely on wall-clock readings. #528 closes the case where the clock is already wrong at the moment a reservation is FIRST written. It does not close the general form (#532): a wall clock stepped FORWARD while a reservation is already live, mid-hold -- an NTP step, a VM pause/resume, an operator setting the clock -- produces no self-contradiction for #528's check to catch, yet the very next prune reads the jump as elapsed time and can retire a bundle's hold while it is still genuinely in flight, with no bound on how far forward the step goes (the #348/#497 double-spend direction). Add ClockGovernor: it disciplines every reservation-lifecycle "now" reading so it cannot advance, between two observations, faster than a monotonic clock says real time has actually elapsed. A forward wall-clock jump is absorbed rather than trusted and the disciplined clock simply runs behind until real time catches up, at which point it resumes tracking the wall clock with no special unfreeze step. A backward step is passed straight through unclamped, since it can only lengthen a hold, never shorten one -- the safe direction #502/#528 already accept elsewhere. The governor lives for the process's lifetime and is not persisted: a restart re-seeds it from the wall clock at that moment, so a clock already wrong at boot remains #528's write-time contradiction check's problem, not this one's. Closes #532 Co-Authored-By: Claude <noreply@anthropic.com> * chore(release): bump workspace version to 0.254.20 Root workspace version, per the main lane -- the minor field's version scheme is being fixed separately under #521/#522; this is the interim number to carry PR #539 (dig-node#532) through the version-increment gate. Cargo.lock refreshed in the same commit (cargo update -w --offline) so dig-node-service's locked entry matches -- every CI job runs --locked, and a manifest-only bump here fails Clippy/Test+coverage/all three package builds together on a change that cannot otherwise break a build. Co-Authored-By: Claude <noreply@anthropic.com> * fix(wallet): route wallet_reset_coin_db's now through ClockGovernor wallet_reset_coin_db read its now_ms from a fresh, undisciplined SystemTime::now() rather than WalletBackend::reservation_now_ms(), so a wall-clock jump mid-hold (an NTP correction, a VM pause/resume) could make its in-flight-spend check see a still-live reservation as already expired and let the reset proceed -- the #348/#497 double-spend direction, no attacker required. reservation_now_ms() is now pub so the control plane (a different crate) can route through it, sharing the same ClockGovernor clamp state every other reservation call site (reserve_coins, prune_reservations) already uses. Swept every reservation-touching path in dig-wallet and dig-node-service for a direct SystemTime::now() read; this was the only production one. Closes #541 Co-Authored-By: Claude <noreply@anthropic.com> * chore(release): bump dig-node 0.254.42 / dig-wallet 0.49.0 dig-wallet: minor -- reservation_now_ms is now a public API surface (dig-node-service routes through it, dig-node#541). dig-node: patch -- behaviour fix, no breaking change. Co-Authored-By: Claude <noreply@anthropic.com> * chore(release): re-bump to 0.254.44 -- coordinator-assigned to avoid collision with #542 #542 keeps 0.254.42 (urgent required-CI-gate PR, merges first); 0.254.43 is reserved for #539, which this branch sits on top of. Version assignment across concurrent PRs is the coordinator's per CLAUDE.md section 1.4. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
Closes #525.
The defect
reserve_pushed_bundle(sage/rpc.rs:2325) reads the clock ONCE and writessubmitted_at = nowandexpires_at = now + RESERVATION_TTL_MS. One reading far in the future (F) stores a deadline decades out, and nothing in the system could retire it:prune_reservationsdeletes onexpires_at <= now, which never arrives.MAX(stored.expires_at, MIN(excluded.expires_at, stored.submitted_at + C)), so a later push under the corrected clockRcomputesMAX(F+T, MIN(R+T, F+C)) = F+T-- unchanged.coin_reservations.transaction_id REFERENCES pending_transactions ON DELETE CASCADEwithforeign_keys(true), so the child row survives too, andreserved_coin_idskeeps excluding the coin fromunreserved_unspent_coinsfor ever.reset_chain_cachedoes not delete either table and refuses while the row exists.The coin is frozen permanently with no recovery available inside the product.
The fix -- repair at OBSERVATION
reserve_spend's upsert is unchanged. A write-time-only fix was rejected on the record: nothing re-pushes an accepted bundle unprompted, so it would leave the headline no-re-push freeze permanent.WalletDb::prune_reservationsnow runs two repairUPDATEs before its two existingDELETEs, all four in ONE write-first transaction (the write lock is taken before any read -- the same orderingreserve_client_coinsdocuments as what prevents aSQLITE_BUSYmis-mapped toUnavailable).UPDATE pending_transactions SET submitted_at = now, expires_at = now + RESERVATION_TTL_MS WHERE expires_at > now + MAX_RESERVATION_HOLD_MSUPDATE client_coin_reservations SET expires_at_ms = now + CLIENT_RESERVATION_MAX_TTL_MS WHERE expires_at_ms > now + MAX_RESERVATION_HOLD_MSReturn value unchanged:
rows_affectedof the first DELETE only. A repaired row is not a retired bundle, and folding it in would inflate a count callers read as in-flight spends.Why the predicate is a self-contradiction check, not a heuristic
INSERT writes
expires_at = now + T; theDO UPDATEclamp writes at mostsubmitted_at + C. So every healthy row satisfiesexpires_at <= submitted_at + C, and withsubmitted_at <= nowit satisfiesexpires_at <= now + C. Henceexpires_at > now + Cimpliessubmitted_at > now-- the row claims to have been submitted after the present moment.The false-fire floor
Let
E = expires_at - submitted_at = min(e + T, C)whereeis the row's age at the last push. The detector fires iffnow < submitted_at + E - C, so a backwards clock step ofdat ageefires iffd > max(C - T, e).The minimum backwards step that can trip the repair on a legitimately created row is
C - T= 50 minutes, and more for an older row. A claim that a 1 ms backwards step trips an at-cap row is false: an at-cap row is ~50 minutes old, so it needsd > 50 mintoo.The genuine cost, stated not argued away
Past that floor a live bundle's row IS re-anchored and gets only
Tmore from the repair instant, where the unrepaired code would have held it until the clock caught up (d + E - elonger). So a backwards step exceedingMAX_RESERVATION_HOLD_MS - RESERVATION_TTL_MScan return a still-live bundle's inputs after one further TTL instead of on clock recovery. That is a real move toward the #348/#497 double-spend direction, bounded by a gross (>50 min) clock anomaly, and accepted because the alternative it replaces is a permanent, in-product-unrecoverable freeze -- the failureRESERVATION_TTL_MS' own doc andSPEC.md18.9a both name as the worse one. This fix does not "never release anything early".The four decisions inside the shape
now + T, notnow + C-- precisely what an honest push atnowwould have written.now + Cwould hand a repaired row six times a healthy row's first hold.submitted_atis re-anchored, and this is not optional. Left atF,stored.submitted_at + C = F + C, so A repushed bundle re-arms its reservation from now, so a retrying caller can hold coins indefinitely #502's cap clause stops binding on that row for ever and a retrying caller renews the hold indefinitely -- resurrecting the A repushed bundle re-arms its reservation from now, so a retrying caller can hold coins indefinitely #502 lockout. This is not a regression of A repushed bundle re-arms its reservation from now, so a retrying caller can hold coins indefinitely #502's stable-anchor rule: that rule keeps the column out ofDO UPDATE SETso a CALLER cannot move the anchor. This write is not a caller; it is the row's owner replacing a value the system has proven impossible with one from the currently trusted clock. Stated in the doc comment.MAX_RESERVATION_HOLD_MS, including the client one. Keying the client detector on its own 10-minute ceiling would let a ~5-minute backwards step re-clamp a healthy client hold and reopen the cross-process selection window dig_ecosystem#3127 closes. One hour is the longest hold of any kind this wallet grants. Pinned by aconst _: () = assert!(CLIENT_RESERVATION_MAX_TTL_MS <= MAX_RESERVATION_HOLD_MS, ...)beside the constant, so the detector cannot silently become unsound if either moves.now + CLIENT_RESERVATION_MAX_TTL_MS, notnow + T: there is nosubmitted_atcolumn and the caller's requested TTL is unrecoverable, so it grants the ceiling a caller could legitimately have asked for. That errs long, the safe direction for a build-window hold. The asymmetry is stated in the comment.Blast radius checked
prune_reservationscallers, via grep (gitnexus indexes are stale by construction here --commitsBehindondig-nodeis in the hundreds and points at the primary checkout, soimpactwould return a false-safe zero):rpc.rs:875, 893, 2961, 3228, 3695-- all already prune before a reservation-sensitive read, and all are unaffected: the signature, the return value's meaning, and the two DELETE predicates are unchanged.rpc.rs:1198(reset_coin_db) -- the one entry point that did NOT prune, now does. Without it a user whose first action after correcting the clock is a reset getsResetRefusal::SpendInFlighttelling them to wait, and waiting alone triggers no prune.SpendInFlight's counting predicate (db.rs:3005) and its message text are deliberately unchanged: excluding contradicted rows from the count would let a reset wipecoinsbeneath a bundle that may genuinely be in a mempool, and the sentence "Wait for them to confirm or expire, then retry" is false today but becomes TRUE under this fix.db.rs.Risk: MEDIUM. It is custody-adjacent (it decides when a coin returns to selection) but it strictly shrinks an unbounded hold to a bounded one, and its one early-release path is the >50-minute-clock-anomaly case stated above.
Evidence
RED, before the fix (
cargo test -p dig-wallet --lib, real counts, not a zero-match run):The headline failure line:
assertion left == right failed: one TTL after the repair the coin returns to selection; before this fix it never did.GREEN, after:
test result: ok. 800 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out(baseline atf0a9726was 794 passed / 1 ignored; +6 new tests).cargo fmt --checkon the two touched files only: CLEAN. No workspace-widecargo fmt --all.Tests added (6)
a_reservation_anchored_to_a_far_future_clock_stops_freezing_its_coin_for_everunreserved_unspent_coins(None)-- the selection surface where the money is frozen, not apending_transactionsrow count one layer below the decisionthe_clock_contradiction_repair_grants_one_fresh_ttl_not_the_maximum_holdthe_clock_contradiction_repair_re_anchors_submitted_at_so_the_cap_still_bindscorrected + 5.5Tmust land oncorrected + C; without the re-anchor it lands oncorrected + 6.5Ta_client_hold_written_under_a_far_future_clock_is_repaired_and_then_lapsesa_one_millisecond_backwards_clock_step_repairs_nothingthe_clock_contradiction_detector_fires_strictly_past_the_maximum_holdnow + Cuntouched, atnow + C + 1repairedMutation table (each run in full, then reverted)
UPDATE pending_transactionsstatement entirely796 passed; 4 faileda_reservation_anchored_to_a_far_future_clock_stops_freezing_its_coin_for_ever,the_clock_contradiction_repair_grants_one_fresh_ttl_not_the_maximum_hold,the_clock_contradiction_repair_re_anchors_submitted_at_so_the_cap_still_binds,the_clock_contradiction_detector_fires_strictly_past_the_maximum_holdnow + MAX_RESERVATION_HOLD_MSinstead ofnow + RESERVATION_TTL_MS797 passed; 3 failed..._grants_one_fresh_ttl_not_the_maximum_hold,a_reservation_anchored_to_a_far_future_clock_...,..._detector_fires_strictly_past_the_maximum_holdSET submitted_at = ?now799 passed; 1 failedthe_clock_contradiction_repair_re_anchors_submitted_at_so_the_cap_still_binds(uniquely)Mutation 3 localises to exactly one test, which is the one it must. Mutations 1 and 2 overlap, as expected: mutation 1 removes the whole repair, so every test that observes any part of it goes red; mutation 2 is the strictly narrower "amount" mutation and is the one distinguished by
..._grants_one_fresh_ttl_....Also in this PR
a_repush_under_a_clock_that_stepped_backwards_never_shortens_a_live_hold. Its third push passedsubmitted_at = firstwhile moving onlyexpires_atbackwards -- a statereserve_pushed_bundlecannot produce, since it writes both from one clock read. It is now the production-shapedsubmitted_at = first - 60_000, expires_at = first - 60_000 + T(well under the 50-minute detector floor, so the row must be untouched by the new repair). Its existing assertionrow.expires_at == extended_at + RESERVATION_TTL_MSstill holds unchanged, now for a reachable state.SPEC.md18.9a ("In-flight coin reservation", the reservation-contract section -- note the file re-uses the number18.9aat two other points for the offer/DID-mint suite; that pre-existing collision was NOT touched or renumbered): a normative statement that a recorded deadline exceeding the current instant by more thanMAX_RESERVATION_HOLD_MSMUST be treated as contradicting the clock, MUST be re-anchored to the current instant with a fresh lifetime before any expiry is evaluated (and the client hold re-granted its own maximum), so a reservation is never held beyond the maximum hold measured from an instant the node has observed.Version in
Cargo.tomlis0.252.31, unchanged as instructed.Deliberately NOT done -- reported, not fixed
prune_reservationsfrees LIVE inputs when the clock jumps forward: the DELETE fires onexpires_at <= nowwith no lower bound, so a forward jump past a live deadline retires a bundle that may still be in a mempool. Onmaintoday, unchanged by this fix, and a separate defect in the sec(reservation): gated on an untrusted 'accepted' — the under-claim direction fails OPEN into the double-select window #348/fix(wallet): a peer-local refusal must not free inputs another destination may hold #497 double-select direction.custody::now_ms(). That is the actual root cause of both this ticket and (1): a monotonic anchor for elapsed time, with wall-clock kept only for display, would make both unreachable. Much larger change; not started.reserved_coin_ids' clockless raw read -- over-reserves by design, safe direction, untouched.MAX_RESERVATION_HOLD_MS' doc -- documented, intended, unaffected.Not merged, not undrafted, no review requested -- left DRAFT for the gate round.