feat(wallet): paginate the unspent-coin read - #413
Conversation
|
IN PROGRESS — not the verdict. Gate reading head Confirmed so far:
One GATING item found, detailed inline below: the required |
MichaelTaylor3d
left a comment
There was a problem hiding this comment.
GATE VERDICT: CHANGES-REQUIRED
Head read: 954baa86515be376df38e460f8531227f4768349. Fresh context; verified rather than adopted.
(Recorded as a COMMENT review because a verdict-bearing review event is 422 on a same-identity PR. The two OPEN inline threads below are what bar the merge under required_conversation_resolution.)
What holds
The engineering is sound and each of the three named properties is genuinely enforced and genuinely tested.
- Lock is real.
dig-node-control-interface 0.25.0,source = registry+https://github.com/rust-lang/crates.io-index, checksum7668b2b45f3fa813efddf24cbf872d05851ee5203248a9d18ff8755bc1e3b9b8. No[patch.crates-io]survives in the root manifest -- the remainingpatchgrep hits are retired-patch PROSE, lines 43-55. The adoption is not fictional. - Ascending, cursor, and
LIMIT limit+1are one statement (crates/dig-wallet/src/sage/db.rs:2823-2882), with scope + asset +spent_height IS NULLat the SAME level -- so the page cannot be cut before the filter. The extra row is dropped by.take(page_size)(sage/rpc.rs:1541) and never reaches the wire. completeis derived from what REMAINS:rows.len() <= page_size(DB tier,rpc.rs:1538) andremaining.len() <= page_size(fallback tier,rpc.rs:1592). Never from page fill.- Fallback tier sorts THEN pages (
rpc.rs:1580-1596), matchingcoins_by_parent, and sorts locally rather than trusting a tier that promises no order. - Test 1 asserts coin IDENTITIES,
vec!["cc33","dd44"]-- not a count. A count-only assertion is exactly what an offset implementation would satisfy; this one would not. - Mutation claims check out analytically. With four coins at page size two,
complete = rows.len() < page_sizeyieldsfalseon BOTH pages -- test 2 red, test 1 unaffected, and test 3 (page size 1) still terminates on its trailing empty page and still walks the fixture in order, so it stays green. Each test is separately load-bearing, as claimed. limitREFUSED, not clamped, and pinned from BOTH sides: at-max accepted,MAX+1and0refused with-32602(control.rs:3774-3820). Cursor case and0xprefix normalization asserted too.- Constants are ALIASES, verified in the published 0.25.0 source:
params.rs:889COINS_DEFAULT_LIMIT = COINS_BY_PARENT_DEFAULT_LIMIT,params.rs:894COINS_MAX_LIMIT = COINS_BY_PARENT_MAX_LIMIT. dig-node consumes them (and the contract'seffective_limit()) rather than redefining. closingIssuesReferencesasked of the parser, not read from the body: exactly one entry,dig-node#381. No epic child caught by accident.- §8.6 parity --
cli_covered_control_methods()updated for the new variant, the drift test atcontrol_cli.rs:1739-1755is present and NOT#[ignore]d; no new control method, so the set is unchanged. (Accurate as the body states it: the parity test covers method NAMES, not the new flags.) - §2.4b measured, not assumed. Every
dig-*dep of both touched crates is at its latest published version -- I re-checked all twelve against index.crates.io.chia-*is coherent on the 0.36 line as a set;chia-sdk-driver/-typestop out at 0.36.0, so leaving the others there is correct rather than stale. - No sibling collision.
loop/377-mirror-coin-lifecycle'ssage/rpc.rsdelta is +46 lines, entirely insidemod tests, and its onlycoins_for_addressreference is the unmodifiedcontrol.rs:1802line. The rebase is clean; there is no structural overlap.
What blocks
Two findings, neither in the paging logic. Both are cheap.
- GATING -- the required
Lint commit messagescheck is RED on this head. - GATING -- the PR body's own merge-gate instruction is now false.
Ranked findings are inline. Neither requires touching the implementation.
Not covered, stated plainly
No run against a real funded mainnet address with thousands of coins. Proven against a real sqlite replica and the in-memory fallback path only. The body does not claim otherwise, but it does not say so either -- see the non-gating note.
MichaelTaylor3d
left a comment
There was a problem hiding this comment.
NON-GATING notes from the same gate pass. I am resolving all three myself so they cannot bar the merge under required_conversation_resolution. None affects the verdict; act on them or not.
954baa8 to
22d12e8
Compare
|
GATING (1/2) — FIXED: Reworded the commit from |
|
GATING (2/2) — FIXED: Removed the opening Commit message rewritten: The problematic Both gating items are now resolved. The old review comments on outdated lines can be dismissed. |
MERGE-ORDER CONSTRAINT — do not merge this before dig-app#322Measured in the published contract, pub fn effective_limit(&self) -> u32 { self.limit.unwrap_or(COINS_DEFAULT_LIMIT) }with So an omitted Why that is a money defect, not a nuisanceThe dig-app#294 lane found the live instance: A user with more than 100 coins would be told they cannot afford a mint they can afford, and nothing anywhere would be red. The constraintDIG-Network/dig-app#322 must merge first, or land in the same window. It rewrites that read to walk to the end and to refuse rather than return a prefix. Its gate has not run yet. Consumers are contained — I checked: dig-app has exactly two Worth recording for the next paged readThe contract itself is honest — a 0.25 node emits |
25ac493 to
7788609
Compare
Refs #381 Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…the temporary patch The dig-node-control-interface 0.25.0 is now published on crates.io with the paginated WalletCoinsParams/WalletCoinsResult contract. Remove the temporary [patch.crates-io] redirect and resolve the lock to the registry version. Verified: - Cargo check passes with published 0.25.0 - Lock resolves to registry+https://github.com/rust-lang/crates.io-index - Checksum: 7668b2b45f3fa813efddf24cbf872d05851ee5203248a9d18ff8755bc1e3b9b8 Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
7788609 to
23e8361
Compare
Closes #381. Parent epic: https://github.com/DIG-Network/dig_ecosystem/issues/3170.
Contract half (release-first, merges + publishes FIRST):
DIG-Network/dig-node-control-interface#37 (0.24.0 -> 0.25.0).
The gap, and only the gap
control.wallet.coinsreturned every unspent coin at an address in one response. An address'scoin count is unbounded and every spend's change coin adds one, so the read had unbounded
cardinality on a control plane with no request rate limiting of any kind
(dig_ecosystem#2577) — and on the fallback tier that work lands on a third-party coinset oracle.
Nothing else about the read changed.
Shape — the repo's own
coinsByParentidiom, reused not reinventedafter_coin_id+limit, ascendingcoin_id,completederived from a row fetched BEYOND thepage,
limitREFUSED out of range rather than clamped. The contract'sCOINS_MAX_LIMIT/COINS_DEFAULT_LIMITare defined AS thecoinsByParentconstants, so a frame-limit change movesboth reads and cannot leave them disagreeing.
Cursor, not offset, matters more here than on the sibling. An address's unspent set SHRINKS.
When a coin is spent between two pages, an offset moves every later row one position earlier and
the next page begins one row late — a coin the caller never sees, on the read whose entire purpose
is selecting coins for a spend. That is money the caller cannot spend and a refusal with a
shortfall that is not true.
Where the page is cut, and why it is not one layer up
The ticket flagged this and it was real:
coins_for_addressfilters unspent in Rust while thescope filter is in SQL. Paginating the broader query and filtering afterwards would cut the page
BEFORE the filter — short pages, and a
completecomputed from a count that no longer describeswhat remains.
So the new
WalletDb::unspent_coins_pageapplies scope, asset,spent_height IS NULL,coin_id > cursor,ORDER BY coin_id ASCandLIMIT limit+1in ONE query. The fallback tiercannot push a page down (that tier answers with the whole set in one call and has no cursor), so it
sorts then pages in memory — exactly as
coins_by_parentdoes, and the sort is done locally for thereason that method records: the tier underneath merges peer and coinset answers and promises no
order at all.
Rivals folded in rather than duplicated
page_fields()incontrol.rsnow serves BOTH paged coin reads;coinsByParent's inline copywas folded into it.
page_suffix()does the same for the two CLI summaries.PAGE_LIMIT_ERRORin the contract is now shared by both reads.Two copies of a page extraction can only drift, and a drift in a page boundary is a coin nobody
sees.
Blast radius checked
coins_for_address— grep found 20 references: 1 production call site (control.rs:1802),13 test call sites, 6 doc references. All updated; the 13 test sites take a named
TEST_PAGErather than a bare literal so "does not care about the page" stays distinguishable from a chosen
size.
coins_wire— 3 test literals, all updated.unspent_coins_pageis new, so it has none.coins_scopedwas left alone and still servesget_coins.No new control method, so
CONTROL_METHODSand the #426 CLI-parity set are unchanged — the CLIgains two flags on an existing verb, which the parity test covers by construction.
Evidence
cargo test -p dig-wallet -p dig-node-service --lib: 526 passed / 0 failed and709 passed / 0 failed, 1 ignored.
cargo clippy -p dig-wallet -p dig-node-service --all-targets: clean;cargo fmt --allapplied.Contract crate: 176 lib tests + 9 doc-tests green, clippy
-D warningsclean.Each test proven load-bearing by mutation, committed first, reverted after:
OFFSETinstead ofcoin_id > cursora_coin_spent_between_pages_shifts_no_boundary_and_loses_no_coinanda_cursor_walk_visits_every_coin_exactly_oncecomplete = rows.len() < page_size(length-derived)an_exactly_full_final_page_is_complete_and_the_one_before_it_is_not(alone — the other two stayed green, so each is separately load-bearing)completedefaulted toSome(false)an_unpaged_answer_from_an_older_node_is_not_read_as_a_truncated_pagecompletefrom page length / ignoresafter_coin_idthe_coin_page_and_the_final_page_are_told_apart_by_complete_not_by_lengthWorth recording: the first attempt at the third mutation used
#[serde(default, deserialize_with = ...)]and survived — serde never callsdeserialize_withfor an ABSENT key, so it only mutated the explicit-null path, not the oneunder test. A surviving mutation that was an instrument error rather than a test gap.
§2.4b — measured, not assumed
Every
dig-*dep of the two touched crates is already at its latest published version(
dig-chainsource-interface0.3.2,dig-rpc-protocol0.10.2,dig-clvm0.4.0,dig-keystore0.13.0,
dig-mirror-coin0.7.0,dig-urn-resolver0.5.3,chia-query0.19.0 — the carets alreadyresolve there). The only bump this PR makes is
dig-node-control-interface0.24 -> 0.25.chia-*stays on the 0.36 line as a set:chia-sdk-driver/chia-sdk-typestop out at0.36.0, so moving the others to a 0.48 would split this crate across two chia lines.
Docs
SPEC.md§23control.wallet.coinsrow rewritten normatively (ascending order, cursor-not-offsetwith the shrinking-set reason,
completefrom what remains, same-level filtering, refuse-not-clamp,additive request shape) and the §8.6 CLI line updated.
Version
0.168.0 -> 0.169.0, MINOR: new optional request params and new response fields on an existing
method; a request naming neither page param is byte-identical to before.
Dependency resolution
The temporary
[patch.crates-io]entry fordig-node-control-interfacehas been removed and the lock re-resolved against the published 0.25.0 version from crates.io. The lock is clean.