fix(code-index): isolate workers and keep the reader charge - #1840
ScriptedAlchemy wants to merge 4 commits into
Conversation
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d323531935
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Review lane verdict: REJECT as-is. The graph/text overlap change (mount.rs: text_projection_in_flight treated as 'text owner admitted') inverts the convergence-park invariant two existing tests assert: convergence_park_tests::fresh_graph_activation_waits_while_the_published_text_owner_is_parked and ::fresh_graph_activation_starts_while_the_clone_successor_is_pending fail deterministically (5/5) with this PR's mount.rs/serving.rs and pass with master's. The PR's test plan never ran the crate's --lib suite. Also: clippy -D warnings fails on the added test_gates.rs (question_mark, redundant_closure_for_method_calls); the reservation shrink is applied to only one of the two publication tails (advance_clone_successor still drops and re-reserves); the retained reservation keeps the build component key. The dedicated worker runtime and the reservation shrink (applied to both tails, keeping the reader key) are separable and worth landing as their own PRs; keep the overlap out until the #1103 parked-owner invariant is re-argued with those two tests updated deliberately. Note #1836 stamps a continuation from published_text_projection_outcome before drop(reconcile_pass); under this PR that is None at that point. |
d323531 to
b4e19fe
Compare
5590a59 to
ebc51a4
Compare
PR dogfood on 3f71bcc timed out in bulk_commit with graph pending. The first pass sat unscheduled for 226s on the daemon runtime, then the published text projection was joined before any graph attempt. Run workers on their own runtime and activate the graph during that projection. Shrink the build charge into the reader instead of dropping it, so an overlapping replay cannot refuse the reader admission. Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
The published-projection overlap treated an in-flight text build as graph admission. Activation then started before exact and lexical owners were ready, including while a parked owner still held the source. Those convergence tests fail, so the seat order stays. Both publication tails hand their held charge to the reader component instead of dropping it and reserving again. Measured RSS cannot refuse a charge that never left the ledger. The clone successor reserves at least the reader budget so that handoff does not under-charge the reader. Workers stay on their own runtime so the first pass is not stuck behind project-open. Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
Metadata::len is already u64. try_from is a useless conversion and -D warnings fails the workspace clippy check. Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
The suite mounts many registries in one process. A process-wide runtime of a few threads then holds a waiter behind every other mount. The search decode test hit the 360s deadline, then passed in a fresh process. Tests already have a runtime that is not the daemon serving runtime, so they spawn there. The daemon still uses the dedicated worker runtime. Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
ebc51a4 to
fac0a0a
Compare
|
Second review against the head fac0a0a: the overlap change is already reverted here (946ad3b), the shrink is on both tails and keeps the reader key, and the clippy items are fixed. The reservation shrink is now ported onto master as its own PR with a ledger-level test that discriminates transfer from fresh admission. The dedicated worker runtime stays out: it compiles under cfg(not(test)) so nothing can fail without it, its own commit notes it hit the 360 s deadline when many registries shared few threads, and the worker already offloads heavy work through spawn_blocking, so a starved worker points at whatever blocks the shared runtime rather than at the worker. This PR closes once the port merges. |
|
Closing as superseded by #1873. #1873 is still open but already owns the reservation-shrink port (both publication tails + Prefer close-as-superseded over a zombie PR that only carries rejected worker-runtime delta. Once #1873 merges, nothing from #1840 remains to land. |
Cause
Strict readiness on PR dogfood (tip
3f71bccb2dba, run 35341667443) sat inbulk_commitwithgraph=pendinguntil the 600s deadline. The first pass was also unscheduled for 226s on the daemon runtime (queue_delay_micros=226558274,wake_to_gates_held_micros=15).A follow-up that started graph activation while that text projection was still in flight inverted the parked-owner ordering. Those convergence tests fail when an in-flight projection is treated as graph admission. That overlap is not in this branch.
CI run 35442762975 then failed on the rebased head: workspace clippy (
u64::try_fromofMetadata::lenin the sessions JSONL test), andsearch_never_awaits_an_in_flight_decode_while_a_generation_is_servabletiming out at 360s then passing in 0.3s in a fresh process. Test Linux only echoed that partition failure.Fix
code-text-artifact-readerand shrink it to the reader budget. They do not drop it and reserve again. The clone-successor charge is at least the reader budget.Metadata::lenis used directly. It is alreadyu64.Rebased onto master
993f5e3c9bce4cc2a03dc43ac0621f1f6ab85819.ci.ymlis unchanged.Proof
code_index_scheduler::tests::serving::search_never_awaits_an_in_flight_decode_while_a_generation_is_servable— 1 passed in 0.56s.cargo clippy -p tracedecay-sessions -p tracedecay-code-index-runtime --all-targets --locked -- -D warnings— passed.SHA:
fac0a0a475405b78a0691673db4c3d3085a1de59