Skip to content

feat(547): in-flight slot expiry, backend.daemon.reclaim and typed admission reasons (slices 5–6) - #652

Merged
mengmengmengqiang merged 2 commits into
layerfs_v2_0_0from
fix/547-slice-56
Nov 12, 2026
Merged

mengmengmengqiang merged 2 commits into
layerfs_v2_0_0from
fix/547-slice-56

Conversation

@mengmengmengqiang

Copy link
Copy Markdown
Contributor

Summary

Slices 5–6 of #547, stacked on fix/547-slice-34. An admitted slot's request now has a deadline, the operator gets a one-shot backend.daemon.reclaim verb, and a refused open is typed from the channel error instead of from the refusal message text.

19 files, +731 −175, two commits on top of the slice 3–4 branch.

What this does

Slice 5 — in-flight expiry and reclaim — d1fc7e228

  • A decoded request carries an explicit deadline (Phase::InFlight { started, deadline }, driven by the new request_deadline_ms limit). Past it the daemon frees the slot, closes the worker's socket and records RetainedFailure { cause: Expired, reply_sent: false }, so the caller observes an unknown outcome instead of a silent hang.
  • backend.daemon.reclaim { slot, generation } is the authenticated, generation-checked, one-shot operator verb: it closes the named worker's socket, records Reclaimed, answers Invalid on a generation mismatch, never restarts the daemon and never touches Workspaces. It joins the recovery allow-list.
  • Reclaimed and expired workers are detached (never joined), and slot fences are keyed by generation, so a stale worker's late clear_fence cannot clobber a newer worker's fence.

Slice 6 — typed refusal outcome — 7a24b34c2

  • A failed open is no longer classified by matching the refusal message text. It is typed from the channel error: a notice is Capacity/Unavailable with resource, pool_vacant, reserved_vacant, retryable and a retry_after hint; a legacy EOF answers Unknown { observed } and never guesses "no vacant slot"; a reply that never arrives (reclaimed or expired worker) answers Unavailable { outcome: unknown }.
  • AdmissionReason is #[non_exhaustive] with constructor functions; the capacity model's parse/to_sdk/exhausted are #[must_use].
  • The doc comment records the version-skew matrix: new daemon × old binding, old daemon × new binding, new control × old worker bundle.

Test evidence

Dell aluminium (x86_64, 88 cores), rustc 1.85.1:

cargo +1.85.1 fmt --all -- --check                                              # rc=0
cargo +1.85.1 clippy --workspace --exclude layerfs-node --all-targets -- -D warnings   # rc=0
cargo +1.85.1 test -p layerfs-daemon --lib                                      # ok, 0 tests
cargo +1.85.1 test -p layerfs-daemon --test daemon_slots --test slot_state      # ok
    daemon_slots.rs: 7 passed, 0 failed
    slot_state.rs:  20 passed, 0 failed
cargo +1.85.1 test -p layerfs-api -p layerfs-control -p layerfs-sandbox --no-fail-fast
    # 11 failures — all pre-existing, see below

The 11 failures are not from this branch. I built and ran the same -p layerfs-api -p layerfs-control -p layerfs-sandbox suite on unmodified e9b903eac: base fails 12 of those tests, and this branch's failing set is a strict subset of base's — zero new failures.

They are a host limitation of the test machine, not of the code: the kernel-mount suites need /dev/fuse owned by the test user and fail as a normal user with kernel attach Mount: Operation not permitted (os error 1) / FUSE device ownership/type. Run as root they pass, but the daemon refuses to start when the command uid equals the owner uid, so daemon_slots/slot_state need a non-root user. The suite therefore cannot be green in one configuration on that host.

Refs #547, #540 — not closing either.

Slice 5: an admitted slot's decoded request now carries an explicit
deadline (Phase::InFlight { started, deadline } driven by the new
request_deadline_ms limit). Past it the daemon frees the slot, closes the
worker's socket and records RetainedFailure { cause: Expired,
reply_sent: false }, so the caller observes an unknown outcome instead of a
silent hang.

backend.daemon.reclaim { slot, generation } is the authenticated,
generation-checked, one-shot operator verb: it closes the named worker's
socket, records Reclaimed, answers Invalid on a generation mismatch, never
restarts the daemon and never touches Workspaces. It joins the recovery
allow-list. Reclaimed and expired workers are detached (never joined), and
slot fences are keyed by generation so a stale worker's late clear_fence
cannot clobber a newer worker's fence.
Slice 6: the client no longer classifies a refused open by matching the
refusal message text. A failed open is typed from the channel error: a
notice is Capacity/Unavailable with resource, pool_vacant,
reserved_vacant, retryable and a retry_after hint; a legacy EOF answers
Unknown { observed }, never a guess at "no vacant slot"; a request whose
reply never arrives (reclaimed or expired worker) answers Unavailable
{ outcome: unknown }.

AdmissionReason is #[non_exhaustive] with constructor functions, the
capacity model's parse/to_sdk/exhausted are #[must_use], and the doc
comment records the version-skew matrix (new daemon x old binding, old
daemon x new binding, new control x old worker bundle).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant