Skip to content

feat(547): recovery-class reserved slot and idle reclamation (slices 3–4) - #651

Merged
mengmengmengqiang merged 1 commit into
layerfs_v2_0_0from
fix/547-slice-34
Nov 12, 2026
Merged

mengmengmengqiang merged 1 commit into
layerfs_v2_0_0from
fix/547-slice-34

Conversation

@mengmengmengqiang

Copy link
Copy Markdown
Contributor

Summary

Slices 3–4 of #547, stacked on fix/547-slice-12. When the control pool is full a recovery-class client can now still get in, and idle pool slots are reaped instead of being lost to a parked worker.

12 files, +643 −29, one commit on top of the slice 1–2 branch.

What this does

Slice 3 — recovery-class reserved slot

  • When the pool has no vacant slot and the daemon is ControlReady, one connection is admitted into the reserved slot, and after decode only the recovery operation set is allowed: close (Unmount), workspace status (Status) and backend.daemon.connection_failures. Anything else is refused Capacity and the connection is closed.
  • At most one connection holds the reserved slot, and it closes after one request (invariant 4).

Slice 4 — idle reclamation

  • Each admitted slot tracks its phase at request boundaries (Busy / Idle { since }).
  • Under admission pressure, pool slots parked in receive() past idle_ttl are reaped by closing their socket fence. Nothing is in flight, so nothing is lost: the woken worker finishes with a stale ticket and the client pool quarantines the closed connection. Reserved slots are never reaped.
  • idle_ttl_ms joins the daemon's private limits (wire format), with the pure recovery_request predicate.
  • Unit and integration coverage for tests 8 and 11.

Slices 5 (in-flight expiry / backend.daemon.reclaim) and 6 (typed reasons) are deliberately not here; they are the next PR in the stack.

Test evidence

Dell aluminium (x86_64, 88 cores), rustc 1.85.1:

cargo +1.85.1 fmt --all -- --check                                              # rc=0
cargo +1.85.1 clippy --workspace --exclude layerfs-node --all-targets -- -D warnings   # rc=0
cargo +1.85.1 test -p layerfs-daemon --lib                                      # ok, 0 tests
cargo +1.85.1 test -p layerfs-daemon --test daemon_slots --test slot_state      # ok
    daemon_slots.rs: 5 passed, 0 failed
    slot_state.rs:  15 passed, 0 failed
cargo +1.85.1 test -p layerfs-api -p layerfs-control -p layerfs-sandbox --no-fail-fast
    # 11 failures — all pre-existing, see below

The 11 failures are not from this branch. I built and ran the same -p layerfs-api -p layerfs-control -p layerfs-sandbox suite on unmodified e9b903eac: base fails 12 of those tests, and this branch's failing set is a strict subset of base's — zero new failures.

They are a host limitation of the test machine, not of the code: the kernel-mount suites need /dev/fuse owned by the test user and fail as a normal user with kernel attach Mount: Operation not permitted (os error 1) / FUSE device ownership/type. Run as root they pass, but the daemon refuses to start when the command uid equals the owner uid, so daemon_slots/slot_state need a non-root user. The suite therefore cannot be green in one configuration on that host.

Refs #547, #540 — not closing either.

Slice 3 — recovery-class reserved slot: when the pool has no vacant slot and
the daemon is ControlReady, admit one connection into the reserved slot and,
after decoding, allow only the recovery operation set (close = Unmount,
workspace status = Status, backend.daemon.connection_failures); any other
request is refused Capacity and the connection is closed. At most one
connection holds the reserved slot, and it closes after one request
(invariant 4). backend.daemon.reclaim joins the set in slice 5.

Slice 4 — idle reclamation: track each admitted slot's phase (Busy /
Idle { since }) at request boundaries, and under admission pressure reap pool
slots parked in receive() past idle_ttl by closing their socket fence. Nothing
is in flight, so nothing is lost; the woken worker finishes with a stale
ticket and the client pool quarantines the closed connection. Reserved slots
are never reaped.

Adds idle_ttl_ms to the daemon's private limits (wire format), the pure
recovery_request predicate, and unit + integration coverage (tests 8 and 11).

Do not implement slice 5 (in-flight expiry / backend.daemon.reclaim) or
slice 6 (typed reasons).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant