Skip to content

feat(nfs): batch READDIRPLUS fact resolution and consolidate LOOKUP owner visits (Ephemeral-AI-Lab/layerfs-project#76) - #637

Open
jiangkoumo wants to merge 14 commits into
Ephemeral-AI-Lab:layerfs_v2_0_0from
jiangkoumo:feat/lp76c-readdir-lookup
Open

jiangkoumo wants to merge 14 commits into
Ephemeral-AI-Lab:layerfs_v2_0_0from
jiangkoumo:feat/lp76c-readdir-lookup

Conversation

@jiangkoumo

@jiangkoumo jiangkoumo commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Tier: P1
Risks: risk:kernel
Gate: yes: NFS cache coherence

范围

本 PR 在 engine/crates/layerfs-{workspace,nfs,mount-ports} 上做五件事(前两件是本 PR 原有内容,后三件是 11-14 审阅返修):

  1. READDIRPLUS 条目属性按窗口批量读取。 读窗口的 owner visit 先从内存回答(本地 inode 行;CanonicalCache 里已驻留的继承 inode),只有内存没答完的窗口才取一个 Store reader,并用 inode::lookup_many 逐层读继承 serial、用一笔 provide_lengths 读其中普通文件的长度。批量数 = 「cache 尚未回答的 inode 树层数」(名字数无关、树深相关);长度是一笔,且只在 cache 额度能记住答案时。非 inline 元数据根 / 被列出子目录的子目录数 / 符号链接目标仍逐条读,与 base 相同,本 PR 不改善也不声称改善。
  2. LOOKUP 合并成一次 owner job。 解析名字的 job 已经读过目录自己的行,所以同时回报目录属性(Decided { inode, parent }),回复的 dir_attributes 不再花第二个 visit。
  3. (返修,阻断)失败 LOOKUP 不再答过期目录属性。 过去逃逸到 kernel.directory_wcc(parent)(directory_stats,本导出最后一次观测到的属性)——那张表只在再次观测时替换、只在 unlink 时删除,没有时间界,所以「at most one observation old」不成立:install 的 refresh 漏掉某目录、或同一 Workspace 的其他写入者改过它之后,每条失败的 LOOKUP(含 NFS3ERR_NOENT)都把该目录的旧 mtime/ctime 交给客户端,macOS 客户端把它装进 attribute cache 并重启 actimeo,负查询又是常态,陈旧视图可以无限延长。现在:
    • 失败 LOOKUP 用同一个 job 已读到的目录行(KernelReadDecision::Refused { refusal, directory } → KernelRead::refused_directory() → Refused { status, directory }),新鲜、no extra job;
    • .、..、silly name、以及在 visit 之前就拒绝的情况不带 dir_attributes,让客户端退回自己的 actimeo;
    • directory_wcc 只留给 AppleDouble companion(companion 变化确实不碰目录)。
  4. (返修,非阻断) finish_with 先把窗口批成「名字 + kind」、按 offer.existing() 截断,再只对要发的名字读 facts(重发回复不再为不会发送的名字读长度/元数据);memory_stat 不再重复 listing 已做的 base.inode 查找。
  5. (返修 R3B)READDIR/READDIRPLUS 的 dir_attributes 也不再回退到 directory_wcc。 回复只答本窗口读到的目录行;继承目录的窗口读不到本地行,就不带属性(详见第 11 节)。此后 directory_wcc 只服务 AppleDouble companion。

文件行数:kernel.rs 989 行逼近 999 上限,因此把「一次读的回答/拒绝类型 + port 失败→status 映射」拆到新文件 engine/crates/layerfs-nfs/src/read.rs(79 行),kernel.rs 最终 987 行;tools/check_product_boundary.py 全量 PASS。

有意变更(行为变化)

  • LOOKUP 的 dir_attributes 来源变了:成功与失败都由同一次 job 读到的目录行给出;.、..、silly name、visit 之前的拒绝一律不带目录属性(RFC 1813 允许缺省)。不再出现「答本导出最后一次观测」的情形。
  • READDIR/READDIRPLUS 的 dir_attributes 只来自本窗口读到的行:继承目录的窗口不读本地行,因此不带目录属性(RFC 1813 允许缺省),客户端退回自己的 actimeo;不再出现「答本导出最后一次观测」的情形。
  • dircount 不参与限制:READDIRPLUS 的 dircount 读取后丢弃,只有 maxcount 界住回复;文档与注释不再声称 dircount 界住名字字节。
  • READDIRPLUS 的 facts 读取顺序:names/kinds → offer 截断 → 只读发出名字的 facts。批量数与层深相关,与名字数无关。

历史

分支在 upstream/layerfs_v2_0_0(af2ad92bc)上重建;本次再合入 upstream/layerfs_v2_0_0(27920d3bb),clean merge,无冲突。历史里没有方案 a 及其回退 commit,bisect 不会落到已知回退上。推送需要 force-with-lease。(合入时上游头是 27920d3bb;之后上游又前进到 3a6e84265(#656),实测与本头 clean merge,按需补合。)

Production LOC

python3 -B tools/production_loc.py(仅第一方产品 Rust/SQL/JS;测试/文档/工具不计;reference 按 R9c absent):

  • af2ad92bc -> d0e21d073:159,977 -> 160,158(+181,本分支原有记录)
  • 其后的测试/注释/文档 commit 与两次 merge:160,158 -> 160,158(0)
  • 本次返修:67559fdfe(160,158) -> merge 27920d3bb(160,802) -> 阻断 1(160,869) -> 非阻断 2(160,847) -> 非阻断 3(160,864) -> 文档(160,864) -> R3B(160,863),最终 160,863

验证(Linux,本机;每个数字都来自本节的命令)

  • fast+slow+mount,/data/work/build2.sh /data/work/lf-ci.sh /data/wt/pr637-r3 pr637-r3b(最终头 e12c5cfda):
    • fast 1944 tests run: 1944 passed (2 slow, 1 leaky), 26 skipped rc=0(62.673s)
    • slow 148 tests run: 148 passed (3 slow), 11 skipped rc=0(61.278s)
    • mount mount binaries pass=45 fail=0 —— 这是本机 lf-ci mount lane(docker layerfs-mount-test 容器、每二进制 timeout 120 --test-threads=1、sudo modprobe fuse),不是远端机器、也不是 macOS 真机
  • PostgreSQL,/data/work/build2.sh /data/work/lf-ci-pg.sh /data/wt/pr637-r3 pr637-r3b:layerfs-persistence 25/25、layerfs-store-service 6/6、layerfs-project 1/1、layerfs-api 2/2、layerfs-cli 2/2、layerfs-control 3/3,全部 rc=0、NOT_RUN=0
  • cargo +1.85.1 clippy --locked --workspace --all-targets -- -D warnings → exit 0(最终头 e12c5cfda)
  • cargo +1.85.1 fmt --all -- --check → exit 1,唯一 diff 为上游自带的 api/crates/layerfs-cli/tests/concurrent_reads.rs:16(与 upstream/layerfs_v2_0_0 逐字节相同,上游 commit 14f62997b),本分支文件全干净
  • python3 -B tools/check_product_boundary.py PASS(1364 个生产文件);check_fuser_integrity.py exit 0;python3 -B -m unittest discover -s tools -p 'test_*.py' 182 tests OK
  • 新增测试:
    • export.rs::a_failed_lookup_answers_fresh_directory_facts_never_a_stale_observation(第二个 engine mount 改目录、本导出未观测 → 失败 LOOKUP 必须给新鲜属性、绝不能给旧属性;./../silly 给 None)
    • kernel_visit.rs::a_refused_lookup_answers_the_directory_its_job_read
    • kernel_directory.rs::a_re_sent_reply_reads_no_fact_of_a_name_it_does_not_send
    • export.rs::an_inherited_directorys_listing_never_answers_a_stale_observation(继承目录的首份 READDIRPLUS 回复必须是「无属性」,继续枚举的回复只能是当前属性或「无属性」,绝不是旧观测)
  • 三个反证(临时改回旧行为后测试确实失败):失败 LOOKUP 旧行为 → nlink 2/旧 mtime 对 nlink 3/新 mtime;facts 挪回截断前 → 读了 8 条长度而不是 2 条;READDIRPLUS 加回 directory_wcc 回退 → 继承目录的首份回复答了 nlink 2 的旧观测而不是「无属性」
  • 一次 PG 偶发:首次 PG lane 的 postgresql_case_attribution 因并发 CREATE EXTENSION IF NOT EXISTS pg_stat_statements(E23505)失败,同一命令复跑与最终头复跑均 6/6 通过

已知限制与 NOT_RUN

  • 非 inline 元数据根 / 子目录子目录数 / 符号链接目标仍逐条读(与 base 相同)。
  • 窗口的 inode 批量数 = cache 尚未回答的 inode 树层数;测试里的 1/1 依赖前置 visit 已把上层读入同一 cache 且 fixture 元数据 inline / 子目录数由 Commit 记住。
  • Durable profile:NOT_RUN — disabled by owner until explicit reauthorization。
  • macOS 真机 live 挂载 / sudo purge / 云端 Linux+PG live NFS:NOT_RUN。
  • 性能测量:NOT_RUN(本 PR 所有性能数字均为诊断值,不构成资格结论)。

Copilot AI balanced review requested due to automatic review settings November 9, 2026 04:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@yifanxuaaa

Copy link
Copy Markdown
Contributor

Review (static, at c13b065). Approve after small fixes. I found no correctness blocker:

  • the owner job does no Store I/O (memory_stat runs through CanonicalClient::resident; the Store work happens in finish_with);
  • cookies, verifier and handles are unchanged;
  • the directory_wcc fallback can't push a reply past maxcount.

The claims overstate the batching, though, and the history and measurements need cleaning up.

To fix

  1. "Any size, same number of demands" is false for directories and symlinks. kernel_directory.rs:372 base_facts runs per entry:

    • a directory calls base.subdirs(), which reads its root object and may scan the child directory;
    • a symlink calls stat_resolved_with_lengths, which reads the target;
    • each distinct non-inline metadata root is read once.

    A find window with 10 cold subdirectories therefore costs 10+ extra demands. This is unchanged from the base, but nfs.md, the code comments ("never one demand per entry") and the commit message now claim otherwise. The PostgreSQL test has one one-file subdirectory and asserts only object_batches <= small+2, while the commit says 1/1. Fix the claim, or batch those reads too.

  2. LOOKUP dir_attributes changed (nfs.rs:120-128, kernel.rs:395-420). A failed LOOKUP (ENOENT) and ./../silly names now answer the export's cached directory_stats, or no attributes, instead of a fresh GETATTR. That is RFC-legal, but state it as a deliberate change.

  3. Measurement. "Multi-sample rotations", "6/6 lower" and "+73.4% first run, +78% recheck" (a confirming rerun) break policy §3.1/§3.6, and there are no receipts or pinned identities.

  4. History. 8e8f3ff and e3667ab are the +73% approach a, reverted by 47f49be. Please squash so a bisect doesn't land on a known regression.

  5. Body vs code.

    • "dircount strictly bounds name bytes": only maxcount bounds the reply.
    • "SQLite test still runs": the whole file is #![cfg(feature = "postgresql")].
    • docs/architecture/engine/kernel-routes/ports.md:70 still shows Value(inode); it is now Decided.

Rules

merge-tree is clean on the current base, and with #636 and #642, but nothing was built.

A READDIRPLUS entry carries its attributes and its handle. On the macOS
NFSv3 client an entry without them builds no vnode, so a cold traversal
of a committed tree falls back to one synchronous LOOKUP per listed name
(diagnostic numbers only; see the pull request's measurement section).

What the reply pays is bounded per window instead of per entry wherever
the base has a batch port. The reading visit answers from memory what it
holds: the window's local inode rows, whose kinds and facts come from the
one statement that listed the names, and any inherited inode whose value,
metadata, remembered length and remembered child-directory count are
resident. Only a window memory left undecided takes the one Store reader,
and it then reads the window's inherited serials in one `lookup_many` and
its ordinary file lengths in one `provide_lengths`, so 8 and 200 names
cost the same inode and length batches.

What stays per entry, as before this change: a distinct non-inline
metadata root is read once, a listed subdirectory's child-directory count
lists that subdirectory, and a symlink entry reads its target. A window
of ordinary files with inline metadata therefore costs exactly the two
batches; a window naming those entries costs more, growing with them and
not with the number of names.

LOOKUP is one owner job (B4): the job that resolves a name has already
read the directory's row, so it reports both. Deliberate change: a
failed LOOKUP, `.`, `..` and the export's silly names answer the
directory facts this export last observed for it, or no attributes when
it has never seen the directory, instead of a fresh GETATTR of the
parent. RFC 1813 keeps those attributes optional, so both forms are
legal; the answer is now at most one observation old.

The listed directory's own `dir_attributes` are its local row or the
facts the export last observed in OPENDIR, never a read of its own, and
the publishing visit takes one kernel lookup reference per handed-out
handle in the job that makes its offset valid.

Tests: layerfs-workspace 184 passed, layerfs-nfs 28 passed,
layerfs-overlay 79 passed (1 skipped), layerfs-mount-ports 0 tests;
clippy -D warnings clean over the changed crates. Deleted tests: none.
The workspace test asserts one batch length demand, no single-key length
demand, the same object batch count at 8 and 200 names, facts for every
entry including a subdirectory, and no Store reader for a window of local
rows; the export test asserts attributes and handles on every listed
name, one owner visit for the whole enumeration, at most one Store reader
per reply and none for a local directory; the daemon's Linux mount
observer forwards the new `attributes` argument, which the previous
version of this change missed on every target that is not macOS.

Production LOC: 159977 -> 160158 (delta +181)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at upstream/layerfs_v2_0_0 (af2ad92)
and at this tree; tests, docs and tools are excluded, and the v0.1.6
reference scope is absent (retired R9c).
… batches

The batched-window tests read an in-memory fixture store, so they say
nothing about the Store a mounted Workspace actually reads. This adds one
test-only file: it mounts a Workspace on a real PostgreSQL Store and on a
real SQLite Store over the same tree, lists the root over READDIRPLUS
through the product's mount path (OPENDIR, the reading visit, the
window's facts outside the owner), and counts the port demands the window
itself makes.

Both Stores spend one batch inode demand and one batch length demand for
the whole window, and the number of names does not change either count:
`object_batches`/`length_batches` are exactly 1/1 at 8, 40 and 200
inherited names, and the one length demand names each file once (8, 200
and 40 ids). The assertions are equalities, not bounds, and the test says
why they hold: this fixture keeps every metadata root inline and builds
its subdirectory's child-directory count with the Commit, so no entry
here pays a read of its own. An entry whose metadata root is an object, a
subdirectory counted by a scan and a symlink's target are still read one
at a time.

Every entry's facts are compared field by field with a per-name LOOKUP of
the same serial and with the bytes the fixture wrote, and the reply's
offsets are published, reused unchanged and resumed.

Needs LAYERFS_TEST_POSTGRES_URL; without it the two PostgreSQL cases
report NOT_RUN and assert nothing, while the SQLite cases still run. The
whole file is behind `#![cfg(feature = "postgresql")]`, so a build
without that feature runs none of it.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at d0e21d0 and at this tree; this
change adds one test file, which is not counted.
The batching this route adds is real but the count was still overstated:
`lookup_many` walks the inode table level by level and reads each level's
pages in one batch demand, so a window pays one batch inode demand for
each level the cache does not already answer, not one for the window.
The number of names adds no demand and a deeper tree adds one.

The windows the tests count are also not a fresh mount's first window:
the visits before them read the inode table's upper level into the same
cache, so only the levels they left cost a demand. Both places now say
so, and `nfs.md` no longer calls the cost "two demands whatever its size";
it states the per-level rule and what the earlier visits make resident.

No behaviour changes: comments and documentation only, and the two
PostgreSQL assertions and the workspace assertions are untouched.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at f2973a6 and at this tree; this
change touches comments and documentation only.
The length batch pays only while the cache can remember what it answers:
`remember_length` stores nothing when the allowance is smaller than one
entry, and `file_length` then asks the provider for each file in turn.
The row that promises one length demand now says so. Documentation only.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at 64ad98b and at this tree; this
change touches one documentation row only.
The comment above the batch assertion named a measured count. The test
asserts the two sizes agree and stay below three, and what a run sees
depends on the inode tree levels the visits before the window left
unread, so the comment now says that instead of fixing a number.
Comments only; assertions unchanged.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at 71606c1 and at this tree; test
comments are outside that scope.
Update the follow-on row in nfs.md so it no longer says a window of ordinary files with inline metadata costs "exactly the two batches" (inode demands scale per unread tree level, and length batching requires cache capacity), and clarify in kernel_directory::window_reads that OPENDIR and the reading visit fill the upper inode tree level before the window's reads are counted.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at f87d032 and at this tree; this change touches docs and test comments only.
Resolved doc conflicts:
- docs/architecture/engine/kernel-routes/nfs.md: kept the new upstream structure while merging in the READDIRPLUS per-level batching/length-cache/dircount/dir_attributes rules and the LOOKUP and GETATTR section.
- docs/architecture/engine/kernel-routes/ports.md: accepted upstream deletion (its substantive Decided/LOOKUP change is ported to README.md in the following commit).

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: first-parent comparison at 871fb04 vs this merge tree via python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c).
Port the KernelRead::prepare Value(Decided) and LOOKUP parent-directory note from the removed docs/architecture/engine/kernel-routes/ports.md into docs/architecture/engine/kernel-routes/README.md.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at 4fa8381 and at this tree; documentation only.
@jiangkoumo
jiangkoumo force-pushed the feat/lp76c-readdir-lookup branch from c13b065 to 67559fd Compare November 13, 2026 10:41
@jiangkoumo

Copy link
Copy Markdown
Collaborator Author

已更新到 67559fd(在 af2ad92 上重建,再 merge 上游 de1d77e),PR 描述也已按新内容重写。逐条回复审阅:

  1. 「任意大小同样 demand 数」不成立:改正说法,不把非 inline 元数据根、子目录数、符号链接目标批量化(需给 BaseView/content 加批量端口,超出本 PR);nfs.md 与代码注释已区分「批量的部分」与「仍按条目计、与 base 相同的部分」;PG 测试断言改为等式。
  2. LOOKUP dir_attributes:写成有意变更——失败 LOOKUP(含 ENOENT)、.、..、silly 名答最后观测到的目录属性(或空),不再做新的 GETATTR;RFC 1813 允许缺省。
  3. 测量:不重测,全部降级为诊断值,逐条标明违反 §3.1/§3.6;新头性能为 NOT_RUN。
  4. 历史:在 af2ad92 上重建,方案 a 与回退 commit 不在历史里;之后 merge 上游 de1d77e(nfs.md 冲突已解,ports.md 被上游删除,说明移到 kernel-routes/README.md)。已 force-with-lease 推送。
  5. 描述与代码不符:dircount 改为只有 maxcount 界住回复;SQLite 测试的 cfg 条件如实写明;Value(inode) 改为 Value(Decided)。
  6. LOC:新 base 重算 159,977,d0e21d073 到 160,158(+181),其余提交 delta 0。
  7. kernel.rs:新头 989 行(base 969),与 fix(nfs): spill AppleDouble companion bodies to disk and expand workspace capacity to 65,536 #636 合并后约 996,仍在 999 以内。
  8. 额外发现:mounted_sqlite_full.rs 转发宏缺 attributes 参数,原 PR 头在 Linux --all-targets 编不过,已补。
  9. 自查:inode 批量数 = cache 未回答的层数(名字数无关、树深相关),测试里的 1/1 是 OPENDIR 已读入上层后的计数;「一笔长度 demand」以 cache 额度能记住为前提。均已写进 nfs.md 与测试注释。

验证(Linux,head 67559fd):fast 1900/1900,slow 146/146,mount 二进制 45 过 0 失败,PG 六包全过;postgresql_readdirplus_window 2/2。

@yifanxuaaa yifanxuaaa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review record

  • Head reviewed: 67559fdfe0. It merges cleanly with layerfs_v2_0_0 at 27920d3bb6.
  • Tier: P1 (risk:kernel: layerfs-nfs, layerfs-mount-ports). No path gate applies, but I'd add one by hand: gate: yes: NFS cache coherence, for the LOOKUP change below.
  • Read: the whole diff. Re-run: tools/check_product_boundary.py (full, without --skip-sdk) PASS; tools/production_loc.py gives 159977 -> 160158 (+181). Not run: cargo, mount tests.

Verified as correct:

  • lookup_many returns results in input order.
  • provide_lengths sends at most 1024 deduplicated ids per window.
  • The Store reader is taken outside the owner (finish_with).
  • Names, local rows and inherited facts all come from the same page root.
  • FUSE is unaffected by the Decided { inode, parent } port change.
  • dircount was already ignored on the base.

The READDIRPLUS batching is good work. One change blocks the merge:

1. A failed LOOKUP returns directory attributes that never expire (blocking)

nfs.rs (LOOKUP, around lines 105–140) answers a failure (including NFS3ERR_NOENT), ., .. and silly names with .or_else(|| kernel.directory_wcc(parent).0). That reads directory_stats (kernel.rs:550-575), which holds the last stat the export observed.

Nothing ages that entry. It is replaced only when the export observes the directory again, and removed only on unlink. The comment "at most one observation old" is therefore not a time bound.

Scenario: an install changes directory D, but the refresh misses it. That happens when more than REFRESHED inodes changed, with LAYERFS_NFS_REFRESH=0, or before the refresh SETATTR arrives.

  • Every negative LOOKUP in D then returns D's old mtime/ctime as dir_attributes.
  • The macOS client loads lookup post-op attributes into its attribute cache and restarts the timer.
  • Negative lookups are constant (.DS_Store, PATH probes), so the stale listing and name cache can outlive actimeo=60 indefinitely.

On the base, the client got fresh attributes. With no attributes, it would at least fall back to actimeo. Stale-but-present is the worst of the three.

Fix, either one:

  • carry the directory row the job already read on Refused(Missing) as well (fresh, and no extra job); or
  • send None for failures, ./.. and silly names, and keep directory_wcc for AppleDouble companions only.

If you want to keep the current behaviour, it needs the owner's explicit approval as a coherence change.

2. A re-sent reply reads facts for names it won't send (non-blocking)

kernel_directory.rs:272 calls listing_with_facts (:472) for the whole window, up to PAGE_ROWS = 1024, before offer.existing() truncates it (:289). The base read facts after truncation. Fix: batch the kinds, truncate, then run facts().

3. Duplicate inode lookup inside the owner job (non-blocking)

memory_stat (kernel_directory.rs:417) repeats a base.inode lookup the listing already did, for each name in the window, inside the owner visit. It is bounded and does no I/O, but please drop the duplicate, or report the hold time or count.

Policy

  • The PR body has no Tier: line.
  • Test results are given as summaries, without exact commands or the ≤120 s timeouts.
  • Clippy is reported for d0e21d073, not the head.
  • The Linux mount proof is the author's claim and has not been re-run by a reviewer. At P1 it must be re-run, or waived by the owner.
  • kernel.rs is 989 lines.

Reviewer: Claude Opus 5.5 (independent-context review, finding 1 spot-checked against the code). Requesting changes for 1.

jiangkoumo and others added 6 commits November 14, 2026 08:59
Clean merge: no code or documentation conflict, so no resolution was
needed. The upstream side brings Ephemeral-AI-Lab#678, Ephemeral-AI-Lab#679/Ephemeral-AI-Lab#680, Ephemeral-AI-Lab#663, Ephemeral-AI-Lab#642, Ephemeral-AI-Lab#681 and
Ephemeral-AI-Lab#647.

Production LOC: 160158 -> 160802 (delta +644)
Scope and method: first-parent comparison at 67559fd vs this merge tree
via python3 -B tools/production_loc.py over first-party product
Rust/SQL/JavaScript only (reference absent, retired R9c).
…ale one

A refused LOOKUP answered `.or_else(|| kernel.directory_wcc(parent).0)`:
`directory_stats`, the last stat this export observed for the directory.
Nothing ages that entry — it is replaced only when the export observes the
directory again and removed only on unlink — so the comment claiming "at
most one observation old" was not a time bound. An install that changes a
directory and is not observed, or any other writer to the same Workspace,
left every negative LOOKUP in it answering the directory's old mtime/ctime;
a macOS client loads those lookup post-op attributes, restarts `actimeo` and
keeps the stale listing and name cache indefinitely.

The deciding job already reads the parent's row, so its refusal now carries
it: `KernelReadDecision::Refused { refusal, directory }` sends the fresh
Inode from `decide` (a name that binds nothing) through `KernelRead`, and
`nfs.rs` answers the failure's `dir_attributes` from it — one owner visit,
fresh facts, no second job. `.`, `..`, silly names and refusals decided
before any visit read no directory row and answer `None`, so the client
falls back to its own `actimeo` instead of restarting the timer on facts
nothing refreshes. `directory_wcc` now serves AppleDouble companions only,
which is what its own comment describes, and the false time-bound comments
are gone.

`Kernel::lookup` now returns `Refused { status, directory }`; `Kernel`,
`mount-ports` and the workspace decision type carry the new payload. The
refusal types, the answer type and the port-failure mapping move from
`kernel.rs` to the new `read.rs` so no file exceeds 999 lines (kernel.rs
989 -> 987, read.rs 79).

Tests: `export.rs::a_failed_lookup_answers_fresh_directory_facts_never_a_stale_observation`
adds a name to a directory from a second engine mount of the same Workspace,
which the first export does not observe, and requires the failed LOOKUP's
`dir_attributes` to be the directory's current facts (nlink 3, current
mtime) and never the pre-change ones (nlink 2), plus `None` for `.`, `..`
and a silly name; it fails against the previous behaviour (nlink 2 and the
old mtime).
`kernel_visit.rs::a_refused_lookup_answers_the_directory_its_job_read` pins
the decision payload to the directory's current row.

Production LOC: 160802 -> 160869 (delta +67)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at the
merge tree 877a1aa and at the committed tree of this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`KernelDirectoryWindow::finish_with` called `listing_with_facts` for the
whole window — up to PAGE_ROWS = 1024 names — and only then truncated it to
the names `offer.existing()` publishes. A re-sent reply, which is exactly
the truncated case, read the facts of every name in the window and sent a
few: one batch length demand and the per-level inode batches for all of
them. The base route read facts after truncation.

Names and kinds now come first: `listing` merges the window and reads the
inherited kinds in one batch inode demand per inode-tree level (no length
demand, which belongs to facts), so both the reading visit and the request
keep their bounded per-level count. The offer truncates that window, and
`facts` then reads only the entries the reply sends. `listing_with_facts`
is gone; `inherited_values` keeps its one length demand and `inode_values`
is the kinds-only form it now splits into.

Test: `kernel_directory.rs::a_re_sent_reply_reads_no_fact_of_a_name_it_does_not_send`
publishes a two-name reply over a nine-name window and requires the re-sent
READDIRPLUS to remember exactly two lengths (`file_lengths`); reading facts
before truncation read eight.

Production LOC: 160869 -> 160847 (delta -22)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at
917e554e6 and at the staged tree of this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`memory_stat` repeated the `base.inode(serial)` lookup `listing` had just
done for every name of a READDIRPLUS window, inside the owner visit. It is
bounded and reads no Store (`resident` answers memory only), but it is the
same page walk twice per name.

`listing` now returns those base values with the listing it built
(`Kinds { listing, values }`), and `memory_stat` takes the value by
reference, so the visit reads each inherited inode once. The cold path keeps
`Kinds.values` unused: `facts` reads the names the reply sends, after the
offer truncates the window.

No new test: the duplicate was an in-memory lookup with no observable I/O,
and `kernel_directory.rs`'s window tests cover both paths.

Production LOC: 160847 -> 160864 (delta +17)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at
272fa823c and at the staged tree of this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The NFS guide still described the retired behaviour: a failed LOOKUP, `.`,
`..` and a silly name answered the directory facts the export last published
or observed, "trading up to one observation period of staleness". Nothing
ages that observation, so there is no such period; the export answered it
until it happened to observe the directory again.

Now the guide says what the source does:
- a LOOKUP that fails answers the directory's own fresh row whenever its job
  read that row, which every looked-up name does;
- `.`, `..`, a silly name and a refusal decided before any visit send no
  `dir_attributes`, so the client falls back to its own `actimeo`;
- no LOOKUP answer repeats the export's last observation;
- the remembered directory facts now serve only the AppleDouble companion
  `wcc_data` and a listed inherited directory's `dir_attributes`;
- READDIRPLUS reads names and kinds first and only the sent names' facts.

`read.rs` is added to the source list and its LOOKUP row, and the ports
guide says a refused LOOKUP carries the directory row its job read.

Production LOC: 160864 -> 160864 (delta 0)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c);
documentation only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
READDIR and READDIRPLUS answered an inherited directory's `post_op_attr` from
`.or_else(|| kernel.directory_wcc(serial).0)`: `directory_stats`, the facts
this export last observed for that directory, which its OPENDIR visit read
whenever the descriptor was opened. Nothing ages that entry — it is replaced
only when the export observes the directory again — so a directory whose
facts moved on after the open was answered as it was then, for as long as the
descriptor lived: the client loads those reply attributes, restarts
`actimeo` and keeps its listing and name cache. The upstream base has no such
fallback; this PR introduced it while changing LOOKUP.

A reply's `dir_attributes` are now exactly the row its listing window read:
fresh facts from that visit, costing no further read. An inherited directory
has no local row, so its window reads none and the reply carries no
attributes at all — RFC 1813 makes the field optional — which leaves the
client's own `actimeo` in charge instead of restarting it on an observation
nothing refreshes. `directory_wcc` now serves the AppleDouble companion
`wcc_data` alone, which is all its own comment describes.

Tests: `export.rs::an_inherited_directorys_listing_never_answers_a_stale_observation`
adds a name to an inherited directory from a second export of the same
Workspace after this export's OPENDIR observed it, and requires the first
reply's directory attributes to be absent and the continued reply's to be the
current facts (nlink 3) and never the pre-change ones (nlink 2); it fails
against the previous behaviour, which answered the observed nlink 2.
`export.rs::readdirplus_lists_a_directory_with_attributes_and_handles` now
expects no `dir_attributes` for its inherited `src`. The NFS guide's
directories table says the same, and the remembered-directory-facts row no
longer lists a listing as one of their readers.

Production LOC: 160864 -> 160863 (delta -1)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at the
parent tree 2fc186e and at this commit's staged tree. The four comment
lines and the guide rows are not production code; the net -1 is the removed
fallback line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants