Skip to content

Use VecDeque for Sandbox log tails - #504

Closed
jiang1997 wants to merge 1 commit into
Ephemeral-AI-Lab:mainfrom
jiang1997:fix/sandbox-log-ring
Closed

jiang1997 wants to merge 1 commit into
Ephemeral-AI-Lab:mainfrom
jiang1997:fix/sandbox-log-ring

Conversation

@jiang1997

@jiang1997 jiang1997 commented Oct 29, 2026 •

Copy link
Copy Markdown

Sandbox::log_tail previously appended each incoming chunk to a Vec and drained excess bytes from the front, repeatedly shifting retained data once the buffer filled.

Use VecDeque to discard old bytes without shifting the retained suffix. When a chunk reaches or exceeds the byte cap, replace the buffer with that chunk's suffix.

Add tests covering capacity boundaries, repeated wraps, interleaved stdout/stderr, fragmented frames, binary bytes, and malformed responses with zero capacity.

Validation: all 3 tests in engine_log_tail pass on macOS with Rust 1.85.1; git diff --check passes.

Production LOC: 102831 -> 102838 (delta +7)
Scope: core 38395 -> 38395; engine 61777 -> 61784; api 2659 -> 2659.
Method: tools/production_loc.py --root --json on archived first-parent and staged trees, using the same counter from 582bc78.
Reference absent (retired R9c; last counted 65,417 at tree 498dd1917812).

Keep the last cap bytes without repeatedly shifting retained data.

Production LOC: 102831 -> 102838 (delta +7)
Scope: core 38395 -> 38395; engine 61777 -> 61784; api 2659 -> 2659.
Method: tools/production_loc.py --root <snapshot> --json on archived
first-parent and staged trees, using the same counter from 582bc78.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
@Mola-maker

Mola-maker commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

感谢贡献,也感谢 AI 协作工具在分析 O(n) 搬移开销时做的推演 🙏

真实评估(已读 diff):listener.rs 把 log tail 的保留缓冲从 Vec(每次追加后 drain(..) 头部搬移)换成 VecDeque,且对 n >= cap 的整窗覆盖走了 clear + 单次 extend 的捷径,避免逐段拷贝。语义保持"最后 cap 字节的精确后缀",新增的 engine_log_tail.rs 测试覆盖得很扎实:0/1/7/2048/8191/8192/8193/65536/全长/超长十种 cap、跨窗口与重复回绕、多路复用流交错、UTF-8 与二进制字节、cap=0 时仍完整校验响应。没有发现缺陷。

对 AI agent 的使用价值:log_tail 是 agent 轮询 sandbox 输出的高频路径,长输出任务的每次读取都受益。

建议:合入前在真实 Docker 后端跑一遍长输出(>16 MiB、多路复用交错)的 exec,实测确认。功能性调整等 LayerFS 2.0(#446)发布后再做。

@yhcedpn yhcedpn added this to the LayerFS v2.0.0 milestone Nov 6, 2026
@yhcedpn yhcedpn assigned jiangkoumo and unassigned jiangkoumo Nov 6, 2026
@jiangkoumo

Copy link
Copy Markdown
Collaborator

Superseded by 2.0: the Docker listener this changes was removed, and log_tail now reads a bounded window of the daemon log file (control/crates/layerfs-sandbox-backend). Thanks for the thorough tests.

(Reviewed against layerfs_v2_0_0 @ 716197c during the Staging cleanup.)

@jiangkoumo jiangkoumo closed this Nov 8, 2026
@yifanxuaaa yifanxuaaa added the new feature A new user-facing capability label Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new feature A new user-facing capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants