Repository navigation
0.6.0: shared_context(threads=) - #10
Merged
Merged
Conversation
… own (0.6.0) chunkmirage's server keeps one context per cache budget for every store it opens, at its own --threads, and under `bsub -n 1` the default cap gives it one thread per pool. threads=None keeps 0.5's thread_budget() limit, N limits both pools to N, 0 writes no limit. Each (cache_bytes, threads) is one watched context per process, and a pickled handle comes back on the same one. Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
… blocks c-elegans measured on /nrs, op by op from one process in a two-slot job: one-chunk blocks are the same at limits 1 and 4, 8-chunk blocks read and write about twice as fast at 4. A one-slot worker is unmeasured. Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
davidackerman
marked this pull request as ready for review
October 23, 2026 16:45
c-elegans' bsub -n 1 run: 8-chunk blocks read 147 against 119 ms and write 325 against 302 ms at limits 1 and 4, against 2x in a two-slot job. Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
0.6.0: one new argument,
wa.shared_context(cache_bytes, threads=), at chunkmirage's request. No placement change and no unit table change. Nothing changes for a caller that doesn't pass it.threads=None(the default) keeps 0.5's limit ofthread_budget()on the decoding and file I/O pools.threads=Nlimits both pools to N.threads=0writes no limit, so tensorstore sizes the pools itself, at every CPU on the node.Each
(cache_bytes, threads)pair gets one watched context per process. A pickled handle on one comes back on the same one:recipe_ofgives("shared", (cache_bytes, threads)). A shared context at the default limit still pickles as("shared", cache_bytes), so the other process sizes it by its own job.Why: chunkmirage's server keeps one context per cache budget, shared by every store it opens, at its own
--threads. Underbsub -n 1the default cap gives it one thread per pool. It can't usecapped_contexthere, because that needs a count and returns a new context on every call. Its draft PR #53 builds and watches its own contexts in about 12 lines. Once this ships, those lines becomewa.shared_context(cache_bytes, threads=0).This change also updates:
0.6.0 (unreleased).Also documented in spec Behaviour 7 and the
capped_context_specdocstring: what a pool of one costs. c-elegans measured it on /nrs on 2026-10-23, from one process, alternating the limit op by op:So most of the gain is a core the worker wasn't given.
185 tests pass locally (1 opt-in stress test skipped), and ruff passes.
https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64