Skip to content

feat: expose blosc.use_threads via zarr.config - #4534

Open
sheikhayaan wants to merge 6 commits into
zarr-developers:mainfrom
sheikhayaan:blosc-config-use-threads
Open

sheikhayaan wants to merge 6 commits into
zarr-developers:mainfrom
sheikhayaan:blosc-config-use-threads

Conversation

@sheikhayaan

Copy link
Copy Markdown
Contributor

Fixes #4531.

Following the suggestion by @rabernat, this PR exposes blosc.use_threads through the zarr.config system.

Added blosc.use_threads (default False) to zarr.core.config.

Updated zarr.codecs.blosc to read this config value.

Overrode Config.set and Config.refresh in zarr.core.config to dynamically sync the numcodecs.blosc.use_threads global state whenever the config is updated.

Added a test in tests/test_config.py to verify both the default state and dynamic updates.

Note: I couldn't run the full pytest suite locally due to a Windows Application Control policy blocking the hypothesis native DLL, but I verified the config logic manually with a script (default False -> set True -> set False) and expect CI to run the full suite.

@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.71%. Comparing base (ce10c0b) to head (0abb755).
⚠️ Report is 137 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4534      +/-   ##
==========================================
+ Coverage   94.12%   94.71%   +0.59%     
==========================================
  Files          92       94       +2     
  Lines       12830    13673     +843     
==========================================
+ Hits        12076    12951     +875     
+ Misses        754      722      -32     
Files with missing lines Coverage Δ
src/zarr/codecs/blosc.py 97.63% <100.00%> (+0.01%) ⬆️
src/zarr/core/config.py 100.00% <100.00%> (ø)

... and 50 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread src/zarr/core/config.py

def refresh(self, *args, **kwargs):
result = super().refresh(*args, **kwargs)
numcodecs.blosc.use_threads = self.get("blosc.use_threads", False)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will behave counter-intuitively if we change the default value in numcodecs to True

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @d-v-b. You're right about the default behavior. Right now, Zarr hardcodes numcodecs.blosc.use_threads = False. By exposing it in zarr.config with a default of False, we are explicitly preserving Zarr's current behavior.

If numcodecs changes its default in the future, we would just need to update the Zarr config default to match it, or users can override it via zarr.config.set({"blosc.use_threads": True}).

Since you mentioned the comprehensive thread budget design in the issue thread, should I add a note in the docs here warning that this is a temporary escape hatch until the budget system is implemented?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Large Blosc chunks decode about 3x slower than in zarr 2 because numcodecs.blosc.use_threads is set to False on import

2 participants