Repository navigation
feat: expose blosc.use_threads via zarr.config - #4534
sheikhayaan wants to merge 6 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4534 +/- ##
==========================================
+ Coverage 94.12% 94.71% +0.59%
==========================================
Files 92 94 +2
Lines 12830 13673 +843
==========================================
+ Hits 12076 12951 +875
+ Misses 754 722 -32
🚀 New features to boost your workflow:
|
|
|
||
| def refresh(self, *args, **kwargs): | ||
| result = super().refresh(*args, **kwargs) | ||
| numcodecs.blosc.use_threads = self.get("blosc.use_threads", False) |
There was a problem hiding this comment.
this will behave counter-intuitively if we change the default value in numcodecs to True
There was a problem hiding this comment.
Thanks for the review @d-v-b. You're right about the default behavior. Right now, Zarr hardcodes numcodecs.blosc.use_threads = False. By exposing it in zarr.config with a default of False, we are explicitly preserving Zarr's current behavior.
If numcodecs changes its default in the future, we would just need to update the Zarr config default to match it, or users can override it via zarr.config.set({"blosc.use_threads": True}).
Since you mentioned the comprehensive thread budget design in the issue thread, should I add a note in the docs here warning that this is a temporary escape hatch until the budget system is implemented?
Fixes #4531.
Following the suggestion by @rabernat, this PR exposes blosc.use_threads through the zarr.config system.
Added blosc.use_threads (default False) to zarr.core.config.
Updated zarr.codecs.blosc to read this config value.
Overrode Config.set and Config.refresh in zarr.core.config to dynamically sync the numcodecs.blosc.use_threads global state whenever the config is updated.
Added a test in tests/test_config.py to verify both the default state and dynamic updates.
Note: I couldn't run the full pytest suite locally due to a Windows Application Control policy blocking the hypothesis native DLL, but I verified the config logic manually with a script (default False -> set True -> set False) and expect CI to run the full suite.