`to_thread` is slow

[d-v-b]

We use to_thread in the BatchedCodecPipeline for running CPU-heavy work on a separate thread. to_thread is slow, according to python/cpython#136084 and also our observations in the development of the synchronous codec pipeline. Something faster than to_thread (e.g., a plain thread pool!) but also "coroutiney" for CPU-heavy work would be worth investigating.

[ngoldbaum]

I'm interested in this, however I'm not terribly familiar with zarr. I do see quite a bit of to_thread uses in the zarr implementation.

Is there any chance you could put together a runnable example that demonstrates the slowness? That way I'll have a concrete example to share with the asyncio experts.