[timtreis]

Summary

First tutorial that builds something on top of the whole catalog: every Visium Spatial Gene Expression table (4,008 sections, 7 mammals) goes through the client into one integrated, GPU-computed atlas. The tutorial asks whether spots cluster by species or by organ.

Client: to_anndata() / concat_tables()

New keyword arguments. Defaults keep the current behaviour.

• max_workers: members are read in parallel. Reading all 4,008 Visium tables with 16 workers takes about 85 minutes.
• obs_columns: catalog columns (organism, tissue, …) are copied into obs as categoricals.
• elements: reads only some table elements, e.g. ("X", "obs", "var"), so duplicate count layers aren't downloaded.
• adapter: a new TableAdapter protocol, the table-level counterpart of TaskAdapter. requirements() sets the elements to read, validate_meta(row) runs before the download, validate(adata, row) runs on the table as read, and build(adata, row) maps the table into the shared space (None drops the member). Skipped members and their reasons are recorded in uns["sddb"] together with the adapter's name and version.

Tests cover parallel ordering, the unknown-column error, element selection, adapter skips at both stages, build returning None, requirements selecting elements, and the protocol check.

Closes #29.

Tutorial: docs/guides/visium-atlas/

• pixi.toml and pixi.lock: CUDA 13 / RAPIDS 26.10 plus the prebuilt rapids-singlecell-cu13 wheel, following rapids-singlecell's recommended conda env. Nothing is compiled.
  • Strict channel priority is disabled, as RAPIDS supports.
  • PYTHONNOUSERSITE=1 stops ~/.local CUDA wheels from shadowing the env.
  • Needs an Ampere or newer GPU (CUDA 13 dropped Volta).
• The notebook:
  1. Uses a pinned catalog (2026-10-22) and reads tables only.
  2. Maps genes to human 1:1 orthologs in an OrthologAdapter (the reference TableAdapter) (Ensembl Compara 116), keyed by Ensembl ID prefix. This handles rat sections aligned to mouse and GRCh38_-prefixed IDs.
  3. Restricts every section to the 12,162 genes with an ortholog in all species. Otherwise human-only genes would separate species by construction.
  4. Runs everything in rapids-singlecell with RMM managed memory: normalize, log1p, HVG, PCA, Harmony (4 batch definitions), the all_neighbors kNN graph and UMAP, on all 10.6M spots.
• nb_execution_mode is off, so RTD renders the committed outputs. The run was done on the Helmholtz HPC.

Result

After Harmony, spots group by organ: brain/cortex, liver, kidney, heart and gut each form their own region, with human and mouse mixed inside each. Spots do not split by species.

Test plan

• pytest: 328 passed, 4 skipped
• Full run on a B300: 4,008 sections → 10.6M spots × 12,162 genes. Preprocessing to PCA took about 20 minutes; integration, kNN and UMAP took about 10 minutes.
• Notebook executed end to end on a B300 and committed with outputs: 3,954 of 4,008 sections kept (35 gene-symbols-only, 19 dropped by QC, as listed in uns["sddb"]), 10.6M spots. Read + map took about 60 minutes, preprocessing to PCA about 20 minutes, and integration plus plots about 6 minutes.

Follow-ups: #30 (streaming/cached to_anndata), #31 (embedding contract and evaluation), theislab/spatialdata-db#241 (normalize gene IDs at ingest).

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[read-the-docs-community]

Documentation build overview

📚 spatialdata-db | 🛠️ Build #35360405 | 📁 Comparing 266769c against latest (88846e1)

  🔍 Preview build  

10 files changed · + 2 added · ± 8 modified

+ Added

• guides/visium-atlas/visium-atlas.html
• reference/generated/sddb.TableAdapter.html

± Modified

• changelog.html
• genindex.html
• guides/cli-and-agents.html
• guides/ml.html
• reference/api.html
• reference/generated/sddb.Manifest.html
• reference/generated/sddb.SpatialDataCohort.html
• reference/generated/sddb.TaskAdapter.html

[codecov]

Codecov Report

❌ Patch coverage is 96.42857% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.24%. Comparing base (88846e1) to head (266769c).

Files with missing lines	Patch %	Lines
src/sddb/concat.py	95.45%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #28      +/-   ##
==========================================
+ Coverage   92.15%   92.24%   +0.08%     
==========================================
  Files          18       18              
  Lines        1466     1508      +42     
==========================================
+ Hits         1351     1391      +40     
- Misses        115      117       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.