Add py3-lowest tox env and CI mode; tighten dependency lower bounds - #1897
Merged
Conversation
Introduces a py3-lowest tox env that installs the lowest declared-compatible version of every direct dependency using uv's `lowest-direct` resolution, run on Python 3.11 (the minimum supported version per `requires-python`). Purpose is to verify that the lower bounds in pyproject.toml are actually valid and that dandi still works against them. Wires into GitHub Actions as a new matrix mode `lowest-deps` on ubuntu-latest / Python 3.11. Direct deps that currently lack a lower bound will resolve to their oldest release and cause the env to fail; that failure is the intended signal to add real bounds. Co-Authored-By: Claude Code 2.1.221 / Claude Opus 4.7 <noreply@anthropic.com>
Now that the `py3-lowest` tox env exercises `uv --resolution lowest-direct`,
every direct dep needs a real lower bound — otherwise uv picks the oldest
published release (often from ~2015) and everything breaks. This adds bounds
targeting the first release with plausible Python 3.11 support (typically
released on or after 2022-10-24) plus fixes for API-compat issues surfaced
by actually installing and importing:
- pydantic ~= 2.0 → ~= 2.9 (dandi uses `BeforeValidator(json_schema_input_type=...)`
added in 2.9 and `FtpUrl` added in 2.8; and dandischema 0.12 requires
a pydantic that properly validates `field_serializer("*")`, which is 2.7+)
- anys ~= 0.2 → ~= 0.3 (0.2.x references non-existent `types.Union` on 3.11)
- duecredit >= 0.6.0 → >= 0.9.2 (0.6 imports `collections.Iterator`, gone in 3.10)
- pyout >= 0.5 → >= 0.7.3 (0.5 uses jsonschema `additionalItems`, rejected by
modern jsonschema Draft 2020-12 metaschema)
- tensorstore: added >= 0.1.62 (first release built for numpy 2.x ABI)
- opencv-python >= 4.7.0.72 → >= 4.10.0.84 (older wheels built against
numpy 1.x ABI segfault on `import cv2` under numpy 2.x, which
tensorstore >= 0.1.62 now brings in)
- vcrpy >= 4.3.0 → >= 5.0.0 (4.3 references
`urllib3.connectionpool.VerifiedHTTPSConnection`, removed in urllib3 2.x;
5.0 switched to `urllib3.connection.VerifiedHTTPSConnection`)
For the remaining 20 direct deps without a lower bound, pick the earliest
release published on or after Python 3.11's release date (2022-10-24) whose
metadata admits 3.11, as determined by querying PyPI.
Verified: with these bounds `tox -e py3-lowest` resolves and installs cleanly,
`dandi` imports, `pytest --collect-only` gathers 1018 tests, and 601 unit
tests pass under the lowest-direct resolution on Python 3.11.
Co-Authored-By: Claude Code 2.1.221 / Claude Opus 4.7 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1897 +/- ##
=======================================
Coverage 76.96% 76.96%
=======================================
Files 88 88
Lines 12882 12882
=======================================
Hits 9914 9914
Misses 2968 2968
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…solves `uv --resolution lowest-direct` picks the *lowest satisfying all constraints*, so if a peer transitively requires more, the direct dep's real lowest is whatever the resolver ends up picking — not what we declared. Inspecting the first successful `lowest-deps` CI run against SHA 272b8c5 showed that 8 direct deps floored ABOVE their declared minimums. That means those declared minimums were never actually being exercised by CI. Fix by raising the floors to match what the resolver is picking, so the declared floor is the tested floor: | Package | Declared -> Raised to | Forced-up by (transitively) | |--------------|------------------------|-------------------------------------------------| | hdmf | 3.4.7 -> 4.1.0 | modern pynwb pins hdmf >= 4 | | pynwb | 1.0.3 -> 3.1.0 | dandischema 0.12 / nwbinspector 0.7 | | platformdirs | 2.5.3 -> 4.1.0 | keyring 23.10 jaraco.classes chain requires >= 4 | | requests | ~= 2.20 -> ~= 2.30 | urllib3 2.x requires requests >= 2.30 | | urllib3 | 2.0.0 -> 2.0.2 | trivial transitive peer | | yarl | ~= 1.9 -> >= 1.9.1 | trivial transitive peer | | ruamel.yaml | >= 0.15 -> >= 0.16 | modern hdmf/pynwb chain | | zarr | >= 2.10 -> >= 2.18 | numcodecs 0.11 / hdmf-zarr chain | The `!= 3.5.0`/`!= 3.14.4` exclusions on hdmf and `!= 1.1.0`/`!= 2.3.0` on pynwb are dropped because the new floors are above all excluded versions. No functional change: the resolver was already picking these versions on Python 3.11; this just makes `pyproject.toml` honest about the tested floor. Co-Authored-By: Claude Code 2.1.221 / Claude Opus 4.7 <noreply@anthropic.com>
yarikoptic
marked this pull request as ready for review
August 5, 2026 03:00
yarikoptic-gitmate
approved these changes
Aug 5, 2026
yarikoptic-gitmate
approved these changes
Aug 5, 2026
|
🚀 PR was released in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Inspired by work on "downgrade metadata" PR functionality so we could ensure that minimal dandischema version dependency holds.
Summary
Adds a
py3-lowesttox environment and matchinglowest-depsCI matrix modethat install every direct dependency at its declared lower bound (via
uv's--resolution lowest-direct) on Python 3.11 — the minimum supported version.This turns "the lower bounds in
pyproject.tomlare valid" into something CIactually verifies, instead of something we assume.
Bringing that env up green surfaced 26 direct deps with no lower bound at all
(so
uvwould happily pick a release from ~2015), plus five cases where theexisting bound was too loose to import cleanly on 3.11. Both are addressed in
the second commit.
Notable version bumps (with reason)
pydantic~= 2.0→~= 2.9— dandi usesBeforeValidator(json_schema_input_type=…)(added in 2.9) andFtpUrl(2.8); dandischema 0.12 requires a pydantic that validatesfield_serializer("*")properly, which is 2.7+.anys~= 0.2→~= 0.3— 0.2.x referencestypes.Union, which does not exist on Python 3.11.duecredit>= 0.6.0→>= 0.9.2— 0.6 importscollections.Iterator, removed in 3.10.pyout>= 0.5→>= 0.7.3— 0.5 uses jsonschemaadditionalItems, rejected by the Draft 2020-12 metaschema.tensorstore: pinned>= 0.1.62— first release built for the numpy 2.x ABI.opencv-python>= 4.7.0.72→>= 4.10.0.84— older wheels were built against the numpy 1.x ABI andimport cv2fails oncetensorstore >= 0.1.62pulls numpy 2.x in transitively.vcrpy>= 4.3.0→>= 5.0.0— 4.3 referencesurllib3.connectionpool.VerifiedHTTPSConnection, removed in urllib3 2.x; 5.0 switched tourllib3.connection.VerifiedHTTPSConnection.For the remaining 20 previously-unbounded direct deps, picked the earliest
release published on or after Python 3.11's release date (2022-10-24) whose
metadata admits 3.11, determined by querying PyPI.
Local verification (on Python 3.11)
tox -e py3-lowest --notest— resolves cleanly, installs 104 packages.dandiimports cleanly under the resulting env.pytest --collect-onlycollects 1018 tests.Full-suite runs (docker-compose fixture, obolibrary, etc.) are left to CI.
Test plan
lowest-depsmatrix job on this PR passes (or surfaces a further bump weshould add to
pyproject.tomlas a direct-dep lower bound).dandi-api,dev-deps,nfs,obolibrary-only) unaffected — the bumps only raise floors, nevercap upper bounds.