Repository navigation
Remove dandi-archive compatibility kludges that no known instance needs - #1973
Open
yarikoptic-gitmate wants to merge 6 commits into
Open
yarikoptic-gitmate wants to merge 6 commits into
yarikoptic-gitmate wants to merge 6 commits into
Conversation
'dandi-staging' was removed from known_instances in 0.74.0 (#1750), and the check meant to tell users so came after `known_instances[dandi_id]`, which raises KeyError for it first. On the command line `-i` is a choice of known instances, so it never got there either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PvaGRXmMFKsLrDKSpivgHD
The retry was added for Girder (#87), which was retired in 2021. A 400 from dandi-archive or S3 says the request is wrong, and repeating it does not help. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PvaGRXmMFKsLrDKSpivgHD
The x-amz-tagging header is still sent only if signed, without failing an embargoed upload when it is not: LINC, which is still in use, runs a fork of dandi-archive 0.3.115 that does not sign it (dandi/dandi-archive#2503 added it in 0.12.14) and accepts embargoed Zarrs without it. Name LINC as what the TODO waits for, since DANDI and EMBER do sign it. mkzarr() matches both error messages for a duplicate Zarr because current dandi-archive sends both, "Zarr already exists" when it loses a race. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PvaGRXmMFKsLrDKSpivgHD
When the server's schema is newer than the client's, check_schema_version() looked the client's schema up in the server's `allowed_schema_versions`, or, for servers before dandi-archive 0.19.0 (dandi/dandi-archive#2625), in the client's own ALLOWED_INPUT_SCHEMAS. DANDI and EMBER, production and sandbox, all report the field now. LINC (schema 0.6.9) does not, but its schema is older than any client's (dandischema >= 0.12.0, schema >= 0.7.0), so it never gets here. A server without the field now allows no older client schema, as before the field was introduced. The existing test only exercised the fallback; give it the field, and test both a server-provided list and its absence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PvaGRXmMFKsLrDKSpivgHD
LINC, still in use, deploys a fork of dandi-archive 0.3.115 with schema 0.6.9, so name it alongside DANDI and EMBER as an instance to stay compatible with, and give the command showing what each instance deploys. A fallback can go when no known instance can reach it, not only when all support the feature, as with `allowed_schema_versions`. Replace the example comment quoting the reworded Zarr tagging TODO. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PvaGRXmMFKsLrDKSpivgHD
As reported by api-dandi.emberarchive.org/api/info/ on 2026-10-28. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PvaGRXmMFKsLrDKSpivgHD
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1973 +/- ##
==========================================
+ Coverage 90.82% 90.91% +0.09%
==========================================
Files 92 92
Lines 14190 14211 +21
==========================================
+ Hits 12888 12920 +32
+ Misses 1302 1291 -11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This follows up on the review of server-compatibility conditionals in #1963 ("Existing conditionals: removable now?"). Here is what each instance currently reports at
/api/info/(2026-10-28):versionschema_versionallowed_schema_versionsLINC is still in use, so anything it needs stays.
Removed
dandi-stagingcheck inget_instance(). It could never run:known_instances[dandi_id]raisesKeyErrorfirst, and on the command line-ionly accepts known instances. The id itself was removed in 0.74.0 (Add support for all known instances in short URL format, removed dandi-staging and renamed EMBER* ones into EMBER-DANDI* #1750).allowed_schema_versions. When the server's schema was newer than the client's, the client's ownALLOWED_INPUT_SCHEMASwas used in place of the missing field./info/endpoint dandi-archive#2625, 0.19.0).Kept, with comments saying why
x-amz-taggingfor Zarr uploads is still sent only when the upload URL signs it, and an embargoed upload does not fail without it.mkzarr()matching both "Zarr already exists" and "dandiset, name must make a unique set". Current dandi-archive sends both. The "unique set" text comes from the DRF upgrade in 1.0.0 (Upgradedjangorestframeworkdandi-archive#2589), but "Zarr already exists" still comes back when two creations race./server-info→/api/info/fallback and the pagination fallback. Neither is about server versions.DEVELOPMENT.md
versioncannot be compared with dandi-archive's any more than LINC's can, so deciding support by server version works only for DANDI's own instances.Not addressed here
Since 0.74.0,
dandi uploadto LINC has failed incheck_schema_version()withServer uses older incompatible schema version 0.6.9; client supports 0.7.0(or0.8.0with dandischema 0.14). The cause is that 0.6.9 is not inALLOWED_TARGET_SCHEMASand is in a different MAJOR.MINOR band. Fixing it would mean dandischema downgrading metadata to 0.6.9 (today it goes no lower than 0.6.10), or LINC upgrading its deployment. This PR does not change that behaviour.Testing
flake8,codespellandmypy dandipass.check_schema_version(), the metadata downgrade,_check_attempts_and_sleep(),dandi/tests/test_utils.pyanddandi/cli/tests/test_instances.pypass locally.test_utils.py::test_server_infofails there only because the environment cannot reach dandiarchive.org; it fails the same way on master.Generated by Claude Code