Repository navigation
filer: import files the way the legacy import-file endpoint did - #7034
Merged
Merged
Conversation
/workspace/import with format=AUTO types files by name before looking at content: it rejects names reserved for other assets (.geniespace.json, .dbjob.json, .lwquery.ipynb, ...) and imports .zip archives as folders. This broke Genie Space deploys in the nightly after the switch from import-file. Notebooks are now imported without their extension with an explicit SOURCE/JUPYTER format, dashboards, alerts and Lakeflow Designer files keep AUTO, and everything else uses RAW, which always stores the bytes as a file. Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
Integration test reportCommit: 1de7093
Top 6 slowest tests (at least 2 minutes):
|
import-file extracts a .zip into its parent directory and overwrites existing entries. /workspace/import imports it as a new folder at the archive's path that cannot be overwritten, which breaks re-syncing a bundle's .zip and the ssh tunnel binary upload. Keep zips on import-file until /workspace/import supports that. The testserver now extracts zips posted to import-file, so the ssh tests stage valid (empty) placeholder archives. Co-authored-by: Isaac <no-reply@databricks.com>
Extracting them made the ssh tests' empty placeholder archives fail. The acceptance test notes the real endpoint's extraction instead, as it already does for dashboards. Co-authored-by: Isaac <no-reply@databricks.com>
…ives AUTO extracts a .zip into a folder at the path without its extension and never overwrites an existing object there. Both the JSON and multipart variants now share one implementation. The new acceptance test documents this, and the reserved-name rejection, against the testserver and a real workspace. Co-authored-by: Isaac <no-reply@databricks.com>
Sankalp-Mittal
requested review from
janniklasrose and
shreyas-goenka
and removed request for
denik and
renaudhartert-db
October 13, 2026 12:40
janniklasrose
approved these changes
Oct 13, 2026
Sankalp-Mittal
added a commit
that referenced
this pull request
Oct 28, 2026
The revert of #7034 restored a direct comparison that errorlint now flags. Co-authored-by: Isaac <no-reply@databricks.com>
Sankalp-Mittal
added a commit
that referenced
this pull request
Oct 28, 2026
The revert restored the pre-#7034 shape of DetectWithFS, which behaves the same for files on disk. Keep the main version instead so the PR does not touch libs/notebook, and let the testserver use DetectBytes for JUPYTER imports again. Co-authored-by: Isaac <no-reply@databricks.com>
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Oct 29, 2026
## Release v1.23.0 ### CLI * Fix GitHub API rate limit errors in `aitools list` and plugin installs by pinning skills to `0.2.32`. ([#7159](#7159)) * `databricks ssh connect` now makes `claude` and `codex` available on dedicated clusters. ([#7190](#7190)) * `databricks ssh connect` on serverless now makes `claude` and `codex` available in IDE sessions (`--ide`) and to non-interactive commands (e.g. `databricks ssh connect -- 'claude -p "..."'`). ([#7123](#7123)) ### AI Runtime * Add workload support for the GPU_1xRTX6000 accelerator type in enabled workspaces. ([#7214](#7214)) * Show only the accelerator types available in the current workspace in `databricks air run -h config.compute`. ([#6993](#6993)) ### Bundles * Fixed `bundle validate` failing with "expected a map to index" when a job `schedule`, `trigger` or `continuous` is a reference to a whole block, such as `${resources.jobs.base.schedule}`. ([#7225](#7225)) * Added `lifecycle.depends_on` to order resources without referencing their fields. ([#7120](#7120)) * Immutable-folder deploys now refuse to deploy onto a snapshot that was modified outside the bundle (break glass); `bundle deploy --force` moves the deployment to a fresh snapshot. ([#6774](#6774)) * Fix `--include-locations` failing when a resource is missing a required field. ([#7185](#7185)) * Removed spurious "required field is not set" warnings for a field whose value is a reference to a whole block, such as `schedule: ${resources.jobs.base.schedule}`. ([#7226](#7226)) * Revert bundle file uploads to the legacy `/workspace-files/import-file` endpoint, rolling back [#6149](#6149) and [#7034](#7034) released in [v1.21.0](https://github.com/databricks/cli/releases/tag/v1.21.0). This fixes notebooks being uploaded as plain files and deployments failing (for example SQL notebooks in [#7216](#7216)). A remote file or notebook whose type differs from the local one is now deleted and replaced instead of blocking the deploy. ([#7218](#7218)) * Support selecting resources for deletion with `bundle deploy --select`. ([#7023](#7023)) * Add the `workspace.sync_mode` setting with the values `overwrite` (default), `in-place`, and `immutable`. It replaces `presets.source_linked_deployment` (use `in-place`) and `experimental.immutable_folder` (use `immutable`), which are now deprecated and emit a warning. ([#7183](#7183)) ### Dependency Updates * Bump dependencies with known vulnerabilities. ([#7213](#7213)) * Update the Go SDK to [v0.190.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.190.0). ([#7155](#7155)) * Update the Go SDK to [v0.191.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.191.0). ([#7211](#7211)) * Bump Node.js downloaded by the `ssh` agent shim from v24.21.0 to v24.22.2. ([#7246](#7246))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
After #6149 moved uploads to
/workspace/importwithformat=AUTO, the nightly broke:Requested node type [UNKNOWN] is not supportedon.geniespace.json. AUTO types a file by its name before looking at its content, and it rejects names reserved for asset types it can't import (.geniespace.json,.dbjob.json,.dbpipeline.json,.lwquery.ipynb, ...).ssh/bulk-transferfails withOverwrite cannot be used for AUTO format when importing a folder. AUTO imports a.zipas a new folder at the archive's path that can't be overwritten. The legacyimport-fileendpoint extracted it into its parent folder instead, which the ssh tunnel binary upload relies on.The same
.zipproblem also breaks re-syncing any bundle that contains a zip.The filer now uploads each file the way
import-filestored it:libs/notebook): imported without the extension, with an explicitSOURCE+ language orJUPYTERformat..lvdash.json,.dbalert.json,.designer.py/.ipynb: keepAUTO, so they still become a dashboard, an alert or a Lakeflow Designer file..zip: stays on the legacyimport-fileendpoint until/workspace/importcan extract into an existing folder.RAW, which always stores the bytes as a file.The testserver now accepts
RAW/SOURCE/JUPYTER, rejects reserved names underAUTO, and mirrors AUTO's.zipfolder import, including rejecting a re-import (with or without overwrite), as the real endpoint does.Tests
sync-upload-edge-casesnow also syncs.geniespace.json,.dbjob.json, a.zipand Jupyter notebooks (includingquery.lwquery.ipynb), and records the format and endpoint used for each upload.cmd/workspace/import-zip-auto(Cloud = true) documents the endpoint's behaviour: AUTO extracts a.zipinto a folder and rejects re-importing it; RAW stores it as a file; AUTO rejects.geniespace.jsonand RAW accepts it. It produces identical output against the testserver and a real workspace.notebook.DetectBytes.import-fileproduced, including the zip being extracted and overwritten on re-sync.deco env run -n aws-cli:ssh/bulk-transfer, allgenie_spacestests, the dashboard and alert resource tests,cmd/workspace, and thedefault-sql/lakeflow-pipelinestemplates all pass.This pull request and its description were written by Isaac.