Skip to content

Add git provenance to bundle deploy telemetry - #6842

Merged
atreyadbrx merged 1 commit into
mainfrom
telemetry-bundle-deploy-git
Nov 2, 2026
Merged

atreyadbrx merged 1 commit into
mainfrom
telemetry-bundle-deploy-git

Conversation

@atreyadbrx

@atreyadbrx atreyadbrx commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Adds git provenance to the bundle deploy telemetry event (BundleDeployEvent):

  • git (BundleGitInfo): origin_url, branch and commit, taken from b.Config.Bundle.Git. The load_git_details mutator has already resolved these and stripped credentials from the origin URL.
  • from_workspace_folder: true when the CLI runs on DBR with the bundle source under /Workspace/. This is the same check the source-linked deployment preset uses.

Why

The deploy telemetry doesn't record which git source a deployment came from. The CLI already resolves this on every deploy and writes it to metadata.json, and sends it to DMS for experimental.deployment_history bundles. None of it reaches Lumberjack, so deploys can't be tied to repos or commits at scale.

The matching universe proto change has merged: databricks-eng/universe#2674496. This PR should merge only after the Lumberjack schema pipeline has deployed those columns. Until then the new fields are dropped server-side.

Tests

acceptance/bundle/telemetry/deploy-git deploys inside a git-repo-init repo with an origin remote. It asserts the git block and from_workspace_folder in the recorded /telemetry-ext request, following the existing per-field telemetry tests (deploy-target-count, deploy-mode).


This pull request and its description were written by Isaac.

@github-actions github-actions Bot added the DABs DABs related issues label Sep 24, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: efde107

Run: 40980423949

Env 🔄​flaky ✅​pass 🙈​skip Time
🔄​ aws linux-2core-8gb 2 279 16 5:30
✅​ aws-windows-latest-4core-16gb 283 14 3:27
✅​ azure linux-2core-8gb 280 16 5:18
✅​ azure-windows-latest-4core-16gb 282 14 3:36
✅​ gcp linux-2core-8gb 281 16 5:16
✅​ gcp-windows-latest-4core-16gb 283 14 3:22
Test Name aws linux-2core-8gb
🔄​ TestFsCpFileToDirFileNotOverwritten 🔄​f
🔄​ TestFsCpFileToDirFileNotOverwritten/uc-volumes_to_dbfs 🔄​f
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:55 gcp linux-2core-8gb TestAccept
3:48 azure linux-2core-8gb TestAccept
3:42 aws linux-2core-8gb TestAccept
3:31 azure-windows-latest-4core-16gb TestAccept
3:21 aws-windows-latest-4core-16gb TestAccept
3:16 gcp-windows-latest-4core-16gb TestAccept

@atreyadbrx
atreyadbrx force-pushed the telemetry-bundle-deploy-git branch 2 times, most recently from c911d63 to cf4dc1a Compare September 25, 2026 00:33
@atreyadbrx
atreyadbrx force-pushed the telemetry-bundle-deploy-git branch from cf4dc1a to 0fcdce3 Compare October 8, 2026 21:23
@atreyadbrx
atreyadbrx marked this pull request as ready for review October 22, 2026 00:12
@atreyadbrx
atreyadbrx requested review from a team as code owners October 22, 2026 00:12
Populate the new git and from_workspace_folder fields on
BundleDeployEvent from b.Config.Bundle.Git and the deploy environment,
so a deploy's origin, branch and commit are captured in telemetry.
Requires the matching universe proto change to the frontend log.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: atreyadbrx <atreya.misra@databricks.com>
@atreyadbrx
atreyadbrx force-pushed the telemetry-bundle-deploy-git branch from 0fcdce3 to efde107 Compare October 30, 2026 00:32
@github-actions

Copy link
Copy Markdown
Contributor

Commit: efde10799 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0

test branch main (aed6702) latest (v1.23.0)
TestAccept/bundle/telemetry/deploy-git/DMS= ✅ ❌ ❌
TestAccept/bundle/telemetry/deploy-git/DMS=: branch ✅ | main (aed6702) ❌ | latest (v1.23.0) ❌

main (aed6702):

acceptance_test.go:1341: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1341
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1082
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:659
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle/default/files...\nFiles: 1 uploaded, 0 deleted\nResources: 0 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> cat [OUTPUT_DIR]/out.requests.txt\n{\n  \"git\": {\n    \"origin_url\": \"https://github.com/databricks/cli\",\n    \"branch\": \"main\",\n    \"commit\": \"[COMMIT_SHA]\"\n  },\n  \"from_workspace_folder\": null\n}\n"
        	            	actual  : "\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle/default/files...\nFiles: 1 uploaded, 0 deleted\nResources: 0 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> cat [OUTPUT_DIR]/out.requests.txt\n{\n  \"git\": null,\n  \"from_workspace_folder\": null\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -8,7 +8,3 @@
        	            	 {
        	            	-  "git": {
        	            	-    "origin_url": "https://github.com/databricks/cli",
        	            	-    "branch": "main",
        	            	-    "commit": "[COMMIT_SHA]"
        	            	-  },
        	            	+  "git": null,
        	            	   "from_workspace_folder": null
        	Test:       	TestAccept/bundle/telemetry/deploy-git/DMS=
        	Messages:   	bundle/telemetry/deploy-git/output.txt vs /tmp/TestAcceptbundletelemetrydeploy-gitDMS=2454429109/002/output.txt
FAIL

latest (v1.23.0): same output as main (aed6702)

@atreyadbrx
atreyadbrx added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 66ed6bc Nov 2, 2026
27 checks passed
@atreyadbrx
atreyadbrx deleted the telemetry-bundle-deploy-git branch November 2, 2026 23:11
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 66ed6bc

Run: 41543598726

Env ❌​FAIL 🔄​flaky ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 2 1502 1227 137:29
❌​ aws-windows-latest-4core-16gb 2 1427 1252 143:00
🔄​ azure linux-2core-8gb 4 1347 1279 116:46
✅​ azure-windows-latest-4core-16gb 1276 1304 127:28
✅​ gcp linux-2core-8gb 1338 1284 116:31
✅​ gcp-windows-latest-4core-16gb 1263 1309 133:36
6 interesting tests: 4 flaky, 2 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb
❌​ TestAccept ❌​F ❌​F ✅​p
❌​ TestAccept/bundle/resources/postgres_projects/factcheck/update_mask_spec ❌​F ❌​F 🙈​s
🔄​ TestFsCatOnADir ✅​p ✅​p 🔄​f
🔄​ TestFsCatOnADir/uc-volumes ✅​p ✅​p 🔄​f
🔄​ TestFsRmFile ✅​p ✅​p 🔄​f
🔄​ TestFsRmFile/uc-volumes ✅​p ✅​p 🔄​f
Top 50 slowest tests (at least 2 minutes):
duration env testname
12:26 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
11:53 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
10:40 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
10:33 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
10:25 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
10:19 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:58 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
9:46 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
9:39 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:31 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:28 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:21 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:09 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:06 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:02 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:00 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
8:57 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:31 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:30 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:12 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
8:08 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:07 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:57 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:36 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:34 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
7:25 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:10 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=
7:02 azure-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
6:53 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
6:52 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
6:49 aws linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
6:47 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
6:45 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
6:44 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:37 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:35 azure-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:31 aws linux-2core-8gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:25 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:23 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:18 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:15 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:13 azure-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:12 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:09 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:06 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:04 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:59 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:48 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:29 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:12 aws-windows-latest-4core-16gb TestAccept/bundle/deploy/spark-jar-task/DMS=true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants