Skip to content

Drop permission and grant entries that reference undeployed resources on migration - #7229

Draft
denik wants to merge 2 commits into
mainfrom
denik/migrate-permission-undeployed-ref
Draft

Drop permission and grant entries that reference undeployed resources on migration#7229
denik wants to merge 2 commits into
mainfrom
denik/migrate-permission-undeployed-ref

Conversation

@denik

@denik denik commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Changes

Follow-up to #7157. When migrating terraform state, a permission or grant entry that references a resource not yet deployed (e.g. service_principal_name: ${resources.apps.new_app.service_principal_client_id}) had its principal cleared, leaving an entry without a key in the migrated state. The entry was never deployed, so it is now dropped and the first direct plan adds it.

Tests

Unit tests in bundle/migrate/build_state_test.go and acceptance test bundle/migrate/auto/new-resource-ref-permissions.

This pull request and its description were written by Isaac.

@denik Isaac
Drop permission and grant entries that reference undeployed resources…
2ffcaa6 
… on migration

Clearing the principal of an entry left an entry without a key in the
migrated state. The entry was not deployed, so drop it and let the first
direct plan add it.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the DABs DABs related issues label Oct 28, 2026
@denik Isaac
Add changelog fragment
fafd106 
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

github-actions Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Commit: fafd10646 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0

test branch main (35ed095) latest (v1.22.1)
TestAccept/bundle/migrate/auto/new-resource-ref-permissions/DMS= ✅ ❌ ❌
TestAccept/bundle/migrate/auto/new-resource-ref-permissions/DMS=: branch ✅ | main (35ed095) ❌ | latest (v1.22.1) ❌

main (35ed095):

acceptance_test.go:1341: Diff:
        --- bundle/migrate/auto/new-resource-ref-permissions/output.txt
        +++ /tmp/TestAcceptbundlemigrateautonew-resource-ref-permissionsDMS=2610634363/002/output.txt
        @@ -5,235 +5,10 @@
         === Add a new app and a job permission that references its service principal
         === Plan shows the new app and the added permission
         >>> [CLI] bundle plan -o json
        -Notice: migrating your bundle to direct deployment engine (https://github.com/databricks/cli/issues/6765).
        -{
        -  "plan_version": [PLAN_VERSION],
        -  "cli_version": "[CLI_VERSION]",
        -  "lineage": "[UUID]",
        -  "serial": 7,
        -  "plan": {
        -    "resources.apps.new_app": {
        -      "action": "create",
        -      "new_state": {
        -        "value": {
        -          "description": "",
        -          "name": "new-app",
        -          "source_code_path": "/Workspace/Users/[USERNAME]/.bundle/migrate-basic-test/default/files"
        -        }
        -      }
        -    },
        -    "resources.jobs.test_job": {
        -      "action": "skip",
        -      "remote_state": {
        -        "created_time": [UNIX_TIME_MILLIS][0],
        -        "creator_user_name": "[USERNAME]",
        -        "deployment": {
        -          "kind": "BUNDLE",
        -          "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/migrate-basic-test/default/state/metadata.json"
        -        },
        -        "edit_mode": "UI_LOCKED",
        -        "email_notifications": {},
        -        "format": "MULTI_TASK",
        -        "job_id": [TEST_JOB_ID],
        -        "max_concurrent_runs": 1,
        -        "name": "Test Migration Job",
        -        "queue": {
        -          "enabled": true
        -        },
        -        "run_as_user_name": "[USERNAME]",
        -        "tasks": [
        -          {
        -            "email_notifications": {},
        -            "notebook_task": {
        -              "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/migrate-basic-test/default/files/notebook",
        -              "source": "WORKSPACE"
        -            },
        -            "run_if": "ALL_SUCCESS",
        -            "task_key": "main",
        -            "timeout_seconds": 0
        -          }
        -        ],
        -        "timeout_seconds": 0,
        -        "webhook_notifications": {}
        -      },
        -      "changes": {
        -        "email_notifications": {
        -          "action": "skip",
        -          "reason": "empty",
        -          "remote": {}
        -        },
        -        "tasks[task_key='main'].email_notifications": {
        -          "action": "skip",
        -          "reason": "empty",
        -          "remote": {}
        -        },
        -        "tasks[task_key='main'].notebook_task.source": {
        -          "action": "skip",
        -          "reason": "backend_default",
        -          "remote": "WORKSPACE"
        -        },
        -        "tasks[task_key='main'].run_if": {
        -          "action": "skip",
        -          "reason": "backend_default",
        -          "remote": "ALL_SUCCESS"
        -        },
        -        "tasks[task_key='main'].timeout_seconds": {
        -          "action": "skip",
        -          "reason": "empty",
        -          "remote": 0
        -        },
        -        "timeout_seconds": {
        -          "action": "skip",
        -          "reason": "empty",
        -          "remote": 0
        -        },
        -        "webhook_notifications": {
        -          "action": "skip",
        -          "reason": "empty",
        -          "remote": {}
        -        }
        -      }
        -    },
        -    "resources.jobs.test_job.permissions": {
        -      "depends_on": [
        -        {
        -          "node": "resources.apps.new_app",
        -          "label": "${resources.apps.new_app.service_principal_client_id}"
        -        },
        -        {
        -          "node": "resources.jobs.test_job",
        -          "label": "${resources.jobs.test_job.id}"
        -        }
        -      ],
        -      "action": "update",
        -      "new_state": {
        -        "value": {
        -          "object_id": "/jobs/[TEST_JOB_ID]",
        -          "__embed__": [
        -            {
        -              "level": "CAN_VIEW",
        -              "user_name": "viewer@databricks.com"
        -            },
        -            {
        -              "level": "CAN_MANAGE",
        -              "service_principal_name": "${resources.apps.new_app.service_principal_client_id}"
        -            },
        -            {
        -              "level": "IS_OWNER",
        -              "user_name": "[USERNAME]"
        -            }
        -          ]
        -        },
        -        "vars": {
        -          "[1].service_principal_name": "${resources.apps.new_app.service_principal_client_id}"
        -        }
        -      },
        -      "remote_state": {
        -        "object_id": "/jobs/[TEST_JOB_ID]",
        -        "__embed__": [
        -          {
        -            "level": "CAN_VIEW",
        -            "user_name": "viewer@databricks.com"
        -          },
        -          {
        -            "level": "IS_OWNER",
        -            "user_name": "[USERNAME]"
        -          }
        -        ]
        -      },
        -      "changes": {
        -        "[service_principal_name='${resources.apps.new_app.service_principal_client_id}']": {
        -          "action": "update",
        -          "new": {
        -            "level": "CAN_MANAGE",
        -            "service_principal_name": "${resources.apps.new_app.service_principal_client_id}"
        -          }
        -        }
        -      }
        -    },
        -    "resources.pipelines.test_pipeline": {
        -      "action": "skip",
        -      "remote_state": {
        -        "channel": "CURRENT",
        -        "creator_user_name": "[USERNAME]",
        -        "deployment": {
        -          "kind": "BUNDLE",
        -          "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/migrate-basic-test/default/state/metadata.json"
        -        },
        -        "edition": "ADVANCED",
        -        "effective_publishing_mode": "DEFAULT_PUBLISHING_MODE",
        -        "id": "[TEST_PIPELINE_ID]",
        -        "last_modified": [UNIX_TIME_MILLIS][1],
        -        "libraries": [
        -          {
        -            "notebook": {
        -              "path": "/Workspace/Users/[USERNAME]/.bundle/migrate-basic-test/default/files/pipeline"
        -            }
        -          }
        -        ],
        -        "name": "Test Migration Pipeline",
        -        "pipeline_id": "[TEST_PIPELINE_ID]",
        -        "run_as_user_name": "[USERNAME]",
        -        "state": "IDLE",
        -        "storage": "dbfs:[PIPELINE_TEST_PIPELINE_ID]"
        -      },
        -      "changes": {
        -        "storage": {
        -          "action": "skip",
        -          "reason": "immutable_backend_value",
        -          "remote": "dbfs:[PIPELINE_TEST_PIPELINE_ID]"
        -        }
        -      }
        -    },
        -    "resources.pipelines.test_pipeline.permissions": {
        -      "depends_on": [
        -        {
        -          "node": "resources.pipelines.test_pipeline",
        -          "label": "${resources.pipelines.test_pipeline.id}"
        -        }
        -      ],
        -      "action": "skip",
        -      "remote_state": {
        -        "object_id": "[PIPELINE_TEST_PIPELINE_ID]",
        -        "__embed__": [
        -          {
        -            "level": "CAN_MANAGE",
... (45 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.22.1): same output as main (35ed095)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: fafd106

Run: 40707509489

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 281 16 6:49
✅​ aws-windows-latest-4core-16gb 283 14 3:53
✅​ azure linux-2core-8gb 280 16 6:53
✅​ azure-windows-latest-4core-16gb 282 14 3:42
✅​ gcp linux-2core-8gb 281 16 6:35
✅​ gcp-windows-latest-4core-16gb 283 14 4:16
Top 6 slowest tests (at least 2 minutes):
duration env testname
4:04 gcp-windows-latest-4core-16gb TestAccept
3:55 gcp linux-2core-8gb TestAccept
3:48 aws-windows-latest-4core-16gb TestAccept
3:46 aws linux-2core-8gb TestAccept
3:45 azure linux-2core-8gb TestAccept
3:38 azure-windows-latest-4core-16gb TestAccept

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

DABs DABs related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@denik @eng-dev-ecosystem-bot