Skip to content

Document the Airflow 3 Azure Data Factory connection form - #84

Open
gams87 wants to merge 6 commits into
mainfrom
airflow3-adf-connection-docs
Open

gams87 wants to merge 6 commits into
mainfrom
airflow3-adf-connection-docs

Conversation

@gams87

@gams87 gams87 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The Run ADF Pipeline page only showed Airflow 2. Airflow 3 environments (available since 6.0) have a different connections UI, so the page no longer matched what customers see:

  • The Admin menu is in the left sidebar (Admin -> Connections).
  • In the Add Connection form, Client ID and Secret (the connection's login and password) are under Standard Fields; Tenant ID, Subscription ID, Resource Group Name and Factory Name are extras. The page has people fill them in Extra Fields JSON, whose editor already lists the Azure Data Factory keys with null values.
  • The connections list shows the connection type by its id, azure_data_factory, while the type picker shows "Azure Data Factory".

Changes:

  • Steps 1 and 2 of "Create a Microsoft Azure Data Factory Connection in Airflow" get Airflow 2 / Airflow 3 tabs (Airflow 2 first and selected by default, the two tab groups stay in sync). The Airflow 2 content and screenshots are unchanged.
  • New Airflow 3 screenshots (Admin menu, standard fields, extra fields JSON, connections list), taken on a local environment with example values.
  • Fixes a "Databricks connection" leftover in the example DAG section.
  • Replaces the example DAG with Noel's: it starts the pipeline, waits for it with a deferrable sensor (poke_interval 60 s, timeout 2 h) and then runs a dbt task. Resource Group Name and Factory Name come from the connection, so the code doesn't set them. The example has Airflow 2 / Airflow 3 tabs synced with the others (only the import changes: airflow.decorators vs airflow.sdk), and "Understanding the Airflow DAG" explains each part. Both versions were checked with mocks on Airflow 2.11.2 + azure provider 12.6.1 and on Airflow 3.3.1 + 14.1.0.

The Azure provider (apache-airflow-providers-microsoft-azure) ships in every release (12.6.1 in 5.0, 6.0 and 6.1; 14.1.0 for Airflow 3 in 6.2), so this applies to all current releases and does not need to wait for a release.

npm run build passes; its only broken-anchor warning (dag-parse-times -> datacoves-my-parse-logs) is also on main.

Checklist

  • [Mandatory] My pull request represents one logical piece of work and my commits are related and look clean.

🤖 Generated with Claude Code

gams87 and others added 2 commits November 18, 2026 16:50
…s under Extra Fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ithout dev overlays

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alejandro-datacoves pushed a commit that referenced this pull request Nov 18, 2026
This reverts commit 52299de.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent ID and Secret as login and password

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gams87
gams87 requested a review from noel November 19, 2026 14:26
…sor and then runs dbt

Adds Airflow 2 and Airflow 3 tabs to the example, synced with the connection tabs.
The DAG reads Resource Group Name and Factory Name from the connection.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants