Skip to content

Drop Python 3.10 from CI, Breeze and the container images - #74151

Open
shahar1 wants to merge 2 commits into
mainfrom
drop-py310/1-ci-breeze-docker
Open

Drop Python 3.10 from CI, Breeze and the container images#74151
shahar1 wants to merge 2 commits into
mainfrom
drop-py310/1-ci-breeze-docker

Conversation

@shahar1

@shahar1 shahar1 commented Oct 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Python 3.10 reached EOL on Oct. 1, 2026, and according to Apache Airflow policy we should drop its support as well. This PR removes 3.10 from the CI matrix, Breeze, and the Docker images, changing the min. version to 3.11.

related: #74148

AI Summary Python 3.10 reaches end of life in October 2026 and a growing number of dependencies have already dropped it. This layer removes 3.10 from the CI matrix, Breeze and the images, and moves the default Python to 3.11.
  • Breeze: ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS / ALL_PYTHON_MAJOR_MINOR_VERSIONS are 3.11–3.14, providers compatibility matrix runs on 3.11, dead PYTHON_TO_MIN_AIRFLOW_MAPPING + unused is_airflow_version_supported_for_python removed. Breeze command output docs regenerated (the ~100 SVG/TXT files are hook output).
  • CI: default 3.11 in composite actions and workflows; the --no-explain-why workaround for 3.10 in finalize-tests.yml goes away.
  • Images: default AIRFLOW_PYTHON_VERSION is 3.11; the PGP branch for verifying the Python tarball is removed (3.11+ publish Sigstore bundles), python-3.10.asc and its key entry go with it.
  • Scripts/tests: 3.11 defaults in scripts/, test constants and prek utility messages; get_min_airflow_version_for_python keyed from 3.11.
  • README/prerequisites regenerated by the update-tested-versions hook; newsfragment added.

Deliberately kept: .github/actions/prepare_all_ci_images still restores 3.9/3.10 images (shared with release branches, #45268); ALL_HISTORICAL_PYTHON_VERSIONS, PYTHON_3_10_TO_3_14, ALL_PYTHON_VERSION_TO_PATCHLEVEL_VERSION["3.10"] stay for older Airflow releases.

Distributions still declare Python 3.10 in this layer; layer 7 bumps them once the code carries no 3.10-only constructs.

Layer 1 of 9 of the Python 3.10 drop (supersedes #74144, which carried the whole change in one PR). Each layer is meant to be green on its own: the script metadata bump comes first because prek picks the hook interpreter from it, and the ruff fixes land before the requires-python bump because ruff infers its target per distribution from the nearest pyproject.toml.

  1. Drop Python 3.10 from CI, Breeze and the container images
  2. Bump inline script metadata to Python 3.11
  3. Remove Python 3.10 compatibility shims
  4. Apply ruff Python 3.11 fixes to core, Task SDK, shared and airflowctl
  5. Apply ruff Python 3.11 fixes to providers
  6. Apply ruff Python 3.11 fixes to dev tooling, scripts and test suites
  7. Require Python 3.11 in all distributions
  8. Update provider READMEs for the Python 3.11 baseline
  9. Update docs for the Python 3.11 baseline
Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Fable 5.1)

Generated-by: Claude Code (Fable 5.1) following the guidelines

@shahar1
shahar1 force-pushed the drop-py310/1-ci-breeze-docker branch from 2c24528 to f1f9d33 Compare October 3, 2026 13:07
@shahar1
shahar1 added this pull request to stack #74160 October 3, 2026 13:08
@shahar1 shahar1 added full tests needed We need to run full set of tests for this PR to merge all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs labels Oct 3, 2026
@shahar1 shahar1 mentioned this pull request Oct 3, 2026
Closed
1 task done
shahar1
shahar1 commented Oct 3, 2026
View reviewed changes
Comment thread dev/refresh_images.sh
shahar1
shahar1 commented Oct 3, 2026
View reviewed changes
Comment thread scripts/ci/prek/update_docker_gpg_keys.py
shahar1
shahar1 commented Oct 3, 2026
View reviewed changes
Comment thread scripts/ci/testing/get_min_airflow_version_for_python.py
@shahar1
shahar1 marked this pull request as ready for review October 3, 2026 13:40
@shahar1
shahar1 requested a review from vatsrahul1001 as a code owner October 3, 2026 13:40
@shahar1 shahar1 added this to the Airflow 3.4.0 milestone Oct 3, 2026
@shahar1 shahar1 removed this from the Airflow 3.4.0 milestone Oct 3, 2026
This was referenced Oct 3, 2026
Merged
Merged
Merged
@zozo123 zozo123 mentioned this pull request Oct 3, 2026
Open
1 task done
@shahar1
shahar1 force-pushed the drop-py310/1-ci-breeze-docker branch 2 times, most recently from 0528099 to 0965104 Compare October 4, 2026 06:41
@shahar1
shahar1 requested a review from o-nikolas as a code owner October 4, 2026 06:41
@shahar1 shahar1 mentioned this pull request Oct 4, 2026
Merged
1 task done
@shahar1
shahar1 force-pushed the drop-py310/1-ci-breeze-docker branch from 0965104 to edfd62b Compare October 4, 2026 17:53
@shahar1
shahar1 requested a review from mobuchowski as a code owner October 4, 2026 17:53
@shahar1 shahar1 added the ready for maintainer review Set after triaging when all criteria pass. label Oct 4, 2026
shahar1 added 2 commits October 5, 2026 01:43
@shahar1
[DO NOT MERGE] Cherry-pick #74200 (deflake backfill e2e on WebKit) to…
c66c7a3 
… verify CI

Temporary: drops out on rebase once #74200 is merged.
@shahar1
Drop Python 3.10 from CI, Breeze and the container images
1cd3fc1 
Python 3.10 reaches end of life in October 2026 and more and more of
Airflow's dependencies have already dropped it, which makes the 3.10
leg the slowest and most fragile part of CI. This is the first layer
of the stack that moves the baseline to Python 3.11: Breeze stops
offering 3.10, the default Python for CI, Breeze and the images moves
to 3.11, and the PGP path for verifying the Python tarball goes away
because every remaining version ships Sigstore bundles.

Python 3.10 stays in the historical version lists that Breeze uses to
reason about older Airflow releases, and in the CI image restore
action that release branches still rely on. The distributions still
declare Python 3.10 in this layer; layer 7 bumps them once the code
no longer carries 3.10-only constructs. Until then the provider
metadata generator keeps its own 3.10-inclusive version list, and the
prek minimum-Python check keeps 3.10 because its script header still
pins it to a 3.10 interpreter until layer 2.

The shared-distribution and scripts test jobs run uv without an
interpreter pin, so uv picks the runner's system Python, which is 3.10
on the current images. Pinning them to the default CI Python keeps them
on a supported interpreter regardless of the runner image.

The image build speed check builds the CI image from the registry cache
within a fixed budget. A PR that changes install_os_dependencies.sh or
the embedded GPG keys misses the cache for the layer that compiles
Python from source. That compile took about 4.5 minutes for Python 3.10
but 11 to 18 minutes for 3.11 and newer on hosted runners, so the
budget is raised to fit a cold Python 3.11+ build.
@potiuk
potiuk force-pushed the drop-py310/1-ci-breeze-docker branch from edfd62b to 1cd3fc1 Compare October 4, 2026 23:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@potiuk potiuk Awaiting requested review from potiuk potiuk is a code owner
@ashb ashb Awaiting requested review from ashb ashb is a code owner
@gopidesupavan gopidesupavan Awaiting requested review from gopidesupavan gopidesupavan is a code owner
@amoghrajesh amoghrajesh Awaiting requested review from amoghrajesh amoghrajesh is a code owner
@bugraoz93 bugraoz93 Awaiting requested review from bugraoz93 bugraoz93 is a code owner
@jason810496 jason810496 Awaiting requested review from jason810496 jason810496 is a code owner
@henry3260 henry3260 Awaiting requested review from henry3260 henry3260 is a code owner
@jscheffl jscheffl Awaiting requested review from jscheffl jscheffl is a code owner
@jedcunningham jedcunningham Awaiting requested review from jedcunningham jedcunningham is a code owner
@ephraimbuddy ephraimbuddy Awaiting requested review from ephraimbuddy ephraimbuddy is a code owner
@choo121600 choo121600 Awaiting requested review from choo121600 choo121600 is a code owner
@vatsrahul1001 vatsrahul1001 Awaiting requested review from vatsrahul1001 vatsrahul1001 is a code owner
@o-nikolas o-nikolas Awaiting requested review from o-nikolas
@mobuchowski mobuchowski Awaiting requested review from mobuchowski

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs area:dev-tools area:docker-tests area:e2e-tests area:production-image Production image improvements and fixes area:providers area:registry full tests needed We need to run full set of tests for this PR to merge kind:documentation provider:amazon AWS/Amazon - related issues ready for maintainer review Set after triaging when all criteria pass.

Projects

Airflow Registry
Status: Backlog

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@shahar1