Skip to content

Drop support for Python 3.10 - #74144

Closed
shahar1 wants to merge 2 commits into
apache:mainfrom
shahar1:drop-python-3-10
Closed

shahar1 wants to merge 2 commits into
apache:mainfrom
shahar1:drop-python-3-10

Conversation

@shahar1

@shahar1 shahar1 commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Drop Python 3.10 support from Airflow, Task SDK, airflowctl, providers, shared libraries, the Helm chart tooling and the dev/CI toolchain. The minimum supported Python is now 3.11; the default CI/breeze Python moves from 3.10 to 3.11.

Python 3.10 reaches end of life in October 2026, and an increasing number of our dependencies have already dropped it (see the --no-explain-why workaround in finalize-tests.yml, which this PR removes along with 3.10).

What changes

  • requires-python = ">=3.11" and the Programming Language :: Python :: 3.10 classifier removed in every pyproject.toml (core, task-sdk, airflow-ctl, all providers, shared/*, dev/test distributions). uv.lock files regenerated.
  • Breeze: ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS / ALL_PYTHON_MAJOR_MINOR_VERSIONS are 3.11–3.14; providers compatibility matrix runs on 3.11; the dead PYTHON_TO_MIN_AIRFLOW_MAPPING constant and its unused is_airflow_version_supported_for_python helper are removed.
  • CI/Docker: default Python 3.11 in composite actions, workflows, devcontainer, Dockerfile/Dockerfile.ci and scripts/docker. The PGP branch for verifying the Python source tarball is removed (3.11+ ships Sigstore bundles); the python-3.10.asc key and its entry in update_docker_gpg_keys.py go with it.
  • Code: tomli fallbacks replaced by tomllib; typing_extensions imports of Self/Unpack/assert_never replaced by typing; the asyncio.Runner re-implementation in BaseOperator.event_loop() removed; sys.version_info < (3, 11) test skips removed. tomli dropped from build-system.requires and prek hook dependencies.
  • Ruff: with the new baseline ruff rewrites timezone.utc to datetime.UTC (UP017) and Unpack[tuple[...]] to *tuple[...] (UP044) across the repo - that is the bulk of the diff. UP042 (str, Enum -> StrEnum) and ASYNC109 become reportable on 3.11 and are added to the ruff ignore list with a rationale: both change runtime behaviour or public signatures and need per-site judgement.
  • Inline script metadata (# requires-python) in scripts/ci/prek/*.py and dev/** bumped to 3.11; docs, READMEs, contributing docs and generated files updated (update-tested-versions, update-supported-versions, update-pyproject-toml, generate-pypi-readme, update-breeze-cmd-output, update-inlined-dockerfile-scripts).

Deliberately kept 3.10 references

  • .github/actions/prepare_all_ci_images/action.yml still restores a 3.10 (and 3.9) CI image: the action is shared with release branches that still support those versions (see the comment in the file and Update ASF "stash" action to allow list of keys #45268).
  • ALL_HISTORICAL_PYTHON_VERSIONS, PYTHON_3_10_TO_3_14 and ALL_PYTHON_VERSION_TO_PATCHLEVEL_VERSION["3.10"] in global_constants.py: used to reason about older Airflow releases (SBOM, compatibility matrix) and to build images for release branches.
  • python:3.10-slim-buster in the Kubernetes decorator/pod operator tests and python_version="3.10" in the Vertex AI Ray hook: those are container/Ray runtime versions, not the Airflow interpreter.
  • Changelogs and dev/breeze/doc/adr/0002-*.md keep their historical wording.

Checks run

  • uv run --project dev/breeze pytest dev/breeze/tests (1160 passed; test_publish_docs_to_s3.py skipped locally due to a botocore/Python 3.14 import issue unrelated to this change).
  • uv run --project scripts pytest scripts/tests (1602 passed), uv run --project dev/registry pytest dev/registry/tests (271 passed).
  • Targeted task-sdk and shared/logging unit tests for the simplified code paths pass; the provider tests that lost their skipif markers need a DB and were not run locally.
  • ruff check / ruff format clean; prek run --from-ref main --to-ref HEAD --stage pre-commit - see the PR checks for the result on CI.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Fable 5.1)

Generated-by: Claude Code (Fable 5.1) following the guidelines

@shahar1 shahar1 added full tests needed We need to run full set of tests for this PR to merge all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs labels Oct 3, 2026
Python 3.10 reaches end of life in October 2026 and more and more of
Airflow's dependencies have already dropped it, which makes the 3.10
constraint resolution the slowest and most fragile leg of CI. Moving
the baseline to Python 3.11 also lets the code base drop the compat
shims that only existed for 3.10: the ``tomli`` fallbacks for
``tomllib``, the ``typing_extensions`` imports of ``Self``, ``Unpack``
and ``assert_never``, the ``asyncio.Runner`` re-implementation in the
task SDK, the PGP path for verifying the Python tarball in the images
(3.11+ ships Sigstore bundles) and the ``timezone.utc`` spelling that
ruff now rewrites to ``datetime.UTC``.

Python 3.10 stays in the historical version lists that breeze uses to
reason about older Airflow releases and release branches, and in the
CI image restore action that release branches still rely on.
@shahar1 shahar1 added the skip common compat check Skips common compat provider modification check label Oct 3, 2026
@shahar1

shahar1 commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by a stacked set of smaller PRs so each piece can be reviewed on its own (stack #74160):

  1. Drop Python 3.10 from CI, Breeze and the container images #74151 Drop Python 3.10 from CI, Breeze and the container images
  2. Bump inline script metadata to Python 3.11 #74152 Bump inline script metadata to Python 3.11
  3. Remove Python 3.10 compatibility shims #74153 Remove Python 3.10 compatibility shims
  4. Apply ruff Python 3.11 fixes to core, Task SDK, shared and airflowctl #74154 Apply ruff Python 3.11 fixes to core, Task SDK, shared and airflowctl
  5. Apply ruff Python 3.11 fixes to providers #74155 Apply ruff Python 3.11 fixes to providers
  6. Apply ruff Python 3.11 fixes to dev tooling, scripts and test suites #74156 Apply ruff Python 3.11 fixes to dev tooling, scripts and test suites
  7. Require Python 3.11 in all distributions #74157 Require Python 3.11 in all distributions
  8. Update provider READMEs for the Python 3.11 baseline #74158 Update provider READMEs for the Python 3.11 baseline
  9. Update docs for the Python 3.11 baseline #74159 Update docs for the Python 3.11 baseline

The combined diff of the stack is identical to this PR (plus the newsfragment rename). Closing in favour of the stack.


Drafted-by: Claude Code (Fable 5.1) (no human review before posting)

@shahar1 shahar1 closed this Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs full tests needed We need to run full set of tests for this PR to merge skip common compat check Skips common compat provider modification check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant