Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/breeze/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description: 'Sets up Python and Breeze'
inputs:
python-version:
description: 'Python version to use'
default: "3.10"
default: "3.11"
outputs:
host-python-version:
description: Python version used in host
Expand Down
4 changes: 2 additions & 2 deletions .github/actions/prepare_all_ci_images/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ runs:
# TODO: Currently we cannot loop through the list of python versions and have dynamic list of
# tasks. Instead we hardcode all possible python versions and they - but
# this should be implemented in stash action as list of keys to download.
# That includes 3.9 - 3.12 as we are backporting it to v3-0-test branch
# That includes 3.9 - 3.14 as we are backporting to the v3-*-test branches
# This is captured in https://github.com/apache/airflow/issues/45268
# So we actually need 3.9 even if 3.9 support on main is dropped!
# So we actually need 3.9 and 3.10 even if their support on main is dropped!
- name: "Restore CI docker image ${{ inputs.platform }}:3.9"
uses: ./.github/actions/prepare_single_ci_image
with:
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/additional-ci-image-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,9 @@ jobs:
# Check that the image builds quickly from the registry cache. This build is slow from
# scratch, so we only run it on regular PRs.
check-that-image-builds-quickly:
timeout-minutes: 25
# A PR that touches install_os_dependencies.sh or scripts/docker/keys misses the cache for the
# layer that builds Python from source, which takes 11-18 minutes for Python 3.11+ on hosted runners.
timeout-minutes: 35
name: Check that image builds quickly
runs-on: ${{ fromJSON(inputs.runners) }}
env:
Expand All @@ -83,5 +85,5 @@ jobs:
uses: ./.github/actions/breeze
- name: "Check that image builds quickly"
# Synchronize to be a little bit shorter than above timeout-minutes to make sure that
# if the build takes too long the job will fail with logs. 22 minutes * 60 s = 1320 seconds
run: breeze shell --max-time 1320 --platform "${PLATFORM}"
# if the build takes too long the job will fail with logs. 33 minutes * 60 s = 1980 seconds
run: breeze shell --max-time 1980 --platform "${PLATFORM}"
2 changes: 1 addition & 1 deletion .github/workflows/airflow-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ on: # yamllint disable-line rule:truthy
default-python-version:
description: "Which version of python should be used by default"
type: string
default: '3.10'
default: '3.11'
use-uv:
description: "Whether to use uv to build the image (true/false)"
type: string
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/basic-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,9 @@ jobs:
tests-shared-distributions:
timeout-minutes: 10
name: Shared ${{ matrix.shared-distribution }} tests
env:
# Without a pin uv picks the runner's system Python, which can be older than Airflow supports.
UV_PYTHON: "${{ inputs.default-python-version }}"
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -157,6 +160,9 @@ jobs:
name: Scripts tests
runs-on: ${{ fromJSON(inputs.runners) }}
if: inputs.run-scripts-tests == 'true'
env:
# Without a pin uv picks the runner's system Python, which can be older than Airflow supports.
UV_PYTHON: "${{ inputs.default-python-version }}"
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand Down
6 changes: 1 addition & 5 deletions .github/workflows/finalize-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -154,14 +154,10 @@ jobs:
run: >
breeze release-management constraints-version-check
--python "${MATRIX_PYTHON_VERSION}"
--airflow-constraints-mode "${MATRIX_CONSTRAINTS_MODE}" "${EXPLAIN_WHY_FLAG}"
--airflow-constraints-mode "${MATRIX_CONSTRAINTS_MODE}" --explain-why
env:
MATRIX_PYTHON_VERSION: "${{ matrix.python-version }}"
MATRIX_CONSTRAINTS_MODE: "${{ matrix.constraints-mode }}"
# Explaining an outdated package resolves its dependency tree; on Python 3.10 more and more
# dependencies have dropped support, so there is far more to explain and the job takes much
# longer. Not worth it weeks before 3.10 support is dropped - remove this along with 3.10.
EXPLAIN_WHY_FLAG: ${{ matrix.python-version == '3.10' && '--no-explain-why' || '--explain-why' }}
VERBOSE: "false"

push-buildx-cache-to-github-registry:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release_dockerhub_image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ on: # yamllint disable-line rule:truthy
default: false
limitPythonVersions:
type: string
description: 'Force python versions (e.g. "3.10 3.11")'
description: 'Force python versions (e.g. "3.11 3.12")'
default: ''
permissions:
contents: read
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release_single_dockerhub_image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ on: # yamllint disable-line rule:truthy
type: string
required: true
pythonVersion:
description: 'Python version (e.g. 3.10, 3.11)'
description: 'Python version (e.g. 3.11, 3.12)'
type: string
required: true
skipLatest:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ui-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ on: # yamllint disable-line rule:truthy
default-python-version:
description: "Which version of python should be used by default"
type: string
default: '3.10'
default: '3.11'
use-uv:
description: "Whether to use uv to build the image (true/false)"
type: string
Expand Down Expand Up @@ -90,7 +90,7 @@ jobs:
name: ${{ inputs.workflow-name || 'UI E2E Tests' }}
runs-on: ${{ fromJSON(inputs.runners || '["ubuntu-24.04"]') }}
env:
PYTHON_MAJOR_MINOR_VERSION: "${{ inputs.default-python-version || '3.10' }}"
PYTHON_MAJOR_MINOR_VERSION: "${{ inputs.default-python-version || '3.11' }}"
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_USERNAME: ${{ github.actor }}
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ formats: []
sphinx:
configuration: devel-common/src/docs/rtd-deprecation/conf.py
python:
version: "3.10"
version: "3.11"
install:
- method: pip
path: .
Expand Down
79 changes: 29 additions & 50 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ if [[ "$#" != 1 ]]; then
exit 1
fi

AIRFLOW_PYTHON_VERSION=${AIRFLOW_PYTHON_VERSION:-3.10.18}
AIRFLOW_PYTHON_VERSION=${AIRFLOW_PYTHON_VERSION:-3.11.16}
PYTHON_LTO=${PYTHON_LTO:-true}
GOLANG_MAJOR_MINOR_VERSION=${GOLANG_MAJOR_MINOR_VERSION:-1.24.4}
TEMURIN_VERSION=${TEMURIN_VERSION:-11}
Expand Down Expand Up @@ -398,56 +398,35 @@ function install_python() {
wget --tries=3 --waitretry=5 -O python.tar.xz "https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz"
local major_minor_version
major_minor_version="${AIRFLOW_PYTHON_VERSION%.*}"
local major minor
major="${major_minor_version%.*}"
minor="${major_minor_version#*.}"
echo "Verifying Python ${AIRFLOW_PYTHON_VERSION} (${major_minor_version})"
if [[ "${major}" -gt 3 ]] || [[ "${major}" -eq 3 && "${minor}" -ge 11 ]]; then
# Sigstore verification for Python >= 3.11 (PEP 761)
declare -A sigstore_identities=(
# https://peps.python.org/pep-0664/#release-manager-and-crew
[3.11]="pablogsal@python.org"
# https://peps.python.org/pep-0693/#release-manager-and-crew
[3.12]="thomas@python.org"
# https://peps.python.org/pep-0719/#release-manager-and-crew
[3.13]="thomas@python.org"
# https://peps.python.org/pep-0745/#release-manager-and-crew
[3.14]="hugo@python.org"
)
declare -A sigstore_issuers=(
[3.11]="https://accounts.google.com"
[3.12]="https://accounts.google.com"
[3.13]="https://accounts.google.com"
[3.14]="https://github.com/login/oauth"
)
wget --tries=3 --waitretry=5 -O python.tar.xz.sigstore \
"https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz.sigstore"
install_cosign
local identity="${sigstore_identities[${major_minor_version}]}"
local issuer="${sigstore_issuers[${major_minor_version}]}"
/tmp/cosign verify-blob \
--bundle python.tar.xz.sigstore \
--certificate-identity "${identity}" \
--certificate-oidc-issuer "${issuer}" \
python.tar.xz
rm -f python.tar.xz.sigstore /tmp/cosign
else
# PGP verification for Python 3.10
declare -A keys=(
# gpg: key 64E628F8D684696D: public key "Pablo Galindo Salgado <pablogsal@gmail.com>" imported
# https://peps.python.org/pep-0619/#release-manager-and-crew
[3.10]="A035C8C19219BA821ECEA86B64E628F8D684696D"
)
wget --tries=3 --waitretry=5 -O python.tar.xz.asc \
"https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz.asc"
GNUPGHOME="$(mktemp -d)"; export GNUPGHOME
local gpg_key="${keys[${major_minor_version}]}"
echo "Using GPG key ${gpg_key}"
gpg --batch --import "/scripts/docker/keys/python-${major_minor_version}.asc"
gpg --batch --verify python.tar.xz.asc python.tar.xz
gpgconf --kill all
rm -rf "${GNUPGHOME}" python.tar.xz.asc
fi
# Sigstore verification (PEP 761)
declare -A sigstore_identities=(
# https://peps.python.org/pep-0664/#release-manager-and-crew
[3.11]="pablogsal@python.org"
# https://peps.python.org/pep-0693/#release-manager-and-crew
[3.12]="thomas@python.org"
# https://peps.python.org/pep-0719/#release-manager-and-crew
[3.13]="thomas@python.org"
# https://peps.python.org/pep-0745/#release-manager-and-crew
[3.14]="hugo@python.org"
)
declare -A sigstore_issuers=(
[3.11]="https://accounts.google.com"
[3.12]="https://accounts.google.com"
[3.13]="https://accounts.google.com"
[3.14]="https://github.com/login/oauth"
)
wget --tries=3 --waitretry=5 -O python.tar.xz.sigstore \
"https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz.sigstore"
install_cosign
local identity="${sigstore_identities[${major_minor_version}]}"
local issuer="${sigstore_issuers[${major_minor_version}]}"
/tmp/cosign verify-blob \
--bundle python.tar.xz.sigstore \
--certificate-identity "${identity}" \
--certificate-oidc-issuer "${issuer}" \
python.tar.xz
rm -f python.tar.xz.sigstore /tmp/cosign
mkdir -p /usr/src/python
tar --extract --directory /usr/src/python --strip-components=1 --file python.tar.xz
rm python.tar.xz
Expand Down
81 changes: 30 additions & 51 deletions Dockerfile.ci
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ if [[ "$#" != 1 ]]; then
exit 1
fi

AIRFLOW_PYTHON_VERSION=${AIRFLOW_PYTHON_VERSION:-3.10.18}
AIRFLOW_PYTHON_VERSION=${AIRFLOW_PYTHON_VERSION:-3.11.16}
PYTHON_LTO=${PYTHON_LTO:-true}
GOLANG_MAJOR_MINOR_VERSION=${GOLANG_MAJOR_MINOR_VERSION:-1.24.4}
TEMURIN_VERSION=${TEMURIN_VERSION:-11}
Expand Down Expand Up @@ -338,56 +338,35 @@ function install_python() {
wget --tries=3 --waitretry=5 -O python.tar.xz "https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz"
local major_minor_version
major_minor_version="${AIRFLOW_PYTHON_VERSION%.*}"
local major minor
major="${major_minor_version%.*}"
minor="${major_minor_version#*.}"
echo "Verifying Python ${AIRFLOW_PYTHON_VERSION} (${major_minor_version})"
if [[ "${major}" -gt 3 ]] || [[ "${major}" -eq 3 && "${minor}" -ge 11 ]]; then
# Sigstore verification for Python >= 3.11 (PEP 761)
declare -A sigstore_identities=(
# https://peps.python.org/pep-0664/#release-manager-and-crew
[3.11]="pablogsal@python.org"
# https://peps.python.org/pep-0693/#release-manager-and-crew
[3.12]="thomas@python.org"
# https://peps.python.org/pep-0719/#release-manager-and-crew
[3.13]="thomas@python.org"
# https://peps.python.org/pep-0745/#release-manager-and-crew
[3.14]="hugo@python.org"
)
declare -A sigstore_issuers=(
[3.11]="https://accounts.google.com"
[3.12]="https://accounts.google.com"
[3.13]="https://accounts.google.com"
[3.14]="https://github.com/login/oauth"
)
wget --tries=3 --waitretry=5 -O python.tar.xz.sigstore \
"https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz.sigstore"
install_cosign
local identity="${sigstore_identities[${major_minor_version}]}"
local issuer="${sigstore_issuers[${major_minor_version}]}"
/tmp/cosign verify-blob \
--bundle python.tar.xz.sigstore \
--certificate-identity "${identity}" \
--certificate-oidc-issuer "${issuer}" \
python.tar.xz
rm -f python.tar.xz.sigstore /tmp/cosign
else
# PGP verification for Python 3.10
declare -A keys=(
# gpg: key 64E628F8D684696D: public key "Pablo Galindo Salgado <pablogsal@gmail.com>" imported
# https://peps.python.org/pep-0619/#release-manager-and-crew
[3.10]="A035C8C19219BA821ECEA86B64E628F8D684696D"
)
wget --tries=3 --waitretry=5 -O python.tar.xz.asc \
"https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz.asc"
GNUPGHOME="$(mktemp -d)"; export GNUPGHOME
local gpg_key="${keys[${major_minor_version}]}"
echo "Using GPG key ${gpg_key}"
gpg --batch --import "/scripts/docker/keys/python-${major_minor_version}.asc"
gpg --batch --verify python.tar.xz.asc python.tar.xz
gpgconf --kill all
rm -rf "${GNUPGHOME}" python.tar.xz.asc
fi
# Sigstore verification (PEP 761)
declare -A sigstore_identities=(
# https://peps.python.org/pep-0664/#release-manager-and-crew
[3.11]="pablogsal@python.org"
# https://peps.python.org/pep-0693/#release-manager-and-crew
[3.12]="thomas@python.org"
# https://peps.python.org/pep-0719/#release-manager-and-crew
[3.13]="thomas@python.org"
# https://peps.python.org/pep-0745/#release-manager-and-crew
[3.14]="hugo@python.org"
)
declare -A sigstore_issuers=(
[3.11]="https://accounts.google.com"
[3.12]="https://accounts.google.com"
[3.13]="https://accounts.google.com"
[3.14]="https://github.com/login/oauth"
)
wget --tries=3 --waitretry=5 -O python.tar.xz.sigstore \
"https://www.python.org/ftp/python/${AIRFLOW_PYTHON_VERSION%%[a-z]*}/Python-${AIRFLOW_PYTHON_VERSION}.tar.xz.sigstore"
install_cosign
local identity="${sigstore_identities[${major_minor_version}]}"
local issuer="${sigstore_issuers[${major_minor_version}]}"
/tmp/cosign verify-blob \
--bundle python.tar.xz.sigstore \
--certificate-identity "${identity}" \
--certificate-oidc-issuer "${issuer}" \
python.tar.xz
rm -f python.tar.xz.sigstore /tmp/cosign
mkdir -p /usr/src/python
tar --extract --directory /usr/src/python --strip-components=1 --file python.tar.xz
rm python.tar.xz
Expand Down Expand Up @@ -1187,7 +1166,7 @@ chmod 1777 /tmp

AIRFLOW_SOURCES=$(cd "${IN_CONTAINER_DIR}/../.." || exit 1; pwd)

PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.10}
PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.11}

export AIRFLOW_HOME=${AIRFLOW_HOME:=${HOME}}

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ Apache Airflow is tested with:

| | Main version (dev) | Stable version (3.3.2) | Deprecate version (2.11.2) |
|------------|------------------------------------------|------------------------------------|------------------------------|
| Python | 3.10, 3.11, 3.12, 3.13, 3.14 | 3.10, 3.11, 3.12, 3.13, 3.14 | 3.10, 3.11, 3.12 |
| Python | 3.11, 3.12, 3.13, 3.14 | 3.10, 3.11, 3.12, 3.13, 3.14 | 3.10, 3.11, 3.12 |
| Platform | AMD64/ARM64 | AMD64/ARM64 | AMD64/ARM64(\*) |
| Kubernetes | 1.31, 1.32, 1.33, 1.34, 1.35, 1.36, 1.37 | 1.30, 1.31, 1.32, 1.33, 1.34, 1.35 | 1.26, 1.27, 1.28, 1.29, 1.30 |
| PostgreSQL | 14, 15, 16, 17, 18 | 14, 15, 16, 17, 18 | 12, 13, 14, 15, 16 |
Expand Down
2 changes: 1 addition & 1 deletion airflow-core/docs/installation/prerequisites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Airflow® is tested with:

.. Beginning of the auto-generated tested versions

* Python: 3.10, 3.11, 3.12, 3.13, 3.14
* Python: 3.11, 3.12, 3.13, 3.14

* Databases:

Expand Down
17 changes: 17 additions & 0 deletions airflow-core/newsfragments/74151.significant.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Support for Python 3.10 has been removed

Airflow now requires Python 3.11 or newer. Python 3.10 reaches its end of life in October 2026,
and Airflow drops support for a Python version in ``main`` as soon as it reaches end of life.
The ``apache-airflow`` distribution, the Task SDK, ``airflowctl`` and all providers released
from ``main`` declare ``requires-python = ">=3.11"``.

* Types of change

* [ ] Dag changes
* [ ] Config changes
* [ ] API changes
* [ ] CLI changes
* [ ] Behaviour changes
* [ ] Plugin changes
* [x] Dependency changes
* [ ] Code interface changes
2 changes: 1 addition & 1 deletion airflow-ctl-tests/tests/airflowctl_tests/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

AIRFLOW_ROOT_PATH = Path(__file__).resolve().parents[3]

DEFAULT_PYTHON_MAJOR_MINOR_VERSION = "3.10"
DEFAULT_PYTHON_MAJOR_MINOR_VERSION = "3.11"
DEFAULT_DOCKER_IMAGE = f"ghcr.io/apache/airflow/main/prod/python{DEFAULT_PYTHON_MAJOR_MINOR_VERSION}:latest"
DOCKER_IMAGE = os.environ.get("DOCKER_IMAGE") or DEFAULT_DOCKER_IMAGE

Expand Down
2 changes: 1 addition & 1 deletion airflow-e2e-tests/tests/airflow_e2e_tests/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
AIRFLOW_ROOT_PATH = Path(__file__).resolve().parents[3]

DOCKER_COMPOSE_HOST_PORT = os.environ.get("HOST_PORT", "localhost:8080")
DEFAULT_PYTHON_MAJOR_MINOR_VERSION = "3.10"
DEFAULT_PYTHON_MAJOR_MINOR_VERSION = "3.11"
DEFAULT_DOCKER_IMAGE = f"ghcr.io/apache/airflow/main/prod/python{DEFAULT_PYTHON_MAJOR_MINOR_VERSION}:latest"
DOCKER_IMAGE = os.environ.get("DOCKER_IMAGE") or DEFAULT_DOCKER_IMAGE
os.environ["AIRFLOW_UID"] = str(os.getuid())
Expand Down
Loading
Loading