Skip to content

Backport #913: Run the PyAthena suite only when its files change #915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
laughingman7743 merged 1 commit into from
Oct 1, 2026
Merged

Backport #913: Run the PyAthena suite only when its files change #915

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions .github/workflows/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,12 @@ jobs:
- run: just lint

# Selects the AWS suites and Python versions. Draft and external-fork pull
# requests run none. A ready pull request always runs the PyAthena suite; it
# runs the SQLAlchemy tests (the compliance suites and the PyAthena suite's
# SQLAlchemy tests) and the Spark tests only when their code, tests,
# dependencies, this workflow, the top-level modules of pyathena and
# pyathena.aio, or the shared test fixtures change. Pull requests and the
# requests run none. A ready pull request runs the PyAthena suite only when
# the package, the tests, setup.cfg, the dependencies, or this workflow
# change; it runs the SQLAlchemy tests (the compliance suites and the
# PyAthena suite's SQLAlchemy tests) and the Spark tests only when their
# code, tests, dependencies, this workflow, the top-level modules of pyathena
# and pyathena.aio, or the shared test fixtures change. Pull requests and the
# schedule test the newest Python version; a dispatch tests the requested
# versions or every version, and the Release workflow every version.
changes:
Expand All @@ -74,6 +75,7 @@ jobs:
permissions:
pull-requests: read
outputs:
pyathena: ${{ github.event_name != 'pull_request' || steps.paths.outputs.pyathena == 'true' }}
sqla: ${{ github.event_name != 'pull_request' || steps.paths.outputs.sqla == 'true' }}
spark: ${{ github.event_name != 'pull_request' || steps.paths.outputs.spark == 'true' }}
python-versions: ${{ steps.versions.outputs.python-versions }}
Expand Down Expand Up @@ -118,6 +120,13 @@ jobs:
- pyathena/aio/spark/**
- tests/pyathena/spark/**
- tests/pyathena/aio/spark/**
# A superset of sqla and spark, so their tests never run without
# the PyAthena suite.
pyathena:
- *shared
- setup.cfg
- pyathena/**
- tests/**
- id: versions
env:
EVENT_NAME: ${{ github.event_name }}
Expand Down Expand Up @@ -149,6 +158,7 @@ jobs:
# parallel; each is still a separate job for "Re-run failed jobs".
test:
needs: changes
if: needs.changes.outputs.pyathena == 'true'

@laughingman7743 laughingman7743 Oct 1, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review round one (implementation behavior): CLEAN

  • Scope: full pass over git diff 0397685561081561b4da6b2dcc2c4f8fac3ad175..93247b6fda14ab34bbd3c340e8ad1e1fae402836, two files: .github/workflows/test.yaml and docs/testing.md.
  • Faithfulness: the added and removed lines are identical to Run the PyAthena suite only when its files change聽#913's merge delta (7e8616fa^1..7e8616fa). The cherry-pick applied without conflicts.
  • Behavior on 3.x:
    • test.yaml here matches master's except for the missing license header and PYTHON_VERSIONS with 3.10, so the output, the filter, and the if: behave as on master. Non-pull_request events produce pyathena=true.
    • release.yaml calls test.yaml from a v* tag push and release has needs: test, so a release still runs every suite.
    • test-suite.yaml keeps the same skip-sqla/skip-spark inputs, so a running PyAthena suite still ignores those directories as before.
    • The 3.x branch protection has no required checks, so a skipped test blocks nothing.
  • What the suite reads on 3.x: tox in pyproject.toml runs uv sync --group dev and just test pyathena (just lint = ruff and mypy, then pytest on tests/pyathena/). 3.x has no uv workspace. benchmarks/ is excluded from ruff, mypy, and pytest collection. No test or recipe reads scripts/cloudformation/. Nothing outside the allowlist feeds the suite.
  • Simulation against this branch's workflow: the 28 cases from Run the PyAthena suite only when its files change聽#913 all match. The superset invariant holds over the 236 tracked files.
  • Simplicity: none to apply; the backport keeps master's text verbatim.

@laughingman7743 laughingman7743 Oct 1, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review (relayed): CLEAN

  • Reviewer: Codex CLI 0.157.1, model gpt-6-sol, reasoning effort high, --sandbox read-only, --ephemeral, session 01a0f833-d953-7963-aa72-417ccea50be3. Static review only: no build or tests run.
  • Scope: full pass over 0397685561081561b4da6b2dcc2c4f8fac3ad175..93247b6fda14ab34bbd3c340e8ad1e1fae402836 on a detached snapshot, compared with master's merge commit 7e8616fa. The package also contained the pinned dorny/paths-filter v4.0.3 source (ceb8a2b8) outside the repository tree. The prompt omitted the PR number, description, and prior findings. After the run, the snapshot's HEAD was unchanged; its only untracked entry was that reference directory.
  • Covered: test.yaml, test-suite.yaml, release.yaml, the pinned action source, the 3.x justfile, the tox configuration, the tests and fixtures, and the GitHub Actions section of docs/testing.md.
  • Result: the backport matches the master delta. The filter covers the suite's test, configuration, and fixture paths on 3.x, and sqla/spark imply pyathena. Draft and fork PRs skip the AWS jobs. Schedule, dispatch, and the Release call select test, and release (needs: test) still waits on it. The comments and docs are accurate for 3.x. No introduced or pre-existing findings.

uses: ./.github/workflows/test-suite.yaml
with:
test-type: pyathena
Expand Down
11 changes: 6 additions & 5 deletions docs/testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,18 +74,19 @@ It runs the offline checks (`just lint`) on each of them, including Drafts and e
| Trigger | PyAthena suite | SQLAlchemy tests | Spark tests | Python versions |
| --- | --- | --- | --- | --- |
| Draft pull request | No | No | No | None |
| Ready pull request from a branch of this repository | Yes | When related files change | When related files change | Newest supported |
| Ready pull request from a branch of this repository | When related files change | When related files change | When related files change | Newest supported |
| Weekly schedule | Yes | Yes | Yes | Newest supported |
| Manual dispatch | Yes | Yes | Yes | Requested, or all supported |
| Release tag (Release workflow) | Yes | Yes | Yes | All supported |

The SQLAlchemy tests are the compliance suites and the PyAthena suite's `tests/pyathena/sqlalchemy/` and `tests/pyathena/aio/sqlalchemy/`.
The Spark tests are the PyAthena suite's `tests/pyathena/spark/` and `tests/pyathena/aio/spark/`.
When the SQLAlchemy or Spark tests do not run, the PyAthena suite runs without them.
For the SQLAlchemy tests, the related files are `pyathena/sqlalchemy/`, `pyathena/aio/sqlalchemy/`, `tests/sqlalchemy/`, their PyAthena suite test directories, and `setup.cfg`.
When the PyAthena suite runs but the SQLAlchemy or Spark tests do not, it runs without them.
For the PyAthena suite, the related files are `pyathena/`, `tests/`, and `setup.cfg`.

@laughingman7743 laughingman7743 Oct 1, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review round two (claims and operational effects): CLEAN

  • Scope: full claim audit of git diff 0397685561081561b4da6b2dcc2c4f8fac3ad175..93247b6fda14ab34bbd3c340e8ad1e1fae402836 as it reads on 3.x: the workflow comments, the docs/testing.md GitHub Actions section, the commit message, and the PR body.
  • Claims checked against 3.x, not master:
  • Operator: on 3.x only pull requests that touch none of the allowlisted files lose their PyAthena run, for example benchmarks/ or scripts/cloudformation/. Release tag runs and dispatch still run every suite. 3.x has no weekly schedule of its own, because cron runs only on the default branch. A file missing from the allowlist would therefore surface on 3.x at the next release tag run, or through a dispatch.
  • Evidence limits: the selection is checked by local simulation. The Ready run of this PR will select every suite because it changes test.yaml.

For the SQLAlchemy tests, they are `pyathena/sqlalchemy/`, `pyathena/aio/sqlalchemy/`, `tests/sqlalchemy/`, their PyAthena suite test directories, and `setup.cfg`.
For the Spark tests, they are `pyathena/spark/`, `pyathena/aio/spark/`, and their PyAthena suite test directories.
Changes to `pyproject.toml`, `uv.lock`, `justfile`, or the Test workflows run both.
For a pull request from a branch of this repository that still changes files other than `docs/` and Markdown, marking the Draft ready for review starts the AWS jobs, and converting it back to Draft cancels AWS jobs still running.
Changes to the modules directly under `pyathena/` and `pyathena/aio/`, the shared test fixtures such as `tests/pyathena/conftest.py` and `tests/resources/`, `pyproject.toml`, `uv.lock`, `justfile`, or the Test workflows run all three.
For a pull request from a branch of this repository that still changes files other than `docs/` and Markdown, marking the Draft ready for review starts any AWS jobs its changed files select, and converting it back to Draft cancels AWS jobs still running.
To run every suite on a branch, dispatch the workflow; it tests every supported Python version unless `python-versions` lists some of them:

```bash
Expand Down
Loading