Backport #839: Resolve the bare awsathena scheme to the REST dialect

[laughingman7743]

WHAT

Backport #839 by @aminghadersohi and its follow-up #905 to the 3.x maintenance branch. Each is one -x commit, in master order: #839 from master's merge commit 0e72d33 (authored by the contributor), and #905 from its commit d6b9303.

• The bare awsathena SQLAlchemy entry point resolves to pyathena.sqlalchemy.rest:AthenaRestDialect instead of the base AthenaDialect, so awsathena:// URLs behave like awsathena+rest://.
• The "Dialect & driver" table in docs/sqlalchemy.md lists the bare schema with the rest driver.
• test_bare_scheme_uses_rest_driver covers the resolution.
• Align the compliance-suite dialect registry with the entry points #905: tests/sqlalchemy/__init__.py, which registers the dialects for the compliance suite, registers the same classes as the entry points. Bare awsathena now registers AthenaRestDialect, and the missing awsathena.polars is added. test_compliance_suite_registry_matches_entry_points compares the file with the installed entry points.

Conflict resolution: tests/pyathena/sqlalchemy/test_base.py conflicted because master's file carries helpers, imports, and the TestAthenaDialect class from PRs that are not on 3.x. The class was added on master by #777. Only #839's own lines are applied: the AthenaRestDialect import and the test. On 3.x the test goes into a new TestAthenaDialect class, placed where master has it, directly before TestSQLAlchemyAthena. pyproject.toml and docs/sqlalchemy.md apply unchanged. The #905 commit conflicted in the same file for the same reason; only #905's imports and test are applied, and its added lines match #905's commit exactly. 3.x's copy of tests/sqlalchemy/__init__.py has no license header, but the registration lines apply unchanged.

WHY

The maintainer decided to port #839 to 3.x. On 3.x before this change, a bare URL creates the base dialect, and Engine.driver, URL.get_driver_name(), and dialect_description raise AttributeError for bare awsathena:// URLs.

Merge order: #905 must be merged into master before this PR, so that the -x source of its commit is on master.

Release: this is not released immediately. The next 3.x release waits until a fix for #854 or #857 is merged and backported as well.

Release note for that release: bare awsathena:// URLs now create an AthenaRestDialect, a subclass of AthenaDialect. Queries use the same default cursor as before, and isinstance(engine.dialect, AthenaDialect) still holds. Code that checks type(engine.dialect) is AthenaDialect changes. Thanks to @aminghadersohi for the fix.

TEST

Tested at bc5fc93 unless stated (the #839 commit at 68b25aa). Python 3.13.1 and SQLAlchemy 2.0.46 (the locked versions) unless stated. All runs are local and offline; nothing contacted AWS.

• just format, just lint (license headers, ruff check, ruff format --check, mypy, cfn-lint): clean. just docs lint: 0 errors.
• After uv sync, with dummy AWS_* variables and --noconftest:
  • pytest tests/pyathena/sqlalchemy/test_base.py::TestAthenaDialect: 2 passed at bc5fc93 (1 passed at 68b25aa).
  • With 3.x's tests/sqlalchemy/__init__.py, test_compliance_suite_registry_matches_entry_points fails on both differences: awsathena maps to AthenaDialect, and awsathena.polars is missing.
  • import tests.sqlalchemy followed by create_engine("awsathena://...") creates AthenaRestDialect.
  • pytest tests/sqlalchemy/ --collect-only, with and without --dburi async: 759 tests collected each, the same as origin/3.x.
  • With 3.x's pyproject.toml entry point reinstalled: 1 failed, AssertionError: assert <class 'pyathena.sqlalchemy.base.AthenaDialect'> is AthenaRestDialect.
  • test_compiler.py, test_types.py, test_array.py, and TestAthenaDialect: 301 passed at 68b25aa.
• 3.x keeps sqlalchemy>=1.0.0. In isolated environments (uv run --no-project --isolated --with sqlalchemy==<version> --with .), a bare URL creates AthenaRestDialect with driver rest, get_driver_name() rest, dialect_description awsathena+rest, and the same connect arguments as awsathena+rest://, on both SQLAlchemy 1.4.54 and 2.0.46.
• CI at bc5fc93 (run 36679353823, Python 3.14, all three suites because this PR changes pyproject.toml):
  • PyAthena suite, including the Spark tests: 1702 passed, 3 skipped.
  • test-sqla: 324 passed, 153 skipped.
  • test-sqla-async: 324 passed, 153 skipped.
• Not run locally: AWS integration suites. Every PyAthena and compliance-suite fixture connects with an explicit awsathena+rest:// / awsathena+aiorest:// URL, so those suites never resolve the bare scheme, and AthenaRestDialect itself is unchanged. The CI run above covers them on the newest Python version; the Release workflow runs every Python version at tag time.

🤖 Generated with Claude Code

[laughingman7743]

Self-review round one (implementation behavior). Base d6fe836, head 68b25aa.

Covered: all three changed files (pyproject.toml, docs/sqlalchemy.md, tests/pyathena/sqlalchemy/test_base.py); callers that resolve the bare scheme; AthenaRestDialect vs AthenaDialect on 3.x (driver, supports_statement_cache, import_dbapi/dbapi); SQLAlchemy 1.4.54 and 2.0.46 resolution; whether the test fails without the change.

Result: CLEAN for this PR's scope.

• AthenaRestDialect only adds driver = "rest" and delegates import_dbapi to the base, so connect arguments and the cursor are unchanged; isinstance(dialect, AthenaDialect) still holds. Nothing in pyathena/ or tests/ compares type(dialect) except the new test.
• The test fails against 3.x's entry point (AthenaDialect is not AthenaRestDialect) and passes after uv sync.
• The conflict resolution adds only Resolve the bare awsathena scheme to the REST dialect #839's import and test; the added lines match Resolve the bare awsathena scheme to the REST dialect #839's diff apart from the new TestAthenaDialect class line.

Out of scope, pre-existing on master as well: tests/sqlalchemy/__init__.py:3 still calls registry.register("awsathena", "pyathena.sqlalchemy.base", "AthenaDialect"), which overrides the entry point inside the compliance-suite process. Importing tests.sqlalchemy first makes a bare URL create AthenaDialect again. There is no functional impact today: the suites run in separate pytest sessions and the compliance suite connects only with +rest / +aiorest. Aligning it belongs in a master change first, then a backport, so this PR keeps #839's delta unchanged.

[laughingman7743]

Repair follow-up, round-one perspective (implementation behavior). Range: git range-diff is not needed because the base is unchanged; the repair is the added commit 68b25aa..bc5fc93 on merge-base d6fe836.

The out-of-scope note above is now addressed by backporting #905 (commit d6b9303 on master's PR branch, -x).

• tests/sqlalchemy/__init__.py registers the same eleven classes as the 3.x entry points: bare awsathena is AthenaRestDialect, and awsathena.polars is added.
• test_compliance_suite_registry_matches_entry_points runs the file against an empty PluginLoader via monkeypatch, so it does not depend on the global registry or test order. It fails with 3.x's original file (both differences reported) and passes now.
• Importing tests.sqlalchemy no longer changes what a bare URL creates.
• The compliance suites collect 759 tests with and without --dburi async, the same as origin/3.x.
• The applied lines match Align the compliance-suite dialect registry with the entry points #905's commit exactly; only the conflict context differs.

Result: CLEAN.

[laughingman7743]

Self-review round two (claims, callers, operations). Base d6fe836, head 68b25aa.

Claims checked against evidence that could disprove them:

• Before the change on 3.x, a bare URL creates AthenaDialect, and Engine.driver, URL.get_driver_name(), and dialect_description each raise AttributeError: reproduced by installing origin/3.x (d6fe836) into an isolated environment with SQLAlchemy 2.0.46.
• TestAthenaDialect came to master with Fix metadata reflection throttling and reuse listed metadata #777: git log -S finds 6d791e2, which GitHub maps to Fix metadata reflection throttling and reuse listed metadata #777.
• Every PyAthena and compliance-suite fixture uses an explicit driver: tests/__init__.py:6 and :10 use awsathena+rest:// / awsathena+aiorest://, and tests/pyathena/conftest.py:158 / :192 default to rest / aiorest. No other test or doc uses a bare awsathena:// URL.
• The backport commit keeps the contributor as author and records the -x source and conflict resolution.
• Docs: the edited table row is the only place on 3.x that describes the bare schema; the other rows and the async table have no bare entry.

Callers: isinstance(engine.dialect, AthenaDialect) still holds, the cursor and connect arguments are unchanged, and SQLAlchemy 1.4.54 resolves the same class as 2.0.46. Only type(engine.dialect) is AthenaDialect checks change, which the release note states.

Operations: no AWS request path changes.

Corrections made to the PR body: "agreed for both lines" now states that the maintainer decided on the port. The CI note now says, from .github/workflows/test.yaml:118-119, that pyproject.toml is a shared file, so Ready runs all three AWS suites, including Spark, on the newest Python version.

Result: CLEAN after those wording corrections. The round-one out-of-scope note about tests/sqlalchemy/__init__.py stands.

[laughingman7743]

Repair follow-up, round-two perspective (claims, callers, operations), same range 68b25aa..bc5fc93.

Claims added to the PR body and commit message:

• "Align the compliance-suite dialect registry with the entry points #905 from its commit d6b9303": the -x line names d6b9303, the single commit of Align the compliance-suite dialect registry with the entry points #905. Master merges PRs with merge commits, so that commit enters master's history when Align the compliance-suite dialect registry with the entry points #905 merges; the body now states that Align the compliance-suite dialect registry with the entry points #905 must merge first.
• "3.x's copy has no license header, but the registration lines apply unchanged": 3.x's file differs from master's only by the header added in Add MIT headers to existing files written solely by the maintainer #805, and tests/sqlalchemy/__init__.py applied without a conflict.
• "The added lines match Align the compliance-suite dialect registry with the entry points #905's commit exactly": compared the sorted added and removed lines of the backport commit with d6b9303; identical.
• TEST figures (2 passed, the before failure, 759 collected on both branches) come from local offline runs at bc5fc93.

Callers and operations: test harness only; no library behavior changes. CI selection is unchanged, because the pyproject.toml change from #839 already selects all three suites including Spark.

Result: CLEAN.

[laughingman7743]

Independent review, relayed. Reviewer: Codex CLI 0.157.1, model gpt-6-sol, reasoning effort high, session 01a0f0d1-3da1-72c2-bd28-d4c093959025. Static review in a read-only sandbox of a detached snapshot at head 68b25aa (base d6fe836); it received the literal diff without the PR description, commit message, or prior findings, and ran no tests. The snapshot and the PR worktree were unchanged afterwards.

Covered: the three changed files; sync and async dialect classes and entry points; test fixtures, SQLAlchemy registration, CI test selection, and repository conventions. It found that the new test fails against the old installed entry point, the docs row matches the new sync driver, async URLs keep +aiorest, and SQLAlchemy 1.x gains no new runtime path.

Result: FINDINGS (1).

• tests/sqlalchemy/__init__.py:3 still registers bare awsathena as AthenaDialect, overriding the new entry point in any process that imports tests.sqlalchemy. Collecting both test packages in one pytest process makes this bare-URL test resolve to the base class and fail. The reviewer classifies it as introduced by this diff: the registration existed before but became inconsistent with the entry point.

Author verification: confirmed. Importing tests.sqlalchemy first makes a bare URL create AthenaDialect. CI and the just test recipes run the suites in separate processes, so they are unaffected, but the repository sets no testpaths, so a bare pytest from the root collects both packages together. The same line is on master after #839. It stays open here: a fix belongs on master first and then in this backport, and that is pending the maintainer's decision.

[laughingman7743]

Independent follow-up, relayed. Reviewer: Codex CLI 0.157.1, model gpt-6-sol, reasoning effort high, session 01a0f0fb-0a86-7de1-a945-130e9dd7cc0b. Static review in a read-only sandbox of a detached snapshot at bc5fc93, given the bounded diff 68b25aa..bc5fc93 and this thread's finding, without the PR description or author conclusions; no tests run. The snapshot and the PR worktree were unchanged afterwards.

Covered: the two changed files, all eleven dialect entry points in pyproject.toml, the test package imports, and the pytest, xdist, and CI invocation paths.

Result: CLEAN. The finding above is resolved: bare awsathena now registers AthenaRestDialect, and tests/sqlalchemy/__init__.py matches every PyAthena entry point. The new test executes the file against a fresh registry, so a wrong or missing registration fails the comparison. No ordering, xdist, or working-directory defect was found.