Skip to content

Resolve the bare awsathena scheme to the REST dialect - #839

Merged
laughingman7743 merged 4 commits into
pyathena-dev:masterfrom
aminghadersohi:fix/bare-scheme-default-driver
Sep 30, 2026
Merged

laughingman7743 merged 4 commits into
pyathena-dev:masterfrom
aminghadersohi:fix/bare-scheme-default-driver

Conversation

@aminghadersohi

@aminghadersohi aminghadersohi commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

WHAT

Point the bare awsathena SQLAlchemy entry point in pyproject.toml at pyathena.sqlalchemy.rest:AthenaRestDialect instead of pyathena.sqlalchemy.base:AthenaDialect, add a unit test for the bare awsathena:// scheme, and list the bare schema with the rest driver in the "Dialect & driver" table of docs/sqlalchemy.md. The base AthenaDialect is unchanged (an earlier revision gave it a default driver; that was reverted in 91fd175 per review). The branch has master merged in (131e53f).

WHY

The approach was agreed in the review thread on this PR: #839 (comment)

The bare awsathena entry point resolves to AthenaDialect, which defines no driver. SQLAlchemy reads the driver when it describes a dialect, so with a bare URL:

>>> from sqlalchemy import create_engine
>>> engine = create_engine("awsathena://athena.us-west-2.amazonaws.com:443/default?s3_staging_dir=s3://bucket/path/")
>>> engine.driver
AttributeError: 'AthenaDialect' object has no attribute 'driver'
>>> engine.url.get_driver_name()
AttributeError: type object 'AthenaDialect' has no attribute 'driver'

The same happens for dialect_description, which SQLAlchemy uses in statement-cache logging. Resolving the bare scheme to a concrete dialect matches how SQLAlchemy's built-in dialects (postgresql:// resolves to PGDialect_psycopg2) and other third-party dialects handle it. The docs already use awsathena+rest:// as the default URL form, so a bare URL now behaves the same as awsathena+rest://. The other awsathena.* entry points are unchanged.

TEST

Tested commit: 257f3c3 (the entry point and test are unchanged from 91fd175; 131e53f merges master, 257f3c3 is the docs row). Python 3.12.13, SQLAlchemy 2.0.46, dependencies from uv.lock. All results are local runs; nothing contacted AWS.

The entry point change only takes effect after the package is reinstalled, because SQLAlchemy resolves dialects from the installed entry point metadata, not from pyproject.toml directly.

  • uv sync -p 3.12, then, with dummy AWS_DEFAULT_REGION, AWS_ATHENA_S3_STAGING_DIR, AWS_ATHENA_WORKGROUP, AWS_ATHENA_SPARK_WORKGROUP and credentials so tests/__init__.py can import, uv run pytest --noconftest tests/pyathena/sqlalchemy/test_base.py -k test_bare_scheme_uses_rest_driver:
    • this branch: 1 passed, 178 deselected;
    • master's pyproject.toml entry point (uv sync again): 1 failed, AssertionError: assert <class 'pyathena.sqlalchemy.base.AthenaDialect'> is AthenaRestDialect.
  • uv run pytest --noconftest tests/pyathena/sqlalchemy/test_base.py::TestAthenaDialect tests/pyathena/sqlalchemy/test_compiler.py tests/pyathena/sqlalchemy/test_types.py: 212 passed.
  • just lint (license headers, ruff check, ruff format --check, mypy, cfn-lint): clean.
  • just docs lint (markdownlint-cli2 0.18.1, run through npx because mise is not installed here): 0 errors in 25 files.

Not run: the AWS integration tests under tests/pyathena/sqlalchemy/ and just test sqla. Per review they are not needed here, because every fixture connects with an explicit awsathena+rest:// / awsathena+aiorest:// URL and never resolves the bare scheme.

The bare awsathena:// scheme resolves to AthenaDialect, which did not
define a driver. SQLAlchemy reads the driver when describing a dialect,
so Engine.driver, URL.get_driver_name() and dialect_description raised
AttributeError for bare URLs, and tools enumerating dialect entry points
could not use it. Give the base dialect the REST driver, which is what
awsathena+rest:// already selects; the concrete dialects keep their own
driver values.
Comment thread pyathena/sqlalchemy/base.py Outdated
Comment thread tests/pyathena/sqlalchemy/test_base.py Outdated
Revert the default driver on the base AthenaDialect, which stays the
shared base for the concrete dialects. Resolve the bare awsathena://
scheme to AthenaRestDialect through the entry point instead, and assert
the resolved dialect type in the test.
@aminghadersohi aminghadersohi changed the title Default the base SQLAlchemy dialect to the REST driver Resolve the bare awsathena scheme to the REST dialect Sep 26, 2026
…efault-driver

# Conflicts:
#	tests/pyathena/sqlalchemy/test_base.py
The bare awsathena entry point now resolves to AthenaRestDialect, so
the Dialect & driver table lists rest for it.
@aminghadersohi
aminghadersohi marked this pull request as ready for review September 30, 2026 04:15
@aminghadersohi

Copy link
Copy Markdown
Contributor Author

@laughingman7743 since your approval: added the docs table row (257f3c3), merged master (131e53f), and recorded TEST. Ready for your re-approval.

@laughingman7743
laughingman7743 merged commit 0e72d33 into pyathena-dev:master Sep 30, 2026
14 of 15 checks passed
@laughingman7743

Copy link
Copy Markdown
Member

Thanks for the fix, @aminghadersohi. This is merged into master, and the backport to the 3.x maintenance branch is #903.

It will not be released right away. The next 3.x release will wait until a fix for #854 or #857 is in as well; both are small, so they should not hold it for long.

laughingman7743 added a commit that referenced this pull request Sep 30, 2026
…ntry points

tests/sqlalchemy/__init__.py registers the dialects for the SQLAlchemy
compliance suite, and a registration overrides the installed entry point in
any process that imports it. After #839 pointed the bare awsathena entry point
at AthenaRestDialect, the file still registered the base AthenaDialect, and it
never registered awsathena.polars. Register the same classes as pyproject.toml,
and add a test that compares the file's registrations with the installed
entry points.

(cherry picked from commit d6b9303)

Conflict resolution: tests/pyathena/sqlalchemy/test_base.py conflicted because master's import block and the tests around TestAthenaDialect come from PRs that are not on 3.x. Only #905's own lines are applied: the importlib.metadata, runpy, Path, and PluginLoader imports and test_compliance_suite_registry_matches_entry_points, placed after test_bare_scheme_uses_rest_driver as on master. The added lines match #905's commit exactly. tests/sqlalchemy/__init__.py applies unchanged apart from context (3.x's copy has no license header).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
laughingman7743 added a commit that referenced this pull request Sep 30, 2026
Backport #839: Resolve the bare awsathena scheme to the REST dialect
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants