[DevChiniwala]

S3Helper.download_directory() lists a literal key prefix. Downloading exports/train therefore also selects siblings such as exports/train-old/part.parquet, which can silently contaminate the training dataset. Nested S3 folder-marker objects are also written as regular files, so downloading their child parts subsequently fails with FileExistsError.

Use a directory-boundary prefix (exports/train/) for listing and relative paths, and skip folder markers. Keep the public return path, nested file layout, pagination, source-bucket selection, empty-directory errors and AWS error propagation.

Validation on Python 3.12.7 with pytest 8.4.2 and boto3/botocore 1.43.123:

• Upstream a1e05f6: four regression cases fail and four controls pass. Failures demonstrate the incorrect listing prefix, nested marker/file collision and marker-only directory being reported as a successful download.
• With the fix: all eight tests pass using the real boto3 paginator, Botocore Stubber responses and actual local output files. Coverage includes 1,001 parts across two pages, optional trailing slash, nested folders, empty listings, marker-only directories and AccessDenied propagation. No live AWS call or credentials are required.
• Black 25.12.0 and the repository-compatible Ruff 0.14.9 both pass on the changed source and tests.
• git diff --check passes.

Draft while the repository-prescribed staged integration run remains unavailable without Anthropic credentials and Spark/Java. Live S3 and staged provider/training integration have not been claimed as passing.

Repository workflow metadata: the patch version is bumped from 1.4.4 to 1.4.5 as required by the source-change version gate. poetry check --lock and the branch/title/version checks pass locally. Dependencies and the lockfile are unchanged.