Run the MinIO-backed S3 suites in CI, starting with ParquetWriteWithMagicCommitterSuite #7150

What is the problem the feature request solves?

ParquetWriteWithMagicCommitterSuite, added in #6940, is the only suite that runs native Parquet writes through S3A's magic committer against a real S3 API. Like the other MinIO-backed suites, it needs Docker for testcontainers, so it is on the manual list in dev/ci/check-suites.py and no CI job runs it. A change that breaks the magic committer route, such as the close ordering, the abort of a failed file, or which committer configurations the planning gate admits, would pass CI.

The other manual MinIO suites have the same gap: ParquetReadFromS3Suite, CsvReadFromS3Suite, IcebergReadFromS3Suite, CometS3CredentialBridgeSuite and HadoopS3ACredentialProviderAdapterBridgeSuite.

Describe the potential solution

Add a job like contrib-delta-s3 in .github/workflows/delta_contrib_test.yml. It runs a MinIO-backed suite on a plain ubuntu-24.04 runner, where testcontainers can start MinIO. Running ParquetWriteWithMagicCommitterSuite on Spark 3.5 and 4.1 would cover both native writers (CometNativeWriteExec and CometWriteFilesExec) and both Hadoop lines (3.3.4 and 3.4). The suite takes about 20 seconds once Comet is built. The read suites could join the same job.

Two things to keep in mind:

• CometS3TestBase builds its bucket setup client without a region, so the job needs AWS_REGION=us-east-1 on a runner with no AWS config.
• The suite sets fs.s3a.committer.magic.enabled=true for its own session, and the shared test configuration turns the magic committer off for every other suite. The job should run the suite on its own.

Additional context

Raised by @parthchandra in the review of #6940, as a follow-up.