Skip to content

GH-3289: Test conversion and writer configuration through CLI - #3861

Open
zhangxinyao88 wants to merge 1 commit into
apache:masterfrom
zhangxinyao88:codex/parquet-3289-conversion-cli-tests
Open

zhangxinyao88 wants to merge 1 commit into
apache:masterfrom
zhangxinyao88:codex/parquet-3289-conversion-cli-tests

Conversation

@zhangxinyao88

Copy link
Copy Markdown
Contributor

Rationale for this change

Continue #3289 with convert/to-avro coverage and observable writer configuration checks. Existing command tests largely assign fields directly and check file existence. Configuration-loading tests do not establish that a setting changes the resulting file.

What changes are included in this PR?

Add fourteen cases using deterministic Avro and Parquet inputs:

  • Both conversion commands preserve record values and apply column projection, a supplied schema, and a selected codec.
  • Existing output bytes are preserved without --overwrite and replaced by readable records with it.
  • --property and properties/XML configuration files change a UUID field from string storage to 16-byte UUID storage in the Parquet footer.
  • An explicit property overrides the corresponding file setting.

Assertions read records, codec metadata and physical/logical schema types. Paths contain spaces; no help-text or full-log snapshots are added. Read, inspection and CSV scopes in #3858/#3859/#3860 remain separate.

Are these changes tested?

./mvnw -o -B -ntp -pl parquet-cli spotless:check test: 97 cases, 95 passed, 2 skipped, no failures/errors. All fourteen new cases passed; Spotless and RAT checks passed. Java 17 and cached dependencies were used; no full-reactor or multi-JDK build was run.

Eleven temporary local regression controls triggered assertion failures when projection, supplied schemas, codecs, overwrite or configuration propagation/precedence was ignored. All production edits were restored before the successful module run.

Are there any user-facing changes?

No. Tests only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant