#863 Add support for custom AST transformers via the ast_transformers option.

[yruslan]

Closes #863

Summary by CodeRabbit

• New Features
  • Added support for custom AST transformers during COBOL copybook parsing.
  • Configure transformers via the ast_transformers option (comma-separated for chaining).
  • Transformers are loaded from the configured classes and run before Spark schema generation and data processing.
  • Supports transformer classes with either a ReaderParameters constructor or a no-argument constructor.
• Documentation
  • Updated guidance and examples for configuring ast_transformers.
  • Removed “experimental” labels from EBCDIC-related documentation headings.
  • Corrected the transformer usage example and refreshed the options reference.
• Tests
  • Added unit and integration coverage validating transformer execution and resulting schema/output.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3a6ee79a-0ae2-4ee6-8419-e7b13b7a31e2

📥 Commits

Reviewing files that changed from the base of the PR and between 9cb382f and 26d7a8a.

📒 Files selected for processing (1)

• README.md

🚧 Files skipped from review as they are similar to previous changes (1)

• README.md

---

Walkthrough

The PR adds configurable custom AST transformers to parser APIs and Spark reader options. Transformers are reflectively loaded, applied before built-in transformers, documented, and covered by parser and Spark integration tests.

Changes

Custom AST Transformers

Layer / File(s)	Summary
Parser transformer contract cobol-parser/src/main/scala/.../CopybookParser.scala	Parser overloads accept custom transformers and prepend them to the built-in transformation pipeline.
Reader configuration and loading cobol-parser/src/main/scala/.../parameters/*, cobol-parser/src/main/scala/.../schema/CobolSchema.scala, cobol-parser/src/main/scala/.../utils/AstTransformerUtils.scala	The ast_transformers option is parsed, transformer classes are reflectively instantiated, and instances are passed to single- and multi-copybook parsing.
Validation and documentation cobol-parser/src/test/.../asttransform/*, spark-cobol/src/test/.../*, README.md	Parser and Spark tests verify transformed AST and DataFrame output; README documents the option and updates EBCDIC headings.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SparkReader
  participant CobolParametersParser
  participant AstTransformerUtils
  participant CobolSchema
  participant CopybookParser
  SparkReader->>CobolParametersParser: configure ast_transformers
  CobolParametersParser->>CobolSchema: pass transformer class names
  CobolSchema->>AstTransformerUtils: load transformer classes
  AstTransformerUtils-->>CobolSchema: return transformer instances
  CobolSchema->>CopybookParser: parseTree with custom transformers
  CopybookParser-->>SparkReader: transformed schema and records

Loading

Possibly related PRs

• AbsaOSS/cobrix#851: Extends CopybookParser and its AST transformation pipeline with another configurable transformer source.

Poem

A bunny hops through trees of code,
With custom magic on the road.
Before built-ins begin their dance,
New fields bloom with every chance.
“Transform!” cries Bun, and tests agree. 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately summarizes the main change: adding custom AST transformer support via the ast_transformers option.
Linked Issues check	✅ Passed	The changes implement the requested Spark option, support multiple transformers, load them from class names, and apply them before built-ins.
Out of Scope Changes check	✅ Passed	The rest of the diff is test coverage and README updates that support the new feature, with no clearly unrelated code changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/863-ast-transformers

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@README.md`:
- Around line 1664-1668: Update the README Cobrix example to select the data
source with the DataFrameReader format method: replace the reader option for
"format" with the equivalent .format("cobol") call, while preserving the
existing copybook, ast_transformers, and load configuration.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2d6a8584-4e2b-4975-ba89-ef6b58d7490d

📥 Commits

Reviewing files that changed from the base of the PR and between 6dc6e20 and 9cb382f.

📒 Files selected for processing (11)

• README.md
• cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/parser/CopybookParser.scala
• cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/reader/parameters/CobolParameters.scala
• cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/reader/parameters/CobolParametersParser.scala
• cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/reader/parameters/ReaderParameters.scala
• cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/reader/schema/CobolSchema.scala
• cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/utils/AstTransformerUtils.scala
• cobol-parser/src/test/scala/za/co/absa/cobrix/cobol/parser/asttransform/AstTransformerSpy.scala
• cobol-parser/src/test/scala/za/co/absa/cobrix/cobol/parser/asttransform/ParsingWithAstTransformersSuite.scala
• spark-cobol/src/test/scala/za/co/absa/cobrix/spark/cobol/mocks/AstTransformerSpy.scala
• spark-cobol/src/test/scala/za/co/absa/cobrix/spark/cobol/source/integration/Test44CustomAstTransformersSpec.scala

[github-actions]

JaCoCo code coverage report - 'cobol-parser'

Overall Project	90.08% -0.07%	🍏
Files changed	48.91%	🍏

File	Coverage
CopybookParser.scala	84.6%	🍏
CobolSchema.scala	81.93%	🍏
AstTransformerUtils.scala	0%	❌

[github-actions]

JaCoCo code coverage report - 'spark-cobol'

Overall Project	83.52%	🍏

There is no coverage information present for the Files changed