[ahmedabu98]

Making Iceberg AddFiles into a Managed transform to take advantage of automatic upgrades

Changing the URN to the standardized format: beam:schematransform:org.apache.beam:iceberg_add_files:v1
Shouldn't result in any breaking changes

[github-actions]

📋 Review Summary

This PR exposes IcebergAddFiles (beam:schematransform:org.apache.beam:iceberg_add_files:v1) as a Managed IO sink (ICEBERG_ADD_FILES) in both the Java and Python SDKs, standardizes its SchemaTransform URN in external_transforms.proto, and adds SchemaTransformPayloadTranslator support along with unit tests.

🔍 General Feedback

• The changes cleanly follow the established patterns for Managed IO transforms across proto definitions, Java/Python Managed APIs, IcebergSchemaTransformTranslation, Beam YAML standard_io.yaml, and expansion service configuration.
• Unit tests in IcebergSchemaTransformTranslationTest thoroughly cover row reconstruction and proto translation for AddFilesSchemaTransformProvider.

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.91%. Comparing base (2213e54) to head (d166b29).
⚠️ Report is 43 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #40827       +/-   ##
===========================================
+ Coverage        0   79.91%   +79.91%     
===========================================
  Files           0      493      +493     
  Lines           0    83480    +83480     
===========================================
+ Hits            0    66714    +66714     
- Misses          0    16766    +16766     

Flag	Coverage Δ
python	79.91% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Assigning reviewers:

R: @derrickaw for label yaml.
Backup reviewer: chamikaramj (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

This pull request likely touches a core component ("core" label). Please review with scrutiny.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[chamikaramj]

Thanks.

Can we also update AddFiles java documentation and may be examples to use the Managed version ?

beam/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AddFiles.java

Line 150 in b9a783a

* paths.apply(new AddFiles(catalog, "db.sales", null, null, null, null, null, null, evolution));

https://github.com/apache/beam/blob/master/sdks/java/io/iceberg/src/test/java/org/apache/beam/sdk/io/iceberg/AddFilesIT.java

[chamikaramj]

I think to actually use managed here you have to do one of the following.

(1) Add this to YAML I/O and use managed.py there:

beam/sdks/python/apache_beam/yaml/yaml_io.py

Line 569 in b9a783a

return beam.managed.Read(

(2) Use the managed_replacement feature similar to Kafka:

beam/sdks/python/apache_beam/yaml/standard_io.yaml

Line 137 in b9a783a

managed_replacement:

Latter might be easier.

[ahmedabu98]

Sounds good, went with the managed_replacement

[chamikaramj]

Thanks!