Release DataFusion `55.0.0` (Jul / Aug 2026)

[alamb]

Is your feature request related to a problem or challenge?

Previous release is #21080 (May, 2026) so next major release would be approximately Jul / Aug 2026

Release Coordinator: @timsaucer

Steps:

• Create a branch-55 branch:
• Update version and changelog:
• Test with DataFusion Python
• Test with DataFusion Comet deps: [DO NOT MERGE] bump DataFusion 55.0 and Arrow/Parqet 59.1 datafusion-comet#5262
• Test with DataFusion Ballista chore(deps): Upgrade to DataFusion 55.0.0 datafusion-ballista#2241
• Test with delta.rs:
• Test vortex: [WIP] Bump Apache DF to 55 vortex-data/vortex#9334
• Test with iceberg-rust: deps: [DO NOT MERGE] test DataFusion 55.0 and Arrow/Parquet 59.1 iceberg-rust#2958
• Test with LakeSail:
• Test with parquet viewer
• Test with datafusion-materialized-views
• Voting Thread
• Publish to crates.io
• Create ticket for next release:
• Create blog post: Blog post for the DataFusion 55.0.0 release #24216

Features to mention in the blog (if they make it)

• [EPIC] Simplify datetime predicates using "preimages" #19946
• [EPIC] Implement Range Partitioning #22395
• Further improve performance of IN list evaluation #19241 from @geoffreyclaude
• [EPIC] Sort Pushdown: skip sorts and skip IO / decode for ORDER BY / TopK queries #23036 from @zhuqi-lucas
• EPIC: complete GroupValuesColumn type coverage (nested types + remaining primitives) #22715 from @zhuqi-lucas
• feat: add strictness metadata for scalar UDF null propagation and use it in outer join elimination #23148
• feat: Plumb Parquet virtual columns (row_number) through TableSchema and ParquetOpener #22026 from @mbutrovich / Add file_row_index UDF to query file-level row indexes from Parquet files #22604 from @AdamGS
• Add MERGE INTO types to datafusion-expr #20763 from @wirybeaver
• feat: Add SQL planner, physical planner, and TableProvider hook for MERGE INTO #22988 from @wirybeaver
• feat: introduce pluggable SpillFile trait and TempFileFactory for custom spill backends #21882 from @pantShrey
• feat: Add support for unnest_outer function for arrays. #22100
• Optimize "per partition" top-k : ROW_NUMBER < 5 / TopK #6899 from @SubhamSinghal
• Enable parquet filter pushdown (filter_pushdown) by default #3463

Tracking Epics

• [EPIC] Split Aggregation Logic into Dedicated Streams #22710 @alamb / @2010YOUY01 / @Rachelint

Bugs that should be fixed

• input_file_name() in a WHERE clause errors when parquet pushdown_filters is enabled #23531
• Parquet filter pushdown silently drops a get_field predicate when the file needs schema adaptation (wrong results) #24109 from @adriangb
• GroupedHashAggregateStream::emit throws ArrowError: column types must match schema types #24069 from @hsiang-c
• Restore the From / TryFrom proto conversions dropped since 54.1.0 #24205 from @adriangb

Bugs maybe to fix

• RepartitionExec withholds all output from an unbounded input until batch_size rows accumulate #24044

Community Wishlist

• [EPIC] Port ExecutionPlan serialization to try_to_proto / try_from_proto hooks #23494 from @adriangb
• Expose accumulator state to allow prefix scanning #24035 (@avantgardnerio )
• Enable dynamic filters for range-partitioned joins #23854 (@jayshrivastava)
• feat: Add SQL planner, physical planner, and TableProvider hook for MERGE INTO #22988 from @wirybeaver
• feat: Implement FFI_QueryPlanner #24028 (@timsaucer )
• Reapply "Add ExecutionPlan::apply_expressions() (apache#20337)" (apache#22437) #24018 (@jayshrivastava)
• add ExecutionPlan::dynamic_expressions_produced() method #24068 (@jayshrivastava)
• feat: prune unread Parquet leaves when a nested column is cast to a narrower type #24090 (@mbutrovich )
• Prefetch Parquet row groups with a bounded I/O budget #23492 (@alamb / @Dandandan )
• refactor(physical-plan): Simplify ExecutionPlan API with replace_children #23903 (@JSOD11 )
• Cache reusable Parquet pruning setup across files with the same schema #21566 (@alamb / @kosiew )

[alamb]

Updated the title to reflect my prediction we'll get this one done in late July / early August

[AdamGS]

I think #23531 should also be fixed (I have a PR out #23638), especially if filter pushdown is going to be enabled by default.

[wirybeaver]

#22988 should also be merged so we can announce complete MERGE INTO support all the way down to the TableProvider hook.

[alamb]

I think #23531 should also be fixed (I have a PR out #23638), especially if filter pushdown is going to be enabled by default.

Added to the list

#22988 should also be merged so we can announce complete MERGE INTO support all the way down to the TableProvider hook.

Added to the requested features

[mbutrovich]

I will need to coordinate with @adriangb about getting this over the finish line, but Comet would really love to see this in 55.0.0. #23398