Enable Mosaic reads by default#491
Conversation
leaves12138
left a comment
There was a problem hiding this comment.
One compatibility concern: removing the mosaic feature name itself is a breaking change for downstream crates that already depend on it.
After this PR, Mosaic is always enabled, but existing dependency declarations such as:
paimon = { version = "0.3.0", features = ["mosaic"] }
paimon-datafusion = { version = "0.3.0", features = ["mosaic"] }will fail Cargo resolution because neither crate defines a mosaic feature anymore. I verified this locally with a small downstream cargo check, which fails with:
package `check-mosaic-feature` depends on `paimon` with feature `mosaic` but `paimon` does not have that feature
Could we keep a no-op compatibility feature, e.g. mosaic = [] in paimon and mosaic = ["paimon/mosaic"] or mosaic = [] in paimon-datafusion, while still making the dependency non-optional and enabling reads by default? That preserves existing user Cargo.toml files and CI commands while achieving the goal of this PR.
leaves12138
left a comment
There was a problem hiding this comment.
Looks good to me. The Mosaic feature compatibility concern is intentionally not a blocker here.
Summary
Make Mosaic reads always available instead of hiding them behind a Cargo feature, and upgrade
paimon-mosaic-coreto 0.2.0 so array and map columns are supported by the reader.Changes
paimon-mosaic-coreas a regular dependency and remove the Mosaic Cargo feature declarations.paimon-mosaic-coreand add an array/map round-trip test.mosaicfrom CI feature lists and update docs to describe Mosaic reads as always available.Testing
cargo check -p paimoncargo check -p paimon --no-default-features --features storage-memorycargo check -p paimon-datafusion --no-default-featurescargo test -p paimon mosaic --libcargo test -p paimon-datafusion --no-default-features --test mosaic_tablesNotes
cargo check -p paimon --no-default-featuresstill requires at least one storage feature because of existing storage enum behavior, so this PR verifies the no-default path withstorage-memoryenabled.