Repository navigation
Conversation
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Exchange mappings and validation isthmus/src/main/java/io/substrait/isthmus/SubstraitRelVisitor.java, isthmus/src/main/java/io/substrait/isthmus/SubstraitRelNodeConverter.java, isthmus/src/main/java/io/substrait/isthmus/SqlKindFromRel.java, isthmus/src/test/java/io/substrait/isthmus/SubstraitRelNodeConverterTest.java |
Calcite hash, broadcast, round-robin, random, and singleton distributions map to Substrait exchanges. Supported Substrait exchanges map to Calcite distributions; scatter keys must be top-level struct fields in ascending index order. Sort, range, and other unsupported distributions are rejected. Tests cover conversion results, rejected inputs, and preserved root output names. |
Priority:
Estimated code review effort: 3 (Moderate) | ~20 minutes
Change: Feature · Severity of issue fixed: Medium
Sequence Diagram(s)
sequenceDiagram
participant CalciteRel
participant SubstraitRelVisitor
participant SubstraitExchange
participant SubstraitRelNodeConverter
CalciteRel->>SubstraitRelVisitor: Convert Calcite exchange distribution
SubstraitRelVisitor->>SubstraitExchange: Create mapped exchange relation
SubstraitExchange->>SubstraitRelNodeConverter: Convert Substrait exchange
SubstraitRelNodeConverter->>CalciteRel: Apply Calcite distribution
Suggested reviewers: cht42, jacques-n, nielspardon
Merge Risk: 
Minimal · up to 7dac7
-
No actionable issue remains from this review; the exchange conversion is ready to merge after normal checks.
Pre-merge checks: 
4 · 
1
-
Failed checks: 1 warning Passed checks: 4 passed-
Full details: Linked Issues check
-
Explanation
Issue [
#153] requires Isthmus support for CalciteLogicalExchange.SubstraitRelVisitornow converts Calcite hash, broadcast, round-robin, random, singleton, and rejected unsupported distributions.SubstraitRelNodeConverterconverts supported Substrait exchanges back to Calcite and rejects lossy cases.SubstraitRelNodeConverterTestcovers both directions and unsupported cases.SqlKindFromRelreturnsSELECTfor exchange relations, so exchange root output names remain available.
Full details: Out of Scope Changes check
-
Explanation
The changes stay within exchange conversion support for [
#153]. TheSqlKindFromRelchange preserves output names for exchange roots. The converter changes and exchange tests support the requested relation handling. No unrelated change is established by the supplied PR evidence.
-
-
Finishing Touches
-
🧪 Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
Calcite → Substrait:
HASH_DISTRIBUTED→ScatterExchangeBROADCAST_DISTRIBUTED→BroadcastExchangeROUND_ROBIN_DISTRIBUTED→RoundRobinExchangewithexact = true, since Calcite defines it as "records are assigned to instances in turn"RANDOM_DISTRIBUTED→RoundRobinExchangewithexact = false, andSINGLETON→SingleBucketExchangewith a constant bucket and a partition count of 1, as proposed in Questions about exchangeKind of ExchangeRel substrait#519Substrait → Calcite:
ScatterExchange→HASH_DISTRIBUTED,BroadcastExchange→BROADCAST_DISTRIBUTED,RoundRobinExchange→ROUND_ROBIN_DISTRIBUTEDRelDistributionhas nowhere to put themNot supported, rejected instead of converted lossily:
RANGE_DISTRIBUTED:RelDistributioncarries no range boundaries, so the range-bucket expression from Questions about exchangeKind of ExchangeRel substrait#519 can't be builtSortExchange: no Substrait equivalent for its per-partition sortSingleBucketExchangeandMultiBucketExchange→ Calcite: noRelDistributiongives what they ask for (multi-bucket can duplicate or drop rows)ScatterExchangeon a nested field, or on fields not in ascending order: Calcite sorts hash keys, which would change partition alignment between exchangesSqlKindFromRelalso returnedOTHER_DDLfor exchanges. Calcite'sRelRoot.projectskips applying output names for DDL kinds, so an exchange root could lose its output names; exchanges now returnSELECTlike the other relations that only pass rows through.Closes #153