You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by "xiedeyantu (via GitHub)" <gi...@apache.org> on 2025/03/31 10:40:17 UTC
[PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
xiedeyantu opened a new pull request, #4280:
URL: https://github.com/apache/calcite/pull/4280
Please see: [CALCITE-6930](https://issues.apache.org/jira/browse/CALCITE-6930)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2784972066
> > This looks pretty good, and it is surprisingly short given how much it accomplishes. I still have an algorithmic question, about the nature of the expressions in the OR.
>
> Sorry, I may not understand what you mean. What question about OR do I need to reply to?
This expansion is most useful when some of the orConds are conjunctions of table columns.
In fact, I think you don't even need to expand ALL the orConds, you could stop after one of them.
Is it worth analyzing the orConds and making a list of the ones that can lead to conjunctions that can be efficiently handled by equijoins and expanding only these?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2791169193
@mihaibudiu According to your suggestion, I split orConds. According to @suibianwanwank and @asolimando 's suggestions, I adjusted the implementation of right join (in order to implement full join, right anti join expansion needs to be implemented, and right join is supported by the way). And added a test in RelOptRuleTest. Please take a look again. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2037957326
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, relBuilder);
+ break;
+ case ANTI:
+ converted = expandAntiJoin(join, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, relBuilder);
+ break;
+ default:
+ return;
+ }
+
+ if (converted == null) {
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ private boolean skip(Join join) {
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ return orConds.size() <= 1;
+ }
+
+ private List<RexNode> splitCond(Join join) {
+ final RexBuilder builder = join.getCluster().getRexBuilder();
+ final List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ final int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+
+ final List<RexNode> result = new ArrayList<>();
+ final List<RexNode> otherBuffer = new ArrayList<>();
+
+ for (RexNode cond : orConds) {
+ if (isUsefulCond(cond, leftFieldCount)) {
+ if (!otherBuffer.isEmpty()) {
Review Comment:
This small fragment could be a utility function "creatDisjunction".
I wonder whether this exists already somewhere.
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
Review Comment:
just two, or can it be many?
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
Review Comment:
no "would". "this rule expands"
maybe you can add the inner join example here and say that examples for other kinds of joins can be found in the code below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029446365
##########
core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml:
##########
@@ -6357,6 +6357,144 @@ LogicalProject(EMPNO=[$3], ENAME=[$4], JOB=[$5], MGR=[$6], HIREDATE=[$7], SAL=[$
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[scott, EMP]])
LogicalTableScan(table=[[scott, DEPT]])
+]]>
Review Comment:
This is a very good suggestion. I also thought about it when writing the test. Is there any existing test framework that can support this kind of logic? I don’t seem to have found it. Do I need to make an extension for JdbcTest?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029688669
##########
core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml:
##########
@@ -6357,6 +6357,144 @@ LogicalProject(EMPNO=[$3], ENAME=[$4], JOB=[$5], MGR=[$6], HIREDATE=[$7], SAL=[$
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[scott, EMP]])
LogicalTableScan(table=[[scott, DEPT]])
+]]>
Review Comment:
I found a way to verify it, do you think it works?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2032856235
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
Review Comment:
@mihaibudiu I can't think of a good way to split out part of an `OR` and leave the rest unsplit. I can only think of expanding all `OR` or none of them. for example, whether all sub-expressions of an `OR` expansion can be converted into equijoins. Could you give me some advice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu closed pull request #4280: [CALCITE-6930] Implementing JoinConditionOrExpansionRule
URL: https://github.com/apache/calcite/pull/4280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2795379348
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [90.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2036379312
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinCommuteRule.java:
##########
@@ -137,6 +137,12 @@ public JoinCommuteRule(Class<? extends Join> clazz,
.build();
}
+ public static RexNode swapJoinCond(RexNode cond, Join join, RexBuilder rexBuilder) {
Review Comment:
It relies on VariableReplacer, which is a private class, so I wrap a method here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2033709445
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
Review Comment:
I think you can analyze each of the orConds separately, and make a list of the ones that are "useful". Then you rewrite the OR as (useful0 OR useful1 OR ... OR (everythingelse)), where "everythingelse" is an OR which contains the expressions which are not useful. Then you expand only the useful conditions, and treat "everythingelse" as a simple atomic boolean expression, without regards to its content.
Regarding when an expression is "useful": An expression is useful if it is:
- a conjunction that has at least one useful subexpression
- it is a comparison using EQUALS or IS NOT DISTINCT FROM between two columns in the two inputs of the join
There is one more subtlety to worry about: if OR has lazy evaluation, you cannot actually reorder the expressions in the OR, because you may produce runtime exceptions. For example TRUE OR a/0 = b cannot be reordered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2787846228
> Consider this condition left.x = right.y OR (left.x < 5 AND left.y > 6) In this case "everything else" is "left.x < 5 AND left.y > 6". Regarding IS NOT DISTINCT FROM, that's the concern of other optimizations. It clearly defines an equijoin.
In this condition left.x = right.y OR (left.x < 5 AND left.y > 6). In my opinion, it may not be expanded at all. I have not thought about how to expand part of it and not expand the other part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2791571702
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [89.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2797035519
> > I successfully executed `./gradlew build` locally. I don't know why continuous-integration/jenkins/pr-head failed, but it should have nothing to do with pr.
>
> From the error message reported, it appears that RelOptRuleTest is in error.
Yes, I know. But it work locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2021885531
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
Review Comment:
Yes, my friend silun is working on it. I will relax this restriction later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2796682505
I successfully executed `./gradlew build` locally. I don't know why continuous-integration/jenkins/pr-head failed, but it should have nothing to do with pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2797407220
I used `./gradlew build` and `build` in idea to compile locally for more than 10 times and they were all successful, all after clean and rebuild. It should be ruled out that it is a code problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2021793676
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
Review Comment:
why is this necessary? The transformation is legal anyway.
Ideally this notion of "simple" should be parameterized, so one could plug in a different implementation.
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
+ return;
+ }
+ }
+
+ List<RexNode> condsAppend = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(condsAppend.get(j)));
+ }
+ condsAppend.add(orCond);
+ relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(join.getJoinType(), orCond);
+ }
+
+ relBuilder.union(true, orConds.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
Review Comment:
I didn't expect this to be necessary.
Why would types be different?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2791173632
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [90.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029688803
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2791630182
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [89.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2039076878
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,401 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexUtil;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>This rule expands OR conditions in join clauses into
+ * multiple separate join conditions, allowing the optimizer
+ * to handle these conditions more efficiently.
+ *
+ * <p>The following is an example of inner join,
+ * and other examples for other kinds of joins
+ * can be found in the code below.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), inner]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * ├─── TableScan[t1]
+ * └─── TableScan[t2]
+ *
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, relBuilder);
+ break;
+ case ANTI:
+ converted = expandAntiJoin(join, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, relBuilder);
+ break;
+ default:
+ return;
+ }
+
+ if (converted == null) {
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ private boolean skip(Join join) {
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ return orConds.size() <= 1;
+ }
+
+ private List<RexNode> splitCond(Join join) {
+ final RexBuilder builder = join.getCluster().getRexBuilder();
+ final List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ final int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+
+ final List<RexNode> result = new ArrayList<>();
+ final List<RexNode> otherBuffer = new ArrayList<>();
+
+ for (RexNode cond : orConds) {
+ if (isUsefulCond(cond, leftFieldCount)) {
+ if (!otherBuffer.isEmpty()) {
+ result.add(RexUtil.composeDisjunction(builder, otherBuffer));
+ otherBuffer.clear();
+ }
+ result.add(cond);
+ } else {
+ otherBuffer.add(cond);
+ }
+ }
+
+ if (!otherBuffer.isEmpty()) {
+ result.add(RexUtil.composeDisjunction(builder, otherBuffer));
+ }
+
+ return result;
+ }
+
+ private boolean isUsefulCond(RexNode node, int leftFieldCount) {
+ if (!(node instanceof RexCall)) {
+ return false;
+ }
+
+ RexCall call = (RexCall) node;
+ SqlKind kind = call.getKind();
+ switch (kind) {
+ case EQUALS:
+ case IS_NOT_DISTINCT_FROM:
+ RexNode left = call.getOperands().get(0);
Review Comment:
For example, it also includes is not distinct from, and the cast you mentioned, maybe it is not very appropriate to use whether it contains equal conditions, or is there any better name you can recommend?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2034701353
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), left]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * ├── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Project[t1-side cols + NULLs]
+ * └── Join[t1.id=t2.id, anti]
+ * ├── Join[t1.age=t2.age, anti]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── TableScan[t2]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
Review Comment:
Hey, great minds think alike! In the next submission, I will add a case of how to handle RIGHT JOIN through JOIN_COMMUTE_OUTER and JOIN_CONDITION_OR_EXPANSION_RULE. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029451642
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
Review Comment:
If I understand correctly, I may change the comments to tbl1 and tbl2 to make it clearer. Otherwise, I may not understand your meaning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2787832056
> Indeed, isSimpleCondition is pretty close to what you need.
Thanks! The lazy evaluation is ignored. Well, I have two questions that I don't understand. The first is that IS NOT DISTINCT FROM will not generate an equal join, as modified in CALCITE-6904. The second is that OR is expanded into "useful" and "else". It seems that it is not possible to expand only "useful" and "else" also needs to participate in the expansion logic. Could you give me some modification suggestions based on `isSimpleCondition`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2021882415
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
Review Comment:
My initial thought is that all the expanded joins can become HashJoin. If one of them becomes NestedLoopJoin, this conversion has no advantage and CBO may not choose it. I read the jira you replied to me and it seems that there is no restriction here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2021884148
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
Review Comment:
you are thinking about this optimization only, but combined with other optimizations there could be additional benefits.
For example, this could help in combination with this new rule that is being added: https://issues.apache.org/jira/browse/CALCITE-6914
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2034646672
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), left]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * ├── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Project[t1-side cols + NULLs]
+ * └── Join[t1.id=t2.id, anti]
+ * ├── Join[t1.age=t2.age, anti]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── TableScan[t2]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
Review Comment:
Thanks! There is indeed a problem here. My idea is that `RIGHT JOIN` does not support direct `OR` expansion for the time being, and it can be converted to `LEFT JOIN` through `JOIN_COMMUTE_OUTER` and then `OR` expansion, which can avoid repeated `JOIN` commute in `JOIN_CONDITION_OR_EXPANSION_RULE` and some complex adaptation. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029431018
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
Review Comment:
Yes, you're right!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2036363334
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -55,40 +62,92 @@ protected JoinConditionOrExpansionRule(Config config) {
@Override public void onMatch(RelOptRuleCall call) {
Join join = call.rel(0);
RelBuilder relBuilder = call.builder();
- List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
-
- if (orConds.size() <= 1) {
- return;
- }
RelNode converted;
switch (join.getJoinType()) {
case INNER:
- converted = expandInnerJoin(join, orConds, relBuilder);
- break;
- case SEMI:
- converted =
- expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ converted = expandInnerJoin(join, relBuilder);
break;
case ANTI:
- converted =
- expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ converted = expandAntiJoin(join, relBuilder);
break;
case LEFT:
- converted = expandLeftJoin(join, orConds, relBuilder);
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
break;
case RIGHT:
- converted = expandRightJoin(join, orConds, relBuilder);
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
break;
case FULL:
- converted = expandFullJoin(join, orConds, relBuilder);
+ converted = expandFullJoin(join, relBuilder);
break;
default:
return;
}
+
+ if (converted == null) {
+ return;
+ }
call.transformTo(converted);
}
+ private boolean skip(Join join) {
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ return orConds.size() <= 1;
+ }
+
+ private List<RexNode> splitCond(Join join) {
+ RexBuilder builder = join.getCluster().getRexBuilder();
+ List<RexNode> conds = new ArrayList<>();
+ List<RexNode> otherConds = new ArrayList<>();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+
+ for (RexNode cond : orConds) {
+ if (!isUsefulCond(cond, leftFieldCount)) {
Review Comment:
So you are not preserving the order...
This may cause problems with operations that cause runtime exceptions.
Note that almost anything can cause runtime exceptions, e.g., arithmetic overflow.
##########
core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml:
##########
@@ -6613,26 +6613,26 @@ EnumerableProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], SAL
<Resource name="sql">
<![CDATA[select *
from EMP as p1
-inner join EMP as p2 on p1.empno = p2.empno or p1.sal = p2.sal]]>
+inner join EMP as p2 on p1.empno = p2.empno or p1.mgr = p2.mgr]]>
</Resource>
<Resource name="planBefore">
<![CDATA[
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[$9], ENAME0=[$10], JOB0=[$11], MGR0=[$12], HIREDATE0=[$13], SAL0=[$14], COMM0=[$15], DEPTNO0=[$16], SLACKER0=[$17])
- LogicalJoin(condition=[OR(=($0, $9), =($5, $14))], joinType=[inner])
+ LogicalJoin(condition=[OR(=($0, $9), =($3, $12))], joinType=[inner])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
]]>
</Resource>
<Resource name="planAfter">
<![CDATA[
-EnumerableProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[$9], ENAME0=[$10], JOB0=[$11], MGR0=[$12], HIREDATE0=[$13], SAL0=[$14], COMM0=[$15], DEPTNO0=[$16], SLACKER0=[$17])
- EnumerableUnion(all=[true])
- EnumerableHashJoin(condition=[=($0, $9)], joinType=[inner])
- EnumerableTableScan(table=[[CATALOG, SALES, EMP]])
- EnumerableTableScan(table=[[CATALOG, SALES, EMP]])
- EnumerableHashJoin(condition=[AND(=($5, $14), <>($0, $9))], joinType=[inner])
- EnumerableTableScan(table=[[CATALOG, SALES, EMP]])
- EnumerableTableScan(table=[[CATALOG, SALES, EMP]])
+LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], EMPNO0=[$9], ENAME0=[$10], JOB0=[$11], MGR0=[$12], HIREDATE0=[$13], SAL0=[$14], COMM0=[$15], DEPTNO0=[$16], SLACKER0=[$17])
Review Comment:
this is better
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinCommuteRule.java:
##########
@@ -137,6 +137,12 @@ public JoinCommuteRule(Class<? extends Join> clazz,
.build();
}
+ public static RexNode swapJoinCond(RexNode cond, Join join, RexBuilder rexBuilder) {
Review Comment:
Does this have to be public or can it be moved to the file where it's used?
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -256,26 +313,50 @@ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds
* │ └── TableScan[tbl]
* └── TableScan[tbl]
*/
- private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
- List<RexNode> orConds, RelBuilder relBuilder) {
- RelNode left = join.getLeft();
+ private @Nullable RelNode expandAntiJoin(Join join, RelBuilder relBuilder) {
+ if (skip(join)) {
+ return null;
+ }
+ List<RexNode> orConds = splitCond(join);
+ return expandAntiJoinToRelNode(join, orConds, true, false, relBuilder);
+ }
+
+ private RelNode expandAntiJoinToRelNode(Join join, List<RexNode> orConds,
+ boolean isLeftAnti, boolean isAppendNulls, RelBuilder relBuilder) {
+ RelNode left = isLeftAnti ? join.getLeft() : join.getRight();
+ RelNode right = isLeftAnti ? join.getRight() : join.getLeft();
+
+ RelNode top = left;
for (int i = 0; i < orConds.size(); i++) {
RexNode orCond = orConds.get(i);
- relBuilder.push(left)
- .push(join.getRight())
- .join(joinType, orCond);
- left = relBuilder.build();
+ relBuilder.push(top)
+ .push(right)
+ .join(JoinRelType.ANTI,
+ isLeftAnti
+ ? orCond
+ : JoinCommuteRule.swapJoinCond(orCond, join, relBuilder.getRexBuilder()));
+ top = relBuilder.build();
}
- relBuilder.push(left);
- List<RexNode> projects = new ArrayList<>(relBuilder.fields());
- int fieldCount = relBuilder.fields().size();
- while (fieldCount < join.getRowType().getFieldCount()) {
- projects.add(
+ if (!isAppendNulls) {
+ return top;
+ }
+
+ relBuilder.push(top);
+ int fieldCount = 0;
+ List<RexNode> fields = new ArrayList<>(relBuilder.fields());
+ List<RexNode> nulls = new ArrayList<>();
+ while (fieldCount < right.getRowType().getFieldCount()) {
Review Comment:
maybe a for loop is better
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -55,40 +62,92 @@ protected JoinConditionOrExpansionRule(Config config) {
@Override public void onMatch(RelOptRuleCall call) {
Join join = call.rel(0);
RelBuilder relBuilder = call.builder();
- List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
-
- if (orConds.size() <= 1) {
- return;
- }
RelNode converted;
switch (join.getJoinType()) {
case INNER:
- converted = expandInnerJoin(join, orConds, relBuilder);
- break;
- case SEMI:
- converted =
- expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ converted = expandInnerJoin(join, relBuilder);
break;
case ANTI:
- converted =
- expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ converted = expandAntiJoin(join, relBuilder);
break;
case LEFT:
- converted = expandLeftJoin(join, orConds, relBuilder);
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
break;
case RIGHT:
- converted = expandRightJoin(join, orConds, relBuilder);
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
break;
case FULL:
- converted = expandFullJoin(join, orConds, relBuilder);
+ converted = expandFullJoin(join, relBuilder);
break;
default:
return;
}
+
+ if (converted == null) {
+ return;
+ }
call.transformTo(converted);
}
+ private boolean skip(Join join) {
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ return orConds.size() <= 1;
+ }
+
+ private List<RexNode> splitCond(Join join) {
+ RexBuilder builder = join.getCluster().getRexBuilder();
+ List<RexNode> conds = new ArrayList<>();
+ List<RexNode> otherConds = new ArrayList<>();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+
+ for (RexNode cond : orConds) {
+ if (!isUsefulCond(cond, leftFieldCount)) {
Review Comment:
If you want to be conservative you have to:
- either detect whether an expression can cause a runtime exception and only move the ones that never do
- preserve the order and stop at the first expression which is not useful and put everything from that point on in a single big OR which is not expanded
Can you add a test for such a case so we can see what happens?
For example an expression such as `A <= 0 OR LN(A) <= 0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2032168693
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
Review Comment:
This expansion is most useful when some of the orConds are conjunctions of table columns.
In fact, I think you don't even need to expand ALL the orConds, you could stop after one of them.
Is it worth analyzing the orConds and making a list of the ones that can lead to conjunctions that can be efficiently handled by equijoins and expanding only these?
##########
core/src/test/java/org/apache/calcite/test/JdbcTest.java:
##########
@@ -1975,6 +1975,69 @@ private void checkResultSetMetaData(Connection connection, String sql)
"c0=Drink; c1=Dairy; c2=USA; c3=WA; c4=Bellingham");
}
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-6930">[CALCITE-6930]
+ * Implementing JoinConditionOrExpansionRule</a>. */
+ @Test void testJoinConditionOrExpansionRule() {
+ final String sql = ""
+ + "SELECT \"t1\".\"deptno\", \"t1\".\"empid\", \"t2\".\"deptno\", \"t2\".\"empid\"\n"
+ + "FROM \"hr\".\"emps\" AS \"t1\"\n"
+ + "INNER JOIN (\n"
+ + " SELECT (\"deptno\" + 10) AS \"deptno\", (\"empid\" + 100) AS \"empid\" FROM \"hr\".\"emps\"\n"
+ + ") AS \"t2\"\n"
+ + "ON (\"t1\".\"deptno\" = \"t2\".\"deptno\") OR (\"t1\".\"empid\" = \"t2\".\"empid\")";
+
+ String[] returns = new String[] {
+ "deptno=20; empid=200; deptno=20; empid=200",
+ "deptno=20; empid=200; deptno=20; empid=210",
+ "deptno=20; empid=200; deptno=20; empid=250"};
+
+ CalciteAssert.hr()
+ .query(sql)
+ .returnsUnordered(returns);
+
+ CalciteAssert.hr()
+ .query(sql)
+ .withHook(Hook.PLANNER, (Consumer<RelOptPlanner>) planner -> {
+ planner.addRule(CoreRules.JOIN_CONDITION_OR_EXPANSION_RULE);
+ planner.removeRule(EnumerableRules.ENUMERABLE_MERGE_JOIN_RULE);
+ })
+ .returnsUnordered(returns);
+ }
+
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-6930">[CALCITE-6930]
+ * Implementing JoinConditionOrExpansionRule</a>. */
+ @Test void testJoinConditionOrExpansionRuleLeft() {
Review Comment:
could you also add a test where the input table is a multiset, so we can check that multisets are handled correctly?
You can make a multiset for example using `emp UNION ALL emp`.
##########
core/src/test/java/org/apache/calcite/test/JdbcTest.java:
##########
@@ -1975,6 +1975,69 @@ private void checkResultSetMetaData(Connection connection, String sql)
"c0=Drink; c1=Dairy; c2=USA; c3=WA; c4=Bellingham");
}
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-6930">[CALCITE-6930]
+ * Implementing JoinConditionOrExpansionRule</a>. */
+ @Test void testJoinConditionOrExpansionRule() {
+ final String sql = ""
+ + "SELECT \"t1\".\"deptno\", \"t1\".\"empid\", \"t2\".\"deptno\", \"t2\".\"empid\"\n"
+ + "FROM \"hr\".\"emps\" AS \"t1\"\n"
+ + "INNER JOIN (\n"
+ + " SELECT (\"deptno\" + 10) AS \"deptno\", (\"empid\" + 100) AS \"empid\" FROM \"hr\".\"emps\"\n"
+ + ") AS \"t2\"\n"
+ + "ON (\"t1\".\"deptno\" = \"t2\".\"deptno\") OR (\"t1\".\"empid\" = \"t2\".\"empid\")";
+
+ String[] returns = new String[] {
+ "deptno=20; empid=200; deptno=20; empid=200",
+ "deptno=20; empid=200; deptno=20; empid=210",
+ "deptno=20; empid=200; deptno=20; empid=250"};
+
+ CalciteAssert.hr()
+ .query(sql)
+ .returnsUnordered(returns);
+
+ CalciteAssert.hr()
+ .query(sql)
+ .withHook(Hook.PLANNER, (Consumer<RelOptPlanner>) planner -> {
+ planner.addRule(CoreRules.JOIN_CONDITION_OR_EXPANSION_RULE);
+ planner.removeRule(EnumerableRules.ENUMERABLE_MERGE_JOIN_RULE);
+ })
+ .returnsUnordered(returns);
+ }
+
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-6930">[CALCITE-6930]
+ * Implementing JoinConditionOrExpansionRule</a>. */
+ @Test void testJoinConditionOrExpansionRuleLeft() {
+ final String sql = ""
Review Comment:
how about a test as well where some of the values that appear in the condition are null? Mostly to check that the expansion such as `t1.age=t2.age AND t1.id≠t2.id` works correctly when NULLs are involved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2784975242
> > > This looks pretty good, and it is surprisingly short given how much it accomplishes. I still have an algorithmic question, about the nature of the expressions in the OR.
> >
> >
> > Sorry, I may not understand what you mean. What question about OR do I need to reply to?
>
> This expansion is most useful when some of the orConds are conjunctions of table columns. In fact, I think you don't even need to expand ALL the orConds, you could stop after one of them. Is it worth analyzing the orConds and making a list of the ones that can lead to conjunctions that can be efficiently handled by equijoins and expanding only these?
I agree with you. In fact, I considered this in my initial view. You can check if `isSimpleCondition` is what you mean (maybe the name needs to be changed). The link is [here ](https://github.com/apache/calcite/commit/749c0725643cf3eb7953b236c0595966198d606d#diff-5761c993bb47ab9756426564eb187d027412f8e6469a0c0df8ce0935d2222fd8R7-R121)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029452486
##########
core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml:
##########
@@ -6357,6 +6357,144 @@ LogicalProject(EMPNO=[$3], ENAME=[$4], JOB=[$5], MGR=[$6], HIREDATE=[$7], SAL=[$
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[scott, EMP]])
LogicalTableScan(table=[[scott, DEPT]])
+]]>
Review Comment:
Yes, I think JdbcTest is the only place where you can do this right now, although it's a cumbersome process.
I wonder whether there is any support for differential testing, so you don't have to use returnsUnordered.
Ideally you would compute the result using the existing rules, and compare to the result obtained using the new rules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029490902
##########
core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml:
##########
@@ -6357,6 +6357,144 @@ LogicalProject(EMPNO=[$3], ENAME=[$4], JOB=[$5], MGR=[$6], HIREDATE=[$7], SAL=[$
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[scott, EMP]])
LogicalTableScan(table=[[scott, DEPT]])
+]]>
Review Comment:
Thanks, I'll see how to adjust the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2022084700
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
+ return;
+ }
+ }
+
+ List<RexNode> condsAppend = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(condsAppend.get(j)));
+ }
+ condsAppend.add(orCond);
+ relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(join.getJoinType(), orCond);
+ }
+
+ relBuilder.union(true, orConds.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
Review Comment:
Strangely, I didn't get any error messages, and it worked fine without this logic. So I just removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2021883964
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
+ return;
+ }
+ }
+
+ List<RexNode> condsAppend = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(condsAppend.get(j)));
+ }
+ condsAppend.add(orCond);
+ relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(join.getJoinType(), orCond);
+ }
+
+ relBuilder.union(true, orConds.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
Review Comment:
I'll add a detailed error report here later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2787836169
Consider this condition left.x = right.y OR (left.x < 5 AND left.y > 6)
In this case "everything else" is "left.x < 5 AND left.y > 6".
Regarding IS NOT DISTINCT FROM, that's the concern of other optimizations. It clearly defines an equijoin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2780199336
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [84.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029461427
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, semi]
+ * ├── HashJoin[age=age0, semi]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandFullJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandLeftJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ relBuilder.pushAll(joins)
+ .union(true, joins.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
+ .map(field -> {
+ RexNode rexNode = relBuilder.field(field.getIndex());
+ return field.getType().equals(rexNode.getType())
+ ? rexNode
+ : relBuilder.getRexBuilder().makeCast(field.getType(), rexNode, true, false);
+ }).collect(Collectors.toList());
+
+ return relBuilder.project(projects)
+ .build();
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── HashJoin[age=age0 AND id≠id0, inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandInnerJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandInnerJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(orConds.get(j)));
+ }
+ joins.add(relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(JoinRelType.INNER, orCond)
+ .build());
+ }
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), anti]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
+ List<RexNode> orConds, RelBuilder relBuilder) {
+ RelNode left = join.getLeft();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ relBuilder.push(left)
+ .push(join.getRight())
+ .join(joinType, orCond);
+ left = relBuilder.build();
+ }
+
+ relBuilder.push(left);
+ int fieldCount = join.getRowType().getFieldCount();
+ List<RexNode> projects = new ArrayList<>(relBuilder.fields());
+ while (fieldCount > relBuilder.fields().size()) {
+ projects.add(relBuilder.literal(null));
Review Comment:
but can you help the builder by supplying nulls with the correct type?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029452616
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
Review Comment:
yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2797624349
can you try to create another PR with the exact same changes, so we can see whether the CI succeeds there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2038530813
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
Review Comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2787186190
Indeed, isSimpleCondition is pretty close to what you need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "asolimando (via GitHub)" <gi...@apache.org>.
asolimando commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2034687517
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), left]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * ├── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Project[t1-side cols + NULLs]
+ * └── Join[t1.id=t2.id, anti]
+ * ├── Join[t1.age=t2.age, anti]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── TableScan[t2]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
Review Comment:
I haven't looked at all details, but the principle of keeping individual rules as simple as possible and exploit the combined effect of multiple rules is in general a good design.
That's, for instance, the reason why we keep simplification to the minimum in all rules, assuming that rules from `ReduceExpressionsRule` will take care of the rest.
It might be worth writing a test in `RelOptRuleTest` to cover such an example?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029459980
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, semi]
+ * ├── HashJoin[age=age0, semi]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandFullJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandLeftJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ relBuilder.pushAll(joins)
+ .union(true, joins.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
+ .map(field -> {
+ RexNode rexNode = relBuilder.field(field.getIndex());
+ return field.getType().equals(rexNode.getType())
+ ? rexNode
+ : relBuilder.getRexBuilder().makeCast(field.getType(), rexNode, true, false);
+ }).collect(Collectors.toList());
+
+ return relBuilder.project(projects)
+ .build();
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── HashJoin[age=age0 AND id≠id0, inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandInnerJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandInnerJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(orConds.get(j)));
+ }
+ joins.add(relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(JoinRelType.INNER, orCond)
+ .build());
+ }
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), anti]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
+ List<RexNode> orConds, RelBuilder relBuilder) {
+ RelNode left = join.getLeft();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ relBuilder.push(left)
+ .push(join.getRight())
+ .join(joinType, orCond);
+ left = relBuilder.build();
+ }
+
+ relBuilder.push(left);
+ int fieldCount = join.getRowType().getFieldCount();
+ List<RexNode> projects = new ArrayList<>(relBuilder.fields());
+ while (fieldCount > relBuilder.fields().size()) {
+ projects.add(relBuilder.literal(null));
Review Comment:
Builder can not do it. It is a SEMI/ANTI JOIN, there are fewer columns than INNER JOIN, and the columns will not match during UNION. NULL is used here to fill in the gaps, and the JOIN will infer types using derive method later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029479237
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, semi]
+ * ├── HashJoin[age=age0, semi]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandFullJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandLeftJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ relBuilder.pushAll(joins)
+ .union(true, joins.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
+ .map(field -> {
+ RexNode rexNode = relBuilder.field(field.getIndex());
+ return field.getType().equals(rexNode.getType())
+ ? rexNode
+ : relBuilder.getRexBuilder().makeCast(field.getType(), rexNode, true, false);
+ }).collect(Collectors.toList());
+
+ return relBuilder.project(projects)
+ .build();
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── HashJoin[age=age0 AND id≠id0, inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandInnerJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandInnerJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(orConds.get(j)));
+ }
+ joins.add(relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(JoinRelType.INNER, orCond)
+ .build());
+ }
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), anti]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
+ List<RexNode> orConds, RelBuilder relBuilder) {
+ RelNode left = join.getLeft();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ relBuilder.push(left)
+ .push(join.getRight())
+ .join(joinType, orCond);
+ left = relBuilder.build();
+ }
+
+ relBuilder.push(left);
+ int fieldCount = join.getRowType().getFieldCount();
+ List<RexNode> projects = new ArrayList<>(relBuilder.fields());
+ while (fieldCount > relBuilder.fields().size()) {
+ projects.add(relBuilder.literal(null));
Review Comment:
I think I understand what you mean. Using literal directly may cause potential problems in some special cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029485004
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, semi]
+ * ├── HashJoin[age=age0, semi]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandFullJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandLeftJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ relBuilder.pushAll(joins)
+ .union(true, joins.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
+ .map(field -> {
+ RexNode rexNode = relBuilder.field(field.getIndex());
+ return field.getType().equals(rexNode.getType())
+ ? rexNode
+ : relBuilder.getRexBuilder().makeCast(field.getType(), rexNode, true, false);
+ }).collect(Collectors.toList());
+
+ return relBuilder.project(projects)
+ .build();
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── HashJoin[age=age0 AND id≠id0, inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandInnerJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandInnerJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(orConds.get(j)));
+ }
+ joins.add(relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(JoinRelType.INNER, orCond)
+ .build());
+ }
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), anti]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
+ List<RexNode> orConds, RelBuilder relBuilder) {
+ RelNode left = join.getLeft();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ relBuilder.push(left)
+ .push(join.getRight())
+ .join(joinType, orCond);
+ left = relBuilder.build();
+ }
+
+ relBuilder.push(left);
+ int fieldCount = join.getRowType().getFieldCount();
+ List<RexNode> projects = new ArrayList<>(relBuilder.fields());
+ while (fieldCount > relBuilder.fields().size()) {
+ projects.add(relBuilder.literal(null));
Review Comment:
Yes, maybe I can know the types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029688719
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
Review Comment:
Done
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "suibianwanwank (via GitHub)" <gi...@apache.org>.
suibianwanwank commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2796883594
> I successfully executed `./gradlew build` locally. I don't know why continuous-integration/jenkins/pr-head failed, but it should have nothing to do with pr.
From the error message reported, it appears that RelOptRuleTest is in error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2021885531
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
Review Comment:
Yes, my friend silun is working on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2771016194
> When you address comments you should not force-push, but rather add new commits; this makes it much easier to review the changes.
Thank you for pointing out the problem. I will follow your suggestion later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "suibianwanwank (via GitHub)" <gi...@apache.org>.
suibianwanwank commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2038998022
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,401 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexUtil;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>This rule expands OR conditions in join clauses into
+ * multiple separate join conditions, allowing the optimizer
+ * to handle these conditions more efficiently.
+ *
+ * <p>The following is an example of inner join,
+ * and other examples for other kinds of joins
+ * can be found in the code below.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), inner]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * ├─── TableScan[t1]
+ * └─── TableScan[t2]
+ *
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, relBuilder);
+ break;
+ case ANTI:
+ converted = expandAntiJoin(join, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, relBuilder);
+ break;
+ default:
+ return;
+ }
+
+ if (converted == null) {
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ private boolean skip(Join join) {
Review Comment:
This seems to be a precondition for whether this rule is applied or not, maybe we can put it in match?
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinCommuteRule.java:
##########
@@ -137,6 +137,12 @@ public JoinCommuteRule(Class<? extends Join> clazz,
.build();
}
+ public static RexNode swapJoinCond(RexNode cond, Join join, RexBuilder rexBuilder) {
Review Comment:
I don't know if there's a practice where one rule references another rule's method, but generally speaking, rules should be independent. You could try moving the method outside the rules.
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,401 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexUtil;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>This rule expands OR conditions in join clauses into
+ * multiple separate join conditions, allowing the optimizer
+ * to handle these conditions more efficiently.
+ *
+ * <p>The following is an example of inner join,
+ * and other examples for other kinds of joins
+ * can be found in the code below.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), inner]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * ├─── TableScan[t1]
+ * └─── TableScan[t2]
+ *
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, relBuilder);
+ break;
+ case ANTI:
+ converted = expandAntiJoin(join, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, relBuilder);
+ break;
+ default:
+ return;
+ }
+
+ if (converted == null) {
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ private boolean skip(Join join) {
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ return orConds.size() <= 1;
+ }
+
+ private List<RexNode> splitCond(Join join) {
+ final RexBuilder builder = join.getCluster().getRexBuilder();
+ final List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ final int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+
+ final List<RexNode> result = new ArrayList<>();
+ final List<RexNode> otherBuffer = new ArrayList<>();
+
+ for (RexNode cond : orConds) {
+ if (isUsefulCond(cond, leftFieldCount)) {
+ if (!otherBuffer.isEmpty()) {
+ result.add(RexUtil.composeDisjunction(builder, otherBuffer));
+ otherBuffer.clear();
+ }
+ result.add(cond);
+ } else {
+ otherBuffer.add(cond);
+ }
+ }
+
+ if (!otherBuffer.isEmpty()) {
+ result.add(RexUtil.composeDisjunction(builder, otherBuffer));
+ }
+
+ return result;
+ }
+
+ private boolean isUsefulCond(RexNode node, int leftFieldCount) {
+ if (!(node instanceof RexCall)) {
+ return false;
+ }
+
+ RexCall call = (RexCall) node;
+ SqlKind kind = call.getKind();
+ switch (kind) {
+ case EQUALS:
+ case IS_NOT_DISTINCT_FROM:
+ RexNode left = call.getOperands().get(0);
Review Comment:
Perhaps this method name could be clearer, like `containEquivCond`.
BTW, since Calcite is a general-purpose database with a strict type system. In many cases, the inputRef in join conditions will contain casts, which may cause some optimizations to fail. This issue exists across several rules. I'll try to collect such examples and create a new ticket to address.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "suibianwanwank (via GitHub)" <gi...@apache.org>.
suibianwanwank commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2038994412
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinCommuteRule.java:
##########
@@ -137,6 +137,12 @@ public JoinCommuteRule(Class<? extends Join> clazz,
.build();
}
+ public static RexNode swapJoinCond(RexNode cond, Join join, RexBuilder rexBuilder) {
Review Comment:
I don't know if there's a practice where one rule references another rule's method, but generally, rules should be independent. You could try moving the method outside the rules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2039070157
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinCommuteRule.java:
##########
@@ -137,6 +137,12 @@ public JoinCommuteRule(Class<? extends Join> clazz,
.build();
}
+ public static RexNode swapJoinCond(RexNode cond, Join join, RexBuilder rexBuilder) {
Review Comment:
This involves the use of a private class, which I don't want to change. This rule also has some static methods for external use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2791155262
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [90.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2802430078
I will close this PR in favor of #4300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu closed pull request #4280: [CALCITE-6930] Implementing JoinConditionOrExpansionRule
URL: https://github.com/apache/calcite/pull/4280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2797929603
> can you try to create another PR with the exact same changes, so we can see whether the CI succeeds there?
@mihaibudiu Very strange! I created another PR and it passed, and the sonar of this PR also passed with the new PR. New pr is #4300.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2033710269
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
Review Comment:
Notice that SQL does not specify whether OR is lazy, but it is lazy in many dialects. I don't know if other parts of Calcite pay attention to this fact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2786047463
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [84.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2774480130
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4280&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4280&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [84.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4280&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4280)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029480211
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, semi]
+ * ├── HashJoin[age=age0, semi]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandFullJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandLeftJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ relBuilder.pushAll(joins)
+ .union(true, joins.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
+ .map(field -> {
+ RexNode rexNode = relBuilder.field(field.getIndex());
+ return field.getType().equals(rexNode.getType())
+ ? rexNode
+ : relBuilder.getRexBuilder().makeCast(field.getType(), rexNode, true, false);
+ }).collect(Collectors.toList());
+
+ return relBuilder.project(projects)
+ .build();
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── HashJoin[age=age0 AND id≠id0, inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandInnerJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandInnerJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(orConds.get(j)));
+ }
+ joins.add(relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(JoinRelType.INNER, orCond)
+ .build());
+ }
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), anti]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
+ List<RexNode> orConds, RelBuilder relBuilder) {
+ RelNode left = join.getLeft();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ relBuilder.push(left)
+ .push(join.getRight())
+ .join(joinType, orCond);
+ left = relBuilder.build();
+ }
+
+ relBuilder.push(left);
+ int fieldCount = join.getRowType().getFieldCount();
+ List<RexNode> projects = new ArrayList<>(relBuilder.fields());
+ while (fieldCount > relBuilder.fields().size()) {
+ projects.add(relBuilder.literal(null));
Review Comment:
If you already know the type, it's probably best to use it rather than infer it again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029454445
##########
core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml:
##########
@@ -6357,6 +6357,144 @@ LogicalProject(EMPNO=[$3], ENAME=[$4], JOB=[$5], MGR=[$6], HIREDATE=[$7], SAL=[$
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[scott, EMP]])
LogicalTableScan(table=[[scott, DEPT]])
+]]>
Review Comment:
Maybe CalciteASsert.consistentResult is the function needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2027979923
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
Review Comment:
EMP?
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
Review Comment:
can you use two tables here to make it clearer?
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
Review Comment:
this expansion looks wrong to me; the antijoin is not symmetric.
I think it is a chain of antijoins with the tablescane on the left, isn't it?
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
Review Comment:
technically I don't think these are hash joins, they are all logical joins, aren't they?
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, semi]
+ * ├── HashJoin[age=age0, semi]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandFullJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandLeftJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ relBuilder.pushAll(joins)
+ .union(true, joins.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
+ .map(field -> {
+ RexNode rexNode = relBuilder.field(field.getIndex());
+ return field.getType().equals(rexNode.getType())
+ ? rexNode
+ : relBuilder.getRexBuilder().makeCast(field.getType(), rexNode, true, false);
+ }).collect(Collectors.toList());
+
+ return relBuilder.project(projects)
+ .build();
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── HashJoin[age=age0 AND id≠id0, inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandInnerJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandInnerJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(orConds.get(j)));
+ }
+ joins.add(relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(JoinRelType.INNER, orCond)
+ .build());
+ }
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), anti]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
+ List<RexNode> orConds, RelBuilder relBuilder) {
+ RelNode left = join.getLeft();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ relBuilder.push(left)
+ .push(join.getRight())
+ .join(joinType, orCond);
+ left = relBuilder.build();
+ }
+
+ relBuilder.push(left);
+ int fieldCount = join.getRowType().getFieldCount();
+ List<RexNode> projects = new ArrayList<>(relBuilder.fields());
+ while (fieldCount > relBuilder.fields().size()) {
+ projects.add(relBuilder.literal(null));
Review Comment:
does the builder infer types for these?
##########
core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml:
##########
@@ -6357,6 +6357,144 @@ LogicalProject(EMPNO=[$3], ENAME=[$4], JOB=[$5], MGR=[$6], HIREDATE=[$7], SAL=[$
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[scott, EMP]])
LogicalTableScan(table=[[scott, DEPT]])
+]]>
Review Comment:
these tests are very hard for humans to certify.
it would be very nice if we could execute these plans somehow on some input data to compare results with the non-expanded form.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2784967662
> This looks pretty good, and it is surprisingly short given how much it accomplishes. I still have an algorithmic question, about the nature of the expressions in the OR.
Sorry, I may not understand what you mean. What question about OR do I need to reply to?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2029688684
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
Review Comment:
Done
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), left]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), full]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── HashJoin[age=age0 AND id≠id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * ├── Project[tbl-side cols + NULLs]
+ * │ └── HashJoin[id=id0, anti]
+ * │ ├── HashJoin[age=age0, anti]
+ * │ │ ├── TableScan[tbl]
+ * │ │ └── TableScan[tbl]
+ * │ └── TableScan[EMP]
+ * └── Project[tbl-side cols + NULLs]
+ * └── HashJoin[id=id0, semi]
+ * ├── HashJoin[age=age0, semi]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandFullJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandLeftJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder));
+ relBuilder.pushAll(joins)
+ .union(true, joins.size());
+
+ final List<RexNode> projects = join.getRowType().getFieldList().stream()
+ .map(field -> {
+ RexNode rexNode = relBuilder.field(field.getIndex());
+ return field.getType().equals(rexNode.getType())
+ ? rexNode
+ : relBuilder.getRexBuilder().makeCast(field.getType(), rexNode, true, false);
+ }).collect(Collectors.toList());
+
+ return relBuilder.project(projects)
+ .build();
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── HashJoin[id=id0, inner]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── HashJoin[age=age0 AND id≠id0, inner]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandInnerJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandInnerJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandInnerJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(orConds.get(j)));
+ }
+ joins.add(relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(JoinRelType.INNER, orCond)
+ .build());
+ }
+ return joins;
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(id=id0, age=age0), anti]
+ * ├── TableScan[tbl]
+ * └── TableScan[tbl]
+ *
+ * <p>into
+ *
+ * <p>HashJoin[id=id0, anti]
+ * ├── HashJoin[age=age0, anti]
+ * │ ├── TableScan[tbl]
+ * │ └── TableScan[tbl]
+ * └── TableScan[tbl]
+ */
+ private RelNode expandSemiOrAntiJoin(Join join, JoinRelType joinType,
+ List<RexNode> orConds, RelBuilder relBuilder) {
+ RelNode left = join.getLeft();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ relBuilder.push(left)
+ .push(join.getRight())
+ .join(joinType, orCond);
+ left = relBuilder.build();
+ }
+
+ relBuilder.push(left);
+ int fieldCount = join.getRowType().getFieldCount();
+ List<RexNode> projects = new ArrayList<>(relBuilder.fields());
+ while (fieldCount > relBuilder.fields().size()) {
+ projects.add(relBuilder.literal(null));
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "suibianwanwank (via GitHub)" <gi...@apache.org>.
suibianwanwank commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2034353430
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), left]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * ├── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Project[t1-side cols + NULLs]
+ * └── Join[t1.id=t2.id, anti]
+ * ├── Join[t1.age=t2.age, anti]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── TableScan[t2]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = new ArrayList<>();
+ joins.addAll(expandInnerJoinToRelNodes(join, orConds, relBuilder));
+ joins.add(expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder));
+ return joins;
+ }
+
+ private RelNode expandRightJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandRightJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandRightJoinToRelNodes(Join join, List<RexNode> orConds,
Review Comment:
Why is this a semi join here? If I understand correctly, for a right join this should swap the join inputs and use an anti join to ensure any unmatched condition return the right table column.
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), left]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * ├── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Project[t1-side cols + NULLs]
+ * └── Join[t1.id=t2.id, anti]
+ * ├── Join[t1.age=t2.age, anti]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── TableScan[t2]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
Review Comment:
The logic for handling Left Join and Right Join is very similar—perhaps we can merge them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2034649289
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, orConds, relBuilder);
+ break;
+ case SEMI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ break;
+ case ANTI:
+ converted =
+ expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftJoin(join, orConds, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandRightJoin(join, orConds, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, orConds, relBuilder);
+ break;
+ default:
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ /**
+ * This method will make the following conversions.
+ *
+ * <p>Project[*]
+ * └── Join[OR(t1.id=t2.id, t1.age=t2.age), left]
+ * ├── TableScan[t1]
+ * └── TableScan[t2]
+ *
+ * <p>into
+ *
+ * <p>Project[*]
+ * └── UnionAll
+ * ├── Join[t1.id=t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * ├── Join[t1.age=t2.age AND t1.id≠t2.id, inner]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── Project[t1-side cols + NULLs]
+ * └── Join[t1.id=t2.id, anti]
+ * ├── Join[t1.age=t2.age, anti]
+ * │ ├── TableScan[t1]
+ * │ └── TableScan[t2]
+ * └── TableScan[t2]
+ */
+ private RelNode expandLeftJoin(Join join, List<RexNode> orConds,
+ RelBuilder relBuilder) {
+ List<RelNode> joins = expandLeftJoinToRelNodes(join, orConds, relBuilder);
+ return relBuilder.pushAll(joins)
+ .union(true, joins.size())
+ .build();
+ }
+
+ private List<RelNode> expandLeftJoinToRelNodes(Join join, List<RexNode> orConds,
Review Comment:
As above, we can temporarily not support direct `OR` expansion of `RIGHT JOIN`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2037681368
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -55,40 +62,92 @@ protected JoinConditionOrExpansionRule(Config config) {
@Override public void onMatch(RelOptRuleCall call) {
Join join = call.rel(0);
RelBuilder relBuilder = call.builder();
- List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
-
- if (orConds.size() <= 1) {
- return;
- }
RelNode converted;
switch (join.getJoinType()) {
case INNER:
- converted = expandInnerJoin(join, orConds, relBuilder);
- break;
- case SEMI:
- converted =
- expandSemiOrAntiJoin(join, JoinRelType.SEMI, orConds, relBuilder);
+ converted = expandInnerJoin(join, relBuilder);
break;
case ANTI:
- converted =
- expandSemiOrAntiJoin(join, JoinRelType.ANTI, orConds, relBuilder);
+ converted = expandAntiJoin(join, relBuilder);
break;
case LEFT:
- converted = expandLeftJoin(join, orConds, relBuilder);
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
break;
case RIGHT:
- converted = expandRightJoin(join, orConds, relBuilder);
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
break;
case FULL:
- converted = expandFullJoin(join, orConds, relBuilder);
+ converted = expandFullJoin(join, relBuilder);
break;
default:
return;
}
+
+ if (converted == null) {
+ return;
+ }
call.transformTo(converted);
}
+ private boolean skip(Join join) {
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ return orConds.size() <= 1;
+ }
+
+ private List<RexNode> splitCond(Join join) {
+ RexBuilder builder = join.getCluster().getRexBuilder();
+ List<RexNode> conds = new ArrayList<>();
+ List<RexNode> otherConds = new ArrayList<>();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+
+ for (RexNode cond : orConds) {
+ if (!isUsefulCond(cond, leftFieldCount)) {
Review Comment:
I kept the order of the join conditions and also fixed a few details you mentioned. Thanks so much!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2038530666
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+
+ RelNode converted;
+ switch (join.getJoinType()) {
+ case INNER:
+ converted = expandInnerJoin(join, relBuilder);
+ break;
+ case ANTI:
+ converted = expandAntiJoin(join, relBuilder);
+ break;
+ case LEFT:
+ converted = expandLeftOrRightJoin(join, true, relBuilder);
+ break;
+ case RIGHT:
+ converted = expandLeftOrRightJoin(join, false, relBuilder);
+ break;
+ case FULL:
+ converted = expandFullJoin(join, relBuilder);
+ break;
+ default:
+ return;
+ }
+
+ if (converted == null) {
+ return;
+ }
+ call.transformTo(converted);
+ }
+
+ private boolean skip(Join join) {
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ return orConds.size() <= 1;
+ }
+
+ private List<RexNode> splitCond(Join join) {
+ final RexBuilder builder = join.getCluster().getRexBuilder();
+ final List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+ final int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+
+ final List<RexNode> result = new ArrayList<>();
+ final List<RexNode> otherBuffer = new ArrayList<>();
+
+ for (RexNode cond : orConds) {
+ if (isUsefulCond(cond, leftFieldCount)) {
+ if (!otherBuffer.isEmpty()) {
Review Comment:
Done!
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>this rule would expand the OR condition into
Review Comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2021885531
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ int leftFieldCount = join.getLeft().getRowType().getFieldCount();
+ for (RexNode cond : orConds) {
+ if (!isSimpleCondition(cond, leftFieldCount)) {
Review Comment:
Yes, I agree with you. I will relax this restriction later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2025063838
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ List<RexNode> extraConds = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(extraConds.get(j)));
+ }
+ extraConds.add(orCond);
+ relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(join.getJoinType(), orCond);
+ }
+
+ relBuilder.union(true, orConds.size());
+ call.transformTo(relBuilder.build());
+ }
+
+ /** Rule configuration. */
+ @Value.Immutable
+ public interface Config extends RelRule.Config {
+ Config DEFAULT = ImmutableJoinConditionOrExpansionRule.Config.of()
+ .withOperandFor(Join.class);
+
+ @Override default JoinConditionOrExpansionRule toRule() {
+ return new JoinConditionOrExpansionRule(this);
+ }
+
+ /** Defines an operand tree for the given classes. */
+ default Config withOperandFor(Class<? extends Join> joinClass) {
+ return withOperandSupplier(b -> b.operand(joinClass).anyInputs())
Review Comment:
I tried to implement outer join and included semi/anti join, and updated jira. please help me to review it again, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2025063838
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ List<RexNode> extraConds = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(extraConds.get(j)));
+ }
+ extraConds.add(orCond);
+ relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(join.getJoinType(), orCond);
+ }
+
+ relBuilder.union(true, orConds.size());
+ call.transformTo(relBuilder.build());
+ }
+
+ /** Rule configuration. */
+ @Value.Immutable
+ public interface Config extends RelRule.Config {
+ Config DEFAULT = ImmutableJoinConditionOrExpansionRule.Config.of()
+ .withOperandFor(Join.class);
+
+ @Override default JoinConditionOrExpansionRule toRule() {
+ return new JoinConditionOrExpansionRule(this);
+ }
+
+ /** Defines an operand tree for the given classes. */
+ default Config withOperandFor(Class<? extends Join> joinClass) {
+ return withOperandSupplier(b -> b.operand(joinClass).anyInputs())
Review Comment:
I tried to implement outer join and included semi/anti join, I will update jira later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #4280:
URL: https://github.com/apache/calcite/pull/4280#issuecomment-2771012252
When you address comments you should not force-push, but rather add new commits; this makes it much easier to review the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6930] Implementing JoinConditionOrExpansionRule [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4280:
URL: https://github.com/apache/calcite/pull/4280#discussion_r2023871952
##########
core/src/main/java/org/apache/calcite/rel/rules/JoinConditionOrExpansionRule.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+
+import org.immutables.value.Value;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Planner rule that matches a
+ * {@link org.apache.calcite.rel.core.Join}
+ * and expands OR clauses in join conditions.
+ *
+ * <p>For example, for the following SQL query:
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON (t1.id = t2.id OR t1.name = t2.name)
+ * </pre>
+ *
+ * <p>To
+ *
+ * <pre>
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.id = t2.id
+ * UNION ALL
+ * SELECT *
+ * FROM t1
+ * JOIN t2
+ * ON t1.name = t2.name
+ * </pre>
+ *
+ * <p>this rule would expand the OR condition into
+ * two separate join conditions, allowing the optimizer
+ * to handle these conditions more effectively.
+ */
+@Value.Enclosing
+public class JoinConditionOrExpansionRule
+ extends RelRule<JoinConditionOrExpansionRule.Config>
+ implements TransformationRule {
+
+ /** Creates an JoinConditionExpansionOrRule. */
+ protected JoinConditionOrExpansionRule(Config config) {
+ super(config);
+ }
+
+ //~ Methods ----------------------------------------------------------------
+
+ @Override public void onMatch(RelOptRuleCall call) {
+ Join join = call.rel(0);
+ RelBuilder relBuilder = call.builder();
+ List<RexNode> orConds = RelOptUtil.disjunctions(join.getCondition());
+
+ if (orConds.size() <= 1) {
+ return;
+ }
+
+ List<RexNode> extraConds = new ArrayList<>();
+ for (int i = 0; i < orConds.size(); i++) {
+ RexNode orCond = orConds.get(i);
+ for (int j = 0; j < i; j++) {
+ orCond = relBuilder.and(orCond, relBuilder.not(extraConds.get(j)));
+ }
+ extraConds.add(orCond);
+ relBuilder.push(join.getLeft())
+ .push(join.getRight())
+ .join(join.getJoinType(), orCond);
+ }
+
+ relBuilder.union(true, orConds.size());
+ call.transformTo(relBuilder.build());
+ }
+
+ /** Rule configuration. */
+ @Value.Immutable
+ public interface Config extends RelRule.Config {
+ Config DEFAULT = ImmutableJoinConditionOrExpansionRule.Config.of()
+ .withOperandFor(Join.class);
+
+ @Override default JoinConditionOrExpansionRule toRule() {
+ return new JoinConditionOrExpansionRule(this);
+ }
+
+ /** Defines an operand tree for the given classes. */
+ default Config withOperandFor(Class<? extends Join> joinClass) {
+ return withOperandSupplier(b -> b.operand(joinClass).anyInputs())
Review Comment:
I left two design questions in Jira. I should have probably asked them sooner, but I only got time to think about them now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org