[GitHub] [hive] kasakrisz commented on a change in pull request #1878: [DRAFT] Remove HiveSubQRemoveRelBuilder

[GitBox <gi...@apache.org>]

kasakrisz commented on a change in pull request #1878:
URL: https://github.com/apache/hive/pull/1878#discussion_r576803125



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRemoveGBYSemiJoinRule.java
##########
@@ -71,9 +75,21 @@ public HiveRemoveGBYSemiJoinRule() {
     if(!rightAggregate.getAggCallList().isEmpty()) {
       return;
     }
-    final JoinInfo joinInfo = join.analyzeCondition();
 
-    boolean shouldTransform = joinInfo.rightSet().equals(
+    JoinPredicateInfo joinPredInfo;
+    try {
+      joinPredInfo = HiveCalciteUtil.JoinPredicateInfo.constructJoinPredicateInfo(join);
+    } catch (CalciteSemanticException e) {
+      return;

Review comment:
       Should the exception logged as a warning?

##########
File path: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterJoinRule.java
##########
@@ -129,22 +143,331 @@ public void onMatch(RelOptRuleCall call) {
     }
   }
 
-  private boolean filterRefersToBothSidesOfJoin(RexNode filter, Join j) {
-    boolean refersToBothSides = false;
 
-    int joinNoOfProjects = j.getRowType().getFieldCount();
-    ImmutableBitSet filterProjs = ImmutableBitSet.FROM_BIT_SET.apply(new BitSet(joinNoOfProjects));
-    ImmutableBitSet allLeftProjs = filterProjs.union(ImmutableBitSet.range(0, j.getInput(0)
-        .getRowType().getFieldCount()));
-    ImmutableBitSet allRightProjs = filterProjs.union(ImmutableBitSet.range(j.getInput(0)
-        .getRowType().getFieldCount(), joinNoOfProjects));
+  /**
+   * Perform is duplicated from parent class to be able to call the modified
+   * classify filters. The modified classify method can push filter conditions
+   * that refer only to the SJ right input to the corresponding input (the
+   * fix is in the number of fields for the join, which is inferred from the
+   * join in the original method rather than the concatenation of the join
+   * inputs).
+   * TODO: Remove this method once {@link RelOptUtil#classifyFilters} is fixed.
+   */
+  protected void perform(RelOptRuleCall call, Filter filter, Join join) {
+    final List<RexNode> joinFilters =
+        RelOptUtil.conjunctions(join.getCondition());
+    final List<RexNode> origJoinFilters = ImmutableList.copyOf(joinFilters);
+
+    // If there is only the joinRel,
+    // make sure it does not match a cartesian product joinRel
+    // (with "true" condition), otherwise this rule will be applied
+    // again on the new cartesian product joinRel.
+    if (filter == null && joinFilters.isEmpty()) {
+      return;
+    }
+
+    final List<RexNode> aboveFilters =
+        filter != null
+            ? getConjunctions(filter)
+            : new ArrayList<>();
+    final ImmutableList<RexNode> origAboveFilters =
+        ImmutableList.copyOf(aboveFilters);
+
+    // Simplify Outer Joins
+    JoinRelType joinType = join.getJoinType();
+    if (!origAboveFilters.isEmpty()
+        && join.getJoinType() != JoinRelType.INNER) {
+      joinType = RelOptUtil.simplifyJoin(join, origAboveFilters, joinType);
+    }
+
+    final List<RexNode> leftFilters = new ArrayList<>();
+    final List<RexNode> rightFilters = new ArrayList<>();
+
+    // TODO - add logic to derive additional filters.  E.g., from
+    // (t1.a = 1 AND t2.a = 2) OR (t1.b = 3 AND t2.b = 4), you can
+    // derive table filters:
+    // (t1.a = 1 OR t1.b = 3)
+    // (t2.a = 2 OR t2.b = 4)
+
+    // Try to push down above filters. These are typically where clause
+    // filters. They can be pushed down if they are not on the NULL
+    // generating side.
+    boolean filterPushed = false;
+    if (classifyFilters(

Review comment:
       This `if` statement can be removed and `filterPushed` can be initialized by calling `classifyFilters`:
   ```
   boolean filterPushed = classifyFilters(...)
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org