You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2022/01/13 13:12:05 UTC

[GitHub] [ignite] alex-plekhanov commented on a change in pull request #9731: IGNITE-16139 Calcite. Request with ORDER BY plans infinitely.

alex-plekhanov commented on a change in pull request #9731:
URL: https://github.com/apache/ignite/pull/9731#discussion_r783888600



##########
File path: modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/trait/TraitsAwareIgniteRel.java
##########
@@ -59,8 +59,9 @@
      * @return Relational node for given traits combination.
      */
     default RelNode createNode(RelTraitSet nodeTraits, List<RelTraitSet> inTraits) {
-        return copy(nodeTraits, Commons.transform(Ord.zip(inTraits),
-            o -> RelOptRule.convert(getInput(o.i), o.e)));
+        List<RelNode> tr = Commons.transform(Ord.zip(inTraits),

Review comment:
       What the difference between current and old implementation?

##########
File path: modules/calcite/src/test/sql/sqlite/orderby/orderby1_10_2.test_ignore
##########
@@ -1,5 +1,3 @@
-# Ignore https://issues.apache.org/jira/browse/IGNITE-16139

Review comment:
       There is no ignored ticket in this file anymore. If all tests are passed, they should be moved to ".test" file and this file should be deleted.

##########
File path: modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/rule/logical/LogicalOrToUnionRule.java
##########
@@ -166,15 +231,21 @@ private RelNode createUnionAll(RelOptCluster cluster, RelNode input, RexNode op1
         /** */
         Config FILTER = DEFAULT
             .withDescription("FilterLogicalOrToUnionRule")
-            .withOperandSupplier(o -> o.operand(LogicalFilter.class).anyInputs())
+            .withOperandSupplier(o -> o.operand(LogicalFilter.class)
+            .inputs(

Review comment:
       Looks like with the current predicate FILTER_INSTANCE is redundant at all, since Filter/Scan can always be merged into the scan with filter, and for a scan with the filter we have SCAN_INSTANCE.  

##########
File path: modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/rule/CorrelatedNestedLoopJoinRule.java
##########
@@ -142,15 +141,9 @@ public CorrelatedNestedLoopJoinRule(int batchSize) {
     }
 
     /** {@inheritDoc} */
-    @Override public void onMatch(RelOptRuleCall call) {
-        LogicalJoin join = call.rel(0);
+    @Override public boolean matches(RelOptRuleCall call) {
+        LogicalJoin corr = call.rel(0);

Review comment:
       Let's keep variable name 'join'

##########
File path: modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/planner/PlannerTest.java
##########
@@ -809,6 +810,118 @@ public void testSplitterPartiallyColocatedReplicatedAndPartitioned() throws Exce
         assertEquals(3, plan.fragments().size());
     }
 
+    /** Multiple join test. */
+    @Test
+    @Ignore("https://issues.apache.org/jira/browse/IGNITE-16046")
+    public void testMultipleJoin() throws Exception {

Review comment:
       It's just a reproducer for IGNITE-16046 with a lot of duplicated code, the final test should be rewritten and should be added by IGNITE-16046. Let's remove it. 

##########
File path: modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/rule/logical/LogicalOrToUnionRule.java
##########
@@ -155,6 +167,59 @@ private RelNode createUnionAll(RelOptCluster cluster, RelNode input, RexNode op1
             .build();
     }
 
+    /** */
+    private static boolean preMatch(IgniteLogicalTableScan scan) {
+        RexNode cond = scan.condition();
+
+        if (cond == null ||
+            // _key_PK not interesting here, but it`s depend on current PK implementation, in future PK can be removed
+            // and this condition will become incorrect.
+            scan.getTable().unwrap(IgniteTable.class).indexes().size() < 2)
+            return false;
+
+        ImmutableBitSet.Builder builder = ImmutableBitSet.builder();
+
+        new RexShuttle() {
+            @Override public RexNode visitLocalRef(RexLocalRef inputRef) {
+                builder.set(inputRef.getIndex());
+                return inputRef;
+            }
+        }.apply(cond);
+
+        ImmutableBitSet requiredColumns = builder.build();
+
+        assert requiredColumns != null;
+
+        IgniteTable tbl = scan.getTable().unwrap(IgniteTable.class);
+        IgniteTypeFactory typeFactory = Commons.typeFactory(scan.getCluster());
+        int fieldCount = tbl.getRowType(typeFactory).getFieldCount();
+
+        boolean idxPreMatch = false;
+
+        for (IgniteIndex idx : tbl.indexes().values()) {
+            RelCollation col0 = idx.collation();
+
+            ImmutableBitSet scanReqCols = F.isEmpty(scan.requiredColumns()) ? requiredColumns : scan.requiredColumns();

Review comment:
       It's not correct, since requiredColumns built on top of scan.requiredColumns

##########
File path: modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/rel/IgniteCorrelatedNestedLoopJoin.java
##########
@@ -212,10 +214,14 @@ public IgniteCorrelatedNestedLoopJoin(RelInput input) {
         if (!rightCorrIds.containsAll(variablesSet))
             return ImmutableList.of();
 
-        Set<CorrelationId> corrIds = new HashSet<>(rightCorrIds);
+        Set<CorrelationId> leftCorrIds = TraitUtils.correlation(inTraits.get(0)).correlationIds();
+
+        Set<CorrelationId> corrIds = new HashSet<>(rightCorrIds.size() + leftCorrIds.size());

Review comment:
       `new HashSet` -> `U.newHashSet`. But in most cases, I think, rightCorrIds is a subset of leftCorrIds.

##########
File path: modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/trait/TraitUtils.java
##########
@@ -430,18 +429,24 @@ public static IgniteDistribution projectDistribution(IgniteDistribution distribu
 
     /** */
     private static Set<Pair<RelTraitSet, List<RelTraitSet>>> combinations(RelTraitSet outTraits, List<List<RelTraitSet>> inTraits) {
-        Set<Pair<RelTraitSet, List<RelTraitSet>>> out = new HashSet<>();
+        ImmutableSet.Builder<Pair<RelTraitSet, List<RelTraitSet>>> out = new ImmutableSet.Builder<>();
         fillRecursive(outTraits, inTraits, out, new RelTraitSet[inTraits.size()], 0);
-        return out;
+        return out.build();
     }
 
     /** */
-    private static boolean fillRecursive(RelTraitSet outTraits, List<List<RelTraitSet>> inTraits,
-        Set<Pair<RelTraitSet, List<RelTraitSet>>> result, RelTraitSet[] combination, int idx) throws ControlFlowException {
+    private static boolean fillRecursive(
+        RelTraitSet outTraits,
+        List<List<RelTraitSet>> inTraits,
+        ImmutableSet.Builder<Pair<RelTraitSet,
+        List<RelTraitSet>>> result,

Review comment:
       Should be on the previous line

##########
File path: modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/rule/logical/LogicalOrToUnionRule.java
##########
@@ -155,6 +167,59 @@ private RelNode createUnionAll(RelOptCluster cluster, RelNode input, RexNode op1
             .build();
     }
 
+    /** */
+    private static boolean preMatch(IgniteLogicalTableScan scan) {
+        RexNode cond = scan.condition();
+
+        if (cond == null ||

Review comment:
       I have some proposals:
   1. Let's keep only checks from this condition in `preMatch`.
   2. Move index collation check to the `onMatch` method.
   3. Check index collation for both operands and proceed only if both operands match any index.
   4. Since we can effectively use index only if filtering first field of index key - make a bitset of first fields of all indexes and proceed only if the intersection of fields in condition and first fields of indexes is not empty.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org