You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2021/09/20 19:30:18 UTC

Change in asterixdb[master]: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in ...

From Glenn Galvizo <gg...@uci.edu>:

Glenn Galvizo has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303 )


Change subject: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe
......................................................................

[ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe

- user model changes: no
- storage format changes: no
- interface changes: no

If there are non-equijoins in the probe (which lead to NL joins),
disable INLJ acceleration from array indexes. This is a broader scope
than the cross-joins in the probe from the previous fix.

Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
1 file changed, 22 insertions(+), 3 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/13303/1

diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
index b733b52..8d79c7d 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
@@ -30,6 +30,7 @@
 import org.apache.asterix.metadata.entities.Dataset;
 import org.apache.asterix.metadata.entities.Index;
 import org.apache.asterix.metadata.utils.ArrayIndexUtil;
+import org.apache.asterix.om.functions.BuiltinFunctions;
 import org.apache.asterix.om.types.IAType;
 import org.apache.asterix.om.utils.NonTaggedFormatUtil;
 import org.apache.commons.lang3.mutable.Mutable;
@@ -38,9 +39,10 @@
 import org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression;
 import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
 import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalExpressionTag;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
-import org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression;
+import org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractBinaryJoinOperator;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.LeftOuterUnnestOperator;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.UnnestOperator;
@@ -101,7 +103,7 @@
             return false;
         }
 
-        // TODO (GLENN): There is a bug with cross-products originating from the probe. Disable this case for now.
+        // TODO (GLENN): There is a bug with nested-loop joins originating from the probe. Disable this case for now.
         Deque<ILogicalOperator> opStack = new ArrayDeque<>();
         List<ILogicalOperator> visited = new ArrayList<>();
         opStack.add(probeSubTree.getRoot());
@@ -111,8 +113,25 @@
                 if (workingOp.getOperatorTag() == LogicalOperatorTag.INNERJOIN
                         || workingOp.getOperatorTag() == LogicalOperatorTag.LEFTOUTERJOIN) {
                     AbstractBinaryJoinOperator joinOperator = (AbstractBinaryJoinOperator) workingOp;
-                    if (joinOperator.getCondition().getValue().equals(ConstantExpression.TRUE)) {
+                    ILogicalExpression joinCondition = joinOperator.getCondition().getValue();
+                    List<Mutable<ILogicalExpression>> conjuncts = new ArrayList<>();
+                    if (joinCondition.splitIntoConjuncts(conjuncts)) {
+                        for (Mutable<ILogicalExpression> conjunct : conjuncts) {
+                            if (conjunct.getValue().getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
+                                return false;
+                            }
+                            AbstractFunctionCallExpression expr = (AbstractFunctionCallExpression) joinCondition;
+                            if (expr.getFunctionIdentifier() != BuiltinFunctions.EQ) {
+                                return false;
+                            }
+                        }
+                    } else if (joinCondition.getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
                         return false;
+                    } else {
+                        AbstractFunctionCallExpression expr = (AbstractFunctionCallExpression) joinCondition;
+                        if (expr.getFunctionIdentifier() != BuiltinFunctions.EQ) {
+                            return false;
+                        }
                     }
                 }
                 visited.add(workingOp);

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Gerrit-Change-Number: 13303
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in ...

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Glenn Galvizo has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303 )

Change subject: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe
......................................................................

[ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe

- user model changes: no
- storage format changes: no
- interface changes: no

If there are non-equijoins in the probe (which lead to NL joins),
disable INLJ acceleration from array indexes. This is a broader scope
than the cross-joins in the probe from the previous fix.

Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Dmitry Lychagin <dm...@couchbase.com>
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
1 file changed, 22 insertions(+), 3 deletions(-)

Approvals:
  Dmitry Lychagin: Looks good to me, approved
  Jenkins: Verified; Verified

Objections:
  Anon. E. Moose #1000171: Violations found



diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
index b733b52..8d79c7d 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
@@ -30,6 +30,7 @@
 import org.apache.asterix.metadata.entities.Dataset;
 import org.apache.asterix.metadata.entities.Index;
 import org.apache.asterix.metadata.utils.ArrayIndexUtil;
+import org.apache.asterix.om.functions.BuiltinFunctions;
 import org.apache.asterix.om.types.IAType;
 import org.apache.asterix.om.utils.NonTaggedFormatUtil;
 import org.apache.commons.lang3.mutable.Mutable;
@@ -38,9 +39,10 @@
 import org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression;
 import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
 import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalExpressionTag;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
-import org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression;
+import org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractBinaryJoinOperator;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.LeftOuterUnnestOperator;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.UnnestOperator;
@@ -101,7 +103,7 @@
             return false;
         }
 
-        // TODO (GLENN): There is a bug with cross-products originating from the probe. Disable this case for now.
+        // TODO (GLENN): There is a bug with nested-loop joins originating from the probe. Disable this case for now.
         Deque<ILogicalOperator> opStack = new ArrayDeque<>();
         List<ILogicalOperator> visited = new ArrayList<>();
         opStack.add(probeSubTree.getRoot());
@@ -111,8 +113,25 @@
                 if (workingOp.getOperatorTag() == LogicalOperatorTag.INNERJOIN
                         || workingOp.getOperatorTag() == LogicalOperatorTag.LEFTOUTERJOIN) {
                     AbstractBinaryJoinOperator joinOperator = (AbstractBinaryJoinOperator) workingOp;
-                    if (joinOperator.getCondition().getValue().equals(ConstantExpression.TRUE)) {
+                    ILogicalExpression joinCondition = joinOperator.getCondition().getValue();
+                    List<Mutable<ILogicalExpression>> conjuncts = new ArrayList<>();
+                    if (joinCondition.splitIntoConjuncts(conjuncts)) {
+                        for (Mutable<ILogicalExpression> conjunct : conjuncts) {
+                            if (conjunct.getValue().getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
+                                return false;
+                            }
+                            AbstractFunctionCallExpression expr = (AbstractFunctionCallExpression) joinCondition;
+                            if (expr.getFunctionIdentifier() != BuiltinFunctions.EQ) {
+                                return false;
+                            }
+                        }
+                    } else if (joinCondition.getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
                         return false;
+                    } else {
+                        AbstractFunctionCallExpression expr = (AbstractFunctionCallExpression) joinCondition;
+                        if (expr.getFunctionIdentifier() != BuiltinFunctions.EQ) {
+                            return false;
+                        }
                     }
                 }
                 visited.add(workingOp);

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Gerrit-Change-Number: 13303
Gerrit-PatchSet: 3
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-MessageType: merged

Change in asterixdb[master]: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in ...

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Glenn Galvizo has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303 )


Change subject: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe
......................................................................

[ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe

- user model changes: no
- storage format changes: no
- interface changes: no

If there are non-equijoins in the probe (which lead to NL joins),
disable INLJ acceleration from array indexes. This is a broader scope
than the cross-joins in the probe from the previous fix.

Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
1 file changed, 22 insertions(+), 3 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/13303/1

diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
index b733b52..8d79c7d 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
@@ -30,6 +30,7 @@
 import org.apache.asterix.metadata.entities.Dataset;
 import org.apache.asterix.metadata.entities.Index;
 import org.apache.asterix.metadata.utils.ArrayIndexUtil;
+import org.apache.asterix.om.functions.BuiltinFunctions;
 import org.apache.asterix.om.types.IAType;
 import org.apache.asterix.om.utils.NonTaggedFormatUtil;
 import org.apache.commons.lang3.mutable.Mutable;
@@ -38,9 +39,10 @@
 import org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression;
 import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
 import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalExpressionTag;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
-import org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression;
+import org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractBinaryJoinOperator;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.LeftOuterUnnestOperator;
 import org.apache.hyracks.algebricks.core.algebra.operators.logical.UnnestOperator;
@@ -101,7 +103,7 @@
             return false;
         }
 
-        // TODO (GLENN): There is a bug with cross-products originating from the probe. Disable this case for now.
+        // TODO (GLENN): There is a bug with nested-loop joins originating from the probe. Disable this case for now.
         Deque<ILogicalOperator> opStack = new ArrayDeque<>();
         List<ILogicalOperator> visited = new ArrayList<>();
         opStack.add(probeSubTree.getRoot());
@@ -111,8 +113,25 @@
                 if (workingOp.getOperatorTag() == LogicalOperatorTag.INNERJOIN
                         || workingOp.getOperatorTag() == LogicalOperatorTag.LEFTOUTERJOIN) {
                     AbstractBinaryJoinOperator joinOperator = (AbstractBinaryJoinOperator) workingOp;
-                    if (joinOperator.getCondition().getValue().equals(ConstantExpression.TRUE)) {
+                    ILogicalExpression joinCondition = joinOperator.getCondition().getValue();
+                    List<Mutable<ILogicalExpression>> conjuncts = new ArrayList<>();
+                    if (joinCondition.splitIntoConjuncts(conjuncts)) {
+                        for (Mutable<ILogicalExpression> conjunct : conjuncts) {
+                            if (conjunct.getValue().getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
+                                return false;
+                            }
+                            AbstractFunctionCallExpression expr = (AbstractFunctionCallExpression) joinCondition;
+                            if (expr.getFunctionIdentifier() != BuiltinFunctions.EQ) {
+                                return false;
+                            }
+                        }
+                    } else if (joinCondition.getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
                         return false;
+                    } else {
+                        AbstractFunctionCallExpression expr = (AbstractFunctionCallExpression) joinCondition;
+                        if (expr.getFunctionIdentifier() != BuiltinFunctions.EQ) {
+                            return false;
+                        }
                     }
                 }
                 visited.add(workingOp);

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Gerrit-Change-Number: 13303
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in ...

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303 )

Change subject: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe
......................................................................


Patch Set 2: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12533/ : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Gerrit-Change-Number: 13303
Gerrit-PatchSet: 2
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 23 Sep 2021 00:06:17 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in ...

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303 )

Change subject: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe
......................................................................


Patch Set 2:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/t3zEJPywcFWDyx8fA : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Gerrit-Change-Number: 13303
Gerrit-PatchSet: 2
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Wed, 22 Sep 2021 19:58:01 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in ...

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303 )

Change subject: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe
......................................................................


Patch Set 1:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/m2HWnbBmvgyqSR6Q6 : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Gerrit-Change-Number: 13303
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-CC: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Mon, 20 Sep 2021 19:37:33 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in ...

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303 )

Change subject: [ASTERIXDB-2966][IDX] Disabling array index INLJ for non-equjoins in probe
......................................................................


Patch Set 2: Contrib-2

Analytics Compatibility Tests Failed
https://cbjenkins.page.link/3Hjn7k8UksBgyvKA8 : UNSTABLE


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I0d36784272acc43f4eb4d876d8f6ebe07b8014ff
Gerrit-Change-Number: 13303
Gerrit-PatchSet: 2
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 23 Sep 2021 00:49:32 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment