You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2021/09/16 00:38:35 UTC

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

From Glenn Galvizo <gg...@uci.edu>:

Glenn Galvizo has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )


Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................

[WIP][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/BTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/RTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query6.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query7.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-1/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-4/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query4.plan
D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/bulk-loading/on-index-creation/open/composite-atomic/composite-atomic.4.query.sqlpp
R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.ddl.sqlpp
C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.2.update.sqlpp
R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.1.ddl.sqlpp
C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.2.update.sqlpp
R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.3.ddl.sqlpp
C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.4.query.sqlpp
D asterixdb/asterix-app/src/test/resources/runtimets/results/array-index/bulk-loading/on-index-creation/open/composite-atomic/composite-atomic.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/ArrayIndexUtil.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
52 files changed, 3,204 insertions(+), 323 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/13263/1


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 4: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12501/ : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 4
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Thu, 16 Sep 2021 19:07:54 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Glenn Galvizo has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )


Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................

[WIP][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/BTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/RTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query6.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query7.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-1/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-4/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query4.plan
D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/bulk-loading/on-index-creation/open/composite-atomic/composite-atomic.4.query.sqlpp
R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.ddl.sqlpp
C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.2.update.sqlpp
R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.1.ddl.sqlpp
C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.2.update.sqlpp
R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.3.ddl.sqlpp
C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.4.query.sqlpp
D asterixdb/asterix-app/src/test/resources/runtimets/results/array-index/bulk-loading/on-index-creation/open/composite-atomic/composite-atomic.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/ArrayIndexUtil.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
52 files changed, 3,204 insertions(+), 323 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/13263/1


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 1:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/XFCBxtqHUccTMiir8 : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-CC: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 16 Sep 2021 00:44:00 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 3: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12496/ : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 3
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Thu, 16 Sep 2021 09:46:40 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Glenn Galvizo has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................

[ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Adding RQGs for array indexes and the fixes detailed below:
- Handle conjuncts with nested ANDs (for SOME AND EVERY).
- Disable merge-select local rewrite when a join is encountered.
- Analyze for array indexes for each ASSIGN op (as opposed to the last
  ASSIGN).
- Disable INLJ with cross products in probe (ASTERIXDB-2966 bug).
- Enable general theta join INLJ acceleration with array indexes.
- Fix type builder bug for getting array type (at initial ingestion).
- Small fixes to get composite-atomic index initial ingestion working
  (now evaluators and record descriptors between operators are correct).

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Dmitry Lychagin <dm...@couchbase.com>
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
19 files changed, 2,840 insertions(+), 61 deletions(-)

Approvals:
  Dmitry Lychagin: Looks good to me, approved
  Jenkins: Verified; Verified

Objections:
  Anon. E. Moose #1000171: Violations found




-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 11
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-MessageType: merged

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Dmitry Lychagin <dm...@couchbase.com>:

Dmitry Lychagin has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 9:

(1 comment)

https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263/9/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java 
File asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java:

https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263/9/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java@85 
PS9, Line 85:     // Schema first -> creating the nesting structure
we need to remove these comments



-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 9
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Fri, 17 Sep 2021 21:49:37 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 4: Contrib-2

Analytics Compatibility Tests Failed
https://cbjenkins.page.link/xgFoixo89brrv4Mh7 : UNSTABLE


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 4
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 16 Sep 2021 20:02:53 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 1: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12491/ : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 1
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Thu, 16 Sep 2021 02:28:13 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Hello Jenkins, 

I'd like you to reexamine a change. Please visit

    https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263

to look at the new patch set (#2).

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................

[WIP][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/BTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/RTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query6.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query7.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-1/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-4/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query4.plan
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
49 files changed, 3,381 insertions(+), 275 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/13263/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 2
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-MessageType: newpatchset

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 2:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/oVWd9Mwwjp1VerS6A : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 2
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Thu, 16 Sep 2021 03:30:39 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [NO-ISSUE][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Hello Jenkins, Anon. E. Moose #1000171, 

I'd like you to reexamine a change. Please visit

    https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263

to look at the new patch set (#5).

Change subject: [NO-ISSUE][IDX] Adding RQG for array indexes + fixes
......................................................................

[NO-ISSUE][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Adding RQGs for array indexes and the fixes detailed below:
- Handle conjuncts with nested ANDs (for SOME AND EVERY).
- Disable merge-select local rewrite when a join is encountered.
- Analyze for array indexes for each ASSIGN op (as opposed to the last
  ASSIGN).
- Disable INLJ with cross products in probe (ASTERIXDB-2966 bug).
- Enable general theta join INLJ acceleration with array indexes.
- Fix type builder bug for getting array type (at initial ingestion).
- Insert filter for unknowns for atomic fields (at initial ingestion).
- Small fixes to get composite-atomic index initial ingestion working
  (now evaluators and record descriptors between operators are correct).

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/BTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/RTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query6.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query7.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-1/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-quantified-queries/use-case-4/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-1/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-2/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-3/query4.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query1.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query2.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query3.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/join-unnest-queries/use-case-4/query4.plan
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/join-unnest-queries/loj-subquery/loj-subquery.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array-index/select-unnest-queries/open/using-feed/using-feed.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/array-index/join-unnest-queries/loj-subquery/loj-subquery.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryUpsertWithNestedPlanOperatorNodePushable.java
50 files changed, 3,400 insertions(+), 280 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/13263/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 5
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-MessageType: newpatchset

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 7: Contrib-2

Analytics Compatibility Tests Failed
https://cbjenkins.page.link/vFdVyaJkcVr7pcsUA : UNSTABLE


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 7
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Fri, 17 Sep 2021 03:58:16 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Dmitry Lychagin <dm...@couchbase.com>:

Dmitry Lychagin has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 10: Code-Review+2


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 10
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Sat, 18 Sep 2021 16:04:10 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 10: Contrib-2

Analytics Compatibility Tests Failed
https://cbjenkins.page.link/moo3iExejuLC4BGG8 : UNSTABLE


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 10
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Sat, 18 Sep 2021 01:52:04 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [WIP][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [WIP][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 3:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/QEmmoSfihVFvLNvG8 : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 3
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Thu, 16 Sep 2021 06:15:54 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Hello Dmitry Lychagin, Jenkins, Anon. E. Moose #1000171, 

I'd like you to reexamine a change. Please visit

    https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263

to look at the new patch set (#10).

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................

[ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Adding RQGs for array indexes and the fixes detailed below:
- Handle conjuncts with nested ANDs (for SOME AND EVERY).
- Disable merge-select local rewrite when a join is encountered.
- Analyze for array indexes for each ASSIGN op (as opposed to the last
  ASSIGN).
- Disable INLJ with cross products in probe (ASTERIXDB-2966 bug).
- Enable general theta join INLJ acceleration with array indexes.
- Fix type builder bug for getting array type (at initial ingestion).
- Small fixes to get composite-atomic index initial ingestion working
  (now evaluators and record descriptors between operators are correct).

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
19 files changed, 2,840 insertions(+), 61 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/13263/10
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 10
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-MessageType: newpatchset

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Hello Dmitry Lychagin, Jenkins, Anon. E. Moose #1000171, 

I'd like you to reexamine a change. Please visit

    https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263

to look at the new patch set (#8).

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................

[ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Adding RQGs for array indexes and the fixes detailed below:
- Handle conjuncts with nested ANDs (for SOME AND EVERY).
- Disable merge-select local rewrite when a join is encountered.
- Analyze for array indexes for each ASSIGN op (as opposed to the last
  ASSIGN).
- Disable INLJ with cross products in probe (ASTERIXDB-2966 bug).
- Enable general theta join INLJ acceleration with array indexes.
- Fix type builder bug for getting array type (at initial ingestion).
- Insert filter for unknowns for atomic fields (at initial ingestion).
- Small fixes to get composite-atomic index initial ingestion working
  (now evaluators and record descriptors between operators are correct).

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
19 files changed, 2,864 insertions(+), 61 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/13263/8
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 8
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-MessageType: newpatchset

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Glenn Galvizo <gg...@uci.edu>:

Hello Dmitry Lychagin, Jenkins, Anon. E. Moose #1000171, 

I'd like you to reexamine a change. Please visit

    https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263

to look at the new patch set (#9).

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................

[ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

- user model changes: no
- storage format changes: no
- interface changes: no

Adding RQGs for array indexes and the fixes detailed below:
- Handle conjuncts with nested ANDs (for SOME AND EVERY).
- Disable merge-select local rewrite when a join is encountered.
- Analyze for array indexes for each ASSIGN op (as opposed to the last
  ASSIGN).
- Disable INLJ with cross products in probe (ASTERIXDB-2966 bug).
- Enable general theta join INLJ acceleration with array indexes.
- Fix type builder bug for getting array type (at initial ingestion).
- Small fixes to get composite-atomic index initial ingestion working
  (now evaluators and record descriptors between operators are correct).

Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/ArrayBTreeAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/AbstractOperatorFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/JoinFromSubplanRewrite.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/array/MergedSelectRewrite.java
A asterixdb/asterix-app/data/array-index.adm
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayDataset.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayElement.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayIndex.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQuery.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ArrayQueryUtil.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/BaseWisconsinTable.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/SqlppArrayIndexRQGTest.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/array/ValueSupplierFactory.java
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query8.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/array-index/atomic-and-array-queries/query9.plan
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryArrayIndexBTreeOperationsHelper.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/TypeUtil.java
19 files changed, 2,848 insertions(+), 61 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/13263/9
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 9
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-MessageType: newpatchset

Change in asterixdb[master]: [NO-ISSUE][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [NO-ISSUE][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 5:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/bTnpDbHNvePHYBnS7 : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 5
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 16 Sep 2021 20:55:57 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263 )

Change subject: [ASTERIXDB-2956][IDX] Adding RQG for array indexes + fixes
......................................................................


Patch Set 9: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12508/ : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13263
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I25a0f34010bca6296e75c6d6bc04eb515376941f
Gerrit-Change-Number: 13263
Gerrit-PatchSet: 9
Gerrit-Owner: Glenn Galvizo <gg...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Fri, 17 Sep 2021 20:04:29 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment