Change in asterixdb[master]: [ASTERIXDB-2310][STO]Enforce Key Uniquness using PKIndex

["Luo Chen (Code Review)" <do...@asterixdb.incubator.apache.org>]

Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/2453

to look at the new patch set (#14).

Change subject: [ASTERIXDB-2310][STO]Enforce Key Uniquness using PKIndex
......................................................................

[ASTERIXDB-2310][STO]Enforce Key Uniquness using PKIndex

- user model changes: no
- storage format changes: yes. Primary key index
now has bloom filters.
- interface changes: no

Details:
- Enforce primary key uniquness check during INSERT
using the primary key index. This can greatly reduces
the memory footprint (only the pk index needs to be cached),
and thus improves the ingestion performance using INSERT.
- Implementation-wise, we first insert into the primary key index (and
perform X lock and duplicate check), and then insert primary index
and all secodnary indexes (without locking/duplicate check).
- Add bloom filters to primary key index.
- Now a dataset has 3 kinds of index, e.g., primary index,
primary key index, and secodnary index (pk index is no longer treated
as a secondary index).

Change-Id: I7a52bb75ee5b14521972999df2f45ba62adc5af1
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceMaterializationForInsertWithSelfScanRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceSecondaryIndexInsertDeleteRule.java
M asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/primary-key-index/delete-primary-key-index-with-secondary.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/primary-key-index/insert-and-scan-primary-key-index-with-secondary.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/primary-key-index/insert-primary-key-index-with-auto-gen-pk.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/primary-key-index/insert-primary-key-index.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/primary-key-index/load-primary-key-index-with-secondary.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/primary-key-index/load-primary-key-index.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/primary-key-index/upsert-primary-key-index-with-secondary.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/results/primary-key-index/delete-primary-key-index-with-secondary.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/primary-key-index/insert-and-scan-primary-key-index-with-secondary.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/primary-key-index/insert-primary-key-index-with-auto-gen-pk.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/primary-key-index/insert-primary-key-index.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/primary-key-index/load-primary-key-index-with-secondary.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/primary-key-index/load-primary-key-index.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/primary-key-index/upsert-primary-key-index-with-secondary.plan
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/LSMInsertDeleteOperatorNodePushable.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/BTreeResourceFactoryProvider.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryIndexOperationsHelper.java
M asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/EntityLevelTransactionContext.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitorJson.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTree.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/utils/LSMBTreeUtil.java
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/LSMHarness.java
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeScanDiskComponentsTest.java
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtree.java
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeUtil.java
32 files changed, 681 insertions(+), 81 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/53/2453/14
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2453
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7a52bb75ee5b14521972999df2f45ba62adc5af1
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen <cl...@uci.edu>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Ian Maxon <im...@uci.edu>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-Reviewer: Steven Jacobs <sj...@ucr.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>