[jira] [Commented] (ASTERIXDB-1451) Upsert: Open Index test fails with duplicate rows in result when VBC page size is reduced

["Murtadha Hubail (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ASTERIXDB-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419091#comment-15419091 ] 

Murtadha Hubail commented on ASTERIXDB-1451:
--------------------------------------------



I have been investigating this issue for some time. It is not related to the VBC page size nor upsert. When any record, with an enforced index of a type other than INT64, is written to a disk component, a delete operation with an antimatter tuple for that same record will not hide the record written in the disk component of the enforced index. Therefore, duplicate records will be returned in any query using the enforced index.

I haven't found the root cause yet, but I believe it is related to the variable type propagation during query compilation. I will discuss it with [~buyingyi] and [~alamoudi].

The issue can be easily reproduced by the following:

{code}
drop dataverse test if exists;
create dataverse test;
use dataverse test;

create type OrderOpenType as open {
  o_orderkey: int64
}

create dataset OrdersOpen(OrderOpenType)
primary key o_orderkey;

insert into dataset OrdersOpen (
  {"o_orderkey": 1,
  "o_custkey": 1,}
)

create index idx_Orders_Custkey on OrdersOpen(o_custkey:int32) enforced;

for $o in dataset('OrdersOpen')
where
 $o.o_custkey >=-1
return {
  "o_orderkey": $o.o_orderkey
}
{code}

> Upsert: Open Index test fails with duplicate rows in result when VBC page size is reduced
> -----------------------------------------------------------------------------------------
>
>                 Key: ASTERIXDB-1451
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1451
>             Project: Apache AsterixDB
>          Issue Type: Bug
>            Reporter: Michael Blow
>            Assignee: Murtadha Hubail
>
> To repro:
> - configure storage.memorycomponent.pagesize to 8k, and increase storage.memorycomponent.numpages to 24
> - run asterix-app runtime tests
> - observe failure in open-index test with duplicated rows as shown below.
> $ diff -du src/test/resources/runtimets/results/upsert/open-index/open-index.1.adm rttest/results/upsert/open-index.adm
> --- src/test/resources/runtimets/results/upsert/open-index/open-index.1.adm	2016-04-27 20:40:58.000000000 -0700
> +++ rttest/results/upsert/open-index.adm	2016-05-16 19:08:57.000000000 -0700
> @@ -1099,11 +1099,15 @@
>  { "o_orderkey": 5927, "o_custkey": 116 }
>  { "o_orderkey": 5952, "o_custkey": 148 }
>  { "o_orderkey": 5955, "o_custkey": 94 }
> +{ "o_orderkey": 5955, "o_custkey": 94 }
> +{ "o_orderkey": 5957, "o_custkey": 89 }
>  { "o_orderkey": 5957, "o_custkey": 89 }
>  { "o_orderkey": 5958, "o_custkey": 115 }
>  { "o_orderkey": 5984, "o_custkey": 70 }
>  { "o_orderkey": 5985, "o_custkey": 143 }
>  { "o_orderkey": 5986, "o_custkey": 115 }
> +{ "o_orderkey": 5986, "o_custkey": 115 }
> +{ "o_orderkey": 5987, "o_custkey": 64 }
>  { "o_orderkey": 5987, "o_custkey": 64 }
>  { "o_orderkey": 10986, "o_custkey": 115 }
>  { "o_orderkey": 10987, "o_custkey": 64 }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)