[jira] [Created] (DRILL-6871) Enabling runtime filter eliminates more incoming rows than it should.

["Kunal Khatua (JIRA)" <ji...@apache.org>]

Kunal Khatua created DRILL-6871:
-----------------------------------

             Summary: Enabling runtime filter eliminates more incoming rows than it should.
                 Key: DRILL-6871
                 URL: https://issues.apache.org/jira/browse/DRILL-6871
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.15.0
            Reporter: Kunal Khatua
            Assignee: weijie.tong
             Fix For: 1.16.0
         Attachments: 24036b35-7c41-12c9-8c08-9327c8d8fdf2.sys.drill, 24036bbf-ede2-690a-93b7-9b43c28eed52.sys.drill, 24036c15-54a9-7b5d-07be-4d6de4f63731.sys.drill, 24036c22-7f39-ae8b-f8f4-000ecc2572cc.sys.drill

When testing with the following combination on TPC-H dataset (scale factor 100) using a 4 node setup...
{code:java}
exec.hashjoin.bloom_filter.fpp=0.2
exec.hashjoin.enable.runtime_filter=true
exec.hashjoin.runtime_filter.max.waiting.time=20000
{code}
It was observed that the filter eliminates more rows than it should.

 
{code:java}
0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem l, supplier s where l.l_suppkey = s.s_suppkey and s.s_acctbal <1000);
+---------+
| EXPR$0  |
+---------+
| 405566  |
+---------+
1 row selected (10.565 seconds)
0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem l, supplier s where l.l_suppkey = s.s_suppkey and s.s_acctbal <1000);
+---------+
| EXPR$0  |
+---------+
| 405769  |
+---------+
1 row selected (9.845 seconds)
{code}
The expected row count for the above (broadcast-join) query should have been *109307880*

 
{code:java}
0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem l, orders o where o.o_orderkey = l.l_orderkey and o.o_totalprice < 100000);
+-----------+
|  EXPR$0   |
+-----------+
| 37338355  |
+-----------+
1 row selected (44.698 seconds)
0: jdbc:drill:schema=dfs.par100> select count(*) from (select * from lineitem l, orders o where o.o_orderkey = l.l_orderkey and o.o_totalprice < 100000);
+-----------+
|  EXPR$0   |
+-----------+
| 38044874  |
+-----------+
1 row selected (44.871 seconds)
{code}
The expected row count for the above (hash partition-join)  query should have been *96176495*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)