[jira] [Comment Edited] (DRILL-2019) Filter pushdown into the subquery when the subquery also has a filter is resulting in incorrect results

["Aman Sinha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279900#comment-14279900 ] 

Aman Sinha edited comment on DRILL-2019 at 1/16/15 7:54 AM:
------------------------------------------------------------

The FilterPrel's  getSupportedEncoding() returns NONE_AND_TWO  - meaning it accepts inputs that have either no selection vectors or SV2.   In this case the input is another Filter which produces an SV2, so no SVRemover gets added between the filters.  One way to force an SVRemover for each of the filters (doing this produces right result) is  for getSupportedEncoding() to return NONE but this would be sub-optimal. 

I also tried enabling Calcite's MergeFilterRule in DrillRuleSets (it was never added there) but it did not help because Calcite's logical plan consists of the two FilterRels with an intervening ProjectRel, which prevents the MergeFilterRule from getting applied.  


was (Author: amansinha100):
The FilterPrel's  getSupportedEncoding() was returning NONE_AND_TWO  - meaning it would accept inputs that have either no selection vectors or SV2.   This is wrong because a Filter would add its own SV2, so it should accept only inputs with NONE.   Making this change introduces the SVRemover for each of the filters and produces right result. 

I also tried enabling Calcite's MergeFilterRule in DrillRuleSets (it was never added there) but it did not help because Calcite's logical plan consists of the two FilterRels with an intervening ProjectRel, which prevents the MergeFilterRule from getting applied.  

> Filter pushdown into the subquery when the subquery also has a filter is resulting in incorrect results
> -------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-2019
>                 URL: https://issues.apache.org/jira/browse/DRILL-2019
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Operators, Query Planning & Optimization
>            Reporter: Rahul Challapalli
>            Assignee: Aman Sinha
>            Priority: Critical
>
> git.commit.id.abbrev=b491cdb
> The below query on top of tpch 0.01 data should actually return 0 records. (Verified with postgres). However drill returns incorrect result
> {code}
> select count(*) from cp.`tpch/lineitem.parquet` l inner join (select o.o_orderkey, o.o_custkey from cp.`tpch/orders.parquet` o where o.o_custkey < 5) s on l.l_orderkey = s.o_orderkey and s.o_custkey > 5;
> +------------+
> |   EXPR$0   |
> +------------+
> | 189        |
> +------------+
> {code}
> Marked as 'critical' since drill is reporting incorrect results



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)