[jira] [Comment Edited] (HIVE-12462) DPP: DPP optimizers need to run on the TS predicate not FIL

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023228#comment-15023228 ] 

Sergey Shelukhin edited comment on HIVE-12462 at 11/23/15 10:34 PM:
--------------------------------------------------------------------

Test failures are because of the missing cleanup (currently, the code cleans up TS expr separately and filter expr as part of processing, if TS expr is used in processing noone cleans up the filter). Patch soon


was (Author: sershe):
Test failures are because of the missing cleanup (currently, the code cleans up TS expr separately and filter expr as part of processing, if TS expr is used in processing noone cleans up the filter). 

> DPP: DPP optimizers need to run on the TS predicate not FIL 
> ------------------------------------------------------------
>
>                 Key: HIVE-12462
>                 URL: https://issues.apache.org/jira/browse/HIVE-12462
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: 2.0.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>            Priority: Critical
>         Attachments: HIVE-12462.1.patch
>
>
> HIVE-11398 + HIVE-11791, the partition-condition-remover became more effective.
> This removes predicates from the FilterExpression which involve partition columns, causing a miss for dynamic-partition pruning if the DPP relies on FilterDesc.
> The TS desc will have the correct predicate in that condition.
> {code}
> $hdt$_0:$hdt$_1:a
>   TableScan (TS_2)
>     alias: a
>     filterExpr: (((account_id = 22) and year(dt) is not null) and (year(dt)) IN (RS[6])) (type: boolean)
>     Filter Operator (FIL_20)
>       predicate: ((account_id = 22) and year(dt) is not null) (type: boolean)
>       Select Operator (SEL_4)
>         expressions: dt (type: date)
>         outputColumnNames: _col1
>         Reduce Output Operator (RS_8)
>           key expressions: year(_col1) (type: int)
>           sort order: +
>           Map-reduce partition columns: year(_col1) (type: int)
>           Join Operator (JOIN_9)
>             condition map:
>                  Inner Join 0 to 1
>             keys:
>               0 year(_col1) (type: int)
>               1 year(_col1) (type: int)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)