[jira] Updated: (HIVE-1750) Remove Partition Filtering Conditions when Possible

["Siying Dong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siying Dong updated HIVE-1750:
------------------------------

    Attachment: HIVE-1750.1.patch

After generating partition predicates, we add another step to prune partitions and according to the list of pruned partitions, remove partition predicates when possible. A new optimizer (PartitionConditionRemover) is created for it, which walks the operator tree and the expression tree for target filter operator. For every expression including partition column, we evaluate the value for every partition pruned. If the result agrees for all partitions, we replace expression to be the constant result. We further remove the nodes when possible. Finally, if the whole expression tree is always to return true, we remove the filter operator.

We only handle the filter pushed down to near the table scan operators. We'll reply on HIVE-1538 to remove the original one.

Fix the unit test results accordingly and add a new unit test.

> Remove Partition Filtering Conditions when Possible
> ---------------------------------------------------
>
>                 Key: HIVE-1750
>                 URL: https://issues.apache.org/jira/browse/HIVE-1750
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Siying Dong
>         Attachments: HIVE-1750.1.patch
>
>
> For some simple queries, partition filtering constraints take 8% of CPU time (now 16% since we filter twice) even if the result is always true. When possible, we should remove these constraints to save CPU times.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.