[jira] Commented: (HIVE-578) Refator partition pruning code as an optimizer transformation in order to fix bugs that appear in the current code

["Namit Jain (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723804#action_12723804 ] 

Namit Jain commented on HIVE-578:
---------------------------------

Some high-level comments:

1. Can you share some expression walker code from Predicate Pushdown ? seems like a lot of common stuff.
2. Can you always assume that the predicate to be filtered will always be after the tablescan ? 

     Currently, all optimizations are independent of each other, and can be turned off - this way, 
     you are dependent on a new feature -- may have stability issues.

     I don't remember exactly how does predicate pushdown work - if a filter is already present immediately after the table
     scan, will the pushed up filter be merged. Is it pushed across user defined mappers/reducers ? Cant these things change
     in the future ?

     It might be a good idea to look for TS*FIL, and then keep some mapping - or probably use the same mapping that Predicate
     pushdown has to figure out whether the filter is for the table under consideration.

> Refator partition pruning code as an optimizer transformation in order to fix bugs that appear in the current code
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-578
>                 URL: https://issues.apache.org/jira/browse/HIVE-578
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.3.0
>            Reporter: Ashish Thusoo
>            Assignee: Ashish Thusoo
>         Attachments: patch-578.txt
>
>
> Some bugs with partition pruning have been reported and the correct fix for many of them is to rewrite the partition pruning code as an optimizer transformation which gets kicked in after the predicate pushdown code. This refactor also uses the graph walker framework so that the partition pruning code gets consolidated well with the frameworks and does not work on the query block but rather works on the operator tree.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.