[jira] [Commented] (HIVE-18873) Skipping predicate pushdown for MR silently at HiveInputFormat can cause storage handlers to produce erroneous result

["Prasanth Jayachandran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16603497#comment-16603497 ] 

Prasanth Jayachandran commented on HIVE-18873:
----------------------------------------------

{quote}Under the assumption that Hive is ensuring the results are accurate anyways? (e.g. we'd want to check that all storage handlers work this way)
{quote}
Orc PPD works this way, if orc cannot handle the filter it will drop the filter and continue with rest of the filters. The filters will still remain in hive operator tree which will get reapplied anyways.

 

If the test failure is unrelated to the patch and is already happening in the master, I think we can go ahead and commit the latest patch and track the test failure in a separate ticket. 

+1, comment failing test, track it separately. 

> Skipping predicate pushdown for MR silently at HiveInputFormat can cause storage handlers to produce erroneous result
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-18873
>                 URL: https://issues.apache.org/jira/browse/HIVE-18873
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Ankit Singhal
>            Assignee: Ankit Singhal
>            Priority: Major
>         Attachments: HIVE-18873.2.patch, HIVE-18873.2_reattach.patch, HIVE-18873.patch
>
>
> {code:java}
> // disable filter pushdown for mapreduce when there are more than one table aliases,
>     // since we don't clone jobConf per alias
>     if (mrwork != null && mrwork.getAliases() != null && mrwork.getAliases().size() > 1 &&
>       jobConf.get(ConfVars.HIVE_EXECUTION_ENGINE.varname).equals("mr")) {
>       return;
>     }
> {code}
> I believe this needs to be handled at OpProcFactory so that hive doesn't believe that predicate is handled by storage handler.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)