[jira] [Commented] (HIVE-18873) Skipping predicate pushdown for MR silently at HiveInputFormat can cause storage handlers to produce erroneous result

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388243#comment-16388243 ] 

Josh Elser commented on HIVE-18873:
-----------------------------------

[~prasanth_j], [~thejas], I'm really worried about the change that went into HIVE-15680. It seems to have some drastically unintended consequences. As we've seen for two separate StorageHandlers already (Phoenix and Accumulo), suddenly not allowing PPD with the MR engine breaks some pretty basic assumptions on correctness.

I can understand the desire to move away from supporting MR, but is there anything we can do here to make this a less painful transition?

> Skipping predicate pushdown for MR silently at HiveInputFormat can cause storage handlers to produce erroneous result
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-18873
>                 URL: https://issues.apache.org/jira/browse/HIVE-18873
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Ankit Singhal
>            Assignee: Ankit Singhal
>            Priority: Major
>             Fix For: 3.0.0
>
>
> {code:java}
> // disable filter pushdown for mapreduce when there are more than one table aliases,
>     // since we don't clone jobConf per alias
>     if (mrwork != null && mrwork.getAliases() != null && mrwork.getAliases().size() > 1 &&
>       jobConf.get(ConfVars.HIVE_EXECUTION_ENGINE.varname).equals("mr")) {
>       return;
>     }
> {code}
> I believe this needs to be handled at OpProcFactory so that hive doesn't believe that predicate is handled by storage handler.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)