[jira] [Updated] (HIVE-20210) Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal

["Jeffrey(Xilang) Yan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jeffrey(Xilang) Yan updated HIVE-20210:
---------------------------------------
    Attachment: HIVE-20210.patch

> Simple Fetch optimizer should lead to MapReduce when filter on non-partition column and conversion is minimal
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-20210
>                 URL: https://issues.apache.org/jira/browse/HIVE-20210
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Planning
>    Affects Versions: 2.3.0, 2.3.1, 2.3.2
>            Reporter: Jeffrey(Xilang) Yan
>            Assignee: Jeffrey(Xilang) Yan
>            Priority: Major
>         Attachments: HIVE-20210.patch
>
>
> When conversion is minimal, simple fetch can be used only when filter on partition column or no filter at all. But it lead to simple fetch even if filter on non-partition column. Unit test " select * from srcpart where key > 100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test is not correct indeed(it should be Map Reduce  but in test it is Simple Fetch).
> This issue lead to a serious problem when data size is huge. When conversion is more and filter on both partition column and non-partition column, it will not chech hive.fetch.task.conversion.threshold, which lead to the query to takes hours to finish. This issue doesn't exist in 1.2.1, how it works should be a magic...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)