[jira] [Commented] (HIVE-7397) Set the default threshold for fetch task conversion to 1Gb

["Gopal V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059667#comment-14059667 ] 

Gopal V commented on HIVE-7397:
-------------------------------

Candidate queries for my test were 

{code}
select * from store_sales where year(ss_sold_date)=2002 and month(ss_sold_date)=12 limit 200;
select ss_item_sk from store_sales where ss_sold_date = '2002-01-01';
select ss_item_sk from store_sales where ss_sold_date = '2002-01-01' and ss_quantity < 2;
{code}

The current exit criteria for this test is that the first 2 should always hit the "more" criteria irrespective of data-sizes.

The latter should only go there if the whole data-set for 2002-01-01 is < 1Gb.

> Set the default threshold for fetch task conversion to 1Gb
> ----------------------------------------------------------
>
>                 Key: HIVE-7397
>                 URL: https://issues.apache.org/jira/browse/HIVE-7397
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.14.0, 0.13.1
>            Reporter: Gopal V
>            Assignee: Gopal V
>              Labels: Performance
>             Fix For: 0.14.0
>
>
> Currently, modifying the value of hive.fetch.task.conversion to "more" results in a dangerous setting where small scale queries function, but large scale queries crash.
> This occurs because the default threshold of -1 means apply this optimization for a petabyte table.
> I am testing a variety of queries with the setting "more" (to make it the default option as suggested by HIVE-887) change the default threshold for this feature to a reasonable 1Gb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)