You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2013/09/26 17:30:03 UTC

[jira] [Commented] (HIVE-5358) ReduceSinkDeDuplication should ignore column orders when check overlapping part of keys between parent and child

    [ https://issues.apache.org/jira/browse/HIVE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778898#comment-13778898 ] 

Ashutosh Chauhan commented on HIVE-5358:
----------------------------------------

For queries like:
{code}
select key, value from (select key, value from src group by key, value) t order by value, key;
select key, value from (select key, value from src order by key, value) t group by value, key;
{code}

In these case also two RSs can be merged but order of key Columns on RS becomes important and must be preserved. Can you add a test case for this to make sure this works correctly. 

For queries like:
{code}
set hive.optimize.correlation=true;   
select key, value from (select s1.key, s2.value from src s1 join src s2 on s1.key = s2.key and s1.value=s2.value) t group by  key,value;
select key, value from (select s1.key, s2.value from src s1 join src s2 on s1.key = s2.key and s1.value=s2.value) t group by  value,key;
{code}
For these cases also, ordering of key columns in RS is important.

cc: [~navis] [~yhuai]
                
> ReduceSinkDeDuplication should ignore column orders when check overlapping part of keys between parent and child
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-5358
>                 URL: https://issues.apache.org/jira/browse/HIVE-5358
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Chun Chen
>            Assignee: Chun Chen
>         Attachments: D13113.1.patch, HIVE-5358.2.patch, HIVE-5358.patch
>
>
> {code}
> select key, value from (select key, value from src group by key, value) t group by key, value;
> {code}
> This can be optimized by ReduceSinkDeDuplication
> {code}
> select key, value from (select key, value from src group by key, value) t group by value, key;
> {code}
> However the sql above can't be optimized by ReduceSinkDeDuplication currently due to different column orders of parent and child operator.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira