[jira] [Updated] (HIVE-17979) Tez: Improve ReduceRecordSource passDownKey copying

["Ashutosh Chauhan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashutosh Chauhan updated HIVE-17979:
------------------------------------
    Status: Open  (was: Patch Available)

+1

> Tez: Improve ReduceRecordSource passDownKey copying
> ---------------------------------------------------
>
>                 Key: HIVE-17979
>                 URL: https://issues.apache.org/jira/browse/HIVE-17979
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 3.0.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>            Priority: Major
>         Attachments: HIVE-17979.1.patch, HIVE-17979.2.patch
>
>
> Tez does not use a single Key stream for both sides of the join, so each input gets its own ReduceRecordSource 
> {code}
> sources[tag] = new ReduceRecordSource();
> {code}
> And this means for each input stream, there's a deserialized key (because the tag is not part of the Key byte stream), this means for a 2-table join there are 2 ReduceRecordSource objects.
> This means that the passDownKey is only an optimization when the Key, List<Value> has more than 1 value in it. Otherwise the copy is entirely wasted CPU cycles, because it deserializes the entire row to extract the key and discards the row.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)