You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2010/11/22 02:59:13 UTC

[jira] Commented: (HIVE-1802) Encode MapReduce Shuffling Keys Differently for Single string/bigint Key

    [ https://issues.apache.org/jira/browse/HIVE-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12934380#action_12934380 ] 

Namit Jain commented on HIVE-1802:
----------------------------------

The code looks OK, but it is not very easy to add new serde's this way.
Can you refactor PlanUtils change into a factory - so that it is easy to add new changes

> Encode MapReduce Shuffling Keys Differently for  Single string/bigint Key
> -------------------------------------------------------------------------
>
>                 Key: HIVE-1802
>                 URL: https://issues.apache.org/jira/browse/HIVE-1802
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Siying Dong
>         Attachments: HIVE-1802.1.patch
>
>
> Delimiters are not needed if we only have one shuffling key, and in the same time escaping delimiters are not needed. We can save some CPU time on serializing and shuffle slightly less amount of data to save memory footprint and network traffic.
> Also there is a bug that for group-by, we by mistake add a -1 to the end of the key and pay one more unnecessary mem-copy. Can be easily fixed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.