You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Siddharth Seth (JIRA)" <ji...@apache.org> on 2014/04/23 22:02:14 UTC

[jira] [Commented] (TEZ-1073) RLE IFile merge() fastpath

    [ https://issues.apache.org/jira/browse/TEZ-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978804#comment-13978804 ] 

Siddharth Seth commented on TEZ-1073:
-------------------------------------

[~gopalv] - the patch mostly looks good to me. Couple of suggestions and question.

Should the main nextRawKey just be changed to return KeyState rather than the additional method readRawKey ?

Also, should DefaultSorter be changed to enabled RLE by default ?

> RLE IFile merge() fastpath
> --------------------------
>
>                 Key: TEZ-1073
>                 URL: https://issues.apache.org/jira/browse/TEZ-1073
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.5.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>            Priority: Minor
>         Attachments: TEZ-1073.WIP.patch
>
>
> IFile iterators actually consist of a tri-state - NEW_KEY, SAME_KEY and NO_KEY.
> This can be used to avoid rebalancing the priority queue when the same reader is returning SAME_KEY in its reader during the initial merge operation in the reducer.



--
This message was sent by Atlassian JIRA
(v6.2#6252)