You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2017/07/10 04:33:00 UTC

[jira] [Created] (TEZ-3789) Consider avoiding buffer copies in TezMerger when lots of unique keys are present in reducer side

Rajesh Balamohan created TEZ-3789:
-------------------------------------

             Summary: Consider avoiding buffer copies in TezMerger when lots of unique keys are present in reducer side
                 Key: TEZ-3789
                 URL: https://issues.apache.org/jira/browse/TEZ-3789
             Project: Apache Tez
          Issue Type: Improvement
            Reporter: Rajesh Balamohan


Currently TezMerger stores the key details in memory. However, depending on the number of records read, number of unique keys and the merger progress information, it should be possible to deduce whether lots of unique keys are present and based on that buffer copies could be avoided.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)