You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Laurent Goujon (JIRA)" <ji...@apache.org> on 2015/04/21 17:44:59 UTC

[jira] [Created] (MAPREDUCE-6323) Support pass-by-reference for ChainMapper/ChainReducer

Laurent Goujon created MAPREDUCE-6323:
-----------------------------------------

             Summary: Support pass-by-reference for ChainMapper/ChainReducer
                 Key: MAPREDUCE-6323
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6323
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: mrv2, performance
    Affects Versions: 2.4.1
            Reporter: Laurent Goujon
            Priority: Minor


ChainMapper and ChainReducer using the new mapreduce API always copy the (key,value) pair between chain elements. It actually looks the copy is done twice, once when queueing, and once when dequeuing!

When transferred objects are immutable (or not reused), this is a waste of resource (especially when passing big records)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)