You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Runping Qi (JIRA)" <ji...@apache.org> on 2008/03/07 04:58:58 UTC

[jira] Commented: (HADOOP-2959) When a mapper needs to run a combiner, it should create one and reuse it, instead of creating one per partition per spill

    [ https://issues.apache.org/jira/browse/HADOOP-2959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12576029#action_12576029 ] 

Runping Qi commented on HADOOP-2959:
------------------------------------


I should have pounted out the use case why this is matter.
When the combining logic (reducer logic) depends on some thing that is initialized 
in the configure method, and of the configure method call is relative expensive (say initialize a dictionary 
from a file on dfs), then such an optimization makes a huge difference.



> When a mapper needs to run a combiner, it should create one and reuse it, instead of creating one per partition per spill
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-2959
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2959
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.16.0
>            Reporter: Runping Qi
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.