You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Sameer Paranjpye (JIRA)" <ji...@apache.org> on 2007/10/02 21:43:50 UTC

[jira] Commented: (HADOOP-1965) Handle map output buffers better

    [ https://issues.apache.org/jira/browse/HADOOP-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531891 ] 

Sameer Paranjpye commented on HADOOP-1965:
------------------------------------------

We need to benchmark anything we implement for this issue. The interesting use case is tasks that spill since these make no progress while the sort/spill is happening. Running more total maps / more concurrent maps per node are also options. But these seem like orthogonal strategies, improving latency will help regardless, no?



> Handle map output buffers better
> --------------------------------
>
>                 Key: HADOOP-1965
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1965
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Devaraj Das
>            Assignee: Amar Kamat
>
> Today, the map task stops calling the map method while sort/spill is using the (single instance of) map output buffer. One improvement that can be done to improve performance of the map task is to have another buffer for writing the map outputs to, while sort/spill is using the first buffer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.