You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Ruyue Ma (JIRA)" <ji...@apache.org> on 2010/04/02 11:18:27 UTC

[jira] Created: (MAPREDUCE-1667) resource estimation still works badly in some cases

resource estimation still works badly in some cases
---------------------------------------------------

                 Key: MAPREDUCE-1667
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1667
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobtracker
            Reporter: Ruyue Ma
            Assignee: Ruyue Ma


A premise upon which our current implemention of ResourceEstimator is that the MapInputSize and MapOutputSize has relation. 

In many user cases, the premise is not satisfied.  

e.g. 

1. The map input is a file list.
2. Every mapper will download the file, process, then output (into map intermediate output)

if one file name is very long, the estimated output size maybe very big.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.