[jira] Updated: (HADOOP-6439) Shuffle deadlocks on wrong number of maps

["V.V.Chaitanya Krishna (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-6439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

V.V.Chaitanya Krishna updated HADOOP-6439:
------------------------------------------

    Attachment: HADOOP-6439-1.patch

Uploading patch with the following changes implemented:

* The value of the key will be the one which is loaded/set most recently. (Note: As maintained previously, there will be only new keys stored in the Configuration object.)

* If the key is set to final, then that key (or the corresponding new key in case of deprecation) will be marked as final and no further changes to its value will be done. This is different from the way final parameters were handled in HADOOP-6105, where the presence of old key itself mattered(irrespective of which value is set/loaded last) and precedence order had to be considered even for being final.

> Shuffle deadlocks on wrong number of maps
> -----------------------------------------
>
>                 Key: HADOOP-6439
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6439
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: conf
>    Affects Versions: 0.21.0, 0.22.0
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>            Priority: Blocker
>             Fix For: 0.21.0, 0.22.0
>
>         Attachments: HADOOP-6439-1.patch, mr-1252.patch
>
>
> The new shuffle assumes that the number of maps is correct. The new JobSubmitter sets the old value. Something misfires in the middle causing:
> 09/12/01 00:00:15 WARN conf.Configuration: mapred.job.split.file is deprecated. Instead, use mapreduce.job.splitfile
> 09/12/01 00:00:15 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> But my reduces got stuck at 2 maps / 12 when there were only 2 maps in the job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.