[jira] [Commented] (MAPREDUCE-2185) Infinite loop at creating splits using CombineFileInputFormat

["Hudson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045186#comment-13045186 ] 

Hudson commented on MAPREDUCE-2185:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk-Commit #717 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/717/])
    MAPREDUCE-2185. Fix infinite loop at creating splits using CombineFileInputFormat. (Ramkumar Vadali via schen)

schen : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132807
Files : 
* /hadoop/mapreduce/trunk/CHANGES.txt
* /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/input/CombineFileInputFormat.java
* /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java


> Infinite loop at creating splits using CombineFileInputFormat
> -------------------------------------------------------------
>
>                 Key: MAPREDUCE-2185
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2185
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: job submission
>            Reporter: Hairong Kuang
>            Assignee: Ramkumar Vadali
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2185.patch
>
>
> This is caused by a missing block in HDFS. So the block's locations are empty. The following code adds the block to blockToNodes map but not to rackToBlocks map. Later on when generating splits, only blocks in rackToBlocks are removed from blockToNodes map. So blockToNodes map can never become empty therefore causing infinite loop
> {code}
>           // add this block to the block --> node locations map
>           blockToNodes.put(oneblock, oneblock.hosts);
>           // add this block to the rack --> block map
>           for (int j = 0; j < oneblock.racks.length; j++) {
>              ..
>           }
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira