You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Suresh Srinivas (Resolved) (JIRA)" <ji...@apache.org> on 2012/01/25 22:37:38 UTC
[jira] [Resolved] (HDFS-183) MapReduce Streaming job hang when all
replications of the input file has corrupted!
[ https://issues.apache.org/jira/browse/HDFS-183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas resolved HDFS-183.
----------------------------------
Resolution: Won't Fix
This bug has been open for a while, with no reply to the last comment posted. Closing for now. Please re-open the bug or open a new bug, if the problem still persists.
> MapReduce Streaming job hang when all replications of the input file has corrupted!
> -----------------------------------------------------------------------------------
>
> Key: HDFS-183
> URL: https://issues.apache.org/jira/browse/HDFS-183
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ZhuGuanyin
> Priority: Critical
>
> On some special cases, all replications of a given file has truncated to zero but the namenode still hold the original size (we don't know why), the mapreduce streaming job will hang if we don't specified mapred.task.timeout when the input files contain this corrupted file, even the dfs shell "cat" will hang when fetch data from this corrupted file.
> We found that job hang at DFSInputStream.blockSeekTo() when chosing a datanode. The following test will show:
> 1) Copy a small file to hdfs.
> 2) Get the file blocks and login to these datanodes, and truncate these blocks to zero.
> 3) Cat this file through dfs shell "cat"
> 4) Cat command will enter dead loop.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira