You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Rodrigo Schmidt (JIRA)" <ji...@apache.org> on 2010/06/23 02:15:54 UTC
[jira] Created: (HDFS-1259) getCorruptFiles() should double check
blocks that are not really corrupted
getCorruptFiles() should double check blocks that are not really corrupted
--------------------------------------------------------------------------
Key: HDFS-1259
URL: https://issues.apache.org/jira/browse/HDFS-1259
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.22.0
Reporter: Rodrigo Schmidt
Fix For: 0.22.0
the getCorruptFiles() api is based on blocks that are in the queue of under-replicated blocks. However, this queue might be outdated and report blocks that are safe, specially during restarts when block reports might get delayed.
getCorruptFiles() should double check if the block is really corrupted based on blocksMap before reporting it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1259) getCorruptFiles() should double check
blocks that are not really corrupted
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881766#action_12881766 ]
dhruba borthakur commented on HDFS-1259:
----------------------------------------
can we get a unit test case that demonstrates that a file has all 3 required replicas in the blocksMap but is still present in neededReplication?
> getCorruptFiles() should double check blocks that are not really corrupted
> --------------------------------------------------------------------------
>
> Key: HDFS-1259
> URL: https://issues.apache.org/jira/browse/HDFS-1259
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.22.0
>
>
> the getCorruptFiles() api is based on blocks that are in the queue of under-replicated blocks. However, this queue might be outdated and report blocks that are safe, specially during restarts when block reports might get delayed.
> getCorruptFiles() should double check if the block is really corrupted based on blocksMap before reporting it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1259) getCorruptFiles() should double check
blocks that are not really corrupted
Posted by "Rodrigo Schmidt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881987#action_12881987 ]
Rodrigo Schmidt commented on HDFS-1259:
---------------------------------------
It might be a little complicated, but yes.
> getCorruptFiles() should double check blocks that are not really corrupted
> --------------------------------------------------------------------------
>
> Key: HDFS-1259
> URL: https://issues.apache.org/jira/browse/HDFS-1259
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.22.0
> Reporter: Rodrigo Schmidt
> Fix For: 0.22.0
>
>
> the getCorruptFiles() api is based on blocks that are in the queue of under-replicated blocks. However, this queue might be outdated and report blocks that are safe, specially during restarts when block reports might get delayed.
> getCorruptFiles() should double check if the block is really corrupted based on blocksMap before reporting it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.