You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Todd Lipcon (Created) (JIRA)" <ji...@apache.org> on 2012/01/16 21:47:39 UTC
[jira] [Created] (HDFS-2798) Append may race with datanode block
scanner, causing replica to be incorrectly marked corrupt
Append may race with datanode block scanner, causing replica to be incorrectly marked corrupt
---------------------------------------------------------------------------------------------
Key: HDFS-2798
URL: https://issues.apache.org/jira/browse/HDFS-2798
Project: Hadoop HDFS
Issue Type: Bug
Components: data-node
Affects Versions: 0.23.0, 0.22.0
Reporter: Todd Lipcon
Priority: Critical
When a pipeline is setup for append, the block's metadata file is renamed before the block is removed from the datanode block scanner queues. This can cause a race condition where the block scanner incorrectly marks the block as corrupt, since it tries to scan the file corresponding to the old genstamp.
This causes TestFileAppend2 to time out in extremely rare circumstances - the corrupt replica prevents the writer thread from completing the file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-2798) Append may race with datanode block
scanner, causing replica to be incorrectly marked corrupt
Posted by "Brandon Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Li resolved HDFS-2798.
------------------------------
Resolution: Duplicate
Target Version/s: 2.0.0-alpha, 0.23.3, 3.0.0 (was: 0.23.3, 2.0.0-alpha, 3.0.0)
It was fixed in HDFS-2525.
> Append may race with datanode block scanner, causing replica to be incorrectly marked corrupt
> ---------------------------------------------------------------------------------------------
>
> Key: HDFS-2798
> URL: https://issues.apache.org/jira/browse/HDFS-2798
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Brandon Li
> Priority: Critical
>
> When a pipeline is setup for append, the block's metadata file is renamed before the block is removed from the datanode block scanner queues. This can cause a race condition where the block scanner incorrectly marks the block as corrupt, since it tries to scan the file corresponding to the old genstamp.
> This causes TestFileAppend2 to time out in extremely rare circumstances - the corrupt replica prevents the writer thread from completing the file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira