You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Nathan Roberts (JIRA)" <ji...@apache.org> on 2017/08/02 20:52:00 UTC
[jira] [Commented] (HDFS-12102) VolumeScanner throttle dropped
(fast scan enabled) when there is a corrupt block
[ https://issues.apache.org/jira/browse/HDFS-12102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111696#comment-16111696 ]
Nathan Roberts commented on HDFS-12102:
---------------------------------------
[~arpitagarwal] Hi Arpit. To provide a bit more background on this feature - we've seen multiple cases where there are many bad blocks stored on a disk. Just because of the way drives tend to fail, one bad block indicates there are probably many others. The volumeScanner will eventually find them over a multi-week period, but this leaves the cluster susceptible to data-loss due to lots of replicas being corrupt on a single misbehaving disk. The idea with this jira is to use a found corrupt block as a hint that there are likely more and we should do a scan over the drive at a faster rate to more quickly find other corrupt blocks on the drive. Thoughts?
> VolumeScanner throttle dropped (fast scan enabled) when there is a corrupt block
> --------------------------------------------------------------------------------
>
> Key: HDFS-12102
> URL: https://issues.apache.org/jira/browse/HDFS-12102
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, hdfs
> Affects Versions: 2.8.2
> Reporter: Ashwin Ramesh
> Priority: Minor
> Fix For: 2.8.2
>
> Attachments: HDFS-12102-001.patch, HDFS-12102-002.patch, HDFS-12102-003.patch
>
>
> When the Volume scanner sees a corrupt block, it restarts the scan and scans the blocks at much faster rate with a negligible scan period. This is so that it doesn't take 3 weeks to report blocks since a corrupt block means increased likelihood that there are more corrupt blocks.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org