You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-issues@hadoop.apache.org by "Arpit Agarwal (JIRA)" <ji...@apache.org> on 2015/03/06 03:04:41 UTC

[jira] [Commented] (HDFS-7896) HDFS Slow disk detection

    [ https://issues.apache.org/jira/browse/HDFS-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349792#comment-14349792 ] 

Arpit Agarwal commented on HDFS-7896:
-------------------------------------

Based on offline discussion with [~sanjay.radia].

Couple of options:
# Track IO latencies per volume to detect outliers. Unfortunately DNs use buffered IO so IO latency is a meaningless for measuring disk performance.
# Run a microbenchmark on DN startup and tag slow disks. Can affect DN startup performance and does not address disks that develop faults once the DN has been started. 

[~cnauroth] recently added failed volume reporting via HDFS-7604. Ideally we can extend that reporting infrastructure.

> HDFS Slow disk detection
> ------------------------
>
>                 Key: HDFS-7896
>                 URL: https://issues.apache.org/jira/browse/HDFS-7896
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode
>            Reporter: Arpit Agarwal
>
> HDFS should detect slow disks. To start with we can flag this information via the NameNode web UI. Alternatively DNs can avoid using slow disks for writes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)