You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Lisheng Sun (JIRA)" <ji...@apache.org> on 2019/07/13 06:37:00 UTC

[jira] [Comment Edited] (HDFS-13571) Dead DataNode Detector

    [ https://issues.apache.org/jira/browse/HDFS-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884305#comment-16884305 ] 

Lisheng Sun edited comment on HDFS-13571 at 7/13/19 6:36 AM:
-------------------------------------------------------------

Hi [~jojochuang] [~sodonnell]  [~elgoiri] [~xkrogen] Could you have time to pay attention to this issue? XIAOMI HBase availability is very helpful by Dead DataNode Detector. Thank you.

If this issue is accepted, I will update new patch for trunk.


was (Author: leosun08):
Hi [~jojochuang] [~sodonnell]  [~elgoiri] [~xkrogen] Could you have time to pay attention to this issue? XIAOMI HBase usability is very helpful by Dead DataNode Detector. Thank you.

> Dead DataNode Detector
> ----------------------
>
>                 Key: HDFS-13571
>                 URL: https://issues.apache.org/jira/browse/HDFS-13571
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 2.4.0, 2.6.0, 3.0.2
>            Reporter: Gang Xie
>            Assignee: Lisheng Sun
>            Priority: Minor
>             Fix For: 3.0.2
>
>         Attachments: HDFS-13571-2.6.diff
>
>
> Currently, the information of the dead datanode in DFSInputStream in stored locally. So, it could not be shared among the inputstreams of the same DFSClient. In our production env, every days, some datanodes dies with different causes. At this time, after the first inputstream blocked and detect this, it could share this information to others in the same DFSClient, thus, the ohter inputstreams are still blocked by the dead node for some time, which could cause bad service latency.
> To eliminate this impact from dead datanode, we designed a dead datanode detector, which detect the dead ones in advance, and share this information among all the inputstreams in the same client. This improvement has being online for some months and works fine.  So, we decide to port to the 3.0 (the version used in our production env is 2.4 and 2.6).
> I will do the porting work and upload the code later.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org