[jira] Commented: (HADOOP-3339) DFS Write pipeline does not detect defective datanode correctly if it times out.

["Raghu Angadi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12593908#action_12593908 ] 

Raghu Angadi commented on HADOOP-3339:
--------------------------------------

TestDatanodeDeath tests killing different datanodes in the pipeline and it works. The main difference is that whether the downstream datanode's error is detected depends on which of the two threads (the main data receiver or the "PacketResonder") detects it. 

In TestDatanodeDeath, its always the PacketResponder that detects it. But when a downstream datanode timeouts (or when the connection is busy) its the main IO thread that detects it. The fix I am thinking of is to make main thread inform the 'PacketResponder' about the failure.


> DFS Write pipeline does not detect defective datanode correctly if it times out.
> --------------------------------------------------------------------------------
>
>                 Key: HADOOP-3339
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3339
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>
> When DFSClient is writing to DFS, it does not correctly detect the culprit datanode (rather datanodes do not inform) properly if the bad node times out. Say, the last datanode in in 3 node pipeline is is too slow or defective. In this case, pipeline removes the first two datanodes in first two attempts. The third attempt has only the 3rd datanode in the pipeline and it will fail too. If the pipeline detects the bad 3rd node when the first failure occurs, the write will succeed in the second attempt. 
> I will attach example logs of such cases. I think this should be fixed in 0.17.x.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.