You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Wei-Chiu Chuang (JIRA)" <ji...@apache.org> on 2018/08/18 00:18:00 UTC
[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on
dead name node, when ConnectTimeoutException happens.
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584528#comment-16584528 ]
Wei-Chiu Chuang commented on HADOOP-15684:
------------------------------------------
Does it actually affect Hadoop 2.75. and 2.9.1? The MultipleNameNode was added in HDFS-6440 and is not in Hadoop 2.x
> triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-15684
> URL: https://issues.apache.org/jira/browse/HADOOP-15684
> Project: Hadoop Common
> Issue Type: Bug
> Components: ha
> Affects Versions: 2.7.5, 2.9.1
> Reporter: Rong Tang
> Priority: Critical
> Attachments: FixRollEditLog.patch, hadoop--rollingUpgrade-BN2SCH070021402.log
>
>
> When name node call triggerActiveLogRoll, and the cachedActiveProxy is a dead name node, it will throws a ConnectTimeoutException, expected behavior is to try next NN, but current logic doesn't do so, instead, it keeps trying the dead, mistakenly take it as active.
>
> 2018-08-17 10:02:12,001 WARN [Edit log tailer] org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Unable to trigger a roll of the active NN
> org.apache.hadoop.net.ConnectTimeoutException: Call From BN2SCH070021402/25.126.188.193 to BN2SCH070041016.ap.gbl:8020 failed on socket timeout exception: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$2.doWork(EditLogTailer.java:298)
>
> C:\Users\rotang>ping BN2SCH070041016
> Pinging BN2SCH070041016 [25.126.141.79] with 32 bytes of data:
> Request timed out.
> Request timed out.
> Request timed out.
> Request timed out.
>
> Attachment is a log file saying how it repeatedly retries a dead name node, and a fix patch.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org