[jira] [Commented] (YARN-3944) Connection refused to nodemanagers are retried at multiple levels

["Sangjin Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14933594#comment-14933594 ] 

Sangjin Lee commented on YARN-3944:
-----------------------------------

[~l201514], [~zxu], what's your thought on this? Should we close this one and target YARN-3554 for 2.6.2 instead?

> Connection refused to nodemanagers are retried at multiple levels
> -----------------------------------------------------------------
>
>                 Key: YARN-3944
>                 URL: https://issues.apache.org/jira/browse/YARN-3944
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Siqi Li
>            Assignee: Siqi Li
>            Priority: Critical
>         Attachments: YARN-3944.v1.patch
>
>
> This is related to YARN-3238. When NM is down, ipc client will get ConnectException.
> Caused by: java.net.ConnectException: Connection refused
> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
> 	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> 	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
> 	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)
> 	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:607)
> 	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:705)
> 	at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:368)
> 	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1521)
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1438)
> However, retry happens at two layers(ipc retry 40 times and serverProxy retrying 91 times), this could end up with ~1 hour retry interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)