[jira] [Reopened] (HDFS-8708) DFSClient should ignore dfs.client.retry.policy.enabled for HA proxies

["Chengbing Liu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-8708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chengbing Liu reopened HDFS-8708:
---------------------------------

I have different opinion so I'm reopening this issue.

In our production environment, we have both HA and non-HA clusters. A client should be able to access both kinds of clusters. This is our dilemma.

By setting dfs.client.retry.policy.enabled = true, currently we see:
1) HA nameservice: in case of nn1 shutdown, will still attempt connecting to nn1 many times (11min by default) before failover, which is undesired
2) non-HA namenode: keep retrying to connect for 11min by default

By setting dfs.client.retry.policy.enabled = false, currently we see:
1) HA nameservice: fast failover, everything works fine
2) non-HA namenode: no retry will be made in case of connection failure, which is undesired

We would like to ensure fast failover with HA mode as well as multiple retries with non-HA mode, and we cannot achieve this with current implementation.

Proposed code change:
In {{NameNodeProxiesClient.createProxyWithAlignmentContext}}, {{defaultPolicy}} should not be passed to {{ClientProtocol}} when {{withRetries}} is false (HA mode). Instead, TRY_ONCE_THEN_FAIL can be used to ensure fast failover.

> DFSClient should ignore dfs.client.retry.policy.enabled for HA proxies
> ----------------------------------------------------------------------
>
>                 Key: HDFS-8708
>                 URL: https://issues.apache.org/jira/browse/HDFS-8708
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Jitendra Nath Pandey
>            Assignee: Brahma Reddy Battula
>            Priority: Critical
>
> DFSClient should ignore dfs.client.retry.policy.enabled for HA proxies to ensure fast failover. Otherwise, dfsclient retries the NN which is no longer active and delays the failover.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org