Fwd: [apache/hadoop] HDFS-16283: RBF: improve renewLease() to call only a specific NameNod… (PR #3595)

[Aihua Xu <ai...@gmail.com>]

Hi folks,

Want to bring the discussion to add a new call in the HDFS client protocol.
(see the changes https://github.com/apache/hadoop/pull/3595/files).

*Context*: Currently in federated HDFS, since the routers don't know which
subclusters the client is writing files to, the routers have to make a
fan-out call to all the subclusters. If one of the subclusters is slow,
then the routers will accumulate such renewLease calls and eventually cause
router degradation.

*Proposal*: We are proposing to add a new call as follows in client
protocol to make renewLease call against specific subcluster. It maintains
backward compatibility: 1) existing clients still make the fan-out call
(renewLease(String clientName)); 2) The new clients can work with old/new
versions of routers.

void renewLease(String clientName, String nsId) throws IOException;

Please comment.

Thanks,
Aihua


---------- Forwarded message ---------
From: Inigo Goiri <no...@github.com>
Date: Thu, Oct 28, 2021 at 12:09 PM
Subject: Re: [apache/hadoop] HDFS-16283: RBF: improve renewLease() to call
only a specific NameNod… (PR #3595)
To: apache/hadoop <ha...@noreply.github.com>
Cc: aihuaxu <ai...@gmail.com>, Author <au...@noreply.github.com>


*@goiri* requested changes on this pull request.
------------------------------

In
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
<https://github.com/apache/hadoop/pull/3595#discussion_r738674897>:

> @@ -765,6 +765,14 @@ BatchedDirectoryListing getBatchedListing(
   @Idempotent
   void renewLease(String clientName) throws IOException;

+  /**
+   * The functionality is the same as renewLease(clientName). This is
to support
+   * router based FileSystem to newLease against a specific target
FileSystem instead
+   * of all the target FileSystems in each call.
+   */
+  @Idempotent
+  void renewLease(String clientName, String nsId) throws IOException;

Changing the client protocol requires more than just a RBF PR.
We should bring this topic up in the mailing list.
The good news is that this is an additional call and not modifying the
existing.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<https://github.com/apache/hadoop/pull/3595#pullrequestreview-792299305>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AGKDWK5QZVNDDH6TAN7Q2ADUJGNV5ANCNFSM5G3KXPFQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.