[GitHub] [helix] jiajunwang commented on a change in pull request #1526: Leverage zk paginated getChildren API for ZkClient to fetch a large number of children

[GitBox <gi...@apache.org>]

jiajunwang commented on a change in pull request #1526:
URL: https://github.com/apache/helix/pull/1526#discussion_r535589063



##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkConnection.java
##########
@@ -192,4 +230,53 @@ public String getServers() {
   public void addAuthInfo(String scheme, byte[] auth) {
     _zk.addAuthInfo(scheme, auth);
   }
+
+  private void lookupGetChildrenMethod() {
+    if (_getChildrenMethod != null) {
+      // Method is already cached.
+      return;
+    }
+    try {
+      if (GETCHILDREN_PAGINATION_DISABLED) {
+        lookupNonPaginatedGetChildren();
+      } else {
+        // Lookup the paginated getChildren API
+        _getChildrenMethod =
+            ZooKeeper.class.getMethod("getAllChildrenPaginated", String.class, boolean.class);
+      }
+    } catch (NoSuchMethodException e1) {
+      // Pagination API is not supported, fall back to non-paginated API
+      lookupNonPaginatedGetChildren();
+    }
+    LOG.info("Pagination config {}={}, method to be invoked: {}",
+        ZkSystemPropertyKeys.ZK_GETCHILDREN_PAGINATION_DISABLED, GETCHILDREN_PAGINATION_DISABLED,
+        _getChildrenMethod.getName());
+  }
+
+  private void lookupNonPaginatedGetChildren() {
+    try {
+      _getChildrenMethod = ZooKeeper.class.getMethod("getChildren", String.class, boolean.class);
+    } catch (NoSuchMethodException e2) {
+      // We should not expect this exception here.
+      LOG.error("getChildren is not supported in this zookeeper version!");
+    }
+  }
+
+  private void handleInvokedMethodException(Throwable cause)

Review comment:
       One possible way is that, once the call throws an unsupported exception, we fallback to the old getChildren method.
   
   Then on every new session establishment, we reset it based on the configuration and zk client lib method availability.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org