[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1770: SOLR-14763 SolrJ HTTP/2 Async API using CompletableFuture

[GitBox <gi...@apache.org>]

tflobbe commented on a change in pull request #1770:
URL: https://github.com/apache/lucene-solr/pull/1770#discussion_r474402967



##########
File path: solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java
##########
@@ -853,6 +858,18 @@ public RouteException(ErrorCode errorCode, NamedList<Throwable> throwables, Map<
 
   @Override
   public NamedList<Object> request(@SuppressWarnings({"rawtypes"})SolrRequest request, String collection) throws SolrServerException, IOException {
+      // synchronous requests should return an already completed future
+      return getNowOrException(makeRequest(request, collection, false));
+  }
+
+  CompletableFuture<NamedList<Object>> makeRequest(SolrRequest<?> request,
+                                                   String collection,
+                                                   boolean isAsyncRequest) throws SolrServerException, IOException {
+    if (isAsyncRequest && !(getLbClient() instanceof LBHttp2SolrClient)) {
+      log.warn("Asynchronous requests require HTTP/2 SolrJ client, defaulting to synchronous request.");
+      isAsyncRequest = false;

Review comment:
       Could we skip this check here? for existing classes, `requestAsync` already throws `UnsupportedOperationException` if async is not supported, and this could cause issues to custom implementation of LB clients.

##########
File path: solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java
##########
@@ -926,22 +946,24 @@ public RouteException(ErrorCode errorCode, NamedList<Throwable> throwables, Map<
       }
     } // else: ??? how to set this ???
 
-    NamedList<Object> resp = null;
+    CompletableFuture<NamedList<Object>> respFuture = null;
     try {
-      resp = sendRequest(request, inputCollections);
-      //to avoid an O(n) operation we always add STATE_VERSION to the last and try to read it from there
-      Object o = resp == null || resp.size() == 0 ? null : resp.get(STATE_VERSION, resp.size() - 1);
-      if(o != null && o instanceof Map) {
-        //remove this because no one else needs this and tests would fail if they are comparing responses
-        resp.remove(resp.size()-1);
-        @SuppressWarnings({"rawtypes"})
-        Map invalidStates = (Map) o;
-        for (Object invalidEntries : invalidStates.entrySet()) {
+      respFuture = sendRequest(request, inputCollections, isAsyncRequest);
+      if (!isAsyncRequest) {
+        NamedList<Object> resp = getNowOrException(respFuture);
+        //to avoid an O(n) operation we always add STATE_VERSION to the last and try to read it from there
+        Object o = resp == null || resp.size() == 0 ? null : resp.get(STATE_VERSION, resp.size() - 1);
+        if (o != null && o instanceof Map) {
+          //remove this because no one else needs this and tests would fail if they are comparing responses
+          resp.remove(resp.size() - 1);
           @SuppressWarnings({"rawtypes"})
-          Map.Entry e = (Map.Entry) invalidEntries;
-          getDocCollection((String) e.getKey(), (Integer) e.getValue());
+          Map invalidStates = (Map) o;
+          for (Object invalidEntries : invalidStates.entrySet()) {
+            @SuppressWarnings({"rawtypes"})
+            Map.Entry e = (Map.Entry) invalidEntries;
+            getDocCollection((String) e.getKey(), (Integer) e.getValue());

Review comment:
       Maybe we can move this to a new method?

##########
File path: solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java
##########
@@ -1167,8 +1191,38 @@ public RouteException(ErrorCode errorCode, NamedList<Throwable> throwables, Map<
     }
 
     LBSolrClient.Req req = new LBSolrClient.Req(request, theUrlList);
-    LBSolrClient.Rsp rsp = getLbClient().request(req);
-    return rsp.getResponse();
+    if (isAsyncRequest) {
+      CompletableFuture<LBSolrClient.Rsp> lbFuture = ((LBHttp2SolrClient) getLbClient()).requestAsync(req);
+      CompletableFuture<NamedList<Object>> apiFuture = new CompletableFuture<>();
+      lbFuture.whenComplete((result, throwable) -> {
+        if (lbFuture.isCompletedExceptionally()) {
+          apiFuture.completeExceptionally(throwable);
+        } else {
+          apiFuture.complete(result.getResponse());
+        }
+      });
+      return apiFuture;
+    } else {
+      LBSolrClient.Rsp rsp = getLbClient().request(req);
+      return CompletableFuture.completedFuture(rsp.getResponse());
+    }
+  }
+
+  private NamedList<Object> getNowOrException(CompletableFuture<NamedList<Object>> future) throws SolrServerException, IOException {
+    try {
+      return future.getNow(null);

Review comment:
       IIUC, we should never get `null` back in the current code. Maybe we could throw some `IllegalStateException` or `AssertionError` to furure-proof this?

##########
File path: solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java
##########
@@ -1167,8 +1191,38 @@ public RouteException(ErrorCode errorCode, NamedList<Throwable> throwables, Map<
     }
 
     LBSolrClient.Req req = new LBSolrClient.Req(request, theUrlList);
-    LBSolrClient.Rsp rsp = getLbClient().request(req);
-    return rsp.getResponse();
+    if (isAsyncRequest) {
+      CompletableFuture<LBSolrClient.Rsp> lbFuture = ((LBHttp2SolrClient) getLbClient()).requestAsync(req);

Review comment:
       Maybe we want to document somewhere that in order to support async requests, the internal lb client needs to extend `LBHttp2SolrClient`?

##########
File path: solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java
##########
@@ -476,8 +478,11 @@ public void registerDocCollectionWatcher(String collection, DocCollectionWatcher
     assertZKStateProvider().zkStateReader.registerDocCollectionWatcher(collection, watcher);
   }
 
+  // TODO: async direct updates

Review comment:
       I personally don't mind this approach, since it's internal. It makes things easier from the method caller side.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org