[GitHub] [zookeeper] hanm commented on a change in pull request #905: ZOOKEEPER-3359: Batch commits in the CommitProcessor

[GitBox <gi...@apache.org>]

hanm commented on a change in pull request #905:  ZOOKEEPER-3359: Batch commits in the CommitProcessor 
URL: https://github.com/apache/zookeeper/pull/905#discussion_r304686860
 
 

 ##########
 File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/CommitProcessor.java
 ##########
 @@ -248,91 +283,111 @@ public void run() {
                         break;
                     }
                 }
+                ServerMetrics.getMetrics().READS_ISSUED_IN_COMMIT_PROC.add(readsQueued);
+
+                if (!commitIsWaiting) {
+                    commitIsWaiting = !committedRequests.isEmpty();
+                }
 
-                // Handle a single committed request
-                if (commitIsWaiting && !stopped){
+                /*
+                 * Handle commits, if any.
+                 */
+                if (commitIsWaiting && !stopped) {
+                    /*
+                     * Drain outstanding reads
+                     */
                     waitForEmptyPool();
 
-                    if (stopped){
+                    if (stopped) {
                         return;
                     }
 
-                    // Process committed head
-                    if ((request = committedRequests.poll()) == null) {
-                        throw new IOException("Error: committed head is null");
-                    }
+                    int commitsToProcess = maxCommitBatchSize;
 
                     /*
-                     * Check if request is pending, if so, update it with the committed info
+                     * Loop through all the commits, and try to drain them.
                      */
-                    Deque<Request> sessionQueue = pendingRequests
-                            .get(request.sessionId);
-                    ServerMetrics.getMetrics().PENDING_SESSION_QUEUE_SIZE.add(pendingRequests.size());
-                    if (sessionQueue != null) {
-                        ServerMetrics.getMetrics().REQUESTS_IN_SESSION_QUEUE.add(sessionQueue.size());
-                        // If session queue != null, then it is also not empty.
-                        Request topPending = sessionQueue.poll();
-                        if (request.cxid != topPending.cxid) {
 
 Review comment:
   Thanks, this slightly makes sense to me now :)
   
   This check is performed on top item of `blockedRequestQueue` instead of on top item of `sessionQueue ` as the old code was doing. And later in this patch, we don't do another check on the top of `sessionQueue`:
   
   `Request topPending = sessionQueue.poll(); ...... ; topPending.setHdr(request.getHdr());` and so on...
   
   So for this to work, I guess the top of `blockedRequestQueue` must match the top of `sessionQueue ` (if they have same request id and cxid). In other words, the top of the both queue actually queued same request. This looks like the case for me, but just want to double check to make sure this is why we don't do another check later when poll items out of `sessionQueue`?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services