You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@zookeeper.apache.org by GitBox <gi...@apache.org> on 2020/01/16 19:57:15 UTC

[GitHub] [zookeeper] lvfangmin opened a new pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

lvfangmin opened a new pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] eolivelli commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#discussion_r367672640
 
 

 ##########
 File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/ProposalRequestProcessor.java
 ##########
 @@ -89,4 +103,14 @@ public void shutdown() {
         syncProcessor.shutdown();
     }
 
+    private boolean shouldForwardToNextProcessor(Request request) {
+        if (!forwardLearnerRequestsToCommitProcessorDisabled) {
+            return true;
+        }
+        if (request.getOwner() instanceof LearnerHandler) {
 
 Review comment:
   What about a more Object Oriented design like
   
   if (request.getOwner().allowForwardToCommitProcessor(request)).....
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] eolivelli commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#issuecomment-592088923
 
 
   @lvfangmin
   
   We already have two binding +1 (me and you)
   
   I can commit this patch tomorrow
   
   Any other opinion is very welcome as well

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] eolivelli commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#discussion_r367672640
 
 

 ##########
 File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/ProposalRequestProcessor.java
 ##########
 @@ -89,4 +103,14 @@ public void shutdown() {
         syncProcessor.shutdown();
     }
 
+    private boolean shouldForwardToNextProcessor(Request request) {
+        if (!forwardLearnerRequestsToCommitProcessorDisabled) {
+            return true;
+        }
+        if (request.getOwner() instanceof LearnerHandler) {
 
 Review comment:
   What about a more Object Oriented design like
   
   if (request.getOwner().allowForwardToCommitProcessor(request)).....
   
   Or even
   
   
   if (request.getOwner().allowForwardToProcessor(request, nextProcessor)).....

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] nkalmar edited a comment on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
nkalmar edited a comment on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#issuecomment-593871359
 
 
   Can you please rebase @lvfangmin , and ping me or Enrico to merge?
   (Plus fix the nits Michael mentioned, at least about the "prod" part removal.)
   Thanks!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] lvfangmin commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
lvfangmin commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#issuecomment-575429250
 
 
   retest maven please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] lvfangmin commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
lvfangmin commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#issuecomment-585470086
 
 
   retest maven please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] hanm commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
hanm commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#discussion_r385476974
 
 

 ##########
 File path: zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
 ##########
 @@ -1133,6 +1133,15 @@ property, when available, is noted below.
     effect due to TLS handshake timeout when there are too many in-flight TLS 
     handshakes. Set it to something like 250 is good enough to avoid herd effect.
 
+* *forward_learner_requests_to_commit_processor_disabled*
+    (Jave system property only: **zookeeper.forward_learner_requests_to_commit_processor_disabled**)
+    When this property is set, the requests from learners won't be enqueued to
+    CommitProcessor queue, which will help save the resources and GC time on 
+    leader.
+
+    The default value is false, will enable it by default after verified on prod.
 
 Review comment:
   Two nits:
   * I suspect that a config option with name "forward_learner_requests_to_commit_processor_disabled" in zoo.cfg will achieve the same effect of setting a java system property, so this is not precisely a "java system property only" (and i suspect we have a couple of other configuration options which claimed to be "java system property only" in the admin doc but they actually are not.). 
   
   * I think we can remove the "will enable it by default after verified on prod." part.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] nkalmar commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
nkalmar commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#issuecomment-593871359
 
 
   Can you please rebase @lvfangmin , and ping me or Enrico to merge?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] eolivelli commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#discussion_r367777405
 
 

 ##########
 File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/ProposalRequestProcessor.java
 ##########
 @@ -89,4 +103,14 @@ public void shutdown() {
         syncProcessor.shutdown();
     }
 
+    private boolean shouldForwardToNextProcessor(Request request) {
+        if (!forwardLearnerRequestsToCommitProcessorDisabled) {
+            return true;
+        }
+        if (request.getOwner() instanceof LearnerHandler) {
 
 Review comment:
   Makes sense

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] lvfangmin commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
lvfangmin commented on issue #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#issuecomment-591660062
 
 
   @hanm @anmolnar need another +1 here.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [zookeeper] lvfangmin commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor

Posted by GitBox <gi...@apache.org>.
lvfangmin commented on a change in pull request #1223: [ZOOKEEPER-3690] Improving leader efficiency via not processing learner forwarded requests in commit processor
URL: https://github.com/apache/zookeeper/pull/1223#discussion_r367736622
 
 

 ##########
 File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/ProposalRequestProcessor.java
 ##########
 @@ -89,4 +103,14 @@ public void shutdown() {
         syncProcessor.shutdown();
     }
 
+    private boolean shouldForwardToNextProcessor(Request request) {
+        if (!forwardLearnerRequestsToCommitProcessorDisabled) {
+            return true;
+        }
+        if (request.getOwner() instanceof LearnerHandler) {
 
 Review comment:
   That seems a little bit overkill to me, the request owner will be the 'me' Object defined in ServerCnxn or LearnerHandler. In order to do this, we need to define an interface which doesn't have other common behavior, which seems not necessary.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services