[GitHub] markusthoemmes closed pull request #3971: Bump fetch.max.wait of Kafka-consumers to lower the load on Kafka in large systems.

[GitBox <gi...@apache.org>]

markusthoemmes closed pull request #3971: Bump fetch.max.wait of Kafka-consumers to lower the load on Kafka in large systems.
URL: https://github.com/apache/incubator-openwhisk/pull/3971
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/common/scala/src/main/resources/application.conf b/common/scala/src/main/resources/application.conf
index d9e557dafd..0b92573b8b 100644
--- a/common/scala/src/main/resources/application.conf
+++ b/common/scala/src/main/resources/application.conf
@@ -81,10 +81,14 @@ whisk {
 
             max-poll-interval-ms = 1800000 // 30 minutes
 
-            // This value controls the server-side wait time which affects polling latency.
-            // A low value improves latency performance but it is important to not set it too low
-            // as that will cause excessive busy-waiting.
-            fetch-max-wait-ms = 20
+            // The maximum amount of time the server will block before answering
+            // the fetch request if there isn't sufficient data to immediately
+            // satisfy the requirement given by fetch.min.bytes.
+            // (default is 500, default of fetch.min.bytes is 1)
+            // On changing fetch.min.bytes, a high value for fetch.max.wait.ms,
+            // could increase latency of activations.
+            // A low value will cause excessive busy-waiting.
+            fetch-max-wait-ms = 500
         }
 
         topics {
diff --git a/core/invoker/src/main/scala/whisk/core/invoker/InvokerReactive.scala b/core/invoker/src/main/scala/whisk/core/invoker/InvokerReactive.scala
index f643a72d82..de9b4bcbb2 100644
--- a/core/invoker/src/main/scala/whisk/core/invoker/InvokerReactive.scala
+++ b/core/invoker/src/main/scala/whisk/core/invoker/InvokerReactive.scala
@@ -108,7 +108,7 @@ class InvokerReactive(
     maxPollInterval = TimeLimit.MAX_DURATION + 1.minute)
 
   private val activationFeed = actorSystem.actorOf(Props {
-    new MessageFeed("activation", logging, consumer, maximumContainers, 500.milliseconds, processActivationMessage)
+    new MessageFeed("activation", logging, consumer, maximumContainers, 1.second, processActivationMessage)
   })
 
   /** Sends an active-ack. */


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services