[GitHub] [pulsar] andrekramer1 commented on issue #5896: Out of Memory with many consumers

[GitBox <gi...@apache.org>]

andrekramer1 commented on issue #5896: Out of Memory with many consumers
URL: https://github.com/apache/pulsar/issues/5896#issuecomment-576363300
 
 
   The OOM appear to be in reading from Bookkeeper and possibly the broker is issuing too many read requests at a time. We've had some limited success with the dispatcher throttling introduced in 2.5. Using both:
   ./bin/pulsar-admin namespaces set-dispatch-rate public/default \
     --msg-dispatch-rate 500 \
     --dispatch-rate-period 1
   ./bin/pulsar-admin namespaces set-subscription-dispatch-rate public/default \
     --msg-dispatch-rate 50 \
     --dispatch-rate-period 1
   allows runs to not OOM most but not all of the time.
   
   A rate limit is difficult to use as the number of topics and subscriptions will not be known and we would have to plan for the worst case scenario. 
   
   One or more new limits on bookie parallel reads seem to be required? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services