[GitHub] merlimat commented on issue #402: Rate limiting in the dispatcher

[gi...@git.apache.org]

merlimat commented on issue #402: Rate limiting in the dispatcher
URL: https://github.com/apache/incubator-pulsar/issues/402#issuecomment-317092503
 
 
   @rdhabalia Can you add information on the implementation approach? How is the limit going to be enforced? 
   
   In all the cases, global-limit vs namespace limit, the policy apply to each topic. That doesn't include any per tenant, namespace or bundle enforcement.
   
   Another thought, should we differentiate between cached vs uncached reads? By throttling cached reads, we could bet set up for a much bigger amount of work later when we have to fetch it from bookies.
   
   > per-subscriber throttling
   
   I agree that different quotas per subscription are overkill, though it's not clear to me whether the configured limit is per-topic (eg: shared across all subscriptions) or applied to each individual subscription
   
   > Throttling threshold: Message-rate Vs Bytes-rate
   > Broker reads data from bookkeeper and dispatches it to consumer in form of message entity. 
   > Therefore, it makes more sense to define threshold as message-rate over bytes-rate.
   
   If the objective is to limit CPU usage, I agree. But if we're trying to protect network bandwidth, the the size should be considered.  
   I would say both should be allowed to apply. 
   
   
   I think that if the objective is to protect broker, another approach would be to a have a per-broker limit. When the limit is reached, the throttling will start applying to the heaviest users. Also, it would be interesting to integrate with load-manager so that heavy users can be potentially "kicked-out" on a isolated broker asap.
   
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services