[GitHub] [pulsar] Gleiphir2769 commented on pull request #15216: [PIP-74] Support consumer client memory limit

["Gleiphir2769 (via GitHub)" <gi...@apache.org>]

Gleiphir2769 commented on PR #15216:
URL: https://github.com/apache/pulsar/pull/15216#issuecomment-1446360203

   > > Do we need to introduce a publish/consume memory ratio? In most cases, users are consuming messages from the consumer and publish new messages, and ack the consumed messages. I'm worried about if the consumer can use up the memory, the producer not able to publish, so that the consumer queue can't be cleanup, it looks like a deadlock.
   > 
   > Yes, good point. Sharing memory quota does have this kind of isolation problem. Not only between producers and consumers, but also between different consumers. E.g. one consumer use up the memory, other consumers may not be able to receive any message any more.
   > 
   > Currently, I suggest to **enable this feature within a new client instance.**
   > 
   > I am not sure it's the right time to just split publish/consume memory quota, by ratio or other ways. One alternative is to reduce local message buffer directly and make broker resend message later. This may require some change in protocols. Just a brief thought yet. Let's dig deeper in anther PIP(like solving isolation problem when sharing producers and consumers in one client instance).
   > 
   > @codelipenghui
   
   Hi @Jason918, I find this discussion from https://github.com/apache/pulsar-client-go/issues/927. Just some question I want to figure out.
   Why not let the producer/consumer have their own `MemoryLimitController`? And split the `memoryLimit` setting into `consumerMemoryLimit` and `producerMemoryLimit`.  Are there any other considerations?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org