Re: ActiveMQ: producer flow control and temp store

[Tim Bain <tb...@alumni.duke.edu>]

Art,

I've never seen a way to do that, but it sounds like a great idea.

Would you be willing to submit an enhancement request for it in JIRA?

Thanks,
Tim

On Thu, Aug 9, 2018, 8:54 AM art.licis <ar...@gmail.com> wrote:

> Dear community,
>
> We want to enable producer flow control to protect a broker from running
> out
> of memory if slow topic consumers appear (we had OOMs through several runs
> on our staging environment, and so far PFC off is one of the versions why
> this has happened). However, we would want to keep our previous approach:
> if
> memory not enough, use temp store for our topic messages so that the
> producer keeps producing and fast consumers are up-to-date. So we expect
> (almost) no backpressure if ActiveMQ has enough CPU and IO to write those
> messages to disk.
>
> I can see that simply enabling producer flow control keeps utilizing temp
> store:
>
>
> <destinationPolicy>
>   <policyMap>
>     <policyEntries>
>       <policyEntry topic=">" producerFlowControl="true">
>
>
> <systemUsage>
>   <systemUsage>
>     <memoryUsage>
>       <memoryUsage limit="2 gb"/>
>     </memoryUsage>
>     <storeUsage>
>       <storeUsage limit="100 gb"/>
>     </storeUsage>
>     <tempUsage>
>       <tempUsage limit="10 gb"/>
>     </tempUsage>
>   </systemUsage>
> </systemUsage>
>
>
> However, our concerns are that single case of slow consumer can flood most
> of the available heap with messages before flowing to temp store. The
> following might seem as a perfect solution:
> *<policyEntry topic=">" producerFlowControl="true" memoryLimit="20 mb">*
>
> However, with this configuration ActiveMQ start throttling producers as
> soon
> as topic memory limit reached, without going to temp store.
>
> Any chance there's some trick to limit memory per destination, and at the
> same time keep using temp store?
>
> Thank you,
> Art
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>

Re: ActiveMQ: producer flow control and temp store

["art.licis" <ar...@gmail.com>]

Hi Tim,

Sorry, I'm a bit late with this :)
It's great you like the idea! I'll do my quick research and I'd love to
contribute, so I'll make a JIRA and hopefully a PR.

- Art



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html