Making the Java broker cease processing Jobs temporarily

[Aidan Skinner <ai...@apache.org>]

Hi,

I ran across a situation where the broker behaves rather poorly. If a large
number of unprocessed Jobs and Events build up (such as a fast producer and
a slow message store) they can cause high memory usage, which causes the GC
to kick in a lot and the whole thing gets a bit messy.

I've written up a design for temporarily suspending the production of those
when memory is low, comments welcome here:
http://cwiki.apache.org/confluence/display/qpid/Broker+job+queue+limits

I'd prefer to keep the discussion on the wiki so it's all in one place.

Thanks,

- Aidan
-- 
Apache Qpid - World Domination through Advanced Message Queueing
http://cwiki.apache.org/qpid
"Nine-tenths of wisdom consists in being wise in time." - Theodore Roosevelt

Re: Making the Java broker cease processing Jobs temporarily

[Carl Trieloff <cc...@redhat.com>]

Aidan Skinner wrote:
> Hi,
>
> I ran across a situation where the broker behaves rather poorly. If a large
> number of unprocessed Jobs and Events build up (such as a fast producer and
> a slow message store) they can cause high memory usage, which causes the GC
> to kick in a lot and the whole thing gets a bit messy.
>
> I've written up a design for temporarily suspending the production of those
> when memory is low, comments welcome here:
> http://cwiki.apache.org/confluence/display/qpid/Broker+job+queue+limits
>
> I'd prefer to keep the discussion on the wiki so it's all in one place.
>
> Thanks,
>
> - Aidan
>   

The C++ broker has threshold for this case -- yes it is the BIG hammer. 
would be interested to
compare strategies.

Carl.