Max size for a group of queues

[Jeff Armstrong <ja...@avvasi.com>]

Is there a way to set a maximum size for a group of queues, or even all queues?

My dilemma is that I have several queues (lets say 8), and when the consumer application is off/disconnected for a while, these queues will fill up memory (lets say 8GB) fairly quickly. Currently I'm setting a maximum size per queue (eg. 1GB per queue * 8 queues), however the queues fill at different rates. My next approach would be to set the individual queue sizes according to how fast they fill relative to each other (eg. 4 queues @ 500MB max size and the faster filling 4 queues @ 1500MB max size), however the relative rates can also vary quite a lot.

What I want is:
1) Never use more than 8GB total
2) Maximize time before any individual queue starts overflowing so as to maximize the downtime of the consumer application without loss of messages

I was hoping there would be a way to just set a max size for all my queues so that they share the 8GB pool which should solve my problems. Or maybe there is another way to go about this?

Thanks for any help,
Jeff
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: Max size for a group of queues

[Gordon Sim <gs...@redhat.com>]

On 08/22/2011 11:12 PM, Jeff Armstrong wrote:
> Is there a way to set a maximum size for a group of queues, or even all queues?
>
> My dilemma is that I have several queues (lets say 8), and when the consumer application is off/disconnected for a while, these queues will fill up memory (lets say 8GB) fairly quickly. Currently I'm setting a maximum size per queue (eg. 1GB per queue * 8 queues), however the queues fill at different rates. My next approach would be to set the individual queue sizes according to how fast they fill relative to each other (eg. 4 queues @ 500MB max size and the faster filling 4 queues @ 1500MB max size), however the relative rates can also vary quite a lot.
>
> What I want is:
> 1) Never use more than 8GB total
> 2) Maximize time before any individual queue starts overflowing so as to maximize the downtime of the consumer application without loss of messages
>
> I was hoping there would be a way to just set a max size for all my queues so that they share the 8GB pool which should solve my problems. Or maybe there is another way to go about this?

Unfortunately there is no mechanism to do this at present. It would be a 
very useful enhancement however.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org