Controlling VM transport queu size by count?

[DaveCC <dc...@gmail.com>]

I'm new to ActiveMQ and I'm trying to tune a very simple queue setup which
I'm relocating from disk to memory-only (VM transport, embedded broker,
launched from Spring). 

Versions below, but I game for changing them as needed:

ActiveMQ core 5.1.0
Spring core 2.5.6

I have about 12 queues, each has a single consumer and a handful of
transient producers (max of 6). Everything is in one 64-bit JVM. Some of the
messages are quite large (several MB) so I want to have a tight reign on
those queues, allowing only perhaps 8-9 messages in memory in total at any
one time.


1. I've been all over the ActiveMQ site and the draft of the forthcoming
book by Synder, Davies and Bosnac, but I can't find a comprehensive config
reference for ActiveMQ anywhere, and it seems that a number of the config
settings are not documented. Am I missing something? Should I buy the TTM
reference?


2. Is there a way to control the size of in-memory queues based on the
number of messages, rather than the size in megabytes?

Since my messages are never serialized, AMQ has no way to know how big they
are. I found one (and only one) reference to a parameter called
minimumMessageSize which - name notwithstanding - seems to be used as a
proxy for the size of an unserialized message when doing storage management
calculations (link below). 

http://activemq.apache.org/per-destination-policies.html

I looked at constantPendingMessageLimitStrategy but it seems to me that this
throws messages away, rather than blocking the producer, which is what I
want to do when the queue is full.

Is the canonical thing to do here to set minimumMessageSize to (e.g.) 1MB
and then use MB memory limits as a proxy for message count in the
<amq:memoryUsage> tags?


3. It seems to me that if I control memory size using the queue, then I
should set the pre-fetch size to 0 or 1 to prevent prefetch from holding
additional messages - is this correct, or do prefetched messages also remain
in the queue?


4. Is there a way to completely control memory usage with only the prefetch
settings?


Cheers
Dave
-- 
View this message in context: http://old.nabble.com/Controlling-VM-transport-queu-size-by-count--tp27218974p27218974.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Controlling VM transport queu size by count?

[Joe Fernandez <jo...@ttmsolutions.com>]

I would strongly recommend moving up to 5.3. A LOT of issues have been
addressed since 5.1. 

AFAIK, you cannot control the size of in-memory queues based on the number
of messages. If that is indeed the case, you could submit a JIRA requesting
this feature. 

If these are transient messages and you want to block the producer when the
queue is full, set producerFlowControl to true, which I believe is the
default setting for 5.3. If these are transient,  producerFlowControl is
false, and you're not using the vm message cursor, they'll overflow into the
temp store. However, see AMQ-2475 which has been addressed in 5.3.1 and 5.4.
If these are persistent messages, they'll overflow into the store. 

Prefetched messages will remain in the queue until the consumer has
acknowledged them.

Joe
http://www.ttmsolutions.com   


DaveCC wrote:
> 
> I'm new to ActiveMQ and I'm trying to tune a very simple queue setup which
> I'm relocating from disk to memory-only (VM transport, embedded broker,
> launched from Spring). 
> 
> Versions below, but I game for changing them as needed:
> 
> ActiveMQ core 5.1.0
> Spring core 2.5.6
> 
> I have about 12 queues, each has a single consumer and a handful of
> transient producers (max of 6). Everything is in one 64-bit JVM. Some of
> the messages are quite large (several MB) so I want to have a tight reign
> on those queues, allowing only perhaps 8-9 messages in memory in total at
> any one time.
> 
> 
> 1. I've been all over the ActiveMQ site and the draft of the forthcoming
> book by Synder, Davies and Bosnac, but I can't find a comprehensive config
> reference for ActiveMQ anywhere, and it seems that a number of the config
> settings are not documented. Am I missing something? Should I buy the TTM
> reference?
> 
> 
> 2. Is there a way to control the size of in-memory queues based on the
> number of messages, rather than the size in megabytes?
> 
> Since my messages are never serialized, AMQ has no way to know how big
> they are. I found one (and only one) reference to a parameter called
> minimumMessageSize which - name notwithstanding - seems to be used as a
> proxy for the size of an unserialized message when doing storage
> management calculations (link below). 
> 
> http://activemq.apache.org/per-destination-policies.html
> 
> I looked at constantPendingMessageLimitStrategy but it seems to me that
> this throws messages away, rather than blocking the producer, which is
> what I want to do when the queue is full.
> 
> Is the canonical thing to do here to set minimumMessageSize to (e.g.) 1MB
> and then use MB memory limits as a proxy for message count in the
> <amq:memoryUsage> tags?
> 
> 
> 3. It seems to me that if I control memory size using the queue, then I
> should set the pre-fetch size to 0 or 1 to prevent prefetch from holding
> additional messages - is this correct, or do prefetched messages also
> remain in the queue?
> 
> 
> 4. Is there a way to completely control memory usage with only the
> prefetch settings?
> 
> 
> Cheers
> Dave
> 

-- 
View this message in context: http://old.nabble.com/Controlling-VM-transport-queu-size-by-count--tp27218974p27225389.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.