PolicyEntry parse order and constantPendingMessageLimitStrategy and message counts

[Matthew Patton <MP...@CalAmp.com>]

For all the documentation out there I can't find any mention as to parse order of activeMQ.xml as it relates to policy entries.

Is it first match? Last match? Most specific match? For example:

                <policyEntry memoryLimit="64mb" queue="PumpQueue" producerFlowControl="false">
                  <pendingMessageLimitStrategy>
                    <constantPendingMessageLimitStrategy limit="175000"/>
                  </pendingMessageLimitStrategy>
                </policyEntry>
...
                <policyEntry memoryLimit="16mb" producerFlowControl="true" queue="&gt;">
                </policyEntry>

Are my PumpQueue settings being overridden by the wildcard entry that comes later?

Also if the limit is 175000 why does Pending Messages (admin/queues.jsp) keep climbing way past that number? Are the messages really being discarded as intended but it's just that there isn't a metric for discard? How does one verify correct behavior?

RE: PolicyEntry parse order and constantPendingMessageLimitStrategy and message counts

[Tim Bain <tb...@alumni.duke.edu>]

So it sounds like there's a bug in the web console's pending message count;
can you submit a bug in JIRA?  Also look at whether the same bug exists in
the equivalent property in JMX and describe what you find in the bug
description.

Add a (separate) enhancement request in JIRA for being able to see store
usage stats at a per-destination level in JMX, and another for exposing the
number of messages discarded because of the pending message limit.

I wasn't clear about how you were defining the PendingLimitCount and
PendingLimitTime attributes you were proposing; can you explain them?
On Mar 6, 2015 3:06 PM, "Matthew Patton" <MP...@calamp.com> wrote:

> > If the stat is right, you'd see the # bytes used in the store climbing as
> > well.  (That stat is available in JMX, but I'm not sure it's on the web
> > console.)
>
> Except I've got a couple dozen queues and they all end up writing to disk
> (overflowing in-memory limits). So aggregate disk (or store) usage is
> useless to differentiate behavior. In one particular mostly controlled
> setting I can see that as messages continue to come in the Store percent
> used has stopped climbing after the queues hit their PendingLimit. So maybe
> it's working as intended?
>
> I'm looking at the MBean Attributes for the queue and nothing indicates
> disk/store usage. I've got metrics for Dequeue, Dispatch, Enqueue,
> QueueSize, Memory Usage (again not useful), and Cursor usage among others.
> When messages roll off, does it show up in 'ExpiredCount'? That's the only
> one that even vaguely fits. And to answer my question, nope, it doesn't
> move an iota. It probably only increments when one sets a time-based expiry
> setting.
>
> Shouldn't I also be able to tell how many (or portion) of all enqueued
> messages are sitting in memory vs the backing store? I'm using the Storage
> cursor as opposed to VM or File. My MemoryPercentUsed and
> CursorPercentUsage both top out at 70 which is the default.
>
> It seems to me there are some glaring oversights to the MBeans attributes.
> We need a PendingLimitCount, PendingLimitTime, DiscardCount (or
> PendingLimitExceededCount) or just use the ExpiredCount for both time-based
> and count-based limits. And too a 'StorageUsageCount' which reflects the
> number of messages that have been spooled out to disk in excess of
> in-memory footprint or if counting messages is too much work, at least the
> Bytes consumed on disk.
>
>

RE: PolicyEntry parse order and constantPendingMessageLimitStrategy and message counts

[Matthew Patton <MP...@CalAmp.com>]

> If the stat is right, you'd see the # bytes used in the store climbing as
> well.  (That stat is available in JMX, but I'm not sure it's on the web
> console.)

Except I've got a couple dozen queues and they all end up writing to disk (overflowing in-memory limits). So aggregate disk (or store) usage is useless to differentiate behavior. In one particular mostly controlled setting I can see that as messages continue to come in the Store percent used has stopped climbing after the queues hit their PendingLimit. So maybe it's working as intended?

I'm looking at the MBean Attributes for the queue and nothing indicates disk/store usage. I've got metrics for Dequeue, Dispatch, Enqueue, QueueSize, Memory Usage (again not useful), and Cursor usage among others. When messages roll off, does it show up in 'ExpiredCount'? That's the only one that even vaguely fits. And to answer my question, nope, it doesn't move an iota. It probably only increments when one sets a time-based expiry setting. 

Shouldn't I also be able to tell how many (or portion) of all enqueued messages are sitting in memory vs the backing store? I'm using the Storage cursor as opposed to VM or File. My MemoryPercentUsed and CursorPercentUsage both top out at 70 which is the default.

It seems to me there are some glaring oversights to the MBeans attributes. We need a PendingLimitCount, PendingLimitTime, DiscardCount (or PendingLimitExceededCount) or just use the ExpiredCount for both time-based and count-based limits. And too a 'StorageUsageCount' which reflects the number of messages that have been spooled out to disk in excess of in-memory footprint or if counting messages is too much work, at least the Bytes consumed on disk.

Re: PolicyEntry parse order and constantPendingMessageLimitStrategy and message counts

[Tim Bain <tb...@alumni.duke.edu>]

If the stat is right, you'd see the # bytes used in the store climbing as
well.  (That stat is available in JMX, but I'm not sure it's on the web
console.)
On Mar 6, 2015 11:52 AM, "Matthew Patton" <MP...@calamp.com> wrote:

> For all the documentation out there I can't find any mention as to parse
> order of activeMQ.xml as it relates to policy entries.
>
> Is it first match? Last match? Most specific match? For example:
>
>                 <policyEntry memoryLimit="64mb" queue="PumpQueue"
> producerFlowControl="false">
>                   <pendingMessageLimitStrategy>
>                     <constantPendingMessageLimitStrategy limit="175000"/>
>                   </pendingMessageLimitStrategy>
>                 </policyEntry>
> ...
>                 <policyEntry memoryLimit="16mb" producerFlowControl="true"
> queue="&gt;">
>                 </policyEntry>
>
> Are my PumpQueue settings being overridden by the wildcard entry that
> comes later?
>
> Also if the limit is 175000 why does Pending Messages (admin/queues.jsp)
> keep climbing way past that number? Are the messages really being discarded
> as intended but it's just that there isn't a metric for discard? How does
> one verify correct behavior?
>
>