ActiveMQ Classic: Message prioritization affected by persistent/non-persistent

[Endre Stølsvik <En...@stolsvik.com>]

Hi!

This regards ActiveMQ Classic.

Setting:

   1. Prioritization enabled on broker with policyEntry
   2. 1x consumer for "Queue" with selector "JMSPriority = 9"
   3. 1x consumer for "Queue" with selector "JMSPriority <> 9"
   4. Send 10k messages to "Queue" in a transaction, with default pri 4
   5. Then, send 1 message to "Queue" in a transaction, with pri = 9

I expect the prioritized message (from #5) to arrive plenty before all the
10k standard messages (from #4) have arrived. That is, it should "overtake"
the existing messages on queue.

This also holds true!

But NOT if the prioritized message is set to nonPersistent! (That is, it
might still come in before all 10k is finished, but just barely. While if
it is left as persistent, then it comes very fast after being sent, while
the 10k takes their time).

Actually, the situation is that the expectation holds if the normal and
prioritized messages are both non-persistent, or both persistent. If they
are different, then it does not hold.

I have made a demo/test-case, but it currently looks like trash, so I was
just hoping that there was some immediate knowledge about this curious
observation!

Thanks,
Endre.

Re: ActiveMQ Classic: Message prioritization affected by persistent/non-persistent

[Matt Pavlovich <ma...@gmail.com>]

Hi Endre-

Pro-tip— using message priority falls down at scale like your 10k use case (using any broker product). ActiveMQ provides a really great feature to help you with this— a Composite Queue virtual destination with broker-side filtering. 

Producer sends to queue://data.input
Consumer 1 from queue://data.input.9
Consumer 2 from queue://data.input.other

queue://data.input 
   + priority = 9  -> queue://data.input.9
   + priority <> 9 -> queue://data.input.other

Documentation here:
https://activemq.apache.org/virtual-destinations
Hope this helps!

Thanks,
Matt Pavlovich

> On Jan 15, 2023, at 4:46 PM, Endre Stølsvik <En...@stolsvik.com> wrote:
> 
> Hi!
> 
> This regards ActiveMQ Classic.
> 
> Setting:
> 
>   1. Prioritization enabled on broker with policyEntry
>   2. 1x consumer for "Queue" with selector "JMSPriority = 9"
>   3. 1x consumer for "Queue" with selector "JMSPriority <> 9"
>   4. Send 10k messages to "Queue" in a transaction, with default pri 4
>   5. Then, send 1 message to "Queue" in a transaction, with pri = 9
> 
> I expect the prioritized message (from #5) to arrive plenty before all the
> 10k standard messages (from #4) have arrived. That is, it should "overtake"
> the existing messages on queue.
> 
> This also holds true!
> 
> But NOT if the prioritized message is set to nonPersistent! (That is, it
> might still come in before all 10k is finished, but just barely. While if
> it is left as persistent, then it comes very fast after being sent, while
> the 10k takes their time).
> 
> Actually, the situation is that the expectation holds if the normal and
> prioritized messages are both non-persistent, or both persistent. If they
> are different, then it does not hold.
> 
> I have made a demo/test-case, but it currently looks like trash, so I was
> just hoping that there was some immediate knowledge about this curious
> observation!
> 
> Thanks,
> Endre.