Why do my messages get stuck (in StoreQueueCursor?)

[Michel Jung <mi...@gmail.com>]

Hi all,

My software uses prioritized messages and it's working just fine – until it
doesn't.Some unknown condition leads to my higher prioritized messages not
getting processed until I either remove at least a single message from the
queue, or until all other messages have been processed.

I posted this on StackOverflow (
https://stackoverflow.com/questions/63483621/activemq-why-do-my-messages-get-stuck-in-storequeuecursor
).

Is it ok if I just link it rather than repeating it here?

I'm quite desperate to solve this and would really appreciate it if someone
more competent in this area could point me in the right direction. Thank
you very much.

Michel

Re: Why do my messages get stuck (in StoreQueueCursor?)

[Michel Jung <mi...@gmail.com>]

Thank you for the links. So far I've failed to wrap my head around it,
unfortunately.

I read that setUseCache(false) and setExpireMessagesPeriod(0) and
prefetchPolicy.all=0 should achieve true priority dispatching, but they
didn't.

I have brokerService.setPersistent(false) so I'm not sure how "page in from
the store for dispatch" affects my case. But when I call
"removeMatchingMessage()" which calls "doPageIn(true)", my high priority
messages suddenly get processed so you're probably right.

When the case happened, I enabled debug logging and got this output:

    2020-09-17 13:29:00.864 DEBUG 8380 --- [ActiveMQ
BrokerService[localhost] Task-113] o.a.a.b.r.Queue
 : queue://task.standard, subscriptions=200, memory=2%, size=214678,
pending=0 toPageIn: 200, force:false, Inflight: 200, pagedInMessages.size
201070, pagedInPendingDispatch.size 200870, enqueueCount: 258397,
dequeueCount: 43719, memUsage:2322435451, maxPageSize:200

Can anyone see why this would cause my high priority messages not being
paged in?

Also I'd like to mention that I don't need "strict" dispatching; if a few
low-prio messages slip in before the high-prio ones, that's ok. But it
can't be that during hours, ten to hundreds of thousands low-prio messages
get dispatched first until all that's left are high-prio messages.

Any help is much appreciated!

On Thu, Aug 20, 2020 at 1:26 PM Gary Tully <ga...@gmail.com> wrote:

> There is a lot of relevant detail in
> https://issues.apache.org/jira/browse/AMQ-6215 and a test case that shows
> how to best achieve true priority dispatch. It is always a trade off
> between page in from the store for dispatch (or other reasons to have
> messages in memory) and prefetch to active consumers.
>
> note some of the scenarios in this test:
>
> https://git-wip-us.apache.org/repos/asf?p=activemq.git;a=blob;f=activemq-unit-tests/src/test/java/org/apache/activemq/usecases/QueueZeroPrefetchLazyDispatchPriorityTest.java
>
> that may help you understand what is in play and point to the best
> approach.
>
> On Thu, 20 Aug 2020 at 07:31, Michel Jung <mi...@gmail.com> wrote:
>
> > Hi all,
> >
> > My software uses prioritized messages and it's working just fine – until
> it
> > doesn't.Some unknown condition leads to my higher prioritized messages
> not
> > getting processed until I either remove at least a single message from
> the
> > queue, or until all other messages have been processed.
> >
> > I posted this on StackOverflow (
> >
> >
> https://stackoverflow.com/questions/63483621/activemq-why-do-my-messages-get-stuck-in-storequeuecursor
> > ).
> >
> > Is it ok if I just link it rather than repeating it here?
> >
> > I'm quite desperate to solve this and would really appreciate it if
> someone
> > more competent in this area could point me in the right direction. Thank
> > you very much.
> >
> > Michel
> >
>

Re: Why do my messages get stuck (in StoreQueueCursor?)

[Gary Tully <ga...@gmail.com>]

There is a lot of relevant detail in
https://issues.apache.org/jira/browse/AMQ-6215 and a test case that shows
how to best achieve true priority dispatch. It is always a trade off
between page in from the store for dispatch (or other reasons to have
messages in memory) and prefetch to active consumers.

note some of the scenarios in this test:
https://git-wip-us.apache.org/repos/asf?p=activemq.git;a=blob;f=activemq-unit-tests/src/test/java/org/apache/activemq/usecases/QueueZeroPrefetchLazyDispatchPriorityTest.java

that may help you understand what is in play and point to the best approach.

On Thu, 20 Aug 2020 at 07:31, Michel Jung <mi...@gmail.com> wrote:

> Hi all,
>
> My software uses prioritized messages and it's working just fine – until it
> doesn't.Some unknown condition leads to my higher prioritized messages not
> getting processed until I either remove at least a single message from the
> queue, or until all other messages have been processed.
>
> I posted this on StackOverflow (
>
> https://stackoverflow.com/questions/63483621/activemq-why-do-my-messages-get-stuck-in-storequeuecursor
> ).
>
> Is it ok if I just link it rather than repeating it here?
>
> I'm quite desperate to solve this and would really appreciate it if someone
> more competent in this area could point me in the right direction. Thank
> you very much.
>
> Michel
>