You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Arjen van der Meijden <ac...@tweakers.net> on 2009/06/08 15:40:03 UTC
Consumers newer than messages don't receive messages from queues
Hello list,
In a somewhat complicated set of consumption-processing, I have three
queues. One to indicate a new job, which is consumed by a single
long-running consumer that sends messages to a second queue.
That second queue is consumed by four consumers that kill themselves
after having processed a (few) message(s). After each message from the
second queue is processed, they're send to a third queue which has two
similar consumers.
As soon as such a consumer exits, a control process starts a new
consumer, to have a fixed amount of consumers.
The jobs for the initial queue are issued in batches, so there can be a
burst of about 150 messages, resulting in a burst in the second queue.
As soon as those second-queue-consumers exit however, they don't consume
much messages. Most of the time it seems that only one is actually doing
any work and the other three are idle, even if the queue has more than
100 messages. The same holds for the third-queue-consumers.
It seems to have to do with the fact that those consumers actually
subscribe to the queue after the messages have been produced. Obviously,
I want them to start consuming a message as soon as they connect.
All message production and consumption is via Stomp in PHP and the
ActiveMQ is a 5.2.0-version with mostly a default configuration. I have
tried to see if subscribing using the headers 'activemq.retroactive' and
'activemq.dispatchAsync' set to 'true' make any difference, but that
doesn't seem to do anything.
Anyone knows what's going on and how to fix it?
Best regards,
Arjen
Re: Consumers newer than messages don't receive messages from queues
Posted by Arjen van der Meijden <ac...@tweakers.net>.
On 8-6-2009 16:25 Aleksandar Ivanisevic wrote:
> Arjen van der Meijden
> <ac...@tweakers.net> writes:
>
>> As soon as those second-queue-consumers exit however, they don't
>> consume much messages. Most of the time it seems that only one is
>> actually doing any work and the other three are idle, even if the
>> queue has more than 100 messages. The same holds for the
>> third-queue-consumers.
>
> How sure are you that you are not subscribing exclusively?
Well, since you need to do (and/or configure) that explicitly. Very
sure. But as said in my follow-up mail, it seems to have been the
prefetchSize.
Best regards,
Arjen
Re: Consumers newer than messages don't receive messages from queues
Posted by Aleksandar Ivanisevic <al...@ivanisevic.de>.
Arjen van der Meijden
<ac...@tweakers.net> writes:
> As soon as those second-queue-consumers exit however, they don't
> consume much messages. Most of the time it seems that only one is
> actually doing any work and the other three are idle, even if the
> queue has more than 100 messages. The same holds for the
> third-queue-consumers.
How sure are you that you are not subscribing exclusively?
Re: Consumers newer than messages don't receive messages from queues
Posted by Arjen van der Meijden <ac...@tweakers.net>.
Well, I seem to have fixed it... I completely overlooked that the
default for the prefetch is actually larger than 1. Setting it to 1
fixes the problem. Still it seems to be a bit of a waste that several
newly connected consumers can be idle while because a "old" one that is
processing, is actually "locking" the other messages.
Best regards,
Arjen
On 8-6-2009 15:40, Arjen van der Meijden wrote:
> Hello list,
>
> In a somewhat complicated set of consumption-processing, I have three
> queues. One to indicate a new job, which is consumed by a single
> long-running consumer that sends messages to a second queue.
> That second queue is consumed by four consumers that kill themselves
> after having processed a (few) message(s). After each message from the
> second queue is processed, they're send to a third queue which has two
> similar consumers.
> As soon as such a consumer exits, a control process starts a new
> consumer, to have a fixed amount of consumers.
>
> The jobs for the initial queue are issued in batches, so there can be a
> burst of about 150 messages, resulting in a burst in the second queue.
>
> As soon as those second-queue-consumers exit however, they don't consume
> much messages. Most of the time it seems that only one is actually doing
> any work and the other three are idle, even if the queue has more than
> 100 messages. The same holds for the third-queue-consumers.
>
> It seems to have to do with the fact that those consumers actually
> subscribe to the queue after the messages have been produced. Obviously,
> I want them to start consuming a message as soon as they connect.
>
> All message production and consumption is via Stomp in PHP and the
> ActiveMQ is a 5.2.0-version with mostly a default configuration. I have
> tried to see if subscribing using the headers 'activemq.retroactive' and
> 'activemq.dispatchAsync' set to 'true' make any difference, but that
> doesn't seem to do anything.
>
> Anyone knows what's going on and how to fix it?
>
> Best regards,
>
> Arjen
>
>