You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by YoungSoul <yo...@inbox.com> on 2007/01/15 22:29:21 UTC
Messages 'stuck' on queue, until client restart
I seem to be having problems with 'stuck' messages on a queue.
I am running with the latest ActiveMQ release. I have noticed this both
when the server runs on Windows or Linux.
My message frequency is very low, around 1 message every 10 seconds and the
messages are small.
For this test scenario, I have one multi-threaded client pulling from a
single queue. I start my test producer to send a single small text message
once every 10 seconds. I have a multi-threaded ( 5 threads ) client
listening on the queue and just prints the message ID so I can see that they
are not all being consumed. Each thread creates its own transacted Session.
There is a connection pool of 3 connections which is shared across the 5
threads, in this case.
As messages are pulled from the queue, the message ID is printed and that is
all for now. I can see blocks of messages not being consumed.
There seems to be a pattern of the number of missed messages to the number
of client threads. For example message 60, 66,
67,68,69,70,76,77,78,79,80,86,87,88,89,90,96, etc are pulled. And as I
mentioned I have 5 client threads.
I tried to set the prefetch size like:
queue.simpleMsgQueue=simpleMsgRcvQueue?consumer.prefetchSize=1
which I then use like:
session =
connectionFactory.createConnection().createSession(true,
Session.SESSION_TRANSACTED);
Queue readQueue = new ActiveMQQueue(readQueueName);
messageConsumer = session.createConsumer(readQueue);
my processing loop looks like:
public void run() {
while( isRunning() ) {
try {
receiveSync(); // basically does consumer.receive(5000)
session.commit();
} catch( ShutdownException se ) {
log.error(se.getMessage(),se);
} catch(Exception e ) {
log.error(e.getMessage(),e);
session.rollback();
}
}
}
I also started the server with the following config:
<policyEntry queue="simpleMsgRcvQueue">
<dispatchPolicy>
<roundRobinDispatchPolicy/>
</dispatchPolicy>
</policyEntry>
The messages are not persistent. I can run the browse command and see the
messages in the server.
The client was written to call receive(5000); timeout because the client has
to check from time to time to see if it needs to do something else so the
client threads pull from the server, the server does not push to the
clients.
If I restart my client, the client will pick up the messages upon startup.
I saw a bug report about non-delivered queue messages but it said it was
fixed in 4.1. Does anyone know if that is in the latest binary download?
Any suggestions on why I can try?
Thanks in advance for any help and thoughts.
--
View this message in context: http://www.nabble.com/Messages-%27stuck%27-on-queue%2C-until-client-restart-tf3017349.html#a8379662
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Messages 'stuck' on queue, until client restart
Posted by YoungSoul <yo...@inbox.com>.
If it helps, I was tailing the console output from the client as it prints
the message IDs and eventually the messages were consumed from the queue -
albeit late and out of order. Below is a snippet from the log file. As you
can, we are pulling messages in order, and then jump from message id: 3235
to message id: 2313
2007-01-16 11:21:30,827 [Thread-4] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3229
2007-01-16 11:21:32,870 [Thread-5] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3230
2007-01-16 11:21:34,913 [Thread-5] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3231
2007-01-16 11:21:36,956 [Thread-5] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3232
2007-01-16 11:21:38,999 [Thread-5] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3233
2007-01-16 11:21:41,042 [Thread-5] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3234
2007-01-16 11:21:43,085 [Thread-5] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3235
2007-01-16 11:21:43,755 [Thread-1] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:2313
2007-01-16 11:21:43,755 [Thread-2] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:2310
2007-01-16 11:21:43,755 [Thread-3] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:2312
2007-01-16 11:21:43,755 [Thread-4] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:2311
2007-01-16 11:21:43,755 [Thread-5] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:2314
2007-01-16 11:21:45,127 [Thread-1] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3236
2007-01-16 11:21:47,170 [Thread-2] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3237
2007-01-16 11:21:49,203 [Thread-3] SimpleMessageConsumer called with message
id: ID:pryan-1360-1168961497076-0:0:1:1:3238
YoungSoul wrote:
>
>
> I am not using the ResourceAdapter. This application is basically one
> that I had written back in the day with version 3, and I now have the
> opportunity to reuse most of that application and I was trying to get it
> to work with the latest ActiveMQ.
>
> So I know this did work in the 3.x version.
>
> I basically followed the 'CompetingConsumer' pattern. I developed them as
> Spring Beans and my afterPropertiesSet method looks like: ( btw: the
> connectionFactory which is a simple wrapper will make sure that the
> connection is started as part of the create. I am also using pooled
> connections so a client app might decide to have a few connections shared
> by any number of sessions. Some minor logging and error checking removed)
>
> public void afterPropertiesSet() throws Exception {
>
> Connection conn = connectionFactory.createConnection();
> if( conn != null ) {
> session = conn.createSession(true,
> Session.SESSION_TRANSACTED);
>
> Queue readQueue = session.createQueue(readQueueName);
> messageConsumer = session.createConsumer(readQueue);
> if( writeQueueNames != null ) {
> for (Iterator iter = writeQueueNames.iterator();
> iter.hasNext();) {
> String writeQueueName = (String) iter.next();
> Queue writeQueue = session.createQueue(writeQueueName);
> MessageProducer mp = session.createProducer(writeQueue);
> mp.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
> messageProducers.add(mp);
> queue2MessageProducerMap.put(writeQueueName,mp);
> }
> }
>
> if( deadQueueName != null ) {
> Queue deadLetter = session.createQueue(deadQueueName);
> deadMessageQueue = session.createProducer(deadLetter);
>
> deadMessageQueue.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
> }
> initialized = true;
> }
>
> }
>
> I did create a junit to receive messages and a sender tool ( modeled after
> your producertool ) to recreate this. In this case, I am sending one
> message every 2 seconds from a windows client, to a Linux AMQ server, to a
> consumer running on the same windows machine. After about 2300 messages,
> I started seeing messages left in the queue.
>
> I would be happy to zip the entire eclipse project and make that available
> if that is helpful. This might certainly be something in my
> implementation or configuration, so I will continue to try to reduce the
> test case to the bare minimum that I can.
>
> Thanks again.
>
> Pat
>
>
>
> Christopher G. Stach II wrote:
>>
>> YoungSoul wrote:
>>>
>>> To be specific about the release I used:
>>>
>>> I went to the following url:
>>> http://www.activemq.org/site/activemq-410-release.html
>>> and downloaded either apache-activemq-4.1.0-incubator.zip or
>>> apache-activemq-4.1.0-incubator.tar.gz depending upon the environment I
>>> was
>>> interested in.
>>>
>>> Thanks for your reply and help you can provide.
>>>
>>> Pat
>>>
>>>
>>>
>>>
>>> James.Strachan wrote:
>>>> On 1/15/07, YoungSoul <yo...@inbox.com> wrote:
>>>>> I seem to be having problems with 'stuck' messages on a queue.
>>>>>
>>>>> I am running with the latest ActiveMQ release.
>>>> By 'latest release' are you talking about 4.1? There were some fixes
>>>> in this area in 4.1 - could you be more specific please?
>>>>
>>>> --
>>>>
>>>> James
>>>> -------
>>>> http://radio.weblogs.com/0112098/
>>>>
>>>>
>>>
>>
>> Are you using AMQ's ResourceAdapter?
>>
>> --
>> Christopher G. Stach II
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Messages-%27stuck%27-on-queue%2C-until-client-restart-tf3017349.html#a8394883
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Messages 'stuck' on queue, until client restart
Posted by YoungSoul <yo...@inbox.com>.
I am not using the ResourceAdapter. This application is basically one that
I had written back in the day with version 3, and I now have the opportunity
to reuse most of that application and I was trying to get it to work with
the latest ActiveMQ.
So I know this did work in the 3.x version.
I basically followed the 'CompetingConsumer' pattern. I developed them as
Spring Beans and my afterPropertiesSet method looks like: ( btw: the
connectionFactory which is a simple wrapper will make sure that the
connection is started as part of the create. I am also using pooled
connections so a client app might decide to have a few connections shared by
any number of sessions. Some minor logging and error checking removed)
public void afterPropertiesSet() throws Exception {
Connection conn = connectionFactory.createConnection();
if( conn != null ) {
session = conn.createSession(true, Session.SESSION_TRANSACTED);
Queue readQueue = session.createQueue(readQueueName);
messageConsumer = session.createConsumer(readQueue);
if( writeQueueNames != null ) {
for (Iterator iter = writeQueueNames.iterator();
iter.hasNext();) {
String writeQueueName = (String) iter.next();
Queue writeQueue = session.createQueue(writeQueueName);
MessageProducer mp = session.createProducer(writeQueue);
mp.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
messageProducers.add(mp);
queue2MessageProducerMap.put(writeQueueName,mp);
}
}
if( deadQueueName != null ) {
Queue deadLetter = session.createQueue(deadQueueName);
deadMessageQueue = session.createProducer(deadLetter);
deadMessageQueue.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
}
initialized = true;
}
}
I did create a junit to receive messages and a sender tool ( modeled after
your producertool ) to recreate this. In this case, I am sending one
message every 2 seconds from a windows client, to a Linux AMQ server, to a
consumer running on the same windows machine. After about 2300 messages, I
started seeing messages left in the queue.
I would be happy to zip the entire eclipse project and make that available
if that is helpful. This might certainly be something in my implementation
or configuration, so I will continue to try to reduce the test case to the
bare minimum that I can.
Thanks again.
Pat
Christopher G. Stach II wrote:
>
> YoungSoul wrote:
>>
>> To be specific about the release I used:
>>
>> I went to the following url:
>> http://www.activemq.org/site/activemq-410-release.html
>> and downloaded either apache-activemq-4.1.0-incubator.zip or
>> apache-activemq-4.1.0-incubator.tar.gz depending upon the environment I
>> was
>> interested in.
>>
>> Thanks for your reply and help you can provide.
>>
>> Pat
>>
>>
>>
>>
>> James.Strachan wrote:
>>> On 1/15/07, YoungSoul <yo...@inbox.com> wrote:
>>>> I seem to be having problems with 'stuck' messages on a queue.
>>>>
>>>> I am running with the latest ActiveMQ release.
>>> By 'latest release' are you talking about 4.1? There were some fixes
>>> in this area in 4.1 - could you be more specific please?
>>>
>>> --
>>>
>>> James
>>> -------
>>> http://radio.weblogs.com/0112098/
>>>
>>>
>>
>
> Are you using AMQ's ResourceAdapter?
>
> --
> Christopher G. Stach II
>
>
>
--
View this message in context: http://www.nabble.com/Messages-%27stuck%27-on-queue%2C-until-client-restart-tf3017349.html#a8394580
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Messages 'stuck' on queue, until client restart
Posted by "Christopher G. Stach II" <cg...@ldsys.net>.
YoungSoul wrote:
>
> To be specific about the release I used:
>
> I went to the following url:
> http://www.activemq.org/site/activemq-410-release.html
> and downloaded either apache-activemq-4.1.0-incubator.zip or
> apache-activemq-4.1.0-incubator.tar.gz depending upon the environment I was
> interested in.
>
> Thanks for your reply and help you can provide.
>
> Pat
>
>
>
>
> James.Strachan wrote:
>> On 1/15/07, YoungSoul <yo...@inbox.com> wrote:
>>> I seem to be having problems with 'stuck' messages on a queue.
>>>
>>> I am running with the latest ActiveMQ release.
>> By 'latest release' are you talking about 4.1? There were some fixes
>> in this area in 4.1 - could you be more specific please?
>>
>> --
>>
>> James
>> -------
>> http://radio.weblogs.com/0112098/
>>
>>
>
Are you using AMQ's ResourceAdapter?
--
Christopher G. Stach II
Re: Messages 'stuck' on queue, until client restart
Posted by YoungSoul <yo...@inbox.com>.
To be specific about the release I used:
I went to the following url:
http://www.activemq.org/site/activemq-410-release.html
and downloaded either apache-activemq-4.1.0-incubator.zip or
apache-activemq-4.1.0-incubator.tar.gz depending upon the environment I was
interested in.
Thanks for your reply and help you can provide.
Pat
James.Strachan wrote:
>
> On 1/15/07, YoungSoul <yo...@inbox.com> wrote:
>>
>> I seem to be having problems with 'stuck' messages on a queue.
>>
>> I am running with the latest ActiveMQ release.
>
> By 'latest release' are you talking about 4.1? There were some fixes
> in this area in 4.1 - could you be more specific please?
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
>
--
View this message in context: http://www.nabble.com/Messages-%27stuck%27-on-queue%2C-until-client-restart-tf3017349.html#a8390723
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Messages 'stuck' on queue, until client restart
Posted by James Strachan <ja...@gmail.com>.
On 1/15/07, YoungSoul <yo...@inbox.com> wrote:
>
> I seem to be having problems with 'stuck' messages on a queue.
>
> I am running with the latest ActiveMQ release.
By 'latest release' are you talking about 4.1? There were some fixes
in this area in 4.1 - could you be more specific please?
--
James
-------
http://radio.weblogs.com/0112098/