You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by chrajanirao <ra...@gmail.com> on 2008/11/12 20:50:23 UTC
Stuck messages - Dispatch issues
We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message dispatching
from queues. It is easily reproducible even using the out of the box
activemq configuration.
Send messages to the a queue (QueueA) using multiple threads (10 or 20
thread in a loop of 10) using JMeter or your custom code. Have the consumer
setup using spring DMLC (DefaultMessageListenerContainer) or with regular
JMS API sync or async consumption with multiple consumers. Consumer part can
be configured using Camel to consumer from QueueA and put the messages in
QueueB within ActiveMQ configuration as well.
After receiving some messages (the number is different each time), the
consumers stop receiving any messages even though there are some left on the
queue. Basically, the broker don't dispatch the messages and they are stuck
until restart of the broker. Any new messages sent to the queue after this
are sometimes dispatched and other times they are stuck too.
This certainly seems like a major bug in the dispatch mechanism. I have
found below posts that state the exact problem.
http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
This is a very basic use case and I wonder how the version 5.1 is currently
used in production, if anyone is using at all.
We tried with prefetch limit as 1, asyncDispatch as true and false, session
transacted as true and false. In all cases, the dispatch problem still
exists starting after 100 messages until 1000 messages.
I hope any of the active commiters looks into this issue seriously. Would
really appreciate the help.
Thanks,
Rajani.
--
View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20467949.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Pavel <pa...@gmail.com>.
Sounds scary to me, as system I'm working on uses activemq extensively, and
is not production-proven yet.
Do you have/can you build a junit test case that reproduces the issue under
5.3? If so, that looks like a good start for Jira defect.
Disclaimer - I'm not an AMQ committer.
Thanks,
Pavel
On Sat, Feb 20, 2010 at 9:23 AM, Elliot Barlas <el...@gmail.com>wrote:
>
> Unfortunately I am seeing it in 5.3. I just posted on this more current
> thread as well:
>
> http://old.nabble.com/50k-%2B-messages-stuck-in-queue-with-all-consumers-blocking-on-receive-td27162095.html
>
> Thanks,
> Elliot
>
>
>
> rajdavies wrote:
> >
> > This should be resolved in 5.3
> > On 20 Feb 2010, at 07:12, Elliot Barlas wrote:
> >
> >>
> >> Was this issue ever resolved? I am seeing this as well with a large
> >> number
> >> of concurrent consumers. Same symptoms. Lost messages until broker
> >> restart.
> >>
> >> Thanks,
> >> Elliot
> >>
> >>
> >>
> >> chrajanirao wrote:
> >>>
> >>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> >>> dispatching from queues. It is easily reproducible even using the
> >>> out of
> >>> the box activemq configuration.
> >>>
> >>> Send messages to the a queue (QueueA) using multiple threads (10 or
> >>> 20
> >>> thread in a loop of 10) using JMeter or your custom code. Have the
> >>> consumer setup using spring DMLC (DefaultMessageListenerContainer)
> >>> or with
> >>> regular JMS API sync or async consumption with multiple consumers.
> >>> Consumer part can be configured using Camel to consumer from QueueA
> >>> and
> >>> put the messages in QueueB within ActiveMQ configuration as well.
> >>>
> >>> After receiving some messages (the number is different each time),
> >>> the
> >>> consumers stop receiving any messages even though there are some
> >>> left on
> >>> the queue. Basically, the broker don't dispatch the messages and
> >>> they are
> >>> stuck until restart of the broker. Any new messages sent to the queue
> >>> after this are sometimes dispatched and other times they are stuck
> >>> too.
> >>>
> >>> This certainly seems like a major bug in the dispatch mechanism. I
> >>> have
> >>> found below posts that state the exact problem.
> >>>
> >>>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> >>>
> >>>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> >>>
> >>> This is a very basic use case and I wonder how the version 5.1 is
> >>> currently used in production, if anyone is using at all.
> >>>
> >>> We tried with prefetch limit as 1, asyncDispatch as true and false,
> >>> session transacted as true and false. In all cases, the dispatch
> >>> problem
> >>> still exists starting after 100 messages until 1000 messages.
> >>>
> >>> I hope any of the active commiters looks into this issue seriously.
> >>> Would
> >>> really appreciate the help.
> >>>
> >>> Thanks,
> >>> Rajani.
> >>>
> >>
> >> --
> >> View this message in context:
> >>
> http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >
> > Rob Davies
> > http://twitter.com/rajdavies
> > I work here: http://fusesource.com
> > My Blog: http://rajdavies.blogspot.com/
> > I'm writing this: http://www.manning.com/snyder/
> >
> >
> >
> >
> >
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664182.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: Stuck messages - Dispatch issues
Posted by Elliot Barlas <el...@gmail.com>.
Unfortunately I am seeing it in 5.3. I just posted on this more current
thread as well:
http://old.nabble.com/50k-%2B-messages-stuck-in-queue-with-all-consumers-blocking-on-receive-td27162095.html
Thanks,
Elliot
rajdavies wrote:
>
> This should be resolved in 5.3
> On 20 Feb 2010, at 07:12, Elliot Barlas wrote:
>
>>
>> Was this issue ever resolved? I am seeing this as well with a large
>> number
>> of concurrent consumers. Same symptoms. Lost messages until broker
>> restart.
>>
>> Thanks,
>> Elliot
>>
>>
>>
>> chrajanirao wrote:
>>>
>>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>>> dispatching from queues. It is easily reproducible even using the
>>> out of
>>> the box activemq configuration.
>>>
>>> Send messages to the a queue (QueueA) using multiple threads (10 or
>>> 20
>>> thread in a loop of 10) using JMeter or your custom code. Have the
>>> consumer setup using spring DMLC (DefaultMessageListenerContainer)
>>> or with
>>> regular JMS API sync or async consumption with multiple consumers.
>>> Consumer part can be configured using Camel to consumer from QueueA
>>> and
>>> put the messages in QueueB within ActiveMQ configuration as well.
>>>
>>> After receiving some messages (the number is different each time),
>>> the
>>> consumers stop receiving any messages even though there are some
>>> left on
>>> the queue. Basically, the broker don't dispatch the messages and
>>> they are
>>> stuck until restart of the broker. Any new messages sent to the queue
>>> after this are sometimes dispatched and other times they are stuck
>>> too.
>>>
>>> This certainly seems like a major bug in the dispatch mechanism. I
>>> have
>>> found below posts that state the exact problem.
>>>
>>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>>
>>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>>
>>> This is a very basic use case and I wonder how the version 5.1 is
>>> currently used in production, if anyone is using at all.
>>>
>>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>>> session transacted as true and false. In all cases, the dispatch
>>> problem
>>> still exists starting after 100 messages until 1000 messages.
>>>
>>> I hope any of the active commiters looks into this issue seriously.
>>> Would
>>> really appreciate the help.
>>>
>>> Thanks,
>>> Rajani.
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
>
--
View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664182.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Rob Davies <ra...@gmail.com>.
This should be resolved in 5.3
On 20 Feb 2010, at 07:12, Elliot Barlas wrote:
>
> Was this issue ever resolved? I am seeing this as well with a large
> number
> of concurrent consumers. Same symptoms. Lost messages until broker
> restart.
>
> Thanks,
> Elliot
>
>
>
> chrajanirao wrote:
>>
>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>> dispatching from queues. It is easily reproducible even using the
>> out of
>> the box activemq configuration.
>>
>> Send messages to the a queue (QueueA) using multiple threads (10 or
>> 20
>> thread in a loop of 10) using JMeter or your custom code. Have the
>> consumer setup using spring DMLC (DefaultMessageListenerContainer)
>> or with
>> regular JMS API sync or async consumption with multiple consumers.
>> Consumer part can be configured using Camel to consumer from QueueA
>> and
>> put the messages in QueueB within ActiveMQ configuration as well.
>>
>> After receiving some messages (the number is different each time),
>> the
>> consumers stop receiving any messages even though there are some
>> left on
>> the queue. Basically, the broker don't dispatch the messages and
>> they are
>> stuck until restart of the broker. Any new messages sent to the queue
>> after this are sometimes dispatched and other times they are stuck
>> too.
>>
>> This certainly seems like a major bug in the dispatch mechanism. I
>> have
>> found below posts that state the exact problem.
>>
>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>
>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>
>> This is a very basic use case and I wonder how the version 5.1 is
>> currently used in production, if anyone is using at all.
>>
>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>> session transacted as true and false. In all cases, the dispatch
>> problem
>> still exists starting after 100 messages until 1000 messages.
>>
>> I hope any of the active commiters looks into this issue seriously.
>> Would
>> really appreciate the help.
>>
>> Thanks,
>> Rajani.
>>
>
> --
> View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Stuck messages - Dispatch issues
Posted by Elliot Barlas <el...@gmail.com>.
Yes. I am using spring 3.0, but not for JMS. I manage my own connections,
sessions, producers, and consumers. I have tried reproducing the issue in a
test case, but I cannot. It only occurs in my application, when I have
roughly 20 or more concurrent consumers, each associated with a different
session. I am using the VM transport exclusively, as all communication
takes place with an embedded AMQ message broker.
Thanks,
Elliot
rajdavies wrote:
>
> Do you still get problems if not using Spring ?
>
> On 20 Feb 2010, at 07:12, Elliot Barlas wrote:
>
>>
>> Was this issue ever resolved? I am seeing this as well with a large
>> number
>> of concurrent consumers. Same symptoms. Lost messages until broker
>> restart.
>>
>> Thanks,
>> Elliot
>>
>>
>>
>> chrajanirao wrote:
>>>
>>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>>> dispatching from queues. It is easily reproducible even using the
>>> out of
>>> the box activemq configuration.
>>>
>>> Send messages to the a queue (QueueA) using multiple threads (10 or
>>> 20
>>> thread in a loop of 10) using JMeter or your custom code. Have the
>>> consumer setup using spring DMLC (DefaultMessageListenerContainer)
>>> or with
>>> regular JMS API sync or async consumption with multiple consumers.
>>> Consumer part can be configured using Camel to consumer from QueueA
>>> and
>>> put the messages in QueueB within ActiveMQ configuration as well.
>>>
>>> After receiving some messages (the number is different each time),
>>> the
>>> consumers stop receiving any messages even though there are some
>>> left on
>>> the queue. Basically, the broker don't dispatch the messages and
>>> they are
>>> stuck until restart of the broker. Any new messages sent to the queue
>>> after this are sometimes dispatched and other times they are stuck
>>> too.
>>>
>>> This certainly seems like a major bug in the dispatch mechanism. I
>>> have
>>> found below posts that state the exact problem.
>>>
>>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>>
>>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>>
>>> This is a very basic use case and I wonder how the version 5.1 is
>>> currently used in production, if anyone is using at all.
>>>
>>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>>> session transacted as true and false. In all cases, the dispatch
>>> problem
>>> still exists starting after 100 messages until 1000 messages.
>>>
>>> I hope any of the active commiters looks into this issue seriously.
>>> Would
>>> really appreciate the help.
>>>
>>> Thanks,
>>> Rajani.
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
>
--
View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27669784.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
RE: Reply:RE: Stuck messages - Dispatch issues
Posted by swplotner <sw...@amherst.edu>.
Hi - thank you for pointing this out - that was it. I was able to follow the blog's description to the dot and it solved the problem - yeah - we can go into production with this . Thank you all for listening and helping out.
Steffen
_______________________________________________________________________________________________
Steffen Plotner Amherst College Tel (413) 542-2348
Systems/Network Administrator/Programmer PO BOX 5000 Fax (413) 542-2626
Systems & Networking Amherst, MA 01002-5000 swplotner@amherst.edu
From: SuoNayi [via ActiveMQ] [mailto:ml-node+s2283324n4661704h75@n4.nabble.com]
Sent: Sunday, January 13, 2013 8:33 PM
To: Steffen Plotner
Subject: Reply:RE: Stuck messages - Dispatch issues
Hi,there is a well-known reason about stuck messages when brokers are networked.
You may take a look at http://activemq.apache.org/networks-of-brokers.html
At 2013-01-13 00:29:01,swplotner <[hidden email]</user/SendEmail.jtp?type=node&node=4661704&i=0>> wrote:
>Hi
>
>Good point, I actually do have the same network config on the other node2 pointing to node1. However, I have now full duplex enabled and still have the same problem.
>
>Steffen
>
>________________________________
>From: ceposta [via ActiveMQ] [[hidden email]</user/SendEmail.jtp?type=node&node=4661704&i=1>]
>Sent: Saturday, January 12, 2013 09:57
>To: Steffen Plotner
>Subject: Re: Stuck messages - Dispatch issues
>
>Looks like your network connectors are in one direction only, ie, node1 -->
>node2
>What this means is messages will only flow across the network FROM node1 TO
>node2 based on the demand (consumers) on node2. They will not flow the
>other way unless you set up a bridge in the other direction (node2 -->
>node1) or use duplex="true" on your existing network connectors. Not sure
>what you mean when you say restarting the brokers fixes what you were
>seeing.
>
>
>On Fri, Jan 11, 2013 at 8:43 PM, swplotner <[hidden email]<UrlBlockedError.aspx>> wrote:
>
>> Hi,
>>
>> Using 5.7.0 I find that the problem exists in the following way:
>>
>> 2 brokers connected via network.
>>
>> Broker A
>> has consumer connected to a /queue/foo.test
>>
>> Broker B
>> has a producer connected to /queue/foo.test
>>
>> the producer is producing message for /queue/foo.test and the consumer will
>> receive them just fine. The producer is configured to stuff messages into
>> the queue one after the other, the consumer on purpose consumes messages
>> slowly.
>>
>> Once we have several hundred messages in /queue/foo.test we abort the
>> producer and consumer.
>>
>> Looking at queue sizes on both brokers, broker A has a large number of
>> messages and broker B has some number of messages or none.
>>
>> Take the consumer that was previously talking via broker A and connect him
>> to broker B. That consumer will now only see the messages that are in
>> broker
>> B and will NOT receive ANY of the messages stuck in broker A.
>>
>> The only way I can get out of this problem is to restart both brokers and
>> all messages are then process correctly.
>>
>> I have spent quite a bit of time on this problem and if there is only one
>> broker this is not a problem - the moment you have a network of 2 brokers
>> this problem exists.
>>
>> Here are my network connectors, one for topics and one for queues:
>>
>> <networkConnectors>
>> <networkConnector name="net-node1-node2-queues"
>> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>> networkTTL="3"
>> dynamicOnly="true"
>> prefetchSize="1"
>> duplex="false"
>> userName="${netlink.username}"
>> password="${netlink.password}"
>> conduitSubscriptions="false"
>> >
>> <dynamicallyIncludedDestinations>
>> <queue physicalName=">"/>
>> </dynamicallyIncludedDestinations>
>> </networkConnector>
>>
>> <networkConnector name="net-node1-node2-topics"
>> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>> networkTTL="3"
>> dynamicOnly="true"
>> prefetchSize="1"
>> duplex="false"
>> userName="${netlink.username}"
>> password="${netlink.password}"
>> conduitSubscriptions="false"
>> >
>> <dynamicallyIncludedDestinations>
>> <topic physicalName=">"/>
>> </dynamicallyIncludedDestinations>
>> </networkConnector>
>> </networkConnectors>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
>http://www.christianposta.com/blog
>
>
>________________________________
>If you reply to this email, your message will be added to the discussion below:
>
>NAML<http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
>--
>View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661694.html
>Sent from the ActiveMQ - User mailing list archive at Nabble.com.
________________________________
If you reply to this email, your message will be added to the discussion below:
http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661704.html
To unsubscribe from Stuck messages - Dispatch issues, click here<http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2367852&code=c3dwbG90bmVyQGFtaGVyc3QuZWR1fDIzNjc4NTJ8LTMxMzAxNjU4OA==>.
NAML<http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661705.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply:RE: Stuck messages - Dispatch issues
Posted by SuoNayi <su...@163.com>.
Hi,there is a well-known reason about stuck messages when brokers are networked.
You may take a look at http://activemq.apache.org/networks-of-brokers.html
At 2013-01-13 00:29:01,swplotner <sw...@amherst.edu> wrote:
>Hi
>
>Good point, I actually do have the same network config on the other node2 pointing to node1. However, I have now full duplex enabled and still have the same problem.
>
>Steffen
>
>________________________________
>From: ceposta [via ActiveMQ] [ml-node+s2283324n4661693h34@n4.nabble.com]
>Sent: Saturday, January 12, 2013 09:57
>To: Steffen Plotner
>Subject: Re: Stuck messages - Dispatch issues
>
>Looks like your network connectors are in one direction only, ie, node1 -->
>node2
>What this means is messages will only flow across the network FROM node1 TO
>node2 based on the demand (consumers) on node2. They will not flow the
>other way unless you set up a bridge in the other direction (node2 -->
>node1) or use duplex="true" on your existing network connectors. Not sure
>what you mean when you say restarting the brokers fixes what you were
>seeing.
>
>
>On Fri, Jan 11, 2013 at 8:43 PM, swplotner <[hidden email]<UrlBlockedError.aspx>> wrote:
>
>> Hi,
>>
>> Using 5.7.0 I find that the problem exists in the following way:
>>
>> 2 brokers connected via network.
>>
>> Broker A
>> has consumer connected to a /queue/foo.test
>>
>> Broker B
>> has a producer connected to /queue/foo.test
>>
>> the producer is producing message for /queue/foo.test and the consumer will
>> receive them just fine. The producer is configured to stuff messages into
>> the queue one after the other, the consumer on purpose consumes messages
>> slowly.
>>
>> Once we have several hundred messages in /queue/foo.test we abort the
>> producer and consumer.
>>
>> Looking at queue sizes on both brokers, broker A has a large number of
>> messages and broker B has some number of messages or none.
>>
>> Take the consumer that was previously talking via broker A and connect him
>> to broker B. That consumer will now only see the messages that are in
>> broker
>> B and will NOT receive ANY of the messages stuck in broker A.
>>
>> The only way I can get out of this problem is to restart both brokers and
>> all messages are then process correctly.
>>
>> I have spent quite a bit of time on this problem and if there is only one
>> broker this is not a problem - the moment you have a network of 2 brokers
>> this problem exists.
>>
>> Here are my network connectors, one for topics and one for queues:
>>
>> <networkConnectors>
>> <networkConnector name="net-node1-node2-queues"
>> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>> networkTTL="3"
>> dynamicOnly="true"
>> prefetchSize="1"
>> duplex="false"
>> userName="${netlink.username}"
>> password="${netlink.password}"
>> conduitSubscriptions="false"
>> >
>> <dynamicallyIncludedDestinations>
>> <queue physicalName=">"/>
>> </dynamicallyIncludedDestinations>
>> </networkConnector>
>>
>> <networkConnector name="net-node1-node2-topics"
>> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>> networkTTL="3"
>> dynamicOnly="true"
>> prefetchSize="1"
>> duplex="false"
>> userName="${netlink.username}"
>> password="${netlink.password}"
>> conduitSubscriptions="false"
>> >
>> <dynamicallyIncludedDestinations>
>> <topic physicalName=">"/>
>> </dynamicallyIncludedDestinations>
>> </networkConnector>
>> </networkConnectors>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
>http://www.christianposta.com/blog
>
>
>________________________________
>If you reply to this email, your message will be added to the discussion below:
>http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661693.html
>To unsubscribe from Stuck messages - Dispatch issues, click here<http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2367852&code=c3dwbG90bmVyQGFtaGVyc3QuZWR1fDIzNjc4NTJ8LTMxMzAxNjU4OA==>.
>NAML<http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
>--
>View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661694.html
>Sent from the ActiveMQ - User mailing list archive at Nabble.com.
RE: Stuck messages - Dispatch issues
Posted by swplotner <sw...@amherst.edu>.
Hi
Good point, I actually do have the same network config on the other node2 pointing to node1. However, I have now full duplex enabled and still have the same problem.
Steffen
________________________________
From: ceposta [via ActiveMQ] [ml-node+s2283324n4661693h34@n4.nabble.com]
Sent: Saturday, January 12, 2013 09:57
To: Steffen Plotner
Subject: Re: Stuck messages - Dispatch issues
Looks like your network connectors are in one direction only, ie, node1 -->
node2
What this means is messages will only flow across the network FROM node1 TO
node2 based on the demand (consumers) on node2. They will not flow the
other way unless you set up a bridge in the other direction (node2 -->
node1) or use duplex="true" on your existing network connectors. Not sure
what you mean when you say restarting the brokers fixes what you were
seeing.
On Fri, Jan 11, 2013 at 8:43 PM, swplotner <[hidden email]<UrlBlockedError.aspx>> wrote:
> Hi,
>
> Using 5.7.0 I find that the problem exists in the following way:
>
> 2 brokers connected via network.
>
> Broker A
> has consumer connected to a /queue/foo.test
>
> Broker B
> has a producer connected to /queue/foo.test
>
> the producer is producing message for /queue/foo.test and the consumer will
> receive them just fine. The producer is configured to stuff messages into
> the queue one after the other, the consumer on purpose consumes messages
> slowly.
>
> Once we have several hundred messages in /queue/foo.test we abort the
> producer and consumer.
>
> Looking at queue sizes on both brokers, broker A has a large number of
> messages and broker B has some number of messages or none.
>
> Take the consumer that was previously talking via broker A and connect him
> to broker B. That consumer will now only see the messages that are in
> broker
> B and will NOT receive ANY of the messages stuck in broker A.
>
> The only way I can get out of this problem is to restart both brokers and
> all messages are then process correctly.
>
> I have spent quite a bit of time on this problem and if there is only one
> broker this is not a problem - the moment you have a network of 2 brokers
> this problem exists.
>
> Here are my network connectors, one for topics and one for queues:
>
> <networkConnectors>
> <networkConnector name="net-node1-node2-queues"
> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
> networkTTL="3"
> dynamicOnly="true"
> prefetchSize="1"
> duplex="false"
> userName="${netlink.username}"
> password="${netlink.password}"
> conduitSubscriptions="false"
> >
> <dynamicallyIncludedDestinations>
> <queue physicalName=">"/>
> </dynamicallyIncludedDestinations>
> </networkConnector>
>
> <networkConnector name="net-node1-node2-topics"
> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
> networkTTL="3"
> dynamicOnly="true"
> prefetchSize="1"
> duplex="false"
> userName="${netlink.username}"
> password="${netlink.password}"
> conduitSubscriptions="false"
> >
> <dynamicallyIncludedDestinations>
> <topic physicalName=">"/>
> </dynamicallyIncludedDestinations>
> </networkConnector>
> </networkConnectors>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
http://www.christianposta.com/blog
________________________________
If you reply to this email, your message will be added to the discussion below:
http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661693.html
To unsubscribe from Stuck messages - Dispatch issues, click here<http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2367852&code=c3dwbG90bmVyQGFtaGVyc3QuZWR1fDIzNjc4NTJ8LTMxMzAxNjU4OA==>.
NAML<http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661694.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Christian Posta <ch...@gmail.com>.
Looks like your network connectors are in one direction only, ie, node1 -->
node2
What this means is messages will only flow across the network FROM node1 TO
node2 based on the demand (consumers) on node2. They will not flow the
other way unless you set up a bridge in the other direction (node2 -->
node1) or use duplex="true" on your existing network connectors. Not sure
what you mean when you say restarting the brokers fixes what you were
seeing.
On Fri, Jan 11, 2013 at 8:43 PM, swplotner <sw...@amherst.edu> wrote:
> Hi,
>
> Using 5.7.0 I find that the problem exists in the following way:
>
> 2 brokers connected via network.
>
> Broker A
> has consumer connected to a /queue/foo.test
>
> Broker B
> has a producer connected to /queue/foo.test
>
> the producer is producing message for /queue/foo.test and the consumer will
> receive them just fine. The producer is configured to stuff messages into
> the queue one after the other, the consumer on purpose consumes messages
> slowly.
>
> Once we have several hundred messages in /queue/foo.test we abort the
> producer and consumer.
>
> Looking at queue sizes on both brokers, broker A has a large number of
> messages and broker B has some number of messages or none.
>
> Take the consumer that was previously talking via broker A and connect him
> to broker B. That consumer will now only see the messages that are in
> broker
> B and will NOT receive ANY of the messages stuck in broker A.
>
> The only way I can get out of this problem is to restart both brokers and
> all messages are then process correctly.
>
> I have spent quite a bit of time on this problem and if there is only one
> broker this is not a problem - the moment you have a network of 2 brokers
> this problem exists.
>
> Here are my network connectors, one for topics and one for queues:
>
> <networkConnectors>
> <networkConnector name="net-node1-node2-queues"
> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
> networkTTL="3"
> dynamicOnly="true"
> prefetchSize="1"
> duplex="false"
> userName="${netlink.username}"
> password="${netlink.password}"
> conduitSubscriptions="false"
> >
> <dynamicallyIncludedDestinations>
> <queue physicalName=">"/>
> </dynamicallyIncludedDestinations>
> </networkConnector>
>
> <networkConnector name="net-node1-node2-topics"
> uri="static:(ssl://activemq-node2.amherst.edu:61615)"
> networkTTL="3"
> dynamicOnly="true"
> prefetchSize="1"
> duplex="false"
> userName="${netlink.username}"
> password="${netlink.password}"
> conduitSubscriptions="false"
> >
> <dynamicallyIncludedDestinations>
> <topic physicalName=">"/>
> </dynamicallyIncludedDestinations>
> </networkConnector>
> </networkConnectors>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Re: Stuck messages - Dispatch issues
Posted by swplotner <sw...@amherst.edu>.
Hi,
Using 5.7.0 I find that the problem exists in the following way:
2 brokers connected via network.
Broker A
has consumer connected to a /queue/foo.test
Broker B
has a producer connected to /queue/foo.test
the producer is producing message for /queue/foo.test and the consumer will
receive them just fine. The producer is configured to stuff messages into
the queue one after the other, the consumer on purpose consumes messages
slowly.
Once we have several hundred messages in /queue/foo.test we abort the
producer and consumer.
Looking at queue sizes on both brokers, broker A has a large number of
messages and broker B has some number of messages or none.
Take the consumer that was previously talking via broker A and connect him
to broker B. That consumer will now only see the messages that are in broker
B and will NOT receive ANY of the messages stuck in broker A.
The only way I can get out of this problem is to restart both brokers and
all messages are then process correctly.
I have spent quite a bit of time on this problem and if there is only one
broker this is not a problem - the moment you have a network of 2 brokers
this problem exists.
Here are my network connectors, one for topics and one for queues:
<networkConnectors>
<networkConnector name="net-node1-node2-queues"
uri="static:(ssl://activemq-node2.amherst.edu:61615)"
networkTTL="3"
dynamicOnly="true"
prefetchSize="1"
duplex="false"
userName="${netlink.username}"
password="${netlink.password}"
conduitSubscriptions="false"
>
<dynamicallyIncludedDestinations>
<queue physicalName=">"/>
</dynamicallyIncludedDestinations>
</networkConnector>
<networkConnector name="net-node1-node2-topics"
uri="static:(ssl://activemq-node2.amherst.edu:61615)"
networkTTL="3"
dynamicOnly="true"
prefetchSize="1"
duplex="false"
userName="${netlink.username}"
password="${netlink.password}"
conduitSubscriptions="false"
>
<dynamicallyIncludedDestinations>
<topic physicalName=">"/>
</dynamicallyIncludedDestinations>
</networkConnector>
</networkConnectors>
--
View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by "lernen.2007" <er...@googlemail.com>.
We tested with activemq version 5.4.2. The message stuck with this version
too. I think a jms provider in which the messages stuck is not useable. The
activemq team should stop to fix another problems and begin to analyse what
the reason for this problem is and deliver a patch as soon as possible.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p3446638.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by nnprasad <NN...@dwd.IN.gov>.
We have been experiencing the same Issue and using 5.2.0.
Recently upgraded to 5.4.2 because of this, But I am Not Sure this has been
fixed.
Responding to Rob Davis Post..
I think nothing to suspect with Spring here, because when this is happening
I tried hitting the same queue with a new simple consumer(AMQ connections
and NO Spring JMS). This Simple consumer is also waiting on the newly sent
message. created a new queue and subscribed to it while this is happening.
New Producer - new Queue - New Single Consumer = No Receive Happened.
Useful Links:
https://jira.springsource.org/browse/SPR-5110
https://issues.apache.org/jira/browse/AMQ-2009
I created an Issue with good links
https://issues.apache.org/jira/browse/AMQ-3225
I am really afraid of 5.4.2 because Once I saw in it...but Some one also
reported the same in 5.4.2
http://activemq.2283324.n4.nabble.com/Message-stuck-on-queue-td3338786.html#a3354797
We are planning to go to production with 5.4.2, Its a public project related
to claims, what if they were not processed because of activeMQ not
delivering messages ?????????????????? :-(
Please take this as serious.
Thank You,
Nag. P
--
View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p3445066.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by "lernen.2007" <er...@googlemail.com>.
Hi,
the problem we have too. Sometimes the messages stuck in queue and after a
restart it deliver to the consumer.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p3443852.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Rob Davies <ra...@gmail.com>.
Do you still get problems if not using Spring ?
On 20 Feb 2010, at 07:12, Elliot Barlas wrote:
>
> Was this issue ever resolved? I am seeing this as well with a large
> number
> of concurrent consumers. Same symptoms. Lost messages until broker
> restart.
>
> Thanks,
> Elliot
>
>
>
> chrajanirao wrote:
>>
>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>> dispatching from queues. It is easily reproducible even using the
>> out of
>> the box activemq configuration.
>>
>> Send messages to the a queue (QueueA) using multiple threads (10 or
>> 20
>> thread in a loop of 10) using JMeter or your custom code. Have the
>> consumer setup using spring DMLC (DefaultMessageListenerContainer)
>> or with
>> regular JMS API sync or async consumption with multiple consumers.
>> Consumer part can be configured using Camel to consumer from QueueA
>> and
>> put the messages in QueueB within ActiveMQ configuration as well.
>>
>> After receiving some messages (the number is different each time),
>> the
>> consumers stop receiving any messages even though there are some
>> left on
>> the queue. Basically, the broker don't dispatch the messages and
>> they are
>> stuck until restart of the broker. Any new messages sent to the queue
>> after this are sometimes dispatched and other times they are stuck
>> too.
>>
>> This certainly seems like a major bug in the dispatch mechanism. I
>> have
>> found below posts that state the exact problem.
>>
>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>
>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>
>> This is a very basic use case and I wonder how the version 5.1 is
>> currently used in production, if anyone is using at all.
>>
>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>> session transacted as true and false. In all cases, the dispatch
>> problem
>> still exists starting after 100 messages until 1000 messages.
>>
>> I hope any of the active commiters looks into this issue seriously.
>> Would
>> really appreciate the help.
>>
>> Thanks,
>> Rajani.
>>
>
> --
> View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Stuck messages - Dispatch issues
Posted by Elliot Barlas <el...@gmail.com>.
Was this issue ever resolved? I am seeing this as well with a large number
of concurrent consumers. Same symptoms. Lost messages until broker
restart.
Thanks,
Elliot
chrajanirao wrote:
>
> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> dispatching from queues. It is easily reproducible even using the out of
> the box activemq configuration.
>
> Send messages to the a queue (QueueA) using multiple threads (10 or 20
> thread in a loop of 10) using JMeter or your custom code. Have the
> consumer setup using spring DMLC (DefaultMessageListenerContainer) or with
> regular JMS API sync or async consumption with multiple consumers.
> Consumer part can be configured using Camel to consumer from QueueA and
> put the messages in QueueB within ActiveMQ configuration as well.
>
> After receiving some messages (the number is different each time), the
> consumers stop receiving any messages even though there are some left on
> the queue. Basically, the broker don't dispatch the messages and they are
> stuck until restart of the broker. Any new messages sent to the queue
> after this are sometimes dispatched and other times they are stuck too.
>
> This certainly seems like a major bug in the dispatch mechanism. I have
> found below posts that state the exact problem.
>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>
> This is a very basic use case and I wonder how the version 5.1 is
> currently used in production, if anyone is using at all.
>
> We tried with prefetch limit as 1, asyncDispatch as true and false,
> session transacted as true and false. In all cases, the dispatch problem
> still exists starting after 100 messages until 1000 messages.
>
> I hope any of the active commiters looks into this issue seriously. Would
> really appreciate the help.
>
> Thanks,
> Rajani.
>
--
View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Joe Fernandez <jo...@ttmsolutions.com>.
What are the queue's "DequeueCount" and "InFlightCount" attributes
registering on the jconsole?
Joe
Get a free ActiveMQ user guide @ http://www.ttmsolutions.com
chrajanirao wrote:
>
> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> dispatching from queues. It is easily reproducible even using the out of
> the box activemq configuration.
>
> Send messages to the a queue (QueueA) using multiple threads (10 or 20
> thread in a loop of 10) using JMeter or your custom code. Have the
> consumer setup using spring DMLC (DefaultMessageListenerContainer) or with
> regular JMS API sync or async consumption with multiple consumers.
> Consumer part can be configured using Camel to consumer from QueueA and
> put the messages in QueueB within ActiveMQ configuration as well.
>
> After receiving some messages (the number is different each time), the
> consumers stop receiving any messages even though there are some left on
> the queue. Basically, the broker don't dispatch the messages and they are
> stuck until restart of the broker. Any new messages sent to the queue
> after this are sometimes dispatched and other times they are stuck too.
>
> This certainly seems like a major bug in the dispatch mechanism. I have
> found below posts that state the exact problem.
>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>
> This is a very basic use case and I wonder how the version 5.1 is
> currently used in production, if anyone is using at all.
>
> We tried with prefetch limit as 1, asyncDispatch as true and false,
> session transacted as true and false. In all cases, the dispatch problem
> still exists starting after 100 messages until 1000 messages.
>
> I hope any of the active commiters looks into this issue seriously. Would
> really appreciate the help.
>
> Thanks,
> Rajani.
>
--
View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20469492.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
RE: Stuck messages - Dispatch issues
Posted by Ma...@sungard.com.
Hi Jacques,
I've just recently logged the following issue:
https://issues.apache.org/activemq/browse/AMQ-2332
It sounds exactly the same as the original email in this thread, but it
is however different from the other threads the original poster linked
to.
Basically it's possible to deadlock queues, and they will then not
deliver or accept any more messages until the broker is restarted.
Regards,
Mats
> -----Original Message-----
> From: couzteau [mailto:couzteau@bitfaeule.net]
> Sent: Thursday, July 30, 2009 2:57 AM
> To: users@activemq.apache.org
> Subject: Re: Stuck messages - Dispatch issues
>
>
> Has this been solved yet? I'm seeing it with 5.2.0.
>
> I'm seeing an issue that occurs on some machines (where machines are
> identical regarding OS, Java version and hardware).
>
> Consumers that are on the same machine as the producer usually work
fine.
>
> It's a major blocker for us - any comments highly appreciated.
>
>
> TIA Jacques
>
>
>
>
> chrajanirao wrote:
> >
> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> > dispatching from queues. It is easily reproducible even using the
out of
> > the box activemq configuration.
> >
> > Send messages to the a queue (QueueA) using multiple threads (10 or
20
> > thread in a loop of 10) using JMeter or your custom code. Have the
> > consumer setup using spring DMLC (DefaultMessageListenerContainer)
or
> with
> > regular JMS API sync or async consumption with multiple consumers.
> > Consumer part can be configured using Camel to consumer from QueueA
and
> > put the messages in QueueB within ActiveMQ configuration as well.
> >
> > After receiving some messages (the number is different each time),
the
> > consumers stop receiving any messages even though there are some
left on
> > the queue. Basically, the broker don't dispatch the messages and
they
> are
> > stuck until restart of the broker. Any new messages sent to the
queue
> > after this are sometimes dispatched and other times they are stuck
too.
> >
> > This certainly seems like a major bug in the dispatch mechanism. I
have
> > found below posts that state the exact problem.
> >
> > http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-
> td20241332.html
> >
> > http://www.nabble.com/Consumer-Listener-stop-receving-message-until-
> ActiveMQ-restart-td20355247.html
> >
> > This is a very basic use case and I wonder how the version 5.1 is
> > currently used in production, if anyone is using at all.
> >
> > We tried with prefetch limit as 1, asyncDispatch as true and false,
> > session transacted as true and false. In all cases, the dispatch
problem
> > still exists starting after 100 messages until 1000 messages.
> >
> > I hope any of the active commiters looks into this issue seriously.
> Would
> > really appreciate the help.
> >
> > Thanks,
> > Rajani.
> >
>
> --
> View this message in context: http://www.nabble.com/Stuck-messages---
> Dispatch-issues-tp20467949p24733104.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Stuck messages - Dispatch issues
Posted by Gary Tully <ga...@gmail.com>.
can you share the details of the compilation issues so we can help diagnose?
2009/7/30 couzteau <co...@bitfaeule.net>
>
> Thanks for the info!
>
> I integrated 5.3-snapshot but ran into compile issues with the latest jar -
> So I abandoned that.
>
>
>
> Gary Tully wrote:
> >
> > in the absence of a test case, the best approach is to try out a
> > 5.3-SNAPSHOT
> >
> > 2009/7/30 couzteau <co...@bitfaeule.net>
> >
> >>
> >> Has this been solved yet? I'm seeing it with 5.2.0.
> >>
> >> I'm seeing an issue that occurs on some machines (where machines are
> >> identical regarding OS, Java version and hardware).
> >>
> >> Consumers that are on the same machine as the producer usually work
> fine.
> >>
> >> It's a major blocker for us - any comments highly appreciated.
> >>
> >>
> >> TIA Jacques
> >>
> >>
> >>
> >>
> >> chrajanirao wrote:
> >> >
> >> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> >> > dispatching from queues. It is easily reproducible even using the out
> >> of
> >> > the box activemq configuration.
> >> >
> >> > Send messages to the a queue (QueueA) using multiple threads (10 or 20
> >> > thread in a loop of 10) using JMeter or your custom code. Have the
> >> > consumer setup using spring DMLC (DefaultMessageListenerContainer) or
> >> with
> >> > regular JMS API sync or async consumption with multiple consumers.
> >> > Consumer part can be configured using Camel to consumer from QueueA
> and
> >> > put the messages in QueueB within ActiveMQ configuration as well.
> >> >
> >> > After receiving some messages (the number is different each time), the
> >> > consumers stop receiving any messages even though there are some left
> >> on
> >> > the queue. Basically, the broker don't dispatch the messages and they
> >> are
> >> > stuck until restart of the broker. Any new messages sent to the queue
> >> > after this are sometimes dispatched and other times they are stuck
> too.
> >> >
> >> > This certainly seems like a major bug in the dispatch mechanism. I
> have
> >> > found below posts that state the exact problem.
> >> >
> >> >
> >>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> >> >
> >> >
> >>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> >> >
> >> > This is a very basic use case and I wonder how the version 5.1 is
> >> > currently used in production, if anyone is using at all.
> >> >
> >> > We tried with prefetch limit as 1, asyncDispatch as true and false,
> >> > session transacted as true and false. In all cases, the dispatch
> >> problem
> >> > still exists starting after 100 messages until 1000 messages.
> >> >
> >> > I hope any of the active commiters looks into this issue seriously.
> >> Would
> >> > really appreciate the help.
> >> >
> >> > Thanks,
> >> > Rajani.
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24733104.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24744031.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Stuck messages - Dispatch issues
Posted by couzteau <co...@bitfaeule.net>.
Thanks for the info!
I integrated 5.3-snapshot but ran into compile issues with the latest jar -
So I abandoned that.
Gary Tully wrote:
>
> in the absence of a test case, the best approach is to try out a
> 5.3-SNAPSHOT
>
> 2009/7/30 couzteau <co...@bitfaeule.net>
>
>>
>> Has this been solved yet? I'm seeing it with 5.2.0.
>>
>> I'm seeing an issue that occurs on some machines (where machines are
>> identical regarding OS, Java version and hardware).
>>
>> Consumers that are on the same machine as the producer usually work fine.
>>
>> It's a major blocker for us - any comments highly appreciated.
>>
>>
>> TIA Jacques
>>
>>
>>
>>
>> chrajanirao wrote:
>> >
>> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>> > dispatching from queues. It is easily reproducible even using the out
>> of
>> > the box activemq configuration.
>> >
>> > Send messages to the a queue (QueueA) using multiple threads (10 or 20
>> > thread in a loop of 10) using JMeter or your custom code. Have the
>> > consumer setup using spring DMLC (DefaultMessageListenerContainer) or
>> with
>> > regular JMS API sync or async consumption with multiple consumers.
>> > Consumer part can be configured using Camel to consumer from QueueA and
>> > put the messages in QueueB within ActiveMQ configuration as well.
>> >
>> > After receiving some messages (the number is different each time), the
>> > consumers stop receiving any messages even though there are some left
>> on
>> > the queue. Basically, the broker don't dispatch the messages and they
>> are
>> > stuck until restart of the broker. Any new messages sent to the queue
>> > after this are sometimes dispatched and other times they are stuck too.
>> >
>> > This certainly seems like a major bug in the dispatch mechanism. I have
>> > found below posts that state the exact problem.
>> >
>> >
>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>> >
>> >
>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>> >
>> > This is a very basic use case and I wonder how the version 5.1 is
>> > currently used in production, if anyone is using at all.
>> >
>> > We tried with prefetch limit as 1, asyncDispatch as true and false,
>> > session transacted as true and false. In all cases, the dispatch
>> problem
>> > still exists starting after 100 messages until 1000 messages.
>> >
>> > I hope any of the active commiters looks into this issue seriously.
>> Would
>> > really appreciate the help.
>> >
>> > Thanks,
>> > Rajani.
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24733104.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
>
--
View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24744031.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Gary Tully <ga...@gmail.com>.
in the absence of a test case, the best approach is to try out a
5.3-SNAPSHOT
2009/7/30 couzteau <co...@bitfaeule.net>
>
> Has this been solved yet? I'm seeing it with 5.2.0.
>
> I'm seeing an issue that occurs on some machines (where machines are
> identical regarding OS, Java version and hardware).
>
> Consumers that are on the same machine as the producer usually work fine.
>
> It's a major blocker for us - any comments highly appreciated.
>
>
> TIA Jacques
>
>
>
>
> chrajanirao wrote:
> >
> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> > dispatching from queues. It is easily reproducible even using the out of
> > the box activemq configuration.
> >
> > Send messages to the a queue (QueueA) using multiple threads (10 or 20
> > thread in a loop of 10) using JMeter or your custom code. Have the
> > consumer setup using spring DMLC (DefaultMessageListenerContainer) or
> with
> > regular JMS API sync or async consumption with multiple consumers.
> > Consumer part can be configured using Camel to consumer from QueueA and
> > put the messages in QueueB within ActiveMQ configuration as well.
> >
> > After receiving some messages (the number is different each time), the
> > consumers stop receiving any messages even though there are some left on
> > the queue. Basically, the broker don't dispatch the messages and they
> are
> > stuck until restart of the broker. Any new messages sent to the queue
> > after this are sometimes dispatched and other times they are stuck too.
> >
> > This certainly seems like a major bug in the dispatch mechanism. I have
> > found below posts that state the exact problem.
> >
> >
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> >
> >
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> >
> > This is a very basic use case and I wonder how the version 5.1 is
> > currently used in production, if anyone is using at all.
> >
> > We tried with prefetch limit as 1, asyncDispatch as true and false,
> > session transacted as true and false. In all cases, the dispatch problem
> > still exists starting after 100 messages until 1000 messages.
> >
> > I hope any of the active commiters looks into this issue seriously. Would
> > really appreciate the help.
> >
> > Thanks,
> > Rajani.
> >
>
> --
> View this message in context:
> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24733104.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Stuck messages - Dispatch issues
Posted by couzteau <co...@bitfaeule.net>.
Has this been solved yet? I'm seeing it with 5.2.0.
I'm seeing an issue that occurs on some machines (where machines are
identical regarding OS, Java version and hardware).
Consumers that are on the same machine as the producer usually work fine.
It's a major blocker for us - any comments highly appreciated.
TIA Jacques
chrajanirao wrote:
>
> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> dispatching from queues. It is easily reproducible even using the out of
> the box activemq configuration.
>
> Send messages to the a queue (QueueA) using multiple threads (10 or 20
> thread in a loop of 10) using JMeter or your custom code. Have the
> consumer setup using spring DMLC (DefaultMessageListenerContainer) or with
> regular JMS API sync or async consumption with multiple consumers.
> Consumer part can be configured using Camel to consumer from QueueA and
> put the messages in QueueB within ActiveMQ configuration as well.
>
> After receiving some messages (the number is different each time), the
> consumers stop receiving any messages even though there are some left on
> the queue. Basically, the broker don't dispatch the messages and they are
> stuck until restart of the broker. Any new messages sent to the queue
> after this are sometimes dispatched and other times they are stuck too.
>
> This certainly seems like a major bug in the dispatch mechanism. I have
> found below posts that state the exact problem.
>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>
> This is a very basic use case and I wonder how the version 5.1 is
> currently used in production, if anyone is using at all.
>
> We tried with prefetch limit as 1, asyncDispatch as true and false,
> session transacted as true and false. In all cases, the dispatch problem
> still exists starting after 100 messages until 1000 messages.
>
> I hope any of the active commiters looks into this issue seriously. Would
> really appreciate the help.
>
> Thanks,
> Rajani.
>
--
View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24733104.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by chrajanirao <ra...@gmail.com>.
Here is the issue that I created. Although the test case may not be of much
help as I could not get it to fail with embedded broker, hope you would look
into the issue.
https://issues.apache.org/activemq/browse/AMQ-2009
Thanks,
Rajani.
Gary Tully wrote:
>
> A test case would be great because it would be good to get this issue
> fully understood. This is a potential blocker for the 5.2.0 release
> IMHO. Can you raise a jira issue to track this?
> thanks.
>
> 2008/11/14 chrajanirao <ra...@gmail.com>:
>>
>> I tried 5.2 RC3 and it seem to have new bugs. It dispatching duplicates.
>> One
>> of my test run resulted below:
>>
>> Messages sent: 1000 (using 50 threads 20 times)
>> Consumers: 2 trasacted
>>
>> Queue Attributes in JConsole:
>> DequeCount: 1000
>> DipatchCount: 4849
>> EnqueueCount:1000
>> InFlightCount: 3849
>> QueueSize: 49
>>
>> Consumer end:
>> Received: 2025
>>
>> Its acting weird. when I tried with Non-transacted+Auto Ack, it sent
>> duplicates and also the broker was still holding on to all of 1000
>> messages.
>>
>> I will try to come up with a JUnit test case. But this is faily staright
>> forward case when messages are sent faster than they can be consumed!!
>>
>> Thanks.
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20492251.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20612764.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Gary Tully <ga...@gmail.com>.
A test case would be great because it would be good to get this issue
fully understood. This is a potential blocker for the 5.2.0 release
IMHO. Can you raise a jira issue to track this?
thanks.
2008/11/14 chrajanirao <ra...@gmail.com>:
>
> I tried 5.2 RC3 and it seem to have new bugs. It dispatching duplicates. One
> of my test run resulted below:
>
> Messages sent: 1000 (using 50 threads 20 times)
> Consumers: 2 trasacted
>
> Queue Attributes in JConsole:
> DequeCount: 1000
> DipatchCount: 4849
> EnqueueCount:1000
> InFlightCount: 3849
> QueueSize: 49
>
> Consumer end:
> Received: 2025
>
> Its acting weird. when I tried with Non-transacted+Auto Ack, it sent
> duplicates and also the broker was still holding on to all of 1000 messages.
>
> I will try to come up with a JUnit test case. But this is faily staright
> forward case when messages are sent faster than they can be consumed!!
>
> Thanks.
>
> --
> View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20492251.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: Stuck messages - Dispatch issues
Posted by chrajanirao <ra...@gmail.com>.
I tried 5.2 RC3 and it seem to have new bugs. It dispatching duplicates. One
of my test run resulted below:
Messages sent: 1000 (using 50 threads 20 times)
Consumers: 2 trasacted
Queue Attributes in JConsole:
DequeCount: 1000
DipatchCount: 4849
EnqueueCount:1000
InFlightCount: 3849
QueueSize: 49
Consumer end:
Received: 2025
Its acting weird. when I tried with Non-transacted+Auto Ack, it sent
duplicates and also the broker was still holding on to all of 1000 messages.
I will try to come up with a JUnit test case. But this is faily staright
forward case when messages are sent faster than they can be consumed!!
Thanks.
--
View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20492251.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Stuck messages - Dispatch issues
Posted by Gary Tully <ga...@gmail.com>.
Once such problem[1] was resolved for 5.2.0 RC3, would it be possible
to validate RC3[2]?
What would really help here is a JUnit test case that demonstrates the problem.
[1] https://issues.apache.org/activemq/browse/AMQ-1984
[2] http://people.apache.org/~gtully/staging-repos/activemq-5.2.0/org/apache/activemq/apache-activemq/5.2.0
2008/11/12 chrajanirao <ra...@gmail.com>:
>
> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message dispatching
> from queues. It is easily reproducible even using the out of the box
> activemq configuration.
>
> Send messages to the a queue (QueueA) using multiple threads (10 or 20
> thread in a loop of 10) using JMeter or your custom code. Have the consumer
> setup using spring DMLC (DefaultMessageListenerContainer) or with regular
> JMS API sync or async consumption with multiple consumers. Consumer part can
> be configured using Camel to consumer from QueueA and put the messages in
> QueueB within ActiveMQ configuration as well.
>
> After receiving some messages (the number is different each time), the
> consumers stop receiving any messages even though there are some left on the
> queue. Basically, the broker don't dispatch the messages and they are stuck
> until restart of the broker. Any new messages sent to the queue after this
> are sometimes dispatched and other times they are stuck too.
>
> This certainly seems like a major bug in the dispatch mechanism. I have
> found below posts that state the exact problem.
>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>
> This is a very basic use case and I wonder how the version 5.1 is currently
> used in production, if anyone is using at all.
>
> We tried with prefetch limit as 1, asyncDispatch as true and false, session
> transacted as true and false. In all cases, the dispatch problem still
> exists starting after 100 messages until 1000 messages.
>
> I hope any of the active commiters looks into this issue seriously. Would
> really appreciate the help.
>
> Thanks,
> Rajani.
> --
> View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20467949.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>