You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by yinghe0101 <yi...@yahoo.com> on 2009/06/12 00:28:14 UTC
Serious dispatch issue
I already put this in a comment on
https://issues.apache.org/activemq/browse/AMQ-2009 which might explain why
people experience this issue.
I have a simple cause which can cause dispatch problem:
1. setup a network of broker1, broker2, bridged by multicast discovery
2. make a producer send 5 msg to queueA to broker2
3. make a consumer to consume from broker1 queueA ( make it slow, so it only
consumer 1 msg) but make sure all 5 msg from broker2 are forwared to broker1
4. stop the consumer to broke1, restart it to consume from broker2 queueA
5. the 4 msgs originally published to broker2 and forwarded to broker1 and
has not yet been consumed will stuck on broker1 and will not forwarded to
broker2 for the consumer to consume.
You can only clear out the 4 left over stuck msg by making consumer consume
from broker1 in this case.
This is a very critical issue because you might restart your application
many times and run into this.
I will look into how the demandfowardbridge handles this case. if you know
anything about this, please help. this is very urgent. thanks
best,
ying
--
View this message in context: http://www.nabble.com/Serious-dispatch-issue-tp23990060p23990060.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Serious dispatch issue
Posted by yinghe0101 <yi...@yahoo.com>.
hi, thank you both for the code base.
What we wish to do is do a check on the broker which has the msgs and if
there is no consumer on the broker, it then can forward to other consumer on
a remote broker even if the msg is coming from the remote broker. This
should be safe, right?
But I encounter the following issue, if the msg is forwarded to the remote
broker where it comes from, KahaReferenceStore will check duplication and
consider it already dispatched. the msg is complete lost this time.
Any idea of how we can go our approach. The overall idea is if the broker
has no consumer at all, it should not hold msgs. it shall dispatch it to
others, even the other is the one where the msg is original coming from.
best,
ying
Gary Tully wrote:
>
> The code in serviceRemoteConsumerAdvisory blocks cyclic subscriptions but
> there is also a filter installed that blocks bouncing messages, enable
> trace
> level logging for org.apache.activemq.command.NetworkBridgeFilter to see
> it
> in action.
>
> It is installed in
> org.apache.activemq.network.DemandForwardingBridgeSupport.configureDemandSubscription
>
> hth,
> Gary.
>
> 2009/6/12 Bruce Snyder <br...@gmail.com>
>
>> On Fri, Jun 12, 2009 at 7:11 AM, yinghe0101<yi...@yahoo.com> wrote:
>> >
>> > hi, Gary,
>> >
>> > thank you for your reply. Could you please point out where in the code
>> base,
>> > this restraint apply. I need to take a look at that and consider your
>> > recommendation improvement.
>>
>> This functionality resides in the
>> DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory() method.
>>
>> Bruce
>> --
>> perl -e 'print
>> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
>> );'
>>
>> ActiveMQ in Action: http://bit.ly/2je6cQ
>> Blog: http://bruceblog.org/
>> Twitter: http://twitter.com/brucesnyder
>>
>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
>
--
View this message in context: http://www.nabble.com/Serious-dispatch-issue-tp23990060p24003521.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Serious dispatch issue
Posted by Gary Tully <ga...@gmail.com>.
The code in serviceRemoteConsumerAdvisory blocks cyclic subscriptions but
there is also a filter installed that blocks bouncing messages, enable trace
level logging for org.apache.activemq.command.NetworkBridgeFilter to see it
in action.
It is installed in
org.apache.activemq.network.DemandForwardingBridgeSupport.configureDemandSubscription
hth,
Gary.
2009/6/12 Bruce Snyder <br...@gmail.com>
> On Fri, Jun 12, 2009 at 7:11 AM, yinghe0101<yi...@yahoo.com> wrote:
> >
> > hi, Gary,
> >
> > thank you for your reply. Could you please point out where in the code
> base,
> > this restraint apply. I need to take a look at that and consider your
> > recommendation improvement.
>
> This functionality resides in the
> DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory() method.
>
> Bruce
> --
> perl -e 'print
> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> );'
>
> ActiveMQ in Action: http://bit.ly/2je6cQ
> Blog: http://bruceblog.org/
> Twitter: http://twitter.com/brucesnyder
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Serious dispatch issue
Posted by Bruce Snyder <br...@gmail.com>.
On Fri, Jun 12, 2009 at 7:11 AM, yinghe0101<yi...@yahoo.com> wrote:
>
> hi, Gary,
>
> thank you for your reply. Could you please point out where in the code base,
> this restraint apply. I need to take a look at that and consider your
> recommendation improvement.
This functionality resides in the
DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory() method.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
Re: Serious dispatch issue
Posted by yinghe0101 <yi...@yahoo.com>.
hi, Gary,
thank you for your reply. Could you please point out where in the code base,
this restraint apply. I need to take a look at that and consider your
recommendation improvement.
best,
Ying
Gary Tully wrote:
>
> this behavior is by design, but it could be improved.
> A network bridge will not forward a message back to where it came from
> because in the scenario you describe, with a slow consumer, messages would
> constantly be forward over and back between broker1 and broker2.
>
> I guess an improvement would be to make this behavior configurable and to
> allow a delay and backoff to be specified so that bouncing messages do not
> consume all available cpu and network resources.
>
> 2009/6/11 yinghe0101 <yi...@yahoo.com>
>
>>
>> I already put this in a comment on
>> https://issues.apache.org/activemq/browse/AMQ-2009 which might explain
>> why
>> people experience this issue.
>>
>> I have a simple cause which can cause dispatch problem:
>>
>> 1. setup a network of broker1, broker2, bridged by multicast discovery
>> 2. make a producer send 5 msg to queueA to broker2
>> 3. make a consumer to consume from broker1 queueA ( make it slow, so it
>> only
>> consumer 1 msg) but make sure all 5 msg from broker2 are forwared to
>> broker1
>> 4. stop the consumer to broke1, restart it to consume from broker2 queueA
>> 5. the 4 msgs originally published to broker2 and forwarded to broker1
>> and
>> has not yet been consumed will stuck on broker1 and will not forwarded to
>> broker2 for the consumer to consume.
>>
>> You can only clear out the 4 left over stuck msg by making consumer
>> consume
>> from broker1 in this case.
>>
>> This is a very critical issue because you might restart your application
>> many times and run into this.
>>
>> I will look into how the demandfowardbridge handles this case. if you
>> know
>> anything about this, please help. this is very urgent. thanks
>>
>> best,
>> ying
>> --
>> View this message in context:
>> http://www.nabble.com/Serious-dispatch-issue-tp23990060p23990060.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
>
--
View this message in context: http://www.nabble.com/Serious-dispatch-issue-tp23990060p23998576.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Serious dispatch issue
Posted by Gary Tully <ga...@gmail.com>.
this behavior is by design, but it could be improved.
A network bridge will not forward a message back to where it came from
because in the scenario you describe, with a slow consumer, messages would
constantly be forward over and back between broker1 and broker2.
I guess an improvement would be to make this behavior configurable and to
allow a delay and backoff to be specified so that bouncing messages do not
consume all available cpu and network resources.
2009/6/11 yinghe0101 <yi...@yahoo.com>
>
> I already put this in a comment on
> https://issues.apache.org/activemq/browse/AMQ-2009 which might explain why
> people experience this issue.
>
> I have a simple cause which can cause dispatch problem:
>
> 1. setup a network of broker1, broker2, bridged by multicast discovery
> 2. make a producer send 5 msg to queueA to broker2
> 3. make a consumer to consume from broker1 queueA ( make it slow, so it
> only
> consumer 1 msg) but make sure all 5 msg from broker2 are forwared to
> broker1
> 4. stop the consumer to broke1, restart it to consume from broker2 queueA
> 5. the 4 msgs originally published to broker2 and forwarded to broker1 and
> has not yet been consumed will stuck on broker1 and will not forwarded to
> broker2 for the consumer to consume.
>
> You can only clear out the 4 left over stuck msg by making consumer consume
> from broker1 in this case.
>
> This is a very critical issue because you might restart your application
> many times and run into this.
>
> I will look into how the demandfowardbridge handles this case. if you know
> anything about this, please help. this is very urgent. thanks
>
> best,
> ying
> --
> View this message in context:
> http://www.nabble.com/Serious-dispatch-issue-tp23990060p23990060.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com