You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Josh Carlson <jc...@e-dialog.com> on 2010/02/24 20:02:10 UTC
Failover causes duplicate messages
When using a shared file system master/server activemq configuration and
client acknoledgements we run into a problem when
our clients fail over to a new server. The problem is that the new
server does not appear to have any knowledge of pending
messages that the old server had dispatched to clients. Consequently all
of these pending messages get dispatched a second
time even though the clients had acknowledged them.
I've filed this as a bug here:
https://issues.apache.org/activemq/browse/AMQ-2627
which contains a reproducer using activemq-cpp. I'm posting here in the
hopes that someone can confirm that they believe this something that is
suppose to be supported and/or if anyone knows of any possible work arounds.
Any advice is appreciated.
Re: Failover causes duplicate messages
Posted by Bruce Snyder <br...@gmail.com>.
On Wed, Feb 24, 2010 at 2:31 PM, Josh Carlson <jc...@e-dialog.com> wrote:
> Hi Gary,
>
> Thanks for the explanation. The order is:
>
> consumer retrieves message
>
> connection is lost and consumer re-establishes connection to new master
> broker
>
> consumer acks the message retrieved from the former broker
>
> consumer retrieves the same message it just ack'ed.
>
> I guess the question is ... is there a way for the new broker on failover to
> recognize the consumers that have retrieved message but not ack'ed messages
> and behave exactly as if they were retrieved from itself (not redispatch
> unless the connection to the consumer is lost)? I suppose that would require
> some heuristics to determine how long to wait before those original
> consumers reconnect.
Given the sequence that you listed above, the redispatch of the
messages is correct as Gary explained. Given the loss of the
connection, the broker will redispatch any pending messages. Any
messages that the client received from the lost connection that are
unacked cannot be acked using the new connection (i.e., the client
can't just hold those messages and swap the underlying connection).
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
Re: Failover causes duplicate messages
Posted by Josh Carlson <jc...@e-dialog.com>.
Hi Gary,
Thanks for the explanation. The order is:
consumer retrieves message
connection is lost and consumer re-establishes connection to new
master broker
consumer acks the message retrieved from the former broker
consumer retrieves the same message it just ack'ed.
I guess the question is ... is there a way for the new broker on
failover to recognize the consumers that have retrieved message but not
ack'ed messages and behave exactly as if they were retrieved from itself
(not redispatch unless the connection to the consumer is lost)? I
suppose that would require some heuristics to determine how long to wait
before those original consumers reconnect.
Thanks
Josh
On 02/24/2010 04:16 PM, Gary Tully wrote:
> On failover, redispatch of unacked messages will occur and this is
> unordered. There is no guarantee that the same consumer will get the
> same messages after failover of a connection.
> On the java client there is a message audit that will suppress
> duplicates that may occur, I think the same is true for the CPP
> client, but this will not be true for STOMP clients.
>
> What is odd is that you say that messages that have been acked are
> replayed. This should only occur if the ack did not actually get to
> the broker.
> If previously acked messages are getting replayed there is something
> wrong.
>
> On 24 February 2010 19:02, Josh Carlson <jcarlson@e-dialog.com
> <ma...@e-dialog.com>> wrote:
>
> When using a shared file system master/server activemq
> configuration and client acknoledgements we run into a problem when
> our clients fail over to a new server. The problem is that the new
> server does not appear to have any knowledge of pending
> messages that the old server had dispatched to clients.
> Consequently all of these pending messages get dispatched a second
> time even though the clients had acknowledged them.
>
> I've filed this as a bug here:
> https://issues.apache.org/activemq/browse/AMQ-2627
>
> which contains a reproducer using activemq-cpp. I'm posting here
> in the hopes that someone can confirm that they believe this
> something that is suppose to be supported and/or if anyone knows
> of any possible work arounds.
>
> Any advice is appreciated.
>
>
>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
Re: Failover causes duplicate messages
Posted by Gary Tully <ga...@gmail.com>.
On failover, redispatch of unacked messages will occur and this is
unordered. There is no guarantee that the same consumer will get the same
messages after failover of a connection.
On the java client there is a message audit that will suppress duplicates
that may occur, I think the same is true for the CPP client, but this will
not be true for STOMP clients.
What is odd is that you say that messages that have been acked are replayed.
This should only occur if the ack did not actually get to the broker.
If previously acked messages are getting replayed there is something wrong.
On 24 February 2010 19:02, Josh Carlson <jc...@e-dialog.com> wrote:
> When using a shared file system master/server activemq configuration and
> client acknoledgements we run into a problem when
> our clients fail over to a new server. The problem is that the new server
> does not appear to have any knowledge of pending
> messages that the old server had dispatched to clients. Consequently all of
> these pending messages get dispatched a second
> time even though the clients had acknowledged them.
>
> I've filed this as a bug here:
> https://issues.apache.org/activemq/browse/AMQ-2627
>
> which contains a reproducer using activemq-cpp. I'm posting here in the
> hopes that someone can confirm that they believe this something that is
> suppose to be supported and/or if anyone knows of any possible work arounds.
>
> Any advice is appreciated.
>
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com