You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by thabach <ba...@evolootion.ch> on 2008/11/28 14:09:06 UTC
Reconciliation after ActiveMQ server crash.
Hey everyone
I have a conceptual question regarding reconciliation after a JMS server
crash. According to the JMS specification (paragraph 4.4.13, see below)
there is room for some ambiguity leaving a producer in doubt if a message
successfully made it to the server or not in case of server failure.
The [X] part is what disquiets me: "It is up to a JMS application to deal
with this ambiguity."
A producer in doubt can only resend the original message. In what way does
ActiveMQ support me in doing so ? Are there any means of having ActiveMQ
sort out potential duplicates sent by the application layers after server
recovery and publisher reconnect ? Or do downstream consumers have to cope
with these kind of duplicate messages somehow ?
Regards and thanks for any insights, Christian.
== JMS spec excerpt ============================================
4.4.13 Duplicate Production of Messages
JMS providers must never produce duplicate messages. This means that a
client that produces a message can rely on its JMS provider to insure that
consumers of the message will receive it only once. No client error can
cause a
provider to duplicate a message.
If a failure occurs between the time a client commits its work on a Session
and
the commit method returns, the client cannot determine if the transaction
was
committed or rolled back. The same ambiguity exists when a failure occurs
between the non-transactional send of a PERSISTENT message and the return
from the sending method.
[X] It is up to a JMS application to deal with this ambiguity. In some
cases, this
may cause a client to produce functionally duplicate messages.
A message that is redelivered due to session recovery is not considered a
duplicate message.
======================================================
--
View this message in context: http://www.nabble.com/Reconciliation-after-ActiveMQ-server-crash.-tp20734165p20734165.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Reconciliation after ActiveMQ server crash.
Posted by thabach <ba...@evolootion.ch>.
rajdavies wrote:
>
> This is handled by the ActiveMQ failover transport (the default in 5.1
> onwards).
> It replays messages on your behalf that haven't been acknowledged -
>
You refer to the case when the connection was only temporarily lost, and the
client runtime would autonomously do a resend on successful (re-)connect to
the same or a fail-over broker, right ?
But what if there is only one broker and the runtime decides to inform
application layers by throwing an exception - then the resend has to be done
by application layers. Is it possible somehow from an application level to
utilize the "internal" duplicate detection in reconciliation when targeting
the original broker a second time after it recovered ?
rajdavies wrote:
>
> ... there is duplicate detection built in to the broker and the clients
> too ...
>
Does this imply that there is a configurable JMS message ID backlog
maintained by ActiveMQ on the broker side as well as in client runtime
instances ? Is this all similar in concept to SwiftMQ duplicate detection
(http://www.swiftmq.com/products/harouter/advanced/duplicate/index.html) ?
I would appreciate if you could elucidate the ActiveMQ support in duplicate
detection even more, thank you very much. Christian.
--
View this message in context: http://www.nabble.com/Reconciliation-after-ActiveMQ-server-crash.-tp20734165p20801684.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Reconciliation after ActiveMQ server crash.
Posted by thabach <ba...@evolootion.ch>.
djencks wrote:
>
> I looked at the failover docs I could find and I think thabach was
> asking a different question that AFAICT the failover docs don't
> discuss... anyway I've wondered about this question :-) :
>
Right, the concrete scenario I was considering was a simple
(non-transactional) producer experiencing a JMSException while sending a
message due to connection loss.
rajdavies wrote:
>
> It replays messages on your behalf that haven't been acknowledged -
> there is duplicate detection built in to the broker and the clients
> too ...
>
You refer to the case when the connection was only temporarily lost, do you
? Then the ActiveMQ client runtime would have done a resend transparently
and some broker side duplicate detection would have set in, right ?
But what if the runtime decides to inform application layers by throwing an
exception - then IMHO all the producer can do is to keep trying to
re-connect and do the resend itself. My question thus actually was if I
could somehow from an application level utilize the "internal" duplicate
detection in reconciliation.
djencks wrote:
>
> The only way I know of to actually guarantee this is to send the
> message inside an xa transaction where the sending of the message is
> recorded "locally" somehow, for instance by marking the data that the
> message is generated from or by inserting a record into a database.
>
So what you are saying is that owing to XA transaction semantics, if the
producer was transactional it could deduce if its message made it to the
broker or not ?
Cool talking to you guys - Christian.
--
View this message in context: http://www.nabble.com/Reconciliation-after-ActiveMQ-server-crash.-tp20734165p20751325.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Reconciliation after ActiveMQ server crash.
Posted by David Jencks <da...@yahoo.com>.
I looked at the failover docs I could find and I think thabach was
asking a different question that AFAICT the failover docs don't
discuss... anyway I've wondered about this question :-) :
Suppose a client sends a message inside a transaction and either the
client or broker crashes before the tx.commit() call has returned.
How can the client determine if the transaction completed, so the
message will be delivered eventually, or failed, so the message needs
to be re-sent?
I recall reading the the big Transaction Processing book that in the
distant past terminals were considered transactional resources so that
unless the user actually saw that the transaction had comitted
successfully, it was not committed: either rolled back or left in the
"prepared" state.
The only way I know of to actually guarantee this is to send the
message inside an xa transaction where the sending of the message is
recorded "locally" somehow, for instance by marking the data that the
message is generated from or by inserting a record into a database.
I would guess that it would often be easier to write message receiving
code that can handle duplicate messages.
There may be other ways to deal with this...
thanks
david jencks
On Nov 28, 2008, at 8:21 AM, Rob Davies wrote:
> This is handled by the ActiveMQ failover transport (the default in
> 5.1 onwards).
> It replays messages on your behalf that haven't been acknowledged -
> there is duplicate detection built in to the broker and the clients
> too ...
>
> cheers,
>
> Rob
>
> cheers,
>
> Rob
>
> Rob Davies
> http://fusesource.com
> http://rajdavies.blogspot.com/
>
>
> On 28 Nov 2008, at 13:09, thabach wrote:
>
>>
>> Hey everyone
>>
>> I have a conceptual question regarding reconciliation after a JMS
>> server
>> crash. According to the JMS specification (paragraph 4.4.13, see
>> below)
>> there is room for some ambiguity leaving a producer in doubt if a
>> message
>> successfully made it to the server or not in case of server failure.
>>
>> The [X] part is what disquiets me: "It is up to a JMS application
>> to deal
>> with this ambiguity."
>>
>> A producer in doubt can only resend the original message. In what
>> way does
>> ActiveMQ support me in doing so ? Are there any means of having
>> ActiveMQ
>> sort out potential duplicates sent by the application layers after
>> server
>> recovery and publisher reconnect ? Or do downstream consumers have
>> to cope
>> with these kind of duplicate messages somehow ?
>>
>> Regards and thanks for any insights, Christian.
>>
>>
>> == JMS spec excerpt ============================================
>>
>> 4.4.13 Duplicate Production of Messages
>> JMS providers must never produce duplicate messages. This means
>> that a
>> client that produces a message can rely on its JMS provider to
>> insure that
>> consumers of the message will receive it only once. No client error
>> can
>> cause a
>> provider to duplicate a message.
>>
>> If a failure occurs between the time a client commits its work on a
>> Session
>> and
>> the commit method returns, the client cannot determine if the
>> transaction
>> was
>> committed or rolled back. The same ambiguity exists when a failure
>> occurs
>> between the non-transactional send of a PERSISTENT message and the
>> return
>> from the sending method.
>>
>> [X] It is up to a JMS application to deal with this ambiguity. In
>> some
>> cases, this
>> may cause a client to produce functionally duplicate messages.
>> A message that is redelivered due to session recovery is not
>> considered a
>> duplicate message.
>>
>> ======================================================
>>
>> --
>> View this message in context: http://www.nabble.com/Reconciliation-after-ActiveMQ-server-crash.-tp20734165p20734165.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
Re: Reconciliation after ActiveMQ server crash.
Posted by Rob Davies <ra...@gmail.com>.
This is handled by the ActiveMQ failover transport (the default in 5.1
onwards).
It replays messages on your behalf that haven't been acknowledged -
there is duplicate detection built in to the broker and the clients
too ...
cheers,
Rob
cheers,
Rob
Rob Davies
http://fusesource.com
http://rajdavies.blogspot.com/
On 28 Nov 2008, at 13:09, thabach wrote:
>
> Hey everyone
>
> I have a conceptual question regarding reconciliation after a JMS
> server
> crash. According to the JMS specification (paragraph 4.4.13, see
> below)
> there is room for some ambiguity leaving a producer in doubt if a
> message
> successfully made it to the server or not in case of server failure.
>
> The [X] part is what disquiets me: "It is up to a JMS application to
> deal
> with this ambiguity."
>
> A producer in doubt can only resend the original message. In what
> way does
> ActiveMQ support me in doing so ? Are there any means of having
> ActiveMQ
> sort out potential duplicates sent by the application layers after
> server
> recovery and publisher reconnect ? Or do downstream consumers have
> to cope
> with these kind of duplicate messages somehow ?
>
> Regards and thanks for any insights, Christian.
>
>
> == JMS spec excerpt ============================================
>
> 4.4.13 Duplicate Production of Messages
> JMS providers must never produce duplicate messages. This means that a
> client that produces a message can rely on its JMS provider to
> insure that
> consumers of the message will receive it only once. No client error
> can
> cause a
> provider to duplicate a message.
>
> If a failure occurs between the time a client commits its work on a
> Session
> and
> the commit method returns, the client cannot determine if the
> transaction
> was
> committed or rolled back. The same ambiguity exists when a failure
> occurs
> between the non-transactional send of a PERSISTENT message and the
> return
> from the sending method.
>
> [X] It is up to a JMS application to deal with this ambiguity. In some
> cases, this
> may cause a client to produce functionally duplicate messages.
> A message that is redelivered due to session recovery is not
> considered a
> duplicate message.
>
> ======================================================
>
> --
> View this message in context: http://www.nabble.com/Reconciliation-after-ActiveMQ-server-crash.-tp20734165p20734165.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>