You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by TLFox <ti...@googlemail.com> on 2008/05/28 20:55:26 UTC
JMS Acknowledgements sent asynchronously?
In a similar vein to the other thread I posted about persistent messages
being sent async, can I confirm whether acknowledgements of message
consumption (sent on auto_acknowledge, dups_ok, or client_acknowledge) are
sent asynchronously or synchronously?
Same setup. I am using QPid Java M2, with Java client.
--
View this message in context: http://www.nabble.com/JMS-Acknowledgements-sent-asynchronously--tp17519565p17519565.html
Sent from the Qpid Developers mailing list archive at Nabble.com.
Re: JMS Acknowledgements sent asynchronously?
Posted by TLFox <ti...@googlemail.com>.
Similarly you can't support client ack semantics if you use async acks.
--
View this message in context: http://www.nabble.com/JMS-Acknowledgements-sent-asynchronously--tp17519565p17520166.html
Sent from the Qpid Developers mailing list archive at Nabble.com.
Re: JMS Acknowledgements sent asynchronously?
Posted by Robert Godfrey <ro...@gmail.com>.
2008/5/28 Robert Greig <ro...@gmail.com>:
> 2008/5/28 TLFox <ti...@googlemail.com>:
>
>> Ok sorry to be a pain. But there's also an issue with doing acknowledgements
>> asynchronously and JMS compliance.
>
In general the goal of Qpid is to provide AMQP brokers and clients in
a number of different languages, Qpid is explictly not aiming to be a
"JMS Broker". However, since the common API for messaging in Java is
JMS we have attempted to layer JMS on top of AMQP. Our work in trying
to do this on the AMQP0-8/0-9 protocols has led directly to the
changes for AMQP0-10. Some of this has involved using Qpid as a
testbed for proposals for protocol extensions.
I believe that the Java Client/Broker on M2 / M2.1 are probably close
enough to the JMS specification that they could pass a TCK; however
for all sorts of reasons we do not claim any sort of JMS compliance.
The 0-10 protocol is such that (apart from one very small and annoying
omission for a particular case of restoring durable subscriptions
after a client reconnection) it should be possible to build a 100%
compliant JMS implementation without any extensions to the core AMQP
protocol.
As we progress AMQP I hope that we can remove any remaining such
incompatibilities and also publish a standard mapping so that JMS
clients from different AMQP vendors will interoperate seamlessly.
Hope this helps,
Rob
Re: JMS Acknowledgements sent asynchronously?
Posted by Robert Greig <ro...@gmail.com>.
2008/5/28 TLFox <ti...@googlemail.com>:
> Ok sorry to be a pain. But there's also an issue with doing acknowledgements
> asynchronously and JMS compliance.
Yes, this is also fixed in the 0-10 version of the protocol (and thus
trunk, which will become M3).
RG
Re: JMS Acknowledgements sent asynchronously?
Posted by Arnaud Simon <as...@redhat.com>.
On Thu, 2008-06-05 at 04:26 -0700, TLFox wrote:
>
>
> Arnaud Simon wrote:
> >
> >
> > If it also applies to receive(), and that's my interpretation, we can
> > ack pseudo-asynchronously i.e. the next message transfer could carry an
> > ack flag confirming that the broker has received the previous message
> > acknowledgment. So we don't have to block on the ack before delivering
> > the message to the receiver. Note that when the client lib is
> > pre-fetching messages we only need to deliver the next message after the
> > sync ack of the previous message has returned.
> >
> > Arnaud
> >
> >
>
> Well, in the above, you're still effectively blocking on the ack, but in a
> more roundabout way, since you can't deliver the next message until the ack
> has reached the server then you are given notification of that by the next
> message transfer- I think you're just moving the same issue from one place
> to another.
Let's say that the time to perform a sync is ts and to process the
message tp (I am not taking into account the message transfer time as
messages may be pre-fetched). If we sync before delivering the message
we'll have the following:
=ts= =tp= =ts= =tp= =ts= =tp=
| | |
M1 M2 M3
That means that the overall time to receive and process 3 messages is 3
* ts + 3 * tp
If we wait on the previous message sync we'll then have something like:
=tp= =Max(0, ts - tp)= =tp= =Max(0, ts - tp)= =tp=
| | |
M1 M2 M3
that means that the overall time is 3 * tp + 2 Max(0, ts - tp). That
looks to me faster than the previous scenario. I may be wrong though.
Arnaud
Re: JMS Acknowledgements sent asynchronously?
Posted by TLFox <ti...@googlemail.com>.
Arnaud Simon wrote:
>
>
> If it also applies to receive(), and that's my interpretation, we can
> ack pseudo-asynchronously i.e. the next message transfer could carry an
> ack flag confirming that the broker has received the previous message
> acknowledgment. So we don't have to block on the ack before delivering
> the message to the receiver. Note that when the client lib is
> pre-fetching messages we only need to deliver the next message after the
> sync ack of the previous message has returned.
>
> Arnaud
>
>
Well, in the above, you're still effectively blocking on the ack, but in a
more roundabout way, since you can't deliver the next message until the ack
has reached the server then you are given notification of that by the next
message transfer- I think you're just moving the same issue from one place
to another.
--
View this message in context: http://www.nabble.com/JMS-Acknowledgements-sent-asynchronously--tp17519565p17667716.html
Sent from the Qpid Developers mailing list archive at Nabble.com.
Re: JMS Acknowledgements sent asynchronously?
Posted by Arnaud Simon <as...@redhat.com>.
On Thu, 2008-06-05 at 10:07 +0100, Robert Greig wrote:
> 2008/6/5 Arnaud Simon <as...@redhat.com>:
>
> > If it also applies to receive(), and that's my interpretation, we can
> > ack pseudo-asynchronously i.e. the next message transfer could carry an
> > ack flag confirming that the broker has received the previous message
> > acknowledgment. So we don't have to block on the ack before delivering
> > the message to the receiver. Note that when the client lib is
> > pre-fetching messages we only need to deliver the next message after the
> > sync ack of the previous message has returned.
>
> Given that 99% of the time people will be using prefetch is it even
> worth bothering with piggybacking the ack flag on the next message
> delivery? ie. you are going to have to sync nearly all the time.
>
> RG
That's exactly what I said, When messages are pre-fetched we only need
to deliver the next message after the previous sync operation has
returned. In other words *receive* may block on the previous sync
operation.
Arnaud
Re: JMS Acknowledgements sent asynchronously?
Posted by Robert Greig <ro...@gmail.com>.
2008/6/5 Arnaud Simon <as...@redhat.com>:
> If it also applies to receive(), and that's my interpretation, we can
> ack pseudo-asynchronously i.e. the next message transfer could carry an
> ack flag confirming that the broker has received the previous message
> acknowledgment. So we don't have to block on the ack before delivering
> the message to the receiver. Note that when the client lib is
> pre-fetching messages we only need to deliver the next message after the
> sync ack of the previous message has returned.
Given that 99% of the time people will be using prefetch is it even
worth bothering with piggybacking the ack flag on the next message
delivery? ie. you are going to have to sync nearly all the time.
RG
Re: JMS Acknowledgements sent asynchronously?
Posted by Arnaud Simon <as...@redhat.com>.
On Wed, 2008-06-04 at 10:15 -0700, TLFox wrote:
>
>
> Arnaud Simon wrote:
> >
> >
> > The JMS spec says:
> > "When a client uses the AUTO_ACKNOWLEDGE mode, it is not in direct
> > control of message acknowledgment. Since such clients cannot know for
> > certain if a particular message has been acknowledged, they must be
> > prepared for redelivery of the last consumed message. This can be caused
> > by the client completing its work just prior to a failure that prevents
> > the message acknowledgment from occurring. Only a session’s last
> > consumed message is subject to this ambiguity"
> >
> > That makes me think that we may not need to sync.
> >
>
> I believe that passage is really referring to the "weird" jms inconsistency
> where the the message is not acked until after onMessage() has finished
> executing in the case of a MesssageListener with AUTO_ACKNOWLEDGE.
>
> But even if it also applies to receive(), then to ensure this statement:
>
> "Only a session’s last
> consumed message is subject to this ambiguity"
>
> You would still have to ack synchronously in order to guarantee that only
> the last message might get redelivered. If you acked async you might get N
> messages redelivered where N can be > 1, depending on how long it takes for
> the async ack to get to the server and the rate of consumption.
>
> Interesting discussion though.
If it also applies to receive(), and that's my interpretation, we can
ack pseudo-asynchronously i.e. the next message transfer could carry an
ack flag confirming that the broker has received the previous message
acknowledgment. So we don't have to block on the ack before delivering
the message to the receiver. Note that when the client lib is
pre-fetching messages we only need to deliver the next message after the
sync ack of the previous message has returned.
Arnaud
Re: JMS Acknowledgements sent asynchronously?
Posted by TLFox <ti...@googlemail.com>.
Arnaud Simon wrote:
>
>
> The JMS spec says:
> "When a client uses the AUTO_ACKNOWLEDGE mode, it is not in direct
> control of message acknowledgment. Since such clients cannot know for
> certain if a particular message has been acknowledged, they must be
> prepared for redelivery of the last consumed message. This can be caused
> by the client completing its work just prior to a failure that prevents
> the message acknowledgment from occurring. Only a session’s last
> consumed message is subject to this ambiguity"
>
> That makes me think that we may not need to sync.
>
I believe that passage is really referring to the "weird" jms inconsistency
where the the message is not acked until after onMessage() has finished
executing in the case of a MesssageListener with AUTO_ACKNOWLEDGE.
But even if it also applies to receive(), then to ensure this statement:
"Only a session’s last
consumed message is subject to this ambiguity"
You would still have to ack synchronously in order to guarantee that only
the last message might get redelivered. If you acked async you might get N
messages redelivered where N can be > 1, depending on how long it takes for
the async ack to get to the server and the rate of consumption.
Interesting discussion though.
--
View this message in context: http://www.nabble.com/JMS-Acknowledgements-sent-asynchronously--tp17519565p17651888.html
Sent from the Qpid Developers mailing list archive at Nabble.com.
Re: JMS Acknowledgements sent asynchronously?
Posted by Martin Ritchie <ri...@apache.org>.
2008/6/4 Arnaud Simon <as...@redhat.com>:
>
> On Wed, 2008-06-04 at 02:37 -0700, TLFox wrote:
>>
>>
>> Rafael Schloming wrote:
>> >
>> >
>> > The JMS spec isn't fully consistent on this point. It clearly states
>> > that with AUTO_ACKNOWLEDGE acks are sent *after* the message listener is
>> > invoked, so (with listeners at least) this window will always exist even
>> > if you sync the acks.
>> >
>>
>> Yes indeed. This is a known glaring inconsistency in the spec.
>>
>>
>>
>> > With receive() of course the ack happens before the application sees the
>> > message, so you could avoid the window by syncing there, however I don't
>> > know offhand of anything that directly states that should be the case.
>> >
> The JMS spec says:
> "When a client uses the AUTO_ACKNOWLEDGE mode, it is not in direct
> control of message acknowledgment. Since such clients cannot know for
> certain if a particular message has been acknowledged, they must be
> prepared for redelivery of the last consumed message. This can be caused
> by the client completing its work just prior to a failure that prevents
> the message acknowledgment from occurring. Only a session's last
> consumed message is subject to this ambiguity"
>
> That makes me think that we may not need to sync.
It makes me think that we need to ensure the ack has been processed on
the broker *before* delivering the next message to onMessage.
Otherwise there will be ambiguity about more than one message which
isn't allowed by the above text.
Sounds like a sync is required.
Martin
>> JMS spec is explicit in that it requires "at most once" semantics in this
>> case, this surely implies you must sync the ack before returning from
>> receive().
>>
>> How else would you guarantee those semantics without syncing?
>>
>
>
--
Martin Ritchie
Re: JMS Acknowledgements sent asynchronously?
Posted by Arnaud Simon <as...@redhat.com>.
On Wed, 2008-06-04 at 02:37 -0700, TLFox wrote:
>
>
> Rafael Schloming wrote:
> >
> >
> > The JMS spec isn't fully consistent on this point. It clearly states
> > that with AUTO_ACKNOWLEDGE acks are sent *after* the message listener is
> > invoked, so (with listeners at least) this window will always exist even
> > if you sync the acks.
> >
>
> Yes indeed. This is a known glaring inconsistency in the spec.
>
>
>
> > With receive() of course the ack happens before the application sees the
> > message, so you could avoid the window by syncing there, however I don't
> > know offhand of anything that directly states that should be the case.
> >
The JMS spec says:
"When a client uses the AUTO_ACKNOWLEDGE mode, it is not in direct
control of message acknowledgment. Since such clients cannot know for
certain if a particular message has been acknowledged, they must be
prepared for redelivery of the last consumed message. This can be caused
by the client completing its work just prior to a failure that prevents
the message acknowledgment from occurring. Only a session’s last
consumed message is subject to this ambiguity"
That makes me think that we may not need to sync.
> JMS spec is explicit in that it requires "at most once" semantics in this
> case, this surely implies you must sync the ack before returning from
> receive().
>
> How else would you guarantee those semantics without syncing?
>
Re: JMS Acknowledgements sent asynchronously?
Posted by TLFox <ti...@googlemail.com>.
Rafael Schloming wrote:
>
>
> The JMS spec isn't fully consistent on this point. It clearly states
> that with AUTO_ACKNOWLEDGE acks are sent *after* the message listener is
> invoked, so (with listeners at least) this window will always exist even
> if you sync the acks.
>
Yes indeed. This is a known glaring inconsistency in the spec.
> With receive() of course the ack happens before the application sees the
> message, so you could avoid the window by syncing there, however I don't
> know offhand of anything that directly states that should be the case.
>
JMS spec is explicit in that it requires "at most once" semantics in this
case, this surely implies you must sync the ack before returning from
receive().
How else would you guarantee those semantics without syncing?
--
View this message in context: http://www.nabble.com/JMS-Acknowledgements-sent-asynchronously--tp17519565p17642794.html
Sent from the Qpid Developers mailing list archive at Nabble.com.
Re: JMS Acknowledgements sent asynchronously?
Posted by Rafael Schloming <ra...@redhat.com>.
TLFox wrote:
>
>
> Robert Greig wrote:
>> In a similar vein, it's asynchronous.
>>
>>
>
> Ok sorry to be a pain. But there's also an issue with doing acknowledgements
> asynchronously and JMS compliance.
>
> I raised the same issue recently with ActiveMQ which suffers from the same
> problem:
> http://www.nabble.com/Asynchronous-acknowledgements---are-they-correct--to17277656s2354.html
>
> To summarise, if you send acks async it's not possible to support JMS
> AUTO_ACKNOWLEDGE semantics properly. AUTO_ACKNOWLEDGE requires "at most
> once" delivery guarantee. This means its ok to lose a message on failure but
> you'll never get duplicates.
>
> In order to guarantee "at most once", basically you need to ack the message
> synchronously to the server before you give the message to the client via a
> receive() call.
>
> If you do that, there's a possibility of lost message if the client fails
> after the ack is processed but before the receive() call returns, but
> there's no possibility of the message getting delivered more than once after
> falure (since it's always acked before receipt).
>
> However if you send the ack async, then the message can actually get acked
> *after* it's been giving to the user. Thus introducing a window during which
> if failure occurs the message can get redelivered in contravention of JMS
> spec.
The JMS spec isn't fully consistent on this point. It clearly states
that with AUTO_ACKNOWLEDGE acks are sent *after* the message listener is
invoked, so (with listeners at least) this window will always exist even
if you sync the acks.
With receive() of course the ack happens before the application sees the
message, so you could avoid the window by syncing there, however I don't
know offhand of anything that directly states that should be the case.
For CLIENT_ACKNOWLEDGE I agree that strictly speaking you should sync
the acks.
--Rafael
Re: JMS Acknowledgements sent asynchronously?
Posted by TLFox <ti...@googlemail.com>.
Robert Greig wrote:
>
> In a similar vein, it's asynchronous.
>
>
Ok sorry to be a pain. But there's also an issue with doing acknowledgements
asynchronously and JMS compliance.
I raised the same issue recently with ActiveMQ which suffers from the same
problem:
http://www.nabble.com/Asynchronous-acknowledgements---are-they-correct--to17277656s2354.html
To summarise, if you send acks async it's not possible to support JMS
AUTO_ACKNOWLEDGE semantics properly. AUTO_ACKNOWLEDGE requires "at most
once" delivery guarantee. This means its ok to lose a message on failure but
you'll never get duplicates.
In order to guarantee "at most once", basically you need to ack the message
synchronously to the server before you give the message to the client via a
receive() call.
If you do that, there's a possibility of lost message if the client fails
after the ack is processed but before the receive() call returns, but
there's no possibility of the message getting delivered more than once after
falure (since it's always acked before receipt).
However if you send the ack async, then the message can actually get acked
*after* it's been giving to the user. Thus introducing a window during which
if failure occurs the message can get redelivered in contravention of JMS
spec.
--
View this message in context: http://www.nabble.com/JMS-Acknowledgements-sent-asynchronously--tp17519565p17520108.html
Sent from the Qpid Developers mailing list archive at Nabble.com.
Re: JMS Acknowledgements sent asynchronously?
Posted by Robert Greig <ro...@gmail.com>.
2008/5/28 TLFox <ti...@googlemail.com>:
>
> In a similar vein to the other thread I posted about persistent messages
> being sent async, can I confirm whether acknowledgements of message
> consumption (sent on auto_acknowledge, dups_ok, or client_acknowledge) are
> sent asynchronously or synchronously?
In a similar vein, it's asynchronous.
RG