You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by "xiaodan.wang" <xi...@salesforce.com> on 2014/09/04 02:20:36 UTC
Synchronous consumer can only dequeue 1 message in AMQP 0-91
Hi guys, we recently switched from protocol version AMQP 0-10 to AMQP 0-91 on
the v0.32 client and noticed that the behavior for synchronous consumers
(consumer.receive) has changed. For example, in AMQP 0-10, if we enqueue 5
messages to queue Q1, then calling "receive" on a consumer that is listening
to Q1 5 times in succession will return all 5 messages. However with AMQP
0-91, calling "receive" will only return the first message and subsequent
invocations will hang/time out even though there are additional messages
sitting on Q1 (we tested with a single consumer).
In both AMQP 0-10 and AMQP 0-91, we are using transacted sessions with max
prefetch of 1. We did notice that changing max prefetch to 3 in AMQP 0-91
allowed us to consume up to 3 messages synchronously and so on. While
intuitively this makes sense, we could not figure out why AMQP 0-10 allows
us to consume multiple messages synchronously with prefetch=1 while AMQP
0-91 does not.
Cheers! Dan
Context on why we switched from AMQP 0-91 to AMQP 0-10 with prefetch=1:
http://qpid.2158936.n2.nabble.com/Re-1-Queue-with-2-Consumers-turn-off-pre-fetching-td6934582.html
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by Rob Godfrey <ro...@gmail.com>.
Great - glad that's working for you.
Let me know if you run into any more issues, or if there is anything else I
can help you with.
Cheers,
Rob
On 9 September 2014 01:24, xiaodan.wang <xi...@salesforce.com> wrote:
> Replying to confirm that synchronous consumer with prefetch of 1 can
> dequeue
> > 1 message. Still need to setup a test for the case in which receive times
> out and a message arrives at a later time, but our existing synchronous
> consume tests are all passing :)
>
> Thanks again Rob!!!
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613328.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by "xiaodan.wang" <xi...@salesforce.com>.
Replying to confirm that synchronous consumer with prefetch of 1 can dequeue
> 1 message. Still need to setup a test for the case in which receive times
out and a message arrives at a later time, but our existing synchronous
consume tests are all passing :)
Thanks again Rob!!!
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613328.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by "xiaodan.wang" <xi...@salesforce.com>.
Awesome :) Will try this out and report back.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613325.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by Rob Godfrey <ro...@gmail.com>.
OK - I've committed a change to the 0-9-1 client code so that if you call
receive() on a transacted or client-ack session and you have already
received all your prefetch, then it will expand the prefetch window...
Probably too late now for 0.30... but it'll be in the next release and is
obviously on trunk now if you wish to try it
-- Rob
That's right, I was thinking of something else and got confused.
No plans to deviate from the one consumer per synchronous session approach
:)
--
View this message in context:
http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613081.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by "xiaodan.wang" <xi...@salesforce.com>.
That's right, I was thinking of something else and got confused.
No plans to deviate from the one consumer per synchronous session approach
:)
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613081.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by Rob Godfrey <ro...@gmail.com>.
On 4 September 2014 19:40, xiaodan.wang <xi...@salesforce.com> wrote:
> > In your case, in the synchronous session do you just have the
> one onsumer on the session?
>
> Yep, one consumer on the synchronous session. But I think this solution
> works even if we decide to use multiple consumer per synchronous session
> because each consumer is listening on a separate named queue.
>
>
Not on 0-9-1 because if your have multiple consumers (lets say for q1, q2
and q3)... you call q3.receive()... that opens up the credit window by one
for the entire session... so you may get a message for q1 or q2 rather than
q3 sent to you... in which case you have prefetched a message you weren't
asking for, and your receive times out because no message from q3 can
arrive because your prefetch buffer is now full :-(
But if you stick with one consumer per session for synchronous mode then
this solution would work for you.
-- Rob
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613079.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by "xiaodan.wang" <xi...@salesforce.com>.
> In your case, in the synchronous session do you just have the
one onsumer on the session?
Yep, one consumer on the synchronous session. But I think this solution
works even if we decide to use multiple consumer per synchronous session
because each consumer is listening on a separate named queue.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613079.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by Rob Godfrey <ro...@gmail.com>.
On 4 September 2014 19:13, xiaodan.wang <xi...@salesforce.com> wrote:
> Looks like I replied before seeing your followup :)
>
>
> > To expand a little further... The behaviour I'm thinking about for 0-9-1
> > would be that on a synchronous session, every time you call receive() the
> > client opens up the credit window enough for you to receive a message if
> > there isn't currently credit enough to get the message. At the point a
> > commit occurs (on a transacted session), or an acknowledge (on a client
> > ack
> > session), the window is reduced again such that no new messages will be
> > prefetched until a receive is called...
>
> And also if the window could be reduced when "receive" times out, mainly to
> prevent a message that arrives after we've started working on the
> previously
> dequeued set of message.
>
>
OK - good point - I'll add that to the requirements :-)
>
> > The other (potentially simpler for me, but hackier) possibility would be
> > to
> > provide some way of allowing you to manually flex the prefetch size on an
> > individual session... this would probably require casting into a qpid
> > specific class for you so you could directly manipulate the session
> > object.
>
> Would the manual re-sizing be done dynamically or only at session creation
> time? For us, over-sizing (setting prefetch of 10 when there are only 8
> messages at the moment) could lead to the starvation of late arrival
> messages mentioned above.
>
>
At the moment I'm thinking the 0-9 behaviour described above is the right
way to go, because, basically it is the sane behaviour anyone would want
from a synchronous session. The only issue is where you have mutliple
consumers on a single session in which case you might ask to receive() on
one consumer, but the prefetched message arrives on a different
consumer... In your case, in the synchronous session do you just have the
one onsumer on the session?
-- Rob
> Cheers! Dan
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613072.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by "xiaodan.wang" <xi...@salesforce.com>.
Looks like I replied before seeing your followup :)
> To expand a little further... The behaviour I'm thinking about for 0-9-1
> would be that on a synchronous session, every time you call receive() the
> client opens up the credit window enough for you to receive a message if
> there isn't currently credit enough to get the message. At the point a
> commit occurs (on a transacted session), or an acknowledge (on a client
> ack
> session), the window is reduced again such that no new messages will be
> prefetched until a receive is called...
And also if the window could be reduced when "receive" times out, mainly to
prevent a message that arrives after we've started working on the previously
dequeued set of message.
> The other (potentially simpler for me, but hackier) possibility would be
> to
> provide some way of allowing you to manually flex the prefetch size on an
> individual session... this would probably require casting into a qpid
> specific class for you so you could directly manipulate the session
> object.
Would the manual re-sizing be done dynamically or only at session creation
time? For us, over-sizing (setting prefetch of 10 when there are only 8
messages at the moment) could lead to the starvation of late arrival
messages mentioned above.
Cheers! Dan
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613072.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by Rob Godfrey <ro...@gmail.com>.
On 4 September 2014 18:47, Rob Godfrey <ro...@gmail.com> wrote:
>
>
>
> On 4 September 2014 18:26, xiaodan.wang <xi...@salesforce.com>
> wrote:
>
>> Thanks Rob, was afraid that was the case. We built a system with mixed
>> asynchronous and synchronous consumers using AMQP 0-10. Most of the time,
>> messages are consumed asynchronously. But if we dequeue a "bulk" message,
>> we
>> create a new session to consume multiple messages synchronously but wrap
>> the
>> acknowledgement/dequeue in the context of a single transaction.
>> Unfortunately, it sounds like delivery and acknowledgement goes
>> hand-in-hand
>> with AMQP 0-91 so "bulk" consume is no longer possible.
>>
>> A couple of quick questions from your reply in the other thread (sorry for
>> jumping around)
>>
>> http://qpid.2158936.n2.nabble.com/Re-1-Queue-with-2-Consumers-turn-off-pre-fetching-td6934582i20.html
>>
>> > It may be possible to code a client library side change without changing
>> > the broker (basically reduce consumer credit to 0 as soon as one
>> consumer
>> > has a message, and release any messages that have been prefetched), but
>> > that probably isn't a trivial piece of work.
>>
>> We might start looking into this as a short term solution. Is it easier to
>> try on an older client (i.e. v0.16)?
>>
>>
> Let me have a think about this over the weekend, I'm not sure whether the
> easier change will be to code up something in 0-10 or 0-9-1 (I'm inclined
> to think the latter may actually be simpler... but I'd need to look at it
> more carefully).
>
To expand a little further... The behaviour I'm thinking about for 0-9-1
would be that on a synchronous session, every time you call receive() the
client opens up the credit window enough for you to receive a message if
there isn't currently credit enough to get the message. At the point a
commit occurs (on a transacted session), or an acknowledge (on a client ack
session), the window is reduced again such that no new messages will be
prefetched until a receive is called...
I've not yet looked how "hard" this would be...
The other (potentially simpler for me, but hackier) possibility would be to
provide some way of allowing you to manually flex the prefetch size on an
individual session... this would probably require casting into a qpid
specific class for you so you could directly manipulate the session object.
I'm maxed out in meetings all of tmr, so I'm not going to have a proper
chance to investigate until Saturday....
Cheers,
Rob
>
>>
>> > Meanwhile I've also knocked up an implementation of the "single consumer
>> > across many queues" idea that I floated the other day. That won't make
>> it
>> > into 0.30 (it's currently only on my laptop), but I would hope it would
>> be
>> > in 0.32, or whatever the next release is called. Note that this change
>> > will require both a new client and a new broker, but should also be
>> easily
>> > available from all the other 0-10 clients.
>>
>> If there is plans to release this in v0.32/trunk, we will be super eager
>> to
>> test it out :)
>>
>
>
> The code is already on trunk, though it's very much "experimental". To
> enable the feature on the broker you need to run with
> -Dqpid.enableMultiQueueConsumers=true
>
> To consume from such an address you need to use an address string like
>
>
> ADDR: '' ; {node : { type : queue }, link : { x-subscribe : { arguments :
> { x-multiqueue : [ q1, q2, q3 ] } } } }
>
> where q1, q2 and q3 are the queues you wish to consumer from.
>
> Note the names of the arguments / system properties are open to change if
> I can think of something better... and like I said it's very experimental -
> I've not had a chance to do much other than a very basic test that it will
> receive messages from any of the given queues.
>
> Hope this helps,
> Rob
>
>>
>> Cheers! Dan
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613067.html
>> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by "xiaodan.wang" <xi...@salesforce.com>.
> Let me have a think about this over the weekend, I'm not sure whether the
> easier change will be to code up something in 0-10 or 0-9-1 (I'm inclined
> to think the latter may actually be simpler... but I'd need to look at it
> more carefully).
Thanks Rob! I assume by changing 0-9-1, you mean decoupling acknowledgement
from delivery on the client?
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613070.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by Rob Godfrey <ro...@gmail.com>.
On 4 September 2014 18:26, xiaodan.wang <xi...@salesforce.com> wrote:
> Thanks Rob, was afraid that was the case. We built a system with mixed
> asynchronous and synchronous consumers using AMQP 0-10. Most of the time,
> messages are consumed asynchronously. But if we dequeue a "bulk" message,
> we
> create a new session to consume multiple messages synchronously but wrap
> the
> acknowledgement/dequeue in the context of a single transaction.
> Unfortunately, it sounds like delivery and acknowledgement goes
> hand-in-hand
> with AMQP 0-91 so "bulk" consume is no longer possible.
>
> A couple of quick questions from your reply in the other thread (sorry for
> jumping around)
>
> http://qpid.2158936.n2.nabble.com/Re-1-Queue-with-2-Consumers-turn-off-pre-fetching-td6934582i20.html
>
> > It may be possible to code a client library side change without changing
> > the broker (basically reduce consumer credit to 0 as soon as one consumer
> > has a message, and release any messages that have been prefetched), but
> > that probably isn't a trivial piece of work.
>
> We might start looking into this as a short term solution. Is it easier to
> try on an older client (i.e. v0.16)?
>
>
Let me have a think about this over the weekend, I'm not sure whether the
easier change will be to code up something in 0-10 or 0-9-1 (I'm inclined
to think the latter may actually be simpler... but I'd need to look at it
more carefully).
>
> > Meanwhile I've also knocked up an implementation of the "single consumer
> > across many queues" idea that I floated the other day. That won't make
> it
> > into 0.30 (it's currently only on my laptop), but I would hope it would
> be
> > in 0.32, or whatever the next release is called. Note that this change
> > will require both a new client and a new broker, but should also be
> easily
> > available from all the other 0-10 clients.
>
> If there is plans to release this in v0.32/trunk, we will be super eager to
> test it out :)
>
The code is already on trunk, though it's very much "experimental". To
enable the feature on the broker you need to run with
-Dqpid.enableMultiQueueConsumers=true
To consume from such an address you need to use an address string like
ADDR: '' ; {node : { type : queue }, link : { x-subscribe : { arguments : {
x-multiqueue : [ q1, q2, q3 ] } } } }
where q1, q2 and q3 are the queues you wish to consumer from.
Note the names of the arguments / system properties are open to change if I
can think of something better... and like I said it's very experimental -
I've not had a chance to do much other than a very basic test that it will
receive messages from any of the given queues.
Hope this helps,
Rob
>
> Cheers! Dan
>
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613067.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by "xiaodan.wang" <xi...@salesforce.com>.
Thanks Rob, was afraid that was the case. We built a system with mixed
asynchronous and synchronous consumers using AMQP 0-10. Most of the time,
messages are consumed asynchronously. But if we dequeue a "bulk" message, we
create a new session to consume multiple messages synchronously but wrap the
acknowledgement/dequeue in the context of a single transaction.
Unfortunately, it sounds like delivery and acknowledgement goes hand-in-hand
with AMQP 0-91 so "bulk" consume is no longer possible.
A couple of quick questions from your reply in the other thread (sorry for
jumping around)
http://qpid.2158936.n2.nabble.com/Re-1-Queue-with-2-Consumers-turn-off-pre-fetching-td6934582i20.html
> It may be possible to code a client library side change without changing
> the broker (basically reduce consumer credit to 0 as soon as one consumer
> has a message, and release any messages that have been prefetched), but
> that probably isn't a trivial piece of work.
We might start looking into this as a short term solution. Is it easier to
try on an older client (i.e. v0.16)?
> Meanwhile I've also knocked up an implementation of the "single consumer
> across many queues" idea that I floated the other day. That won't make it
> into 0.30 (it's currently only on my laptop), but I would hope it would be
> in 0.32, or whatever the next release is called. Note that this change
> will require both a new client and a new broker, but should also be easily
> available from all the other 0-10 clients.
If there is plans to release this in v0.32/trunk, we will be super eager to
test it out :)
Cheers! Dan
--
View this message in context: http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017p7613067.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Synchronous consumer can only dequeue 1 message in AMQP 0-91
Posted by Rob Godfrey <ro...@gmail.com>.
Hi Dan,
in AMQP 0-9-1 there is only one notion of "acknowledgement" which is what
the prefetch is based on, so a new message will only be sent to the client
when it has acknowledged (taken responsibility for / asked to be dequeued)
the previous message.
AMQP 0-10 allows for the separation the notion of accepting the message
from the notion of confirming the receipt. Thus flow control can be based
on knowing that the message has been received (but not yet acknowledged) by
the client application. (If the 0-9-1 client were smarter it could have a
mode where it automatically increases the window when it knows that it has
delivered a message to the application but they have not yet acknowledged
it... however the client is not (yet) that smart).
Hope this helps,
Rob
On 4 September 2014 02:20, xiaodan.wang <xi...@salesforce.com> wrote:
> Hi guys, we recently switched from protocol version AMQP 0-10 to AMQP 0-91
> on
> the v0.32 client and noticed that the behavior for synchronous consumers
> (consumer.receive) has changed. For example, in AMQP 0-10, if we enqueue 5
> messages to queue Q1, then calling "receive" on a consumer that is
> listening
> to Q1 5 times in succession will return all 5 messages. However with AMQP
> 0-91, calling "receive" will only return the first message and subsequent
> invocations will hang/time out even though there are additional messages
> sitting on Q1 (we tested with a single consumer).
>
> In both AMQP 0-10 and AMQP 0-91, we are using transacted sessions with max
> prefetch of 1. We did notice that changing max prefetch to 3 in AMQP 0-91
> allowed us to consume up to 3 messages synchronously and so on. While
> intuitively this makes sense, we could not figure out why AMQP 0-10 allows
> us to consume multiple messages synchronously with prefetch=1 while AMQP
> 0-91 does not.
>
> Cheers! Dan
>
> Context on why we switched from AMQP 0-91 to AMQP 0-10 with prefetch=1:
>
> http://qpid.2158936.n2.nabble.com/Re-1-Queue-with-2-Consumers-turn-off-pre-fetching-td6934582.html
>
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Synchronous-consumer-can-only-dequeue-1-message-in-AMQP-0-91-tp7613017.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>