consumer flow control??

[Fraser Adams <fr...@blueyonder.co.uk>]

Hello,
I seem to recall that it's possible to set consumer flow control with 
Qpid is that correct?

I can't see anything specific in the documentation relating to consumer 
flow control, I thought that it had something to do with message credits 
or consumer prefetching but I'm not sure how this actually relates to 
throttling consumers to a particular rate.

TBH I also thought that there was a "capacity" option in the 
AddressString for link, in 
http://qpid.apache.org/books/0.20/Programming-In-Apache-Qpid/html/ch03s06.html 
there's mention of the |-Dmax_prefetch=1000 and |
|{create: always, link:{capacity: 10}} |but it no longer mentions 
capacity in the Link Properties section in table 2.3 so I'm not sure.

I had a quick play setting capacity and max_prefetch to some low value 
such as 10 or 1 but to be honest I couldn't see much difference to the 
rate that the consumer was pulling messages.

Have I missed something, can anyone offer any tips as to how to throttle 
consumers??

Cheers,
Frase

Re: consumer flow control??

[Gordon Sim <gs...@redhat.com>]

On 05/10/2013 09:54 AM, Fraser Adams wrote:
> On the subject of capacity though do you know why the capacity property
> has been removed from table 2.3 in Programming in Apache Qpid
> (describing link properties), I *thought* that this was documented,
> Clearly with qpid::messaging it's easy to set with setCapacity() but
> with JMS it's necessary to use a JVM property a ConnectionURL property
> or (I thought) a link property.

Only the JMS client I believe recognised the capacity option as a link 
property specified in the address, the others had it as an explicitly 
set on the receiver.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: consumer flow control??

[Fraser Adams <fr...@blueyonder.co.uk>]

On 08/05/13 21:49, Ted Ross wrote:
>
> Consumer flow control is where the consumer throttles the broker. Flow 
> control is credit-based, not rate-based.  When you set the capacity on 
> a receiver (qpid::messaging::Receiver.setCapacity() or equivalent), 
> you are setting the upper limit on the number of messages that will be 
> sent by the broker to the receiver without acknowledgement.  This 
> causes the broker to match the speed of the consumer as it processes 
> messages.
>
> Are you looking for a way to cap the transfer rate to a particular 
> consumer?
>
>
Yep, that's exactly what I'd like. The irony is I *normally* want 
everything to go as fast as possible but I've got one scenario where  
I'm actually bridging to a system that is running into problems with the 
data rate. What I'd really like is for my bridge client to have a 
maximum consume rate. Message loss isn't critical for this scenario and 
I'm using a ring queue so if the messages aren't consumed as fast as 
they are produced so be it. I can clearly write code in my bridge to 
manage this, but it would have been nice if I didn't have to :-)

On the subject of capacity though do you know why the capacity property 
has been removed from table 2.3 in Programming in Apache Qpid 
(describing link properties), I *thought* that this was documented, 
Clearly with qpid::messaging it's easy to set with setCapacity() but 
with JMS it's necessary to use a JVM property a ConnectionURL property 
or (I thought) a link property. I'm still curious why messing with this 
didn't seem to have a noticeable affect, I'm guessing it's because I was 
using my normal test code that has messages of 50K so I suspect 
prefetch/capacity tuning may have marginal effect - I had an inkling 
that the prefetch/capacity model is there to minimise network 
"chattiness" so it likely to be more noticeable when sending smaller 
messages.

Frase

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: consumer flow control??

[Ted Ross <tr...@redhat.com>]

On 05/08/2013 02:42 PM, Fraser Adams wrote:
> Hello,
> I seem to recall that it's possible to set consumer flow control with 
> Qpid is that correct?
>
> I can't see anything specific in the documentation relating to 
> consumer flow control, I thought that it had something to do with 
> message credits or consumer prefetching but I'm not sure how this 
> actually relates to throttling consumers to a particular rate.
>
> TBH I also thought that there was a "capacity" option in the 
> AddressString for link, in 
> http://qpid.apache.org/books/0.20/Programming-In-Apache-Qpid/html/ch03s06.html 
> there's mention of the |-Dmax_prefetch=1000 and |
> |{create: always, link:{capacity: 10}} |but it no longer mentions 
> capacity in the Link Properties section in table 2.3 so I'm not sure.
>
> I had a quick play setting capacity and max_prefetch to some low value 
> such as 10 or 1 but to be honest I couldn't see much difference to the 
> rate that the consumer was pulling messages.
>
> Have I missed something, can anyone offer any tips as to how to 
> throttle consumers??
>
> Cheers,
> Frase
>

Consumer flow control is where the consumer throttles the broker. Flow 
control is credit-based, not rate-based.  When you set the capacity on a 
receiver (qpid::messaging::Receiver.setCapacity() or equivalent), you 
are setting the upper limit on the number of messages that will be sent 
by the broker to the receiver without acknowledgement.  This causes the 
broker to match the speed of the consumer as it processes messages.

Are you looking for a way to cap the transfer rate to a particular consumer?

-Ted


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org