You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Aidan Skinner <ai...@apache.org> on 2008/06/26 16:35:39 UTC
Producer flow control
I wrote up a quick sketch of what I'm planning on implementing for
flow controlling producers to protect the broker from out of control
producers / dead clients causing massive build up. Comments on this
would be welcome, as I'm going to have to start hacking on it RSN.
http://cwiki.apache.org/confluence/display/qpid/Producer+flow+control
Danke,
- Aidan
--
Apache Qpid - World Domination through Advanced Message Queueing
http://cwiki.apache.org/qpid
Re: Producer flow control
Posted by Aidan Skinner <ai...@apache.org>.
On Thu, Jun 26, 2008 at 10:23 PM, Rafael Schloming <ra...@redhat.com> wrote:
> Why throw an exception by default? I'd think you would want a configurable
> timeout, i.e. by default you block for n seconds, and when the time is up
> throw an exception.
JMS API compatability more than anything else, there is no send(int) :/
Having a configurable timeout rather than blocking is a good idea
though, I'll amend the design so that you specify a system property
with the timeout, if that's not default (0) then it blocks for N
milliseconds and then throws, or -1 to block forever?
> What should the client expect when an exception is thrown, did the message
> go through or not?
That the message did not go through, I'll document this.
> Why stop the producer *after* putting the message on the queue? Is it really
> that expensive to check whether the queue is too large *before* putting the
> message on? In general this seems backwards, e.g. if a client were to
> publish a single message to a queue that is full, and then move on to other
> unfull queues, you could end up blocking the client for no reason.
It's not really because of the cost of the check, it's because there's
no PublishOK in 0-8/0-9 to signal success outside of a transaction.
> Are the limits going to be byte based, message based, or both?
Both
> How do you intend to block the producer?
Check if the connection is flowed, if it is then wait on a flow
semaphore that gets signaled when the connection is reflowed.
- Aidan
--
Apache Qpid - World Domination through Advanced Message Queueing
http://cwiki.apache.org/qpid
Re: Producer flow control
Posted by Marnie McCormack <ma...@googlemail.com>.
SonicMQ also does some interesting stuff with flow to store for messages
until flow control is no longer in effect, which is nice:
http://www.psdn.com/library/servlet/KbServlet/download/2679-102-5202/mq_application_program.pdf
Pg 221.
Worth thought I reckon.
Bfn,
Marnie
On 6/27/08, Marnie McCormack <ma...@googlemail.com> wrote:
>
> Hi,
>
> Having thought about it a little, I think the limiting would really need to
> be able to be configured on queue size if possible as this is the most
> useful limit that client applications could set, particularly for users of
> transient messaging who would wish to avoid OoM.
>
> The ability to set it on message number is useful too for applications that
> have business related thresholds i.e. to avoid flooding downstream order
> processing systems etc. I bet the message number check is cheaper :-)
>
> I'm coming around to the view that we need to do this work very carefully,
> and I'm not sure that needing message publication to trigger flow control on
> a producer is ideal. Been looking at the Active MQ producer window size and
> wonder if we can't use a similar approach ?
> (
> http://activemq.apache.org/maven/activemq-core/apidocs/src-html/org/apache/activemq/ActiveMQMessageProducer.html#line.71
> )
>
> Thoughts ?
>
> Bfn,
> Marnie
>
> On 6/26/08, Rafael Schloming <ra...@redhat.com> wrote:
>>
>> Aidan Skinner wrote:
>>
>>> I wrote up a quick sketch of what I'm planning on implementing for
>>> flow controlling producers to protect the broker from out of control
>>> producers / dead clients causing massive build up. Comments on this
>>> would be welcome, as I'm going to have to start hacking on it RSN.
>>>
>>> http://cwiki.apache.org/confluence/display/qpid/Producer+flow+control
>>>
>>
>> A few random questions/comments...
>>
>> Why throw an exception by default? I'd think you would want a configurable
>> timeout, i.e. by default you block for n seconds, and when the time is up
>> throw an exception.
>>
>> What should the client expect when an exception is thrown, did the message
>> go through or not?
>>
>> Why stop the producer *after* putting the message on the queue? Is it
>> really that expensive to check whether the queue is too large *before*
>> putting the message on? In general this seems backwards, e.g. if a client
>> were to publish a single message to a queue that is full, and then move on
>> to other unfull queues, you could end up blocking the client for no reason.
>>
>> Are the limits going to be byte based, message based, or both?
>>
>> How do you intend to block the producer?
>>
>> --Rafael
>>
>
>
Re: Producer flow control
Posted by Alan Conway <ac...@redhat.com>.
On Fri, 2008-06-27 at 08:47 -0400, Rafael Schloming wrote:
> Aidan Skinner wrote:
> > On Fri, Jun 27, 2008 at 8:38 AM, Marnie McCormack
> > <ma...@googlemail.com> wrote:
> >
> >> I'm coming around to the view that we need to do this work very carefully,
> >> and I'm not sure that needing message publication to trigger flow control on
> >> a producer is ideal. Been looking at the Active MQ producer window size and
> >> wonder if we can't use a similar approach ?
> >
> > It's definately not ideal, the window size stuff is, AFAICT, quite
> > similar to how 0-10 works but I don't think we can implement that with
> > 0-8/0-9. :(
>
> One option you have on 0-8/0-9 is to stop reading from the socket. This
> should cause the TCP window to fill up, and the client will eventually
> block (or run out of memory because it's stupid enough to buffer
> indefinitely).
>
> Something like this should allow you to do more sophisticated
> rate-limiting type things in addition to just a binary on/off.
>
> Of course this sort of thing is probably easier said than done given
> that we use the worlds Most Irritating Network Api. ;)
>
> --Rafael\
I added some comments to the Jira about the existing AMQP flow control
and the stop-reading-sockets approach.
Re: Producer flow control
Posted by Rafael Schloming <ra...@redhat.com>.
Aidan Skinner wrote:
> On Fri, Jun 27, 2008 at 8:38 AM, Marnie McCormack
> <ma...@googlemail.com> wrote:
>
>> I'm coming around to the view that we need to do this work very carefully,
>> and I'm not sure that needing message publication to trigger flow control on
>> a producer is ideal. Been looking at the Active MQ producer window size and
>> wonder if we can't use a similar approach ?
>
> It's definately not ideal, the window size stuff is, AFAICT, quite
> similar to how 0-10 works but I don't think we can implement that with
> 0-8/0-9. :(
One option you have on 0-8/0-9 is to stop reading from the socket. This
should cause the TCP window to fill up, and the client will eventually
block (or run out of memory because it's stupid enough to buffer
indefinitely).
Something like this should allow you to do more sophisticated
rate-limiting type things in addition to just a binary on/off.
Of course this sort of thing is probably easier said than done given
that we use the worlds Most Irritating Network Api. ;)
--Rafael
Re: Producer flow control
Posted by Aidan Skinner <ai...@apache.org>.
On Fri, Jun 27, 2008 at 8:38 AM, Marnie McCormack
<ma...@googlemail.com> wrote:
> I'm coming around to the view that we need to do this work very carefully,
> and I'm not sure that needing message publication to trigger flow control on
> a producer is ideal. Been looking at the Active MQ producer window size and
> wonder if we can't use a similar approach ?
It's definately not ideal, the window size stuff is, AFAICT, quite
similar to how 0-10 works but I don't think we can implement that with
0-8/0-9. :(
- Aidan
--
Apache Qpid - World Domination through Advanced Message Queueing
http://cwiki.apache.org/qpid
Re: Producer flow control
Posted by Marnie McCormack <ma...@googlemail.com>.
Hi,
Having thought about it a little, I think the limiting would really need to
be able to be configured on queue size if possible as this is the most
useful limit that client applications could set, particularly for users of
transient messaging who would wish to avoid OoM.
The ability to set it on message number is useful too for applications that
have business related thresholds i.e. to avoid flooding downstream order
processing systems etc. I bet the message number check is cheaper :-)
I'm coming around to the view that we need to do this work very carefully,
and I'm not sure that needing message publication to trigger flow control on
a producer is ideal. Been looking at the Active MQ producer window size and
wonder if we can't use a similar approach ?
(
http://activemq.apache.org/maven/activemq-core/apidocs/src-html/org/apache/activemq/ActiveMQMessageProducer.html#line.71
)
Thoughts ?
Bfn,
Marnie
On 6/26/08, Rafael Schloming <ra...@redhat.com> wrote:
>
> Aidan Skinner wrote:
>
>> I wrote up a quick sketch of what I'm planning on implementing for
>> flow controlling producers to protect the broker from out of control
>> producers / dead clients causing massive build up. Comments on this
>> would be welcome, as I'm going to have to start hacking on it RSN.
>>
>> http://cwiki.apache.org/confluence/display/qpid/Producer+flow+control
>>
>
> A few random questions/comments...
>
> Why throw an exception by default? I'd think you would want a configurable
> timeout, i.e. by default you block for n seconds, and when the time is up
> throw an exception.
>
> What should the client expect when an exception is thrown, did the message
> go through or not?
>
> Why stop the producer *after* putting the message on the queue? Is it
> really that expensive to check whether the queue is too large *before*
> putting the message on? In general this seems backwards, e.g. if a client
> were to publish a single message to a queue that is full, and then move on
> to other unfull queues, you could end up blocking the client for no reason.
>
> Are the limits going to be byte based, message based, or both?
>
> How do you intend to block the producer?
>
> --Rafael
>
Re: Producer flow control
Posted by Rafael Schloming <ra...@redhat.com>.
Aidan Skinner wrote:
> I wrote up a quick sketch of what I'm planning on implementing for
> flow controlling producers to protect the broker from out of control
> producers / dead clients causing massive build up. Comments on this
> would be welcome, as I'm going to have to start hacking on it RSN.
>
> http://cwiki.apache.org/confluence/display/qpid/Producer+flow+control
A few random questions/comments...
Why throw an exception by default? I'd think you would want a
configurable timeout, i.e. by default you block for n seconds, and when
the time is up throw an exception.
What should the client expect when an exception is thrown, did the
message go through or not?
Why stop the producer *after* putting the message on the queue? Is it
really that expensive to check whether the queue is too large *before*
putting the message on? In general this seems backwards, e.g. if a
client were to publish a single message to a queue that is full, and
then move on to other unfull queues, you could end up blocking the
client for no reason.
Are the limits going to be byte based, message based, or both?
How do you intend to block the producer?
--Rafael