You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by Michael Goulish <mg...@redhat.com> on 2013/02/26 18:34:22 UTC
messenger credit concern
One hole that I feel like I'm seeing in the messenger
interface concerns credit.
I have a way of using credit to set a max number of
messages that a recv call should return in one gulp,
or a way of doing ... something that I'm still figuring
out ... by setting credit=-1.
What I don't have is any way of getting guidance about
what effect my credit allocation is having.
A messenger app might have some flexibility in how
much time it spends servicing incoming messages vs.
time it spends doing its own processing, and it might
be able to allocate more time to servicing incoming
messages if it knows more about what's happening.
Alternatively, it might want to set the credit allocated
per recv call based on the number of current incoming
links. ( And assume that the credit will be distributed
round-robin across all incoming links. )
Would it be practical / desirable / catastrophic
to expose current backlog or number of incoming links,
or both, at the messenger level ?
Or would that violate part of the messenger API philosophy?
( And if so, what is that philosophy? I want to be able
to explain it. )
Re: messenger credit concern
Posted by Michael Goulish <mg...@redhat.com>.
Oh crap, you're right.
I have to remember what I saw in the API, and what I saw only in the code.
OK, re-writing.....
----- Original Message -----
From: "Rafael Schloming" <rh...@alum.mit.edu>
I think as soon as you start talking about credit you're already at much
lower level than is appropriate for the messenger API.
Re: messenger credit concern
Posted by Michael Goulish <mg...@redhat.com>.
----- Original Message -----
>I think as soon as you start talking about credit you're already at much
>lower level than is appropriate for the messenger API. The messenger
>interface has API doc in both C and python and they both describe the
>semantics of recv without making any mention of the term credit. In fact
>that term doesn't appear anywhere in the messenger API or docs, only in the
>implementation.
>
>On Tue, Feb 26, 2013 at 9:34 AM, Michael Goulish <mg...@redhat.com>wrote:
>
>>
>> One hole that I feel like I'm seeing in the messenger
>> interface concerns credit.
>>
>> I have a way of using credit to set a max number of
>> messages that a recv call should return in one gulp,
>> or a way of doing ... something that I'm still figuring
>> out ... by setting credit=-1.
>>
>> What I don't have is any way of getting guidance about
>> what effect my credit allocation is having.
>>
>> A messenger app might have some flexibility in how
>> much time it spends servicing incoming messages vs.
>> time it spends doing its own processing, and it might
>> be able to allocate more time to servicing incoming
>> messages if it knows more about what's happening.
>>
>
>What's the distinction between "servicing incoming messages" and "doing its
>own processing"?
Apps that use messaging have work to do other than handling messages.
The work that they do is what they get paid for -- the messaging is
how they communicate with their peers to get new work to do, or subcontract
out tasks, or whatever.
An app has a fixed amount of compute power available that it has to
allocate between doing its payload work, and looking at incoming messages.
( Or maybe nodes can request more compute power, but they have to know
that they need it. )
In some applications you can make tradeoffs. Do your own work faster,
and thus be able to handle more incoming requests. Or do it more
thoroughly and handle fewer tasks. ( example: a machine vision
system where I want to use messaging can choose more expensive operations
in hopes of getting a slightly more accurate result. But if it is building
up a large number of incoming requests, it can choose to go cheap-and-fast.)
It seems to me that one of the fundamental questions that a communicating
node has to ask, to be a good citizen of the network, is "How many people
are *trying* to talk to me?" A count of the messages that *could* have
been given to you if you had made the N in recv() larger.
Yes, so as you say below "a count of remotely blocked messages".
If that could be exposed without a philosophy-violation, then I vote for it.
Otherwise, a node has a way of saying how many messages we will take,
but no way to get knowledge about what effect that decision is having on
its customers.
*** Now maybe *** concerns about congestion in the network should be handled
at a higher level. i.e. the manager sees that node A is building up a backlog
of messages trying to get to it -- so he fires off another copy of node A
and reroutes some of the load.
But I thought it would be nice to allow the network designer the option of
putting some of that intelligence right at node A. So it could say something
like "Oh crap, look at all the requests waiting for me! I've gotta speed up!"
>I'm struggling to imagine a scenario where such a division
>would be discretely controlled as opposed to say things happening in
>different threads. That aside, I'm also not sure how that would play into
>credit. Credit is really a measure of how many messages the receiver can
>handle at any given point, and doesn't necessarily relate at all to how
>many incoming messages are available, e.g. just because you allocate 1000
>units of credit doesn't mean there are any messages to handle. The number
>of actual messages currently being buffered would seem to be much more
>relevant to any sort of manual allocation of processing, and this is
>directly available through the API already via the incoming field.
>
>Alternatively, it might want to set the credit allocated
>> per recv call based on the number of current incoming
>> links. ( And assume that the credit will be distributed
>> round-robin across all incoming links. )
>>
>> Would it be practical / desirable / catastrophic
>> to expose current backlog or number of incoming links,
>> or both, at the messenger level ?
>>
>
>What do you mean by backlog? If you're talking about messages buffered by
>the messenger, this is already available through the incoming field. If
>you're talking about remotely blocked messages, then we could certainly
>expose an aggregate count without violating any philosophy, however I think
>it would be a major issue if any but the most advanced/obscure scenarios
>would actually require using such a thing.
>
>
>> Or would that violate part of the messenger API philosophy?
>> ( And if so, what is that philosophy? I want to be able
>> to explain it. )
>>
>
>I would say the philosophy is the same as in brokered messaging. The user
>just wants to focus on processing messages and shouldn't have to care about
>where they are coming from or how they arrive.
>
>Imagine a messenger app that is configured with one or more subscriptions
>and processes whatever messages arrive. You could pass that app an address
>of "amqp://~0.0.0.0/" or "amqp://foobroker/queue". In the former case
>incoming messages will be arriving to the messenger on potentially
>thousands of connections and links. In the latter case messages will be
>arriving to the messenger on a single link over one connection. If you have
>to alter your application code for the different scenarios then I would say
>we've failed to provide a sufficient messenger implementation.
>
>I would also point out that in the latter scenario you are still funneling
>all the messages into a single queue and you still need to solve the very
>same credit allocation issues. That queue just happens to reside in a
>remote broker rather than locally colocated with the messenger.
>
>--Rafael
>
Re: messenger credit concern
Posted by Rafael Schloming <rh...@alum.mit.edu>.
I think as soon as you start talking about credit you're already at much
lower level than is appropriate for the messenger API. The messenger
interface has API doc in both C and python and they both describe the
semantics of recv without making any mention of the term credit. In fact
that term doesn't appear anywhere in the messenger API or docs, only in the
implementation.
On Tue, Feb 26, 2013 at 9:34 AM, Michael Goulish <mg...@redhat.com>wrote:
>
> One hole that I feel like I'm seeing in the messenger
> interface concerns credit.
>
> I have a way of using credit to set a max number of
> messages that a recv call should return in one gulp,
> or a way of doing ... something that I'm still figuring
> out ... by setting credit=-1.
>
> What I don't have is any way of getting guidance about
> what effect my credit allocation is having.
>
> A messenger app might have some flexibility in how
> much time it spends servicing incoming messages vs.
> time it spends doing its own processing, and it might
> be able to allocate more time to servicing incoming
> messages if it knows more about what's happening.
>
What's the distinction between "servicing incoming messages" and "doing its
own processing"? I'm struggling to imagine a scenario where such a division
would be discretely controlled as opposed to say things happening in
different threads. That aside, I'm also not sure how that would play into
credit. Credit is really a measure of how many messages the receiver can
handle at any given point, and doesn't necessarily relate at all to how
many incoming messages are available, e.g. just because you allocate 1000
units of credit doesn't mean there are any messages to handle. The number
of actual messages currently being buffered would seem to be much more
relevant to any sort of manual allocation of processing, and this is
directly available through the API already via the incoming field.
Alternatively, it might want to set the credit allocated
> per recv call based on the number of current incoming
> links. ( And assume that the credit will be distributed
> round-robin across all incoming links. )
>
> Would it be practical / desirable / catastrophic
> to expose current backlog or number of incoming links,
> or both, at the messenger level ?
>
What do you mean by backlog? If you're talking about messages buffered by
the messenger, this is already available through the incoming field. If
you're talking about remotely blocked messages, then we could certainly
expose an aggregate count without violating any philosophy, however I think
it would be a major issue if any but the most advanced/obscure scenarios
would actually require using such a thing.
> Or would that violate part of the messenger API philosophy?
> ( And if so, what is that philosophy? I want to be able
> to explain it. )
>
I would say the philosophy is the same as in brokered messaging. The user
just wants to focus on processing messages and shouldn't have to care about
where they are coming from or how they arrive.
Imagine a messenger app that is configured with one or more subscriptions
and processes whatever messages arrive. You could pass that app an address
of "amqp://~0.0.0.0/" or "amqp://foobroker/queue". In the former case
incoming messages will be arriving to the messenger on potentially
thousands of connections and links. In the latter case messages will be
arriving to the messenger on a single link over one connection. If you have
to alter your application code for the different scenarios then I would say
we've failed to provide a sufficient messenger implementation.
I would also point out that in the latter scenario you are still funneling
all the messages into a single queue and you still need to solve the very
same credit allocation issues. That queue just happens to reside in a
remote broker rather than locally colocated with the messenger.
--Rafael
Re: messenger credit concern
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Mar 1, 2013 at 9:10 AM, Ken Giusti <kg...@redhat.com> wrote:
> Thanks, I think you've clarified my understanding of the intended behavior
> of N in messenger::recv(N) - it places a limit to the number of messages
> the application can pn_messenger_get() for that invokation of recv(N). If
> N==-1, then the number is unspecified.
>
> My understanding is that this API is not making any explicit guarantees
> about what is happening at the wire level - for example, it's possible that
> more messages may in fact arrive via the network - is that correct?
>
> I can buy that.
>
It's certainly not making explicit guarantees about what happens on the
wire, but it's not good in general for the implementation to ever prefetch
more than it needs to. As I mentioned in my reply to Alan on this subject
prefetching too much can have very negative consequences if you're trying
to load balance messages across multiple consumers, and prefetching in
general introduces additional risk/exposure. If a client app asks for 10
messages, processes them, and then exits, and we optimistically prefetch
1024, we've just caused 1014 messages to be abandoned. Even if they were
sent pre-settled it's still not great, it could mean additional load on the
senders since they need to requeue those messages, and some queue
implementations would hold those messages for an extended period before
allowing them to be requeued.
That said, I don't think we urgently need to solve the pathological case
where you have thousands of incoming links and the app is asking for one
message at a time, but that is actually quite straightforward if we
properly implement and use the "available" field in the protocol, and I
actually think it enables some fairly powerful use cases.
Bingo! That's _exactly_ my problem: the current impl tries to make a
> mapping from the recv(N) argument into the proper distribution of link
> credit. And it does so badly in the multi-link case (w/o the ability to
> really drain, as you point out). Using N makes it hard to optimize credit
> allocation for any scenario that involves more than one underlying link
> (probably the case for scenario(1) and some cases of scenario(3)). Good
> optimization for multi-link is complicated because N is unconstrained
> between calls to recv(N) - very hard to do something predictive and fair in
> this case.
>
> However, establishing bounds not based on recv(N) - as you point out -
> would go a long way to making a very nice multi-link credit distribution
> impl.
>
> From a documentation point of view - yes, certainly, a description of
> recv(N) shouldn't involve describing credit at all.
>
Given that we have recv(), I don't think anyone actually needs to use
recv(N) unless they actually really mean I want N messages and no more
(right now), so I don't think we need to worry about recv(N) being fast for
the case where there are lots of underlying links. I think you would use it
if, e.g., you want to write a script that pops exactly N messages off of a
queue, in which case you really don't want any prefetch.
--Rafael
Re: messenger credit concern
Posted by Rafael Schloming <rh...@alum.mit.edu>.
Oops, this was supposed to go to the list.
On Mon, Mar 4, 2013 at 11:56 AM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Thu, Feb 28, 2013 at 12:55 PM, Alan Conway <ac...@redhat.com> wrote:
>
>> I think this is a common case:
>>
>> (1a) an app wants to receive and process messages indefinitely, but wants
>> the implementation to use a bounded buffer of N messages or B bytes to do
>> so. AKA "credit window" in AMQP 0.10 or "prefetch" in JMS.
>>
>> I'm not familiar enough with Messenger yet to say whether that belongs in
>> the messenger API or in some other configuration, but I think it needs to
>> be a use case that is easy to set up. Agreed that ideally we would have a
>> dynamic flow control algorithm that can figure out the optimal credit
>> settings by itself, but until we do I suspect the simple "bounded buffer"
>> model will cover most cases, and doesn't require exposing the complexity of
>> the underlying flow control.
>>
>
> This is an interesting scenario because while it's a common case, it's
> arguably not a truly general case. In fact it is an anti-pattern in some
> cases.
>
> The problem this approach is that it depends on there being only a small
> penalty from using an oversized prefetch limit. Based on this assumption
> you can avoid computing the optimal prefetch amount and simply advise the
> user to statically assign a number large enough to guarantee in practice
> that there will always be messages sitting in the prefetch buffer. This
> works fine for a simple point to point scenario where the only cost is
> wasted memory on the client side for holding messages that didn't need to
> be prefetched in order to avoid stalling.
>
> Consider, however, what happens if you copy the point to point receiver
> and based on its code try to set up a job queue that load balances messages
> to competing consumers. Even a relatively small prefetch like 1024 can now
> result in pathological behaviour as your first consumer may well suck up
> all the messages in the queue and prevent other consumers from processing
> them.
>
> Now given that it is quite trivial to write message processing code that
> has no clue if it is running in a point to point vs load balanced
> environment, I think it would be unfortunate if our API forced users to
> make that code become specific to the topology in which it happens to be
> deployed. What this amounts to is that any number appearing in the API used
> by application code to fetch messages needs to actually have semantic
> meaning to the *application* and *not* the topology (e.g. the application
> is actually expecting N replies and no more).
>
> A possible consequence of this is that the messenger implementation needs
> to be smarter and track both round trip time and message processing time,
> however measuring these quantities isn't actually all that difficult or
> expensive, and if these numbers are known it is quite trivial to compute an
> optimal prefetch limit. Given this, I don't think there is really a good
> reason to make users guess at what an optimal value might be and I
> certainly wouldn't put it as part of the primary message retrieval API,
> rather as a tuning parameter for the internal algorithm.
>
> --Rafael
>
>
Re: messenger credit concern
Posted by Alan Conway <ac...@redhat.com>.
On 02/28/2013 03:11 PM, Rafael Schloming wrote:
> On Tue, Feb 26, 2013 at 12:18 PM, Ken Giusti <kg...@redhat.com> wrote:
[snip]
> As I mentioned above, I don't think recv should be thought of as a flow
> control thing. It does provide input into what messenger does for flow
> control, but it's really just about a way for the app to fetch messages,
> and so far I've been considering three scenarios:
>
> (1) an app wants to receive and process messages indefinitely, in which
> case pn_messenger_recv(-1) now does that job pretty nicely
> (2) an app wants to make a simple request/response, in which case it
> wants to receive exactly one message back and getting anymore would be a
> bug.
> (3) a generalized form of option (2) where an app makes N requests and
> processes each one as they arrive back. In this case you have to do
> pn_messenger_recv(N - what you've already processed).
>
> I think these are probably the 3 most common scenarios, and I can see how
> using a pattern like (3) to cater to scenario (1) would be awkward, however
> I think it's less awkward when used in scenario (3).
>
> That said I'm open to trying to simplify the API here, but I fundamentally
> don't think of this as a wire level flow control API. I get the impression
> from the comments in this thread that there is an idea that the app
> developer somehow has more knowledge or is in a better position to
> distribute credit than the messenger implementation, whereas I think the
> opposite is true. In my experience, the nature/shape of incoming message
> traffic is not a variable that is well known at development time. Perhaps
> you can define the extreme bounds of what loads you want to be able to
> handle, but at runtime there are many unpredictable factors:
>
> - your service can go from idle to extreme bursts with no warning
> - round trip times can fluctuate based general network activity
> - message processing times might vary due to other activity on
> your machine or degradation in services you depend on
> - your load might be unevenly distributed across different
> links/connections
> - a buggy or malicious app might be (unintentionally) DoSing you
>
> All of these factors and more go into determining the optimal and/or fair
> credit allocation at any given point in time, and that means a robust flow
> control algorithm really needs to be dynamic in nature. Not only that, but
> a robust flow control algorithm is a huge part of the value that messaging
> infrastructure provides, and should really be a fundamentally separate
> concern from how apps logically process messages.
>
>
>> Since Messenger models a queue of incoming messages, I'd rather see flow
>> control configured as thresholds on that queue, and recv() not take any
>> arguments at all.
>>
>> Something like this:
>>
>> Messenger m;
>> ...
>> m.set_flow_stop( 10000 )
>> m.set_flow_resume( 9000 )
>> ...
>> for (;;) {
>> m.recv()
>> while (m.incoming())
>> ....
>>
>> IMHO, this is a lot "cleaner" than the current approach. Of course, some
>> may find my sample names too cryptic :)
>>
>
> I think limits the API to only the first scenario I described above. At
> least it's not clear to me how you'd fetch exactly N messages.
>
>
>>
>> From an implementation point of view, the "flow stop" threshold is really
>> just a suggestion for how much credit should be distributed across the
>> links. We could distribute more, as we would need to if the number of
>> links is greater than the flow stop threshold. Or less, assume a point of
>> diminishing returns.
>>
>> Once the flow stop threshold is hit, credit would be drained from all
>> links. No further credit would be granted until the number of "queued"
>> messages drops below "flow resume".
>>
>> This is the same model we use for queue flow control in the C++ broker.
>>
>
> This is starting to mix two things: (1) how the application fetches
> messages from the messenger, and (2) how to tune the messengers internal
> flow control algorithm in the specific case that the application wants to
> receive messages indefinitely. I think (2) is premature given that we
> haven't really done any performance work yet. Ideally I'd say we don't want
> to have to tune it, rather just give it some bounds to work within, e.g.
> limit to no more than X megabytes or no more than Y messages.
>
> In any case I think we need to be clear on the application scenarios we're
> trying to support. I've given (3) common ones above. Are there cases that
> you think are missing, and do you have a better way to cater to the 3 I've
> mentioned?
>
I think this is a common case:
(1a) an app wants to receive and process messages indefinitely, but wants the
implementation to use a bounded buffer of N messages or B bytes to do so. AKA
"credit window" in AMQP 0.10 or "prefetch" in JMS.
I'm not familiar enough with Messenger yet to say whether that belongs in the
messenger API or in some other configuration, but I think it needs to be a use
case that is easy to set up. Agreed that ideally we would have a dynamic flow
control algorithm that can figure out the optimal credit settings by itself, but
until we do I suspect the simple "bounded buffer" model will cover most cases,
and doesn't require exposing the complexity of the underlying flow control.
Re: messenger credit concern
Posted by Ken Giusti <kg...@redhat.com>.
----- Original Message -----
> On Tue, Feb 26, 2013 at 12:18 PM, Ken Giusti <kg...@redhat.com>
> wrote:
>
> > Hi Mick
> >
> > ----- Original Message -----
> > >
> > > One hole that I feel like I'm seeing in the messenger
> > > interface concerns credit.
> > >
> > > I have a way of using credit to set a max number of
> > > messages that a recv call should return in one gulp,
> > > or a way of doing ... something that I'm still figuring
> > > out ... by setting credit=-1.
> > >
> > > What I don't have is any way of getting guidance about
> > > what effect my credit allocation is having.
> > >
> > > A messenger app might have some flexibility in how
> > > much time it spends servicing incoming messages vs.
> > > time it spends doing its own processing, and it might
> > > be able to allocate more time to servicing incoming
> > > messages if it knows more about what's happening.
> > >
> > > Alternatively, it might want to set the credit allocated
> > > per recv call based on the number of current incoming
> > > links. ( And assume that the credit will be distributed
> > > round-robin across all incoming links. )
> > >
> > > Would it be practical / desirable / catastrophic
> > > to expose current backlog or number of incoming links,
> > > or both, at the messenger level ?
> > >
> > > Or would that violate part of the messenger API philosophy?
> > > ( And if so, what is that philosophy? I want to be able
> > > to explain it. )
> > >
> >
> > I feel your pain - the more I use the "recv(n)" interface, the more
> > I
> > think it's an awkward approach to the credit/flow issue.
> >
>
> I don't think it's aiming to solve a credt/flow issue at all. That's
> the
> job of the messenger implementation. It's simply providing a way for
> the
> application to ask for n messages.
>
>
> >
> > From what I understand, Messenger's model is based on two queues -
> > one for
> > incoming messages and one for outgoing messages.
> >
> > For flow control, the recv(n) method indicates how many messages
> > are
> > allowed on the incoming queue (max n). But recv(n) is also used as
> > the
> > "pend for incoming messages" call. So every time my app needs to
> > fetch
> > some messages, it also needs to compute flow control. It's that
> > dual use
> > that I don't like.
> >
>
> As I mentioned above, I don't think recv should be thought of as a
> flow
> control thing. It does provide input into what messenger does for
> flow
> control, but it's really just about a way for the app to fetch
> messages,
> and so far I've been considering three scenarios:
>
> (1) an app wants to receive and process messages indefinitely, in
> which
> case pn_messenger_recv(-1) now does that job pretty nicely
> (2) an app wants to make a simple request/response, in which case
> it
> wants to receive exactly one message back and getting anymore would
> be a
> bug.
> (3) a generalized form of option (2) where an app makes N requests
> and
> processes each one as they arrive back. In this case you have to do
> pn_messenger_recv(N - what you've already processed).
>
> I think these are probably the 3 most common scenarios, and I can see
> how
> using a pattern like (3) to cater to scenario (1) would be awkward,
> however
> I think it's less awkward when used in scenario (3).
>
Thanks, I think you've clarified my understanding of the intended behavior of N in messenger::recv(N) - it places a limit to the number of messages the application can pn_messenger_get() for that invokation of recv(N). If N==-1, then the number is unspecified.
My understanding is that this API is not making any explicit guarantees about what is happening at the wire level - for example, it's possible that more messages may in fact arrive via the network - is that correct?
I can buy that.
> That said I'm open to trying to simplify the API here, but I
> fundamentally
> don't think of this as a wire level flow control API. I get the
> impression
> from the comments in this thread that there is an idea that the app
> developer somehow has more knowledge or is in a better position to
> distribute credit than the messenger implementation, whereas I think
> the
> opposite is true. In my experience, the nature/shape of incoming
> message
> traffic is not a variable that is well known at development time.
> Perhaps
> you can define the extreme bounds of what loads you want to be able
> to
> handle, but at runtime there are many unpredictable factors:
>
> - your service can go from idle to extreme bursts with no warning
> - round trip times can fluctuate based general network activity
> - message processing times might vary due to other activity on
> your machine or degradation in services you depend on
> - your load might be unevenly distributed across different
> links/connections
> - a buggy or malicious app might be (unintentionally) DoSing you
>
> All of these factors and more go into determining the optimal and/or
> fair
> credit allocation at any given point in time, and that means a robust
> flow
> control algorithm really needs to be dynamic in nature. Not only
> that, but
> a robust flow control algorithm is a huge part of the value that
> messaging
> infrastructure provides, and should really be a fundamentally
> separate
> concern from how apps logically process messages.
>
>
> > Since Messenger models a queue of incoming messages, I'd rather see
> > flow
> > control configured as thresholds on that queue, and recv() not take
> > any
> > arguments at all.
> >
> > Something like this:
> >
> > Messenger m;
> > ...
> > m.set_flow_stop( 10000 )
> > m.set_flow_resume( 9000 )
> > ...
> > for (;;) {
> > m.recv()
> > while (m.incoming())
> > ....
> >
> > IMHO, this is a lot "cleaner" than the current approach. Of
> > course, some
> > may find my sample names too cryptic :)
> >
>
> I think limits the API to only the first scenario I described above.
> At
> least it's not clear to me how you'd fetch exactly N messages.
>
>
> >
> > From an implementation point of view, the "flow stop" threshold is
> > really
> > just a suggestion for how much credit should be distributed across
> > the
> > links. We could distribute more, as we would need to if the number
> > of
> > links is greater than the flow stop threshold. Or less, assume a
> > point of
> > diminishing returns.
> >
> > Once the flow stop threshold is hit, credit would be drained from
> > all
> > links. No further credit would be granted until the number of
> > "queued"
> > messages drops below "flow resume".
> >
> > This is the same model we use for queue flow control in the C++
> > broker.
> >
>
> This is starting to mix two things: (1) how the application fetches
> messages from the messenger, and (2) how to tune the messengers
> internal
> flow control algorithm in the specific case that the application
> wants to
> receive messages indefinitely. I think (2) is premature given that we
> haven't really done any performance work yet. Ideally I'd say we
> don't want
> to have to tune it, rather just give it some bounds to work within,
> e.g.
> limit to no more than X megabytes or no more than Y messages.
>
Bingo! That's _exactly_ my problem: the current impl tries to make a mapping from the recv(N) argument into the proper distribution of link credit. And it does so badly in the multi-link case (w/o the ability to really drain, as you point out). Using N makes it hard to optimize credit allocation for any scenario that involves more than one underlying link (probably the case for scenario(1) and some cases of scenario(3)). Good optimization for multi-link is complicated because N is unconstrained between calls to recv(N) - very hard to do something predictive and fair in this case.
However, establishing bounds not based on recv(N) - as you point out - would go a long way to making a very nice multi-link credit distribution impl.
>From a documentation point of view - yes, certainly, a description of recv(N) shouldn't involve describing credit at all.
> In any case I think we need to be clear on the application scenarios
> we're
> trying to support. I've given (3) common ones above. Are there cases
> that
> you think are missing, and do you have a better way to cater to the 3
> I've
> mentioned?
>
> --Rafael
>
Re: messenger credit concern
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Tue, Feb 26, 2013 at 12:18 PM, Ken Giusti <kg...@redhat.com> wrote:
> Hi Mick
>
> ----- Original Message -----
> >
> > One hole that I feel like I'm seeing in the messenger
> > interface concerns credit.
> >
> > I have a way of using credit to set a max number of
> > messages that a recv call should return in one gulp,
> > or a way of doing ... something that I'm still figuring
> > out ... by setting credit=-1.
> >
> > What I don't have is any way of getting guidance about
> > what effect my credit allocation is having.
> >
> > A messenger app might have some flexibility in how
> > much time it spends servicing incoming messages vs.
> > time it spends doing its own processing, and it might
> > be able to allocate more time to servicing incoming
> > messages if it knows more about what's happening.
> >
> > Alternatively, it might want to set the credit allocated
> > per recv call based on the number of current incoming
> > links. ( And assume that the credit will be distributed
> > round-robin across all incoming links. )
> >
> > Would it be practical / desirable / catastrophic
> > to expose current backlog or number of incoming links,
> > or both, at the messenger level ?
> >
> > Or would that violate part of the messenger API philosophy?
> > ( And if so, what is that philosophy? I want to be able
> > to explain it. )
> >
>
> I feel your pain - the more I use the "recv(n)" interface, the more I
> think it's an awkward approach to the credit/flow issue.
>
I don't think it's aiming to solve a credt/flow issue at all. That's the
job of the messenger implementation. It's simply providing a way for the
application to ask for n messages.
>
> From what I understand, Messenger's model is based on two queues - one for
> incoming messages and one for outgoing messages.
>
> For flow control, the recv(n) method indicates how many messages are
> allowed on the incoming queue (max n). But recv(n) is also used as the
> "pend for incoming messages" call. So every time my app needs to fetch
> some messages, it also needs to compute flow control. It's that dual use
> that I don't like.
>
As I mentioned above, I don't think recv should be thought of as a flow
control thing. It does provide input into what messenger does for flow
control, but it's really just about a way for the app to fetch messages,
and so far I've been considering three scenarios:
(1) an app wants to receive and process messages indefinitely, in which
case pn_messenger_recv(-1) now does that job pretty nicely
(2) an app wants to make a simple request/response, in which case it
wants to receive exactly one message back and getting anymore would be a
bug.
(3) a generalized form of option (2) where an app makes N requests and
processes each one as they arrive back. In this case you have to do
pn_messenger_recv(N - what you've already processed).
I think these are probably the 3 most common scenarios, and I can see how
using a pattern like (3) to cater to scenario (1) would be awkward, however
I think it's less awkward when used in scenario (3).
That said I'm open to trying to simplify the API here, but I fundamentally
don't think of this as a wire level flow control API. I get the impression
from the comments in this thread that there is an idea that the app
developer somehow has more knowledge or is in a better position to
distribute credit than the messenger implementation, whereas I think the
opposite is true. In my experience, the nature/shape of incoming message
traffic is not a variable that is well known at development time. Perhaps
you can define the extreme bounds of what loads you want to be able to
handle, but at runtime there are many unpredictable factors:
- your service can go from idle to extreme bursts with no warning
- round trip times can fluctuate based general network activity
- message processing times might vary due to other activity on
your machine or degradation in services you depend on
- your load might be unevenly distributed across different
links/connections
- a buggy or malicious app might be (unintentionally) DoSing you
All of these factors and more go into determining the optimal and/or fair
credit allocation at any given point in time, and that means a robust flow
control algorithm really needs to be dynamic in nature. Not only that, but
a robust flow control algorithm is a huge part of the value that messaging
infrastructure provides, and should really be a fundamentally separate
concern from how apps logically process messages.
> Since Messenger models a queue of incoming messages, I'd rather see flow
> control configured as thresholds on that queue, and recv() not take any
> arguments at all.
>
> Something like this:
>
> Messenger m;
> ...
> m.set_flow_stop( 10000 )
> m.set_flow_resume( 9000 )
> ...
> for (;;) {
> m.recv()
> while (m.incoming())
> ....
>
> IMHO, this is a lot "cleaner" than the current approach. Of course, some
> may find my sample names too cryptic :)
>
I think limits the API to only the first scenario I described above. At
least it's not clear to me how you'd fetch exactly N messages.
>
> From an implementation point of view, the "flow stop" threshold is really
> just a suggestion for how much credit should be distributed across the
> links. We could distribute more, as we would need to if the number of
> links is greater than the flow stop threshold. Or less, assume a point of
> diminishing returns.
>
> Once the flow stop threshold is hit, credit would be drained from all
> links. No further credit would be granted until the number of "queued"
> messages drops below "flow resume".
>
> This is the same model we use for queue flow control in the C++ broker.
>
This is starting to mix two things: (1) how the application fetches
messages from the messenger, and (2) how to tune the messengers internal
flow control algorithm in the specific case that the application wants to
receive messages indefinitely. I think (2) is premature given that we
haven't really done any performance work yet. Ideally I'd say we don't want
to have to tune it, rather just give it some bounds to work within, e.g.
limit to no more than X megabytes or no more than Y messages.
In any case I think we need to be clear on the application scenarios we're
trying to support. I've given (3) common ones above. Are there cases that
you think are missing, and do you have a better way to cater to the 3 I've
mentioned?
--Rafael
Re: messenger credit concern
Posted by Michael Goulish <mg...@redhat.com>.
+1
This would make sense to me. It seems like it
1. does not break the messenger model
2. keeps the interface simple
3. allows more visibility in whether the messenger is "falling behind"
4. gives new capability -- throttling
5. still hides all fancy stuff about links & credit distribution
under the messenger covers.
----- Original Message -----
From: "Ken Giusti" <kg...@redhat.com>
To: proton@qpid.apache.org
Sent: Tuesday, February 26, 2013 3:18:22 PM
Subject: Re: messenger credit concern
Hi Mick
----- Original Message -----
>
> One hole that I feel like I'm seeing in the messenger
> interface concerns credit.
>
> I have a way of using credit to set a max number of
> messages that a recv call should return in one gulp,
> or a way of doing ... something that I'm still figuring
> out ... by setting credit=-1.
>
> What I don't have is any way of getting guidance about
> what effect my credit allocation is having.
>
> A messenger app might have some flexibility in how
> much time it spends servicing incoming messages vs.
> time it spends doing its own processing, and it might
> be able to allocate more time to servicing incoming
> messages if it knows more about what's happening.
>
> Alternatively, it might want to set the credit allocated
> per recv call based on the number of current incoming
> links. ( And assume that the credit will be distributed
> round-robin across all incoming links. )
>
> Would it be practical / desirable / catastrophic
> to expose current backlog or number of incoming links,
> or both, at the messenger level ?
>
> Or would that violate part of the messenger API philosophy?
> ( And if so, what is that philosophy? I want to be able
> to explain it. )
>
I feel your pain - the more I use the "recv(n)" interface, the more I think it's an awkward approach to the credit/flow issue.
>From what I understand, Messenger's model is based on two queues - one for incoming messages and one for outgoing messages.
For flow control, the recv(n) method indicates how many messages are allowed on the incoming queue (max n). But recv(n) is also used as the "pend for incoming messages" call. So every time my app needs to fetch some messages, it also needs to compute flow control. It's that dual use that I don't like.
Since Messenger models a queue of incoming messages, I'd rather see flow control configured as thresholds on that queue, and recv() not take any arguments at all.
Something like this:
Messenger m;
...
m.set_flow_stop( 10000 )
m.set_flow_resume( 9000 )
...
for (;;) {
m.recv()
while (m.incoming())
....
IMHO, this is a lot "cleaner" than the current approach. Of course, some may find my sample names too cryptic :)
>From an implementation point of view, the "flow stop" threshold is really just a suggestion for how much credit should be distributed across the links. We could distribute more, as we would need to if the number of links is greater than the flow stop threshold. Or less, assume a point of diminishing returns.
Once the flow stop threshold is hit, credit would be drained from all links. No further credit would be granted until the number of "queued" messages drops below "flow resume".
This is the same model we use for queue flow control in the C++ broker.
-K
Re: messenger credit concern
Posted by Ken Giusti <kg...@redhat.com>.
Hi Mick
----- Original Message -----
>
> One hole that I feel like I'm seeing in the messenger
> interface concerns credit.
>
> I have a way of using credit to set a max number of
> messages that a recv call should return in one gulp,
> or a way of doing ... something that I'm still figuring
> out ... by setting credit=-1.
>
> What I don't have is any way of getting guidance about
> what effect my credit allocation is having.
>
> A messenger app might have some flexibility in how
> much time it spends servicing incoming messages vs.
> time it spends doing its own processing, and it might
> be able to allocate more time to servicing incoming
> messages if it knows more about what's happening.
>
> Alternatively, it might want to set the credit allocated
> per recv call based on the number of current incoming
> links. ( And assume that the credit will be distributed
> round-robin across all incoming links. )
>
> Would it be practical / desirable / catastrophic
> to expose current backlog or number of incoming links,
> or both, at the messenger level ?
>
> Or would that violate part of the messenger API philosophy?
> ( And if so, what is that philosophy? I want to be able
> to explain it. )
>
I feel your pain - the more I use the "recv(n)" interface, the more I think it's an awkward approach to the credit/flow issue.
>From what I understand, Messenger's model is based on two queues - one for incoming messages and one for outgoing messages.
For flow control, the recv(n) method indicates how many messages are allowed on the incoming queue (max n). But recv(n) is also used as the "pend for incoming messages" call. So every time my app needs to fetch some messages, it also needs to compute flow control. It's that dual use that I don't like.
Since Messenger models a queue of incoming messages, I'd rather see flow control configured as thresholds on that queue, and recv() not take any arguments at all.
Something like this:
Messenger m;
...
m.set_flow_stop( 10000 )
m.set_flow_resume( 9000 )
...
for (;;) {
m.recv()
while (m.incoming())
....
IMHO, this is a lot "cleaner" than the current approach. Of course, some may find my sample names too cryptic :)
>From an implementation point of view, the "flow stop" threshold is really just a suggestion for how much credit should be distributed across the links. We could distribute more, as we would need to if the number of links is greater than the flow stop threshold. Or less, assume a point of diminishing returns.
Once the flow stop threshold is hit, credit would be drained from all links. No further credit would be granted until the number of "queued" messages drops below "flow resume".
This is the same model we use for queue flow control in the C++ broker.
-K