You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by Robbie Gemmell <ro...@gmail.com> on 2015/07/01 16:47:43 UTC
AMQP 1.0 session outgoing-window usage / meaning
Hi all,
Short intro:
The way we use the outgoing-window feels wrong, and seems to violate
at least one bit of the related [and unclear overall] description in
the spec. The way we use it means we currently can't send messages to
ServiceBus in many cases (likely anything-but-messenger).
Full version:
A user reported being unable to send messages to Service Bus (Azure or
Windows variants) using the new JMS client. Investigating that lead me
to the handling of session outgoing-window in proton (and
incoming-window in ServiceBus). I'd like to discuss how proton uses
this, what the spec actually says on how it should be used since its
not clear to me these are currently in alignment, and possibly
changing how we utilise it in proton.
In Proton the outgoing-window is set based on the amount of
outstanding outgoing bytes for the session (from buffered sends) and
the frame size, which leads to many situations where the advertised
value is 0, particularly in a client that creates sessions entirely
independently from sending messages (such as a JMS client, or the Qpid
Messaging C++ client). We then send transfer frames if a link has
credit, regardless of the session outgoing window, only withholding
them if the remote-incoming-window hits 0.
This causes problems against ServiceBus because it seems to base its
advertised incoming window on the clients [initial] advertised
outgoing window. If it is 0, then it means we can never send any
messages.
The intent/definition of the outgoing-window seems a bit unclear as a
whole in the spec to me, to the point you could almost remove it
without issue given the [remote-]incoming-window seems to govern
overall behaviour. However, it seems wrong to me that we often set the
outgoing-window to 0. Doing this means we violate the outgoing window
(local and at peer) whenever we send a message if the
remote-incoming-window allowed it. If we tried not to violate it, we
would have to flow the new session window before every send, which
again seems odd.
It feels to me like it would be better to define initial values for
the windows, perhaps allowing that to be configurable in the same
manner as e.g max frame size is. Those that want to impose some limit
then could, and those that don't can leave/set it to the max values to
achieve that effect.
Thoughts?
Robbie
RE: AMQP 1.0 session outgoing-window usage / meaning
Posted by James Birdsall <jb...@microsoft.com>.
Yes, that's a good point: all our work with Proton-C to date has been via the Messenger API.
-----Original Message-----
From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
Sent: Wednesday, July 1, 2015 4:16 PM
To: proton@qpid.apache.org
Cc: users@qpid.apache.org
Subject: Re: AMQP 1.0 session outgoing-window usage / meaning
Thanks James. Some expansion which may be useful to add.
When comparing the older JMS client, proton-c via the Messenger API, and the new JMS client using proton-j, its important to note that they aren't all doing the same thing even where their underlying implementations do seem to share the same behaviour in the cases of proton-c and proton-j.
The older JMS client initializes its outgoing window to a fixed number in the session Begin frame and then doesnt seem to ever change it for subsequent Flow frames, and simply manages whether its session can later send transfer frames based on the current value of the remote incoming window. Proton-J and Proton-C similarly only base their session level decision to send transfers on the remote incoming window and not their own outgoing window (which as noted below means they violate their advertised outgoing window, which is often going to be 0).
Proton-C and Proton-J both currently look to set the outgoing window at any given time to a calculated value based on either the number of buffered messages or the buffered bytes divided by frame size. If there are no buffered messages at the point the Begin and Flow frames are generated, then the outgoing-window will be set to 0. This appears to function the same for both proton-c and proton-j. A key point though is that I think much of the historic usage of proton-c against Service Bus has been via the Messenger API, which works somewhat differently than many others in that it looks to create a session and a sender and sends the messages in one pipelined sequence of transport output, which means that by the point the Begin frame actually gets generated there are indeed buffered messages to send which means the outgoing-window is initialised to a value greater than zero. Other APIs which create the session as a distinct step thus wont ever have buffered messages when the Begin frame gets created and so the outgoing-window is initialised to 0, which is the behaviour observed with the new JMS client using proton-j and also what I saw when trying proton-c via the Qpid Messaging C++ client (against qpidd).
Robbie
On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
> FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
>
> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
>
> The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
>
> -----Original Message-----
> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
> Sent: Wednesday, July 1, 2015 7:48 AM
> To: users@qpid.apache.org; proton@qpid.apache.org
> Subject: AMQP 1.0 session outgoing-window usage / meaning
>
> Hi all,
>
> Short intro:
>
> The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
>
>
> Full version:
>
> A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
>
> In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
>
> This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
>
> The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
>
> It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
>
> Thoughts?
>
> Robbie
RE: AMQP 1.0 session outgoing-window usage / meaning
Posted by James Birdsall <jb...@microsoft.com>.
Yes, that's a good point: all our work with Proton-C to date has been via the Messenger API.
-----Original Message-----
From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
Sent: Wednesday, July 1, 2015 4:16 PM
To: proton@qpid.apache.org
Cc: users@qpid.apache.org
Subject: Re: AMQP 1.0 session outgoing-window usage / meaning
Thanks James. Some expansion which may be useful to add.
When comparing the older JMS client, proton-c via the Messenger API, and the new JMS client using proton-j, its important to note that they aren't all doing the same thing even where their underlying implementations do seem to share the same behaviour in the cases of proton-c and proton-j.
The older JMS client initializes its outgoing window to a fixed number in the session Begin frame and then doesnt seem to ever change it for subsequent Flow frames, and simply manages whether its session can later send transfer frames based on the current value of the remote incoming window. Proton-J and Proton-C similarly only base their session level decision to send transfers on the remote incoming window and not their own outgoing window (which as noted below means they violate their advertised outgoing window, which is often going to be 0).
Proton-C and Proton-J both currently look to set the outgoing window at any given time to a calculated value based on either the number of buffered messages or the buffered bytes divided by frame size. If there are no buffered messages at the point the Begin and Flow frames are generated, then the outgoing-window will be set to 0. This appears to function the same for both proton-c and proton-j. A key point though is that I think much of the historic usage of proton-c against Service Bus has been via the Messenger API, which works somewhat differently than many others in that it looks to create a session and a sender and sends the messages in one pipelined sequence of transport output, which means that by the point the Begin frame actually gets generated there are indeed buffered messages to send which means the outgoing-window is initialised to a value greater than zero. Other APIs which create the session as a distinct step thus wont ever have buffered messages when the Begin frame gets created and so the outgoing-window is initialised to 0, which is the behaviour observed with the new JMS client using proton-j and also what I saw when trying proton-c via the Qpid Messaging C++ client (against qpidd).
Robbie
On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
> FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
>
> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
>
> The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
>
> -----Original Message-----
> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
> Sent: Wednesday, July 1, 2015 7:48 AM
> To: users@qpid.apache.org; proton@qpid.apache.org
> Subject: AMQP 1.0 session outgoing-window usage / meaning
>
> Hi all,
>
> Short intro:
>
> The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
>
>
> Full version:
>
> A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
>
> In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
>
> This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
>
> The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
>
> It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
>
> Thoughts?
>
> Robbie
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
On 8 July 2015 at 20:13, Rob Godfrey <ro...@gmail.com> wrote:
> As far as I can recall/reconstruct the only utility given by the
> outgoing window was so that the sender (of transfer frames) can
> indicate to the receiver (of transfer frames) that it will require
> notification of which frames have been seen by the receiver within
> <window size> frames, otherwise the sender will potentially block
> until it has received such information. As such the receiver should
> always be able to safely set its incoming window to be <= the sender's
> outgoing window, since setting it to be larger is pointless. A sender
> which finds itself having previously set the outgoing window size to
> zero, and finding its peer's incoming window size is also zero must, I
> think, send a flow indicating that its window size is now non-zero.
>
> Having said that I'm not sure I ever saw much utility in this value
> and as such I would agree that implementations should probably just
> set the default outgoing window size to a large number, and ignore
> their peer's outgoing window when setting their own incoming window.
> About the only thing that we probably should be doing is ensuring that
> if the implied outgoing window of the peer is zero, but might open up
> if a flow informing the peer of the updated state, then such a flow
> should be sent. (Not sure I would set to MAX_INT in case anyone is
> trying to create an array based on the provided size upon receipt...
> but that's just me).
>
> -- Rob
>
I did wonder if setting it that large was too large (though its only
half the allowed value)...but decided its a bit difficult to pick a
'large enough to forget it exists, but not too large' number, and just
ensured it is configurable in case needed.
>
>
> On 8 July 2015 at 20:46, Xin Chen <da...@gmail.com> wrote:
>> I think one safe thing to do now is to ignore this field. I will update
>> service bus to not setting incoming window based on the received outgoing
>> window field. This is the only time we look at this value.
>>
>>
>> On Wed, Jul 8, 2015 at 10:58 AM, Gordon Sim <gs...@redhat.com> wrote:
>>
>>> On 07/08/2015 06:23 PM, Rafael Schloming wrote:
>>>
>>>> On Wed, Jul 8, 2015 at 8:58 AM, Gordon Sim <gs...@redhat.com> wrote:
>>>>
>>>>> Under your interpretation - i.e. where the sender can change the value of
>>>>> the outgoing window without ever having to inform the receiver - the
>>>>> outgoing window seems to have very little value except as a hint that the
>>>>> sender may be constrained by capacity limits on the number of
>>>>> deliveries/transfers it is tracking. To send an outgoing window of 0 when
>>>>> there have not yet even been any transfers seems to undermine even that
>>>>> limited value.
>>>>>
>>>>>
>>>> I think the way to look at it under that interpretation is not that the
>>>> sender is not notifying the peer, simply that it is notifying the peer
>>>> asynchronously, and that the transfer is implicitly notifying the peer
>>>> (just like it implicitly advances the next-outgoing-id).
>>>>
>>>
>>> I'm not sure I understand. Asynchronous with respect to what? Are you
>>> saying that a transfer implicitly expands the remote-outgoing-window (but
>>> only if it was 0 before the transfer)?
>>>
>>> One of the few concrete rules specified concerning the outgoing window is
>>> that the remote-outgoing-window "MUST be decremented after every incoming
>>> transfer frame is received, and recomputed when informed of the remote
>>> session endpoint state". If the last information we received was that the
>>> outgoing window was 0, what should we do on then receiving a transfer?
>>>
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rob Godfrey <ro...@gmail.com>.
As far as I can recall/reconstruct the only utility given by the
outgoing window was so that the sender (of transfer frames) can
indicate to the receiver (of transfer frames) that it will require
notification of which frames have been seen by the receiver within
<window size> frames, otherwise the sender will potentially block
until it has received such information. As such the receiver should
always be able to safely set its incoming window to be <= the sender's
outgoing window, since setting it to be larger is pointless. A sender
which finds itself having previously set the outgoing window size to
zero, and finding its peer's incoming window size is also zero must, I
think, send a flow indicating that its window size is now non-zero.
Having said that I'm not sure I ever saw much utility in this value
and as such I would agree that implementations should probably just
set the default outgoing window size to a large number, and ignore
their peer's outgoing window when setting their own incoming window.
About the only thing that we probably should be doing is ensuring that
if the implied outgoing window of the peer is zero, but might open up
if a flow informing the peer of the updated state, then such a flow
should be sent. (Not sure I would set to MAX_INT in case anyone is
trying to create an array based on the provided size upon receipt...
but that's just me).
-- Rob
On 8 July 2015 at 20:46, Xin Chen <da...@gmail.com> wrote:
> I think one safe thing to do now is to ignore this field. I will update
> service bus to not setting incoming window based on the received outgoing
> window field. This is the only time we look at this value.
>
>
> On Wed, Jul 8, 2015 at 10:58 AM, Gordon Sim <gs...@redhat.com> wrote:
>
>> On 07/08/2015 06:23 PM, Rafael Schloming wrote:
>>
>>> On Wed, Jul 8, 2015 at 8:58 AM, Gordon Sim <gs...@redhat.com> wrote:
>>>
>>>> Under your interpretation - i.e. where the sender can change the value of
>>>> the outgoing window without ever having to inform the receiver - the
>>>> outgoing window seems to have very little value except as a hint that the
>>>> sender may be constrained by capacity limits on the number of
>>>> deliveries/transfers it is tracking. To send an outgoing window of 0 when
>>>> there have not yet even been any transfers seems to undermine even that
>>>> limited value.
>>>>
>>>>
>>> I think the way to look at it under that interpretation is not that the
>>> sender is not notifying the peer, simply that it is notifying the peer
>>> asynchronously, and that the transfer is implicitly notifying the peer
>>> (just like it implicitly advances the next-outgoing-id).
>>>
>>
>> I'm not sure I understand. Asynchronous with respect to what? Are you
>> saying that a transfer implicitly expands the remote-outgoing-window (but
>> only if it was 0 before the transfer)?
>>
>> One of the few concrete rules specified concerning the outgoing window is
>> that the remote-outgoing-window "MUST be decremented after every incoming
>> transfer frame is received, and recomputed when informed of the remote
>> session endpoint state". If the last information we received was that the
>> outgoing window was 0, what should we do on then receiving a transfer?
>>
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Xin Chen <da...@gmail.com>.
I think one safe thing to do now is to ignore this field. I will update
service bus to not setting incoming window based on the received outgoing
window field. This is the only time we look at this value.
On Wed, Jul 8, 2015 at 10:58 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 07/08/2015 06:23 PM, Rafael Schloming wrote:
>
>> On Wed, Jul 8, 2015 at 8:58 AM, Gordon Sim <gs...@redhat.com> wrote:
>>
>>> Under your interpretation - i.e. where the sender can change the value of
>>> the outgoing window without ever having to inform the receiver - the
>>> outgoing window seems to have very little value except as a hint that the
>>> sender may be constrained by capacity limits on the number of
>>> deliveries/transfers it is tracking. To send an outgoing window of 0 when
>>> there have not yet even been any transfers seems to undermine even that
>>> limited value.
>>>
>>>
>> I think the way to look at it under that interpretation is not that the
>> sender is not notifying the peer, simply that it is notifying the peer
>> asynchronously, and that the transfer is implicitly notifying the peer
>> (just like it implicitly advances the next-outgoing-id).
>>
>
> I'm not sure I understand. Asynchronous with respect to what? Are you
> saying that a transfer implicitly expands the remote-outgoing-window (but
> only if it was 0 before the transfer)?
>
> One of the few concrete rules specified concerning the outgoing window is
> that the remote-outgoing-window "MUST be decremented after every incoming
> transfer frame is received, and recomputed when informed of the remote
> session endpoint state". If the last information we received was that the
> outgoing window was 0, what should we do on then receiving a transfer?
>
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Gordon Sim <gs...@redhat.com>.
On 07/08/2015 06:23 PM, Rafael Schloming wrote:
> On Wed, Jul 8, 2015 at 8:58 AM, Gordon Sim <gs...@redhat.com> wrote:
>> Under your interpretation - i.e. where the sender can change the value of
>> the outgoing window without ever having to inform the receiver - the
>> outgoing window seems to have very little value except as a hint that the
>> sender may be constrained by capacity limits on the number of
>> deliveries/transfers it is tracking. To send an outgoing window of 0 when
>> there have not yet even been any transfers seems to undermine even that
>> limited value.
>>
>
> I think the way to look at it under that interpretation is not that the
> sender is not notifying the peer, simply that it is notifying the peer
> asynchronously, and that the transfer is implicitly notifying the peer
> (just like it implicitly advances the next-outgoing-id).
I'm not sure I understand. Asynchronous with respect to what? Are you
saying that a transfer implicitly expands the remote-outgoing-window
(but only if it was 0 before the transfer)?
One of the few concrete rules specified concerning the outgoing window
is that the remote-outgoing-window "MUST be decremented after every
incoming transfer frame is received, and recomputed when informed of the
remote session endpoint state". If the last information we received was
that the outgoing window was 0, what should we do on then receiving a
transfer?
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Jul 8, 2015 at 8:58 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 07/08/2015 03:49 PM, Rafael Schloming wrote:
>
>> I think what is confusing about the spec language is that it is defining
>> the meaning of the value in terms of the sending endpoint's state and then
>> not saying anything about when the sending endpoint is obligated to
>> communicate the value to the receiver. This is because (at least as
>> originally conceived) it is never directly obligated to communicate the
>> value to the receiver, i.e. it could just choose to grow its array
>> indefinitely.
>>
>> By contrast the incoming-window is defined in terms of the receiver's
>> endpoint state and it is (relatively) clearly communicated that the
>> receiver is obligated to communicate a non-zero incoming-window if it
>> wishes to receive transfer frames.
>>
>> So while I agree that the language as defined requires the outgoing window
>> to be increased from zero, I don't think this implies (at least without a
>> more careful reading) that the sender needs to communicate this fact
>> before
>> sending transfers. It owns the value so it can effectively increase the
>> outgoing-window to 1 momentarily, send the transfer, and then decrease it
>> back to zero without needing to sandwich that transfer in flow frames in
>> order to signal the changing window values.
>>
>
> I think that is a bit of a stretch given the language the spec does use. I
> agree it doesn't explicitly state the rule in those terms, however it does
> say:
>
> The remote-outgoing-window reflects the maximum number of
> incoming transfers that MAY arrive without exceeding the
> remote endpoint’s outgoing-window. This value MUST be
> decremented after every incoming transfer frame is received,
> and recomputed when informed of the remote session endpoint
> state.
>
> I.e. the language suggests that a compliant receiver must do some work to
> keep track of the value and states that this value is tied to the maximum
> number of transfers that may arrive.
>
> Put another way, the rule that says it is illegal for the sender to send
>> transfers when the receiver's incoming window is zero creates an
>> obligation
>> for the receiver to communicate its window, but there is no corresponding
>> rule for the receiver, e.g. nothing says it is illegal for the receiver to
>> send credit when the sender's outgoing-window is zero, nor that the
>> receiver must compute it's incoming-window based on the sender's
>> outgoing-window,
>>
>
> I certainly agree that nothing ties the receivers incoming window to the
> senders outgoing window in any fixed way.
>
> so there is no obligation similarly implied for the sender.
>>
>
> I disagree here. I think the fact that begin and flow have the
> outgoing-window field defined as mandatory, and the fact that there is
> specific (though bewildering) language that appears to mandate the tracking
> of the senders outgoing window by the receiver, *implies* (though does not
> explicitly spell out) that the sender is supposed to keep the receiver
> informed of its outgoing window.
>
> Further I think the sender should not take the lack of credit as grounds
>>> to set a window of 0. The receiver knows it has not issued credit. (At
>>> the
>>> link level, the sender can also indicate that it has messages awaiting
>>> link
>>> credit).
>>>
>>>
>> I agree that we shouldn't do this in our implementation, however I think
>> it
>> is a valid interpretation,
>>
>
> The flow frame describes the outgoing window field as defining:
>
> the maximum number of outgoing transfer frames that the
> endpoint could potentially currently send, if it was not
> constrained by restrictions imposed by its peer’s
> incoming-window.
>
> To me that explicitly states that the value of the senders outgoing window
> is not tied to the value of the receivers incoming window, i.e. that a
> value of 0 for the senders outgoing window cannot reasonably be interpreted
> as an expectation for the receiver to open its incoming window.
>
> I don't see a single sentence that would tie the session outgoing window
> to link level credit, which is an entirely distinct mechanism.
>
> The one thing the spec does say of the remote-outgoing-window is that
> "settling outstanding transfers can cause the window to grow". Settling
> deliveries frees the sender from needing to track them which clearly
> *might* enable a constrained sender to then send some more. Likewise a
> received outcome might allow the sender to make room for further transfers
> (though this isn't stated anywhere I can see it would seem a reasonable
> interpretation).
>
> which implies that making the incoming window
>> dependent on your peers outgoing window (as service-bus is doing) is
>> probably not a safe thing to do.
>>
>
> Tying the incoming window to the peers outgoing window can only make sense
> if the sender is expected to keep the receiver up to date with regards to
> the value of the outgoing window. If that is not the case, then I can't see
> that the receiver can deduce anything from the outgoing window except that
> possibly a value of 0 may be a hint that the sender is held up pending some
> undefined action on the receiver part.
>
> Even if the sender was required to keep the receiver informed of the
> outgoing window, having the receivers incoming window set to the outgoing
> window of the sender is not terribly sensible in my view unless the
> receiver would exapnd its incoming window if/when the sender expanded its
> outgoing window. But doing that would essentially negate the whole purpose
> of flow control anyway (i.e. a means for the receiver to protect itself
> from excessive incoming traffic).
>
> In the case where the sender's implementation involves a fixed amount of
>>> buffer space and requires messages to be settled before it can send more,
>>> the receiver would not be able to know that without getting some signal.
>>> So
>>> to my mind that is the only case for which it would make sense to send an
>>> outgoing window of 0. (I'm not sure how useful this is in practice and I
>>> don't believe it applies to proton at present anyway).
>>>
>>> I think as it stands proton is violating the spec, and should be changed
>>> to send a non-zero outgoing window.
>>>
>>>
>> Per above I don't believe it is violating the spec, but given that it has
>> been misinterpreted at least once, I certainly agree that some behavior
>> change is warranted. I don't think we should do the max-frame-size thing
>> though as this encourages the notion that the incoming-window is somehow
>> dependent on the outgoing-window which as I said above is I think unsafe.
>>
>> I guess setting it to a large constant for now is fairly reasonable, but I
>> do think we should encourage service-bus to make its implementation more
>> robust in this regard.
>>
>
> Under your interpretation - i.e. where the sender can change the value of
> the outgoing window without ever having to inform the receiver - the
> outgoing window seems to have very little value except as a hint that the
> sender may be constrained by capacity limits on the number of
> deliveries/transfers it is tracking. To send an outgoing window of 0 when
> there have not yet even been any transfers seems to undermine even that
> limited value.
>
I think the way to look at it under that interpretation is not that the
sender is not notifying the peer, simply that it is notifying the peer
asynchronously, and that the transfer is implicitly notifying the peer
(just like it implicitly advances the next-outgoing-id). While I agree the
latter part of that is probably a stretch, I don't think the former part is.
> If there is no required behaviour actually specified, and the intention
> was to use the outgoing window as a hint of constraints on the sender then
> sending 0 is the worst possible choice. Whatever the value, it is not
> something the peer can rely on in anyway and using 0 when there are no
> constraints seems to ignore even the apparent intent.
I think the intended effect of always setting it to zero would be forcing
the receiver to not buffer settlement notification.
> So yes, my advice to service-bus and anyone else would be to ignore the
> value. I think the question then is simply what is the least dangerous way
> to set the mandated field for an implementation that wishes to avoid
> interop problems. I think some very large fixed window is the best bet.
> (I'm not sure I understand what the 'max-frame-size thing' is).
>
I think the best advice would be to ignore the value unless it is zero in
which case you should ensure that you flush any buffered settlement
notifications.
--Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Gordon Sim <gs...@redhat.com>.
On 07/08/2015 03:49 PM, Rafael Schloming wrote:
> I think what is confusing about the spec language is that it is defining
> the meaning of the value in terms of the sending endpoint's state and then
> not saying anything about when the sending endpoint is obligated to
> communicate the value to the receiver. This is because (at least as
> originally conceived) it is never directly obligated to communicate the
> value to the receiver, i.e. it could just choose to grow its array
> indefinitely.
>
> By contrast the incoming-window is defined in terms of the receiver's
> endpoint state and it is (relatively) clearly communicated that the
> receiver is obligated to communicate a non-zero incoming-window if it
> wishes to receive transfer frames.
>
> So while I agree that the language as defined requires the outgoing window
> to be increased from zero, I don't think this implies (at least without a
> more careful reading) that the sender needs to communicate this fact before
> sending transfers. It owns the value so it can effectively increase the
> outgoing-window to 1 momentarily, send the transfer, and then decrease it
> back to zero without needing to sandwich that transfer in flow frames in
> order to signal the changing window values.
I think that is a bit of a stretch given the language the spec does use.
I agree it doesn't explicitly state the rule in those terms, however it
does say:
The remote-outgoing-window reflects the maximum number of
incoming transfers that MAY arrive without exceeding the
remote endpoint’s outgoing-window. This value MUST be
decremented after every incoming transfer frame is received,
and recomputed when informed of the remote session endpoint
state.
I.e. the language suggests that a compliant receiver must do some work
to keep track of the value and states that this value is tied to the
maximum number of transfers that may arrive.
> Put another way, the rule that says it is illegal for the sender to send
> transfers when the receiver's incoming window is zero creates an obligation
> for the receiver to communicate its window, but there is no corresponding
> rule for the receiver, e.g. nothing says it is illegal for the receiver to
> send credit when the sender's outgoing-window is zero, nor that the
> receiver must compute it's incoming-window based on the sender's
> outgoing-window,
I certainly agree that nothing ties the receivers incoming window to the
senders outgoing window in any fixed way.
> so there is no obligation similarly implied for the sender.
I disagree here. I think the fact that begin and flow have the
outgoing-window field defined as mandatory, and the fact that there is
specific (though bewildering) language that appears to mandate the
tracking of the senders outgoing window by the receiver, *implies*
(though does not explicitly spell out) that the sender is supposed to
keep the receiver informed of its outgoing window.
>> Further I think the sender should not take the lack of credit as grounds
>> to set a window of 0. The receiver knows it has not issued credit. (At the
>> link level, the sender can also indicate that it has messages awaiting link
>> credit).
>>
>
> I agree that we shouldn't do this in our implementation, however I think it
> is a valid interpretation,
The flow frame describes the outgoing window field as defining:
the maximum number of outgoing transfer frames that the
endpoint could potentially currently send, if it was not
constrained by restrictions imposed by its peer’s
incoming-window.
To me that explicitly states that the value of the senders outgoing
window is not tied to the value of the receivers incoming window, i.e.
that a value of 0 for the senders outgoing window cannot reasonably be
interpreted as an expectation for the receiver to open its incoming window.
I don't see a single sentence that would tie the session outgoing window
to link level credit, which is an entirely distinct mechanism.
The one thing the spec does say of the remote-outgoing-window is that
"settling outstanding transfers can cause the window to grow". Settling
deliveries frees the sender from needing to track them which clearly
*might* enable a constrained sender to then send some more. Likewise a
received outcome might allow the sender to make room for further
transfers (though this isn't stated anywhere I can see it would seem a
reasonable interpretation).
> which implies that making the incoming window
> dependent on your peers outgoing window (as service-bus is doing) is
> probably not a safe thing to do.
Tying the incoming window to the peers outgoing window can only make
sense if the sender is expected to keep the receiver up to date with
regards to the value of the outgoing window. If that is not the case,
then I can't see that the receiver can deduce anything from the outgoing
window except that possibly a value of 0 may be a hint that the sender
is held up pending some undefined action on the receiver part.
Even if the sender was required to keep the receiver informed of the
outgoing window, having the receivers incoming window set to the
outgoing window of the sender is not terribly sensible in my view unless
the receiver would exapnd its incoming window if/when the sender
expanded its outgoing window. But doing that would essentially negate
the whole purpose of flow control anyway (i.e. a means for the receiver
to protect itself from excessive incoming traffic).
>> In the case where the sender's implementation involves a fixed amount of
>> buffer space and requires messages to be settled before it can send more,
>> the receiver would not be able to know that without getting some signal. So
>> to my mind that is the only case for which it would make sense to send an
>> outgoing window of 0. (I'm not sure how useful this is in practice and I
>> don't believe it applies to proton at present anyway).
>>
>> I think as it stands proton is violating the spec, and should be changed
>> to send a non-zero outgoing window.
>>
>
> Per above I don't believe it is violating the spec, but given that it has
> been misinterpreted at least once, I certainly agree that some behavior
> change is warranted. I don't think we should do the max-frame-size thing
> though as this encourages the notion that the incoming-window is somehow
> dependent on the outgoing-window which as I said above is I think unsafe.
>
> I guess setting it to a large constant for now is fairly reasonable, but I
> do think we should encourage service-bus to make its implementation more
> robust in this regard.
Under your interpretation - i.e. where the sender can change the value
of the outgoing window without ever having to inform the receiver - the
outgoing window seems to have very little value except as a hint that
the sender may be constrained by capacity limits on the number of
deliveries/transfers it is tracking. To send an outgoing window of 0
when there have not yet even been any transfers seems to undermine even
that limited value.
If there is no required behaviour actually specified, and the intention
was to use the outgoing window as a hint of constraints on the sender
then sending 0 is the worst possible choice. Whatever the value, it is
not something the peer can rely on in anyway and using 0 when there are
no constraints seems to ignore even the apparent intent.
So yes, my advice to service-bus and anyone else would be to ignore the
value. I think the question then is simply what is the least dangerous
way to set the mandated field for an implementation that wishes to avoid
interop problems. I think some very large fixed window is the best bet.
(I'm not sure I understand what the 'max-frame-size thing' is).
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
On 8 July 2015 at 17:59, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Wed, Jul 8, 2015 at 8:29 AM, Robbie Gemmell <ro...@gmail.com>
> wrote:
>
>> The wording of "This identifies a current maximum outgoing transfer-id
>> that can be computed by subtracting one from the sum of
>> outgoing-window and next-outgoing-id." when describing it, coupled
>> with the requirement for the peer session to track it and decrement
>> the remote-outgoing-window when receiving transfers, does suggest to
>> me that things above the advertised point should not be sent (without
>> first asynchronously communicating an increaseat least) since it would
>> define a 'maximum' below the next-outgoing-id.
>>
>> On the other hand, you are right that it doesnt explicitly define when
>> we need to update the peer, and there is a specific error condition
>> symbol for when the peer exceeds the incoming window but there is no
>> equivalent condition for doing the same with the outgoing window.
>>
>> I think the wording is wooly enough we could be here for a while
>> figuring it out and still end up being unsure what it actually says
>> though :)
>>
>
> Fair enough, and I think the conservative thing to do is certainly to
> asynchronously notify your peer when the window changes, however as in my
> other reply I don't see how doing that actually eliminates the possibility
> of deadlock with service-bus. Asynchronously notifying your peer when you
> change the window is a very different requirement than being obliged to
> notify your peer that the window is nonzero in order to receive credit.
>
For the ServiceBus case I was saying we would need to do it
synchronously, which we obviously wouldnt want to for various reasons.
That would also presumably require us (and SB) to support the echo
flag, which I think I recall raising a JIRA about some time ago.
I'm much more inclined to set the window high and forget it exists :)
>
> I think Gordon is meaning in regard to the fact that we send a
>> transfer when our outgoing window is [initially] 0 without first
>> increasing it to e.g.1, due to the use of the phrasing I mentioned
>> earlier that defines a "maximum" id below our [initial] next outgoing
>> id. If so, I tend to share his view on that.
>>
>
> Gotcha, I certainly agree that the robust thing to do would be to issue the
> flow frame in such cases.
>
>
>> Ultimately I think we just set a big window and essentially forgot it
>> exists for now, we aren't using it at all within proton currently
>> after all. We might as well stop calculating the remote value given we
>> never actually read it.
>>
>> > I don't think we should do the max-frame-size thing
>> > though as this encourages the notion that the incoming-window is somehow
>> > dependent on the outgoing-window which as I said above is I think unsafe.
>> >
>> > I guess setting it to a large constant for now is fairly reasonable, but
>> I
>> > do think we should encourage service-bus to make its implementation more
>> > robust in this regard.
>> >
>> > --Rafael
>>
>> Can I take that as a +1 on the approach I proposed changes for on
>> PROTON-936 / https://github.com/apache/qpid-proton/pull/42 ?
>>
>
> Yes
>
> --Rafael
Great, I'll push that change in tomorrow morning.
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Jul 8, 2015 at 8:29 AM, Robbie Gemmell <ro...@gmail.com>
wrote:
> The wording of "This identifies a current maximum outgoing transfer-id
> that can be computed by subtracting one from the sum of
> outgoing-window and next-outgoing-id." when describing it, coupled
> with the requirement for the peer session to track it and decrement
> the remote-outgoing-window when receiving transfers, does suggest to
> me that things above the advertised point should not be sent (without
> first asynchronously communicating an increaseat least) since it would
> define a 'maximum' below the next-outgoing-id.
>
> On the other hand, you are right that it doesnt explicitly define when
> we need to update the peer, and there is a specific error condition
> symbol for when the peer exceeds the incoming window but there is no
> equivalent condition for doing the same with the outgoing window.
>
> I think the wording is wooly enough we could be here for a while
> figuring it out and still end up being unsure what it actually says
> though :)
>
Fair enough, and I think the conservative thing to do is certainly to
asynchronously notify your peer when the window changes, however as in my
other reply I don't see how doing that actually eliminates the possibility
of deadlock with service-bus. Asynchronously notifying your peer when you
change the window is a very different requirement than being obliged to
notify your peer that the window is nonzero in order to receive credit.
I think Gordon is meaning in regard to the fact that we send a
> transfer when our outgoing window is [initially] 0 without first
> increasing it to e.g.1, due to the use of the phrasing I mentioned
> earlier that defines a "maximum" id below our [initial] next outgoing
> id. If so, I tend to share his view on that.
>
Gotcha, I certainly agree that the robust thing to do would be to issue the
flow frame in such cases.
> Ultimately I think we just set a big window and essentially forgot it
> exists for now, we aren't using it at all within proton currently
> after all. We might as well stop calculating the remote value given we
> never actually read it.
>
> > I don't think we should do the max-frame-size thing
> > though as this encourages the notion that the incoming-window is somehow
> > dependent on the outgoing-window which as I said above is I think unsafe.
> >
> > I guess setting it to a large constant for now is fairly reasonable, but
> I
> > do think we should encourage service-bus to make its implementation more
> > robust in this regard.
> >
> > --Rafael
>
> Can I take that as a +1 on the approach I proposed changes for on
> PROTON-936 / https://github.com/apache/qpid-proton/pull/42 ?
>
Yes
--Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
On 8 July 2015 at 15:49, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Wed, Jul 8, 2015 at 2:03 AM, Gordon Sim <gs...@redhat.com> wrote:
>
>> On 07/08/2015 02:22 AM, Rafael Schloming wrote:
>>
>>> a value of zero is actually what
>>> signals that the receiver needs to take some action here, and arguably an
>>> initial value of zero is correct since it is signaling that the receiver
>>> needs to take action (in this case issue credit).
>>>
>>
>> My interpretation is that if 0 is sent as the initial value, the sender
>> cannot legally send any transfers without first expanding the window by
>> sending a flow with a non-zero value.
>>
>
> I think what is confusing about the spec language is that it is defining
> the meaning of the value in terms of the sending endpoint's state and then
> not saying anything about when the sending endpoint is obligated to
> communicate the value to the receiver. This is because (at least as
> originally conceived) it is never directly obligated to communicate the
> value to the receiver, i.e. it could just choose to grow its array
> indefinitely.
>
> By contrast the incoming-window is defined in terms of the receiver's
> endpoint state and it is (relatively) clearly communicated that the
> receiver is obligated to communicate a non-zero incoming-window if it
> wishes to receive transfer frames.
>
> So while I agree that the language as defined requires the outgoing window
> to be increased from zero, I don't think this implies (at least without a
> more careful reading) that the sender needs to communicate this fact before
> sending transfers. It owns the value so it can effectively increase the
> outgoing-window to 1 momentarily, send the transfer, and then decrease it
> back to zero without needing to sandwich that transfer in flow frames in
> order to signal the changing window values.
>
> Put another way, the rule that says it is illegal for the sender to send
> transfers when the receiver's incoming window is zero creates an obligation
> for the receiver to communicate its window, but there is no corresponding
> rule for the receiver, e.g. nothing says it is illegal for the receiver to
> send credit when the sender's outgoing-window is zero, nor that the
> receiver must compute it's incoming-window based on the sender's
> outgoing-window, so there is no obligation similarly implied for the sender.
>
The wording of "This identifies a current maximum outgoing transfer-id
that can be computed by subtracting one from the sum of
outgoing-window and next-outgoing-id." when describing it, coupled
with the requirement for the peer session to track it and decrement
the remote-outgoing-window when receiving transfers, does suggest to
me that things above the advertised point should not be sent (without
first asynchronously communicating an increaseat least) since it would
define a 'maximum' below the next-outgoing-id.
On the other hand, you are right that it doesnt explicitly define when
we need to update the peer, and there is a specific error condition
symbol for when the peer exceeds the incoming window but there is no
equivalent condition for doing the same with the outgoing window.
I think the wording is wooly enough we could be here for a while
figuring it out and still end up being unsure what it actually says
though :)
>
>>
>> Further I think the sender should not take the lack of credit as grounds
>> to set a window of 0. The receiver knows it has not issued credit. (At the
>> link level, the sender can also indicate that it has messages awaiting link
>> credit).
>>
>
> I agree that we shouldn't do this in our implementation, however I think it
> is a valid interpretation, which implies that making the incoming window
> dependent on your peers outgoing window (as service-bus is doing) is
> probably not a safe thing to do.
>
>
>>
>> In the case where the sender's implementation involves a fixed amount of
>> buffer space and requires messages to be settled before it can send more,
>> the receiver would not be able to know that without getting some signal. So
>> to my mind that is the only case for which it would make sense to send an
>> outgoing window of 0. (I'm not sure how useful this is in practice and I
>> don't believe it applies to proton at present anyway).
>>
>> I think as it stands proton is violating the spec, and should be changed
>> to send a non-zero outgoing window.
>>
>
> Per above I don't believe it is violating the spec, but given that it has
> been misinterpreted at least once, I certainly agree that some behavior
> change is warranted.
I think Gordon is meaning in regard to the fact that we send a
transfer when our outgoing window is [initially] 0 without first
increasing it to e.g.1, due to the use of the phrasing I mentioned
earlier that defines a "maximum" id below our [initial] next outgoing
id. If so, I tend to share his view on that.
Ultimately I think we just set a big window and essentially forgot it
exists for now, we aren't using it at all within proton currently
after all. We might as well stop calculating the remote value given we
never actually read it.
> I don't think we should do the max-frame-size thing
> though as this encourages the notion that the incoming-window is somehow
> dependent on the outgoing-window which as I said above is I think unsafe.
>
> I guess setting it to a large constant for now is fairly reasonable, but I
> do think we should encourage service-bus to make its implementation more
> robust in this regard.
>
> --Rafael
Can I take that as a +1 on the approach I proposed changes for on
PROTON-936 / https://github.com/apache/qpid-proton/pull/42 ?
Robbie
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Jul 8, 2015 at 2:03 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 07/08/2015 02:22 AM, Rafael Schloming wrote:
>
>> a value of zero is actually what
>> signals that the receiver needs to take some action here, and arguably an
>> initial value of zero is correct since it is signaling that the receiver
>> needs to take action (in this case issue credit).
>>
>
> My interpretation is that if 0 is sent as the initial value, the sender
> cannot legally send any transfers without first expanding the window by
> sending a flow with a non-zero value.
>
I think what is confusing about the spec language is that it is defining
the meaning of the value in terms of the sending endpoint's state and then
not saying anything about when the sending endpoint is obligated to
communicate the value to the receiver. This is because (at least as
originally conceived) it is never directly obligated to communicate the
value to the receiver, i.e. it could just choose to grow its array
indefinitely.
By contrast the incoming-window is defined in terms of the receiver's
endpoint state and it is (relatively) clearly communicated that the
receiver is obligated to communicate a non-zero incoming-window if it
wishes to receive transfer frames.
So while I agree that the language as defined requires the outgoing window
to be increased from zero, I don't think this implies (at least without a
more careful reading) that the sender needs to communicate this fact before
sending transfers. It owns the value so it can effectively increase the
outgoing-window to 1 momentarily, send the transfer, and then decrease it
back to zero without needing to sandwich that transfer in flow frames in
order to signal the changing window values.
Put another way, the rule that says it is illegal for the sender to send
transfers when the receiver's incoming window is zero creates an obligation
for the receiver to communicate its window, but there is no corresponding
rule for the receiver, e.g. nothing says it is illegal for the receiver to
send credit when the sender's outgoing-window is zero, nor that the
receiver must compute it's incoming-window based on the sender's
outgoing-window, so there is no obligation similarly implied for the sender.
>
> Further I think the sender should not take the lack of credit as grounds
> to set a window of 0. The receiver knows it has not issued credit. (At the
> link level, the sender can also indicate that it has messages awaiting link
> credit).
>
I agree that we shouldn't do this in our implementation, however I think it
is a valid interpretation, which implies that making the incoming window
dependent on your peers outgoing window (as service-bus is doing) is
probably not a safe thing to do.
>
> In the case where the sender's implementation involves a fixed amount of
> buffer space and requires messages to be settled before it can send more,
> the receiver would not be able to know that without getting some signal. So
> to my mind that is the only case for which it would make sense to send an
> outgoing window of 0. (I'm not sure how useful this is in practice and I
> don't believe it applies to proton at present anyway).
>
> I think as it stands proton is violating the spec, and should be changed
> to send a non-zero outgoing window.
>
Per above I don't believe it is violating the spec, but given that it has
been misinterpreted at least once, I certainly agree that some behavior
change is warranted. I don't think we should do the max-frame-size thing
though as this encourages the notion that the incoming-window is somehow
dependent on the outgoing-window which as I said above is I think unsafe.
I guess setting it to a large constant for now is fairly reasonable, but I
do think we should encourage service-bus to make its implementation more
robust in this regard.
--Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Jul 8, 2015 at 8:28 AM, iPC <da...@gmail.com> wrote:
> // quotes from the spec ---
> 2.5.6 Session Flow Control
> The Session Endpoint assigns each outgoing transfer frame an implicit
> transfer-id from a session scoped sequence.
>
> outgoing-window: The outgoing-window defines the maximum number of outgoing
> transfer frames that the endpoint can currently send. This identifies a
> current maximum outgoing transfer-id that can be computed by subtracting
> one from the sum of outgoing-window and next-outgoing-id.
> // end quotes ---
>
> Outgoing-window is a mandatory field on flow. The sender's outgoing window
> is communicated to the peer. If the sender sends a 0 outgoing window but
> subsequently sends a transfer frame, the implicit outgoing transfer-id
> would exceed the limit implied by the flow frame.
>
> I agree that the sender can maintain a local outgoing window to satisfy any
> local implementation specific logic, but the value communicated to the peer
> must allow further transfer frames to be sent.
>
> I don't think it makes senses to use 0 outgoing window to request for
> credit (assuming it is the link credit mentioned in early mails). Using a 0
> link credit would make more sense.
>
That's certainly a fair interpretation, but even under that interpretation
a potential deadlock with service-bus may still exist. Consider an
implementation that sets its initial outgoing window to zero, waits for
credit, and then upon receiving credit expands its window, sends a flow
frame with the updated window, and then sends a transfer. I expect this
implementation would deadlock with service-bus, but I don't think it is
violating the spec under any of the interpretations proposed in this thread.
--Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by iPC <da...@gmail.com>.
// quotes from the spec ---
2.5.6 Session Flow Control
The Session Endpoint assigns each outgoing transfer frame an implicit
transfer-id from a session scoped sequence.
outgoing-window: The outgoing-window defines the maximum number of outgoing
transfer frames that the endpoint can currently send. This identifies a
current maximum outgoing transfer-id that can be computed by subtracting
one from the sum of outgoing-window and next-outgoing-id.
// end quotes ---
Outgoing-window is a mandatory field on flow. The sender's outgoing window
is communicated to the peer. If the sender sends a 0 outgoing window but
subsequently sends a transfer frame, the implicit outgoing transfer-id
would exceed the limit implied by the flow frame.
I agree that the sender can maintain a local outgoing window to satisfy any
local implementation specific logic, but the value communicated to the peer
must allow further transfer frames to be sent.
I don't think it makes senses to use 0 outgoing window to request for
credit (assuming it is the link credit mentioned in early mails). Using a 0
link credit would make more sense.
On Wed, Jul 8, 2015 at 8:03 AM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Wed, Jul 8, 2015 at 2:38 AM, Robbie Gemmell <ro...@gmail.com>
> wrote:
>
> > On 8 July 2015 at 10:03, Gordon Sim <gs...@redhat.com> wrote:
> > > On 07/08/2015 02:22 AM, Rafael Schloming wrote:
> > >>
> > >> a value of zero is actually what
> > >> signals that the receiver needs to take some action here, and arguably
> > an
> > >> initial value of zero is correct since it is signaling that the
> receiver
> > >> needs to take action (in this case issue credit).
> > >
> > >
> > > My interpretation is that if 0 is sent as the initial value, the sender
> > > cannot legally send any transfers without first expanding the window by
> > > sending a flow with a non-zero value.
> > >
> >
> > Agreed. We don't currently do that, so we are typically violating the
> > window (Messenger being an exception, on its initial send to a node at
> > least).
> >
> > In the case the remote incoming window is also 0 (which it is in the
> > case which prompted this discussion) we would also have to
> > synchronously wait for a flow 'response' increasing it before we could
> > send anything, since we also need to know the remote incoming window
> > actually enabled us to send.
> >
>
> See my reply to Gordon for more details, but I think we need to be careful
> in distinguishing between the rules the spec mandates for
> updating/maintaining the value locally, and the rules the spec mandates for
> when it is necessary to communicate the value to the peer. These definitely
> should not be conflated because you don't want to notify the peer whenever
> you update the value locally, this would be way to chatty.
>
>
> > > Further I think the sender should not take the lack of credit as
> grounds
> > to
> > > set a window of 0. The receiver knows it has not issued credit. (At the
> > link
> > > level, the sender can also indicate that it has messages awaiting link
> > > credit).
> > >
> >
> > That was my take on reading things, that the two are essentially
> > separate mechanisms, and on top the two windows are separate as well.
> >
> > If the initial outgoing window was set to 0 because the reciever hasnt
> > issued, would there be a need for the field to exist on Begin? It
> > would seem like much of the time it wouldnt be used and a Flow would
> > have to be used instead.
> >
> > > In the case where the sender's implementation involves a fixed amount
> of
> > > buffer space and requires messages to be settled before it can send
> more,
> > > the receiver would not be able to know that without getting some
> signal.
> > So
> > > to my mind that is the only case for which it would make sense to send
> an
> > > outgoing window of 0. (I'm not sure how useful this is in practice and
> I
> > > don't believe it applies to proton at present anyway).
> > >
> > > I think as it stands proton is violating the spec, and should be
> changed
> > to
> > > send a non-zero outgoing window.
> >
> > That is what I did for the proposed changes on
> > https://issues.apache.org/jira/browse/PROTON-936 /
> > https://github.com/apache/qpid-proton/pull/42. Essentially setting it
> > initially on begin to max int (unless otherwise configured) and
> > leaving it there for any subsequent flows. We could later (when we
> > arent right before a release) look to make it smarter if needed.
> >
>
> See my other email for more details, but I agree this seems like a
> reasonable change for now.
>
> --Rafael
>
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
On 8 July 2015 at 16:03, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Wed, Jul 8, 2015 at 2:38 AM, Robbie Gemmell <ro...@gmail.com>
> wrote:
>
>> On 8 July 2015 at 10:03, Gordon Sim <gs...@redhat.com> wrote:
>> > On 07/08/2015 02:22 AM, Rafael Schloming wrote:
>> >>
>> >> a value of zero is actually what
>> >> signals that the receiver needs to take some action here, and arguably
>> an
>> >> initial value of zero is correct since it is signaling that the receiver
>> >> needs to take action (in this case issue credit).
>> >
>> >
>> > My interpretation is that if 0 is sent as the initial value, the sender
>> > cannot legally send any transfers without first expanding the window by
>> > sending a flow with a non-zero value.
>> >
>>
>> Agreed. We don't currently do that, so we are typically violating the
>> window (Messenger being an exception, on its initial send to a node at
>> least).
>>
>> In the case the remote incoming window is also 0 (which it is in the
>> case which prompted this discussion) we would also have to
>> synchronously wait for a flow 'response' increasing it before we could
>> send anything, since we also need to know the remote incoming window
>> actually enabled us to send.
>>
>
> See my reply to Gordon for more details, but I think we need to be careful
> in distinguishing between the rules the spec mandates for
> updating/maintaining the value locally, and the rules the spec mandates for
> when it is necessary to communicate the value to the peer. These definitely
> should not be conflated because you don't want to notify the peer whenever
> you update the value locally, this would be way to chatty.
I agree, you wouldnt waat to do that.
To reiterate, I was only saying you would have to do that when both
local outgoing and remote incoming were 0, given you would need to
hear back about the new remote incoming (to ensure it not being 0)
before you could actually send. You could of course asyncronously
increase your outgoing window normally if you needed to, it just
woudlnt do any good in that special case, which is the scenario we
found ourselves to prompt this discussion.
>
>
>> > Further I think the sender should not take the lack of credit as grounds
>> to
>> > set a window of 0. The receiver knows it has not issued credit. (At the
>> link
>> > level, the sender can also indicate that it has messages awaiting link
>> > credit).
>> >
>>
>> That was my take on reading things, that the two are essentially
>> separate mechanisms, and on top the two windows are separate as well.
>>
>> If the initial outgoing window was set to 0 because the reciever hasnt
>> issued, would there be a need for the field to exist on Begin? It
>> would seem like much of the time it wouldnt be used and a Flow would
>> have to be used instead.
>>
>> > In the case where the sender's implementation involves a fixed amount of
>> > buffer space and requires messages to be settled before it can send more,
>> > the receiver would not be able to know that without getting some signal.
>> So
>> > to my mind that is the only case for which it would make sense to send an
>> > outgoing window of 0. (I'm not sure how useful this is in practice and I
>> > don't believe it applies to proton at present anyway).
>> >
>> > I think as it stands proton is violating the spec, and should be changed
>> to
>> > send a non-zero outgoing window.
>>
>> That is what I did for the proposed changes on
>> https://issues.apache.org/jira/browse/PROTON-936 /
>> https://github.com/apache/qpid-proton/pull/42. Essentially setting it
>> initially on begin to max int (unless otherwise configured) and
>> leaving it there for any subsequent flows. We could later (when we
>> arent right before a release) look to make it smarter if needed.
>>
>
> See my other email for more details, but I agree this seems like a
> reasonable change for now.
>
> --Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Jul 8, 2015 at 2:38 AM, Robbie Gemmell <ro...@gmail.com>
wrote:
> On 8 July 2015 at 10:03, Gordon Sim <gs...@redhat.com> wrote:
> > On 07/08/2015 02:22 AM, Rafael Schloming wrote:
> >>
> >> a value of zero is actually what
> >> signals that the receiver needs to take some action here, and arguably
> an
> >> initial value of zero is correct since it is signaling that the receiver
> >> needs to take action (in this case issue credit).
> >
> >
> > My interpretation is that if 0 is sent as the initial value, the sender
> > cannot legally send any transfers without first expanding the window by
> > sending a flow with a non-zero value.
> >
>
> Agreed. We don't currently do that, so we are typically violating the
> window (Messenger being an exception, on its initial send to a node at
> least).
>
> In the case the remote incoming window is also 0 (which it is in the
> case which prompted this discussion) we would also have to
> synchronously wait for a flow 'response' increasing it before we could
> send anything, since we also need to know the remote incoming window
> actually enabled us to send.
>
See my reply to Gordon for more details, but I think we need to be careful
in distinguishing between the rules the spec mandates for
updating/maintaining the value locally, and the rules the spec mandates for
when it is necessary to communicate the value to the peer. These definitely
should not be conflated because you don't want to notify the peer whenever
you update the value locally, this would be way to chatty.
> > Further I think the sender should not take the lack of credit as grounds
> to
> > set a window of 0. The receiver knows it has not issued credit. (At the
> link
> > level, the sender can also indicate that it has messages awaiting link
> > credit).
> >
>
> That was my take on reading things, that the two are essentially
> separate mechanisms, and on top the two windows are separate as well.
>
> If the initial outgoing window was set to 0 because the reciever hasnt
> issued, would there be a need for the field to exist on Begin? It
> would seem like much of the time it wouldnt be used and a Flow would
> have to be used instead.
>
> > In the case where the sender's implementation involves a fixed amount of
> > buffer space and requires messages to be settled before it can send more,
> > the receiver would not be able to know that without getting some signal.
> So
> > to my mind that is the only case for which it would make sense to send an
> > outgoing window of 0. (I'm not sure how useful this is in practice and I
> > don't believe it applies to proton at present anyway).
> >
> > I think as it stands proton is violating the spec, and should be changed
> to
> > send a non-zero outgoing window.
>
> That is what I did for the proposed changes on
> https://issues.apache.org/jira/browse/PROTON-936 /
> https://github.com/apache/qpid-proton/pull/42. Essentially setting it
> initially on begin to max int (unless otherwise configured) and
> leaving it there for any subsequent flows. We could later (when we
> arent right before a release) look to make it smarter if needed.
>
See my other email for more details, but I agree this seems like a
reasonable change for now.
--Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
On 8 July 2015 at 10:03, Gordon Sim <gs...@redhat.com> wrote:
> On 07/08/2015 02:22 AM, Rafael Schloming wrote:
>>
>> a value of zero is actually what
>> signals that the receiver needs to take some action here, and arguably an
>> initial value of zero is correct since it is signaling that the receiver
>> needs to take action (in this case issue credit).
>
>
> My interpretation is that if 0 is sent as the initial value, the sender
> cannot legally send any transfers without first expanding the window by
> sending a flow with a non-zero value.
>
Agreed. We don't currently do that, so we are typically violating the
window (Messenger being an exception, on its initial send to a node at
least).
In the case the remote incoming window is also 0 (which it is in the
case which prompted this discussion) we would also have to
synchronously wait for a flow 'response' increasing it before we could
send anything, since we also need to know the remote incoming window
actually enabled us to send.
> Further I think the sender should not take the lack of credit as grounds to
> set a window of 0. The receiver knows it has not issued credit. (At the link
> level, the sender can also indicate that it has messages awaiting link
> credit).
>
That was my take on reading things, that the two are essentially
separate mechanisms, and on top the two windows are separate as well.
If the initial outgoing window was set to 0 because the reciever hasnt
issued, would there be a need for the field to exist on Begin? It
would seem like much of the time it wouldnt be used and a Flow would
have to be used instead.
> In the case where the sender's implementation involves a fixed amount of
> buffer space and requires messages to be settled before it can send more,
> the receiver would not be able to know that without getting some signal. So
> to my mind that is the only case for which it would make sense to send an
> outgoing window of 0. (I'm not sure how useful this is in practice and I
> don't believe it applies to proton at present anyway).
>
> I think as it stands proton is violating the spec, and should be changed to
> send a non-zero outgoing window.
That is what I did for the proposed changes on
https://issues.apache.org/jira/browse/PROTON-936 /
https://github.com/apache/qpid-proton/pull/42. Essentially setting it
initially on begin to max int (unless otherwise configured) and
leaving it there for any subsequent flows. We could later (when we
arent right before a release) look to make it smarter if needed.
Robbie
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Gordon Sim <gs...@redhat.com>.
On 07/08/2015 02:22 AM, Rafael Schloming wrote:
> a value of zero is actually what
> signals that the receiver needs to take some action here, and arguably an
> initial value of zero is correct since it is signaling that the receiver
> needs to take action (in this case issue credit).
My interpretation is that if 0 is sent as the initial value, the sender
cannot legally send any transfers without first expanding the window by
sending a flow with a non-zero value.
Further I think the sender should not take the lack of credit as grounds
to set a window of 0. The receiver knows it has not issued credit. (At
the link level, the sender can also indicate that it has messages
awaiting link credit).
In the case where the sender's implementation involves a fixed amount of
buffer space and requires messages to be settled before it can send
more, the receiver would not be able to know that without getting some
signal. So to my mind that is the only case for which it would make
sense to send an outgoing window of 0. (I'm not sure how useful this is
in practice and I don't believe it applies to proton at present anyway).
I think as it stands proton is violating the spec, and should be changed
to send a non-zero outgoing window.
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Tue, Jul 7, 2015 at 4:29 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 07/07/2015 07:22 AM, Rafael Schloming wrote:
>
>> IIRC, the definition of the outgoing-window was largely motivated by the
>> need to express to receivers certain conditions under which they may be
>> required to settle deliveries in order to receive more. For example if an
>> implementation uses a fixed sized array to store deliveries, and this
>> array
>> is keyed by the offset of the delivery-id from the smallest unsettled
>> delivery, then although the sender may have sufficient credit to send more
>> transfers, it may not actually be capable of doing this because the next
>> delivery-id would land outside the range of deliveries that are currently
>> represented within its fixed size array.
>>
>
> The outgoing-window is measured in transfers, right? So in this case each
> slot in the array would be a *transfer* with a single delivery possibly
> spanning multiple slots.
>
Yes, good point.
> This could happen for example if
>> the receiver issues N credits (where N is the size of the sender's fixed
>> array) and settles deliveries 2 through N. The sender is then stuck with
>> an
>> unsettled delivery in the first slot of its fixed sized array and cannot
>> send another delivery until that first delivery is settled.
>>
>> Given this, it's certainly true an outgoing-window of 0 is kind of strange
>> and useless.
>>
>
> Isn't that exactly the mechanism by which a sender, such as the one in
> your description above, would indicate its inability to send further
> transfers?
Gah, sorry, jet lag... you are right. I was thinking of the window measured
from the oldest unsettled transfers, however it is actually measured from
the next-outgoing-id, and so as you say a value of zero is actually what
signals that the receiver needs to take some action here, and arguably an
initial value of zero is correct since it is signaling that the receiver
needs to take action (in this case issue credit).
--Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Gordon Sim <gs...@redhat.com>.
On 07/07/2015 07:22 AM, Rafael Schloming wrote:
> IIRC, the definition of the outgoing-window was largely motivated by the
> need to express to receivers certain conditions under which they may be
> required to settle deliveries in order to receive more. For example if an
> implementation uses a fixed sized array to store deliveries, and this array
> is keyed by the offset of the delivery-id from the smallest unsettled
> delivery, then although the sender may have sufficient credit to send more
> transfers, it may not actually be capable of doing this because the next
> delivery-id would land outside the range of deliveries that are currently
> represented within its fixed size array.
The outgoing-window is measured in transfers, right? So in this case
each slot in the array would be a *transfer* with a single delivery
possibly spanning multiple slots.
> This could happen for example if
> the receiver issues N credits (where N is the size of the sender's fixed
> array) and settles deliveries 2 through N. The sender is then stuck with an
> unsettled delivery in the first slot of its fixed sized array and cannot
> send another delivery until that first delivery is settled.
>
> Given this, it's certainly true an outgoing-window of 0 is kind of strange
> and useless.
Isn't that exactly the mechanism by which a sender, such as the one in
your description above, would indicate its inability to send further
transfers?
> It's probably also true that it is never super useful for the
> incoming window of the receiver to be larger than the outgoing window of
> the sender (or vice versa) since one can't ever exceed the other, so I'd
> say your largest-possible-int default and max-frame-like treatment are
> fairly appropriate.
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Gordon Sim <gs...@redhat.com>.
On 07/07/2015 07:22 AM, Rafael Schloming wrote:
> IIRC, the definition of the outgoing-window was largely motivated by the
> need to express to receivers certain conditions under which they may be
> required to settle deliveries in order to receive more. For example if an
> implementation uses a fixed sized array to store deliveries, and this array
> is keyed by the offset of the delivery-id from the smallest unsettled
> delivery, then although the sender may have sufficient credit to send more
> transfers, it may not actually be capable of doing this because the next
> delivery-id would land outside the range of deliveries that are currently
> represented within its fixed size array.
The outgoing-window is measured in transfers, right? So in this case
each slot in the array would be a *transfer* with a single delivery
possibly spanning multiple slots.
> This could happen for example if
> the receiver issues N credits (where N is the size of the sender's fixed
> array) and settles deliveries 2 through N. The sender is then stuck with an
> unsettled delivery in the first slot of its fixed sized array and cannot
> send another delivery until that first delivery is settled.
>
> Given this, it's certainly true an outgoing-window of 0 is kind of strange
> and useless.
Isn't that exactly the mechanism by which a sender, such as the one in
your description above, would indicate its inability to send further
transfers?
> It's probably also true that it is never super useful for the
> incoming window of the receiver to be larger than the outgoing window of
> the sender (or vice versa) since one can't ever exceed the other, so I'd
> say your largest-possible-int default and max-frame-like treatment are
> fairly appropriate.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
On 7 July 2015 at 07:22, Rafael Schloming <rh...@alum.mit.edu> wrote:
> IIRC, the definition of the outgoing-window was largely motivated by the
> need to express to receivers certain conditions under which they may be
> required to settle deliveries in order to receive more. For example if an
> implementation uses a fixed sized array to store deliveries, and this array
> is keyed by the offset of the delivery-id from the smallest unsettled
> delivery, then although the sender may have sufficient credit to send more
> transfers, it may not actually be capable of doing this because the next
> delivery-id would land outside the range of deliveries that are currently
> represented within its fixed size array. This could happen for example if
> the receiver issues N credits (where N is the size of the sender's fixed
> array) and settles deliveries 2 through N. The sender is then stuck with an
> unsettled delivery in the first slot of its fixed sized array and cannot
> send another delivery until that first delivery is settled.
>
I see.
I assume things above are slightly simplified by considering a session
with a single sending link and, to Gordons point, only using messages
with a single transfer frame given link credit and session window
handling actually use different units (which actually seems like it
might pose an issue in certain cases)
> Given this, it's certainly true an outgoing-window of 0 is kind of strange
> and useless. It's probably also true that it is never super useful for the
> incoming window of the receiver to be larger than the outgoing window of
> the sender (or vice versa) since one can't ever exceed the other, so I'd
> say your largest-possible-int default and max-frame-like treatment are
> fairly appropriate.
>
> --Rafael
I added https://issues.apache.org/jira/browse/PROTON-936 for this,
with some initial changes for discussion on
https://github.com/apache/qpid-proton/pull/42
(The earlier change I suggested for proton-j which only changed the
behviour if the new setter was used is still at
https://github.com/gemmellr/qpid-proton/commit/b2b15a617b42bd36eacf9b51fb146223ba516496
though that might get garbase collected at some point as I rebased the
repo and orphaned it)
>
> On Fri, Jul 3, 2015 at 8:57 AM, Robbie Gemmell <ro...@gmail.com>
> wrote:
>
>> Rob, Rafi, as authors of the spec and the related code in proton, do
>> you have any thoughts to add here?
>>
>> Barring any discussion otherwise I will be looking to change proton to
>> at least optionally allow controlling the outgoing window along the
>> lines I mentioned near the end of my original mail.
>>
>> Robbie
>>
>> On 2 July 2015 at 00:15, Robbie Gemmell <ro...@gmail.com> wrote:
>> > Thanks James. Some expansion which may be useful to add.
>> >
>> > When comparing the older JMS client, proton-c via the Messenger API,
>> > and the new JMS client using proton-j, its important to note that they
>> > aren't all doing the same thing even where their underlying
>> > implementations do seem to share the same behaviour in the cases of
>> > proton-c and proton-j.
>> >
>> > The older JMS client initializes its outgoing window to a fixed number
>> > in the session Begin frame and then doesnt seem to ever change it for
>> > subsequent Flow frames, and simply manages whether its session can
>> > later send transfer frames based on the current value of the remote
>> > incoming window. Proton-J and Proton-C similarly only base their
>> > session level decision to send transfers on the remote incoming window
>> > and not their own outgoing window (which as noted below means they
>> > violate their advertised outgoing window, which is often going to be
>> > 0).
>> >
>> > Proton-C and Proton-J both currently look to set the outgoing window
>> > at any given time to a calculated value based on either the number of
>> > buffered messages or the buffered bytes divided by frame size. If
>> > there are no buffered messages at the point the Begin and Flow frames
>> > are generated, then the outgoing-window will be set to 0. This appears
>> > to function the same for both proton-c and proton-j. A key point
>> > though is that I think much of the historic usage of proton-c against
>> > Service Bus has been via the Messenger API, which works somewhat
>> > differently than many others in that it looks to create a session and
>> > a sender and sends the messages in one pipelined sequence of transport
>> > output, which means that by the point the Begin frame actually gets
>> > generated there are indeed buffered messages to send which means the
>> > outgoing-window is initialised to a value greater than zero. Other
>> > APIs which create the session as a distinct step thus wont ever have
>> > buffered messages when the Begin frame gets created and so the
>> > outgoing-window is initialised to 0, which is the behaviour observed
>> > with the new JMS client using proton-j and also what I saw when trying
>> > proton-c via the Qpid Messaging C++ client (against qpidd).
>> >
>> > Robbie
>> >
>> >
>> > On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
>> >> FYI, I have forwarded this and important bits of the preceding
>> discussion to our AMQP stack dev within the ServiceBus team.
>> >>
>> >> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been
>> working fine with Azure SB for years now. Proton-J, however, is not
>> something we have explored previously, and obviously there is something
>> different about its behavior compared to the other clients.
>> >>
>> >> The Qpid JMS client is our recommended JMS client for interop with
>> ServiceBus, and we would like to keep up with the times and not have to
>> direct customers to the legacy client, so we are very interested in
>> figuring out the correct resolution to this issue.
>> >>
>> >> -----Original Message-----
>> >> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
>> >> Sent: Wednesday, July 1, 2015 7:48 AM
>> >> To: users@qpid.apache.org; proton@qpid.apache.org
>> >> Subject: AMQP 1.0 session outgoing-window usage / meaning
>> >>
>> >> Hi all,
>> >>
>> >> Short intro:
>> >>
>> >> The way we use the outgoing-window feels wrong, and seems to violate at
>> least one bit of the related [and unclear overall] description in the spec.
>> The way we use it means we currently can't send messages to ServiceBus in
>> many cases (likely anything-but-messenger).
>> >>
>> >>
>> >> Full version:
>> >>
>> >> A user reported being unable to send messages to Service Bus (Azure or
>> Windows variants) using the new JMS client. Investigating that lead me to
>> the handling of session outgoing-window in proton (and incoming-window in
>> ServiceBus). I'd like to discuss how proton uses this, what the spec
>> actually says on how it should be used since its not clear to me these are
>> currently in alignment, and possibly changing how we utilise it in proton.
>> >>
>> >> In Proton the outgoing-window is set based on the amount of outstanding
>> outgoing bytes for the session (from buffered sends) and the frame size,
>> which leads to many situations where the advertised value is 0,
>> particularly in a client that creates sessions entirely independently from
>> sending messages (such as a JMS client, or the Qpid Messaging C++ client).
>> We then send transfer frames if a link has credit, regardless of the
>> session outgoing window, only withholding them if the
>> remote-incoming-window hits 0.
>> >>
>> >> This causes problems against ServiceBus because it seems to base its
>> advertised incoming window on the clients [initial] advertised outgoing
>> window. If it is 0, then it means we can never send any messages.
>> >>
>> >> The intent/definition of the outgoing-window seems a bit unclear as a
>> whole in the spec to me, to the point you could almost remove it without
>> issue given the [remote-]incoming-window seems to govern overall behaviour.
>> However, it seems wrong to me that we often set the outgoing-window to 0.
>> Doing this means we violate the outgoing window (local and at peer)
>> whenever we send a message if the remote-incoming-window allowed it. If we
>> tried not to violate it, we would have to flow the new session window
>> before every send, which again seems odd.
>> >>
>> >> It feels to me like it would be better to define initial values for the
>> windows, perhaps allowing that to be configurable in the same manner as e.g
>> max frame size is. Those that want to impose some limit then could, and
>> those that don't can leave/set it to the max values to achieve that effect.
>> >>
>> >> Thoughts?
>> >>
>> >> Robbie
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
IIRC, the definition of the outgoing-window was largely motivated by the
need to express to receivers certain conditions under which they may be
required to settle deliveries in order to receive more. For example if an
implementation uses a fixed sized array to store deliveries, and this array
is keyed by the offset of the delivery-id from the smallest unsettled
delivery, then although the sender may have sufficient credit to send more
transfers, it may not actually be capable of doing this because the next
delivery-id would land outside the range of deliveries that are currently
represented within its fixed size array. This could happen for example if
the receiver issues N credits (where N is the size of the sender's fixed
array) and settles deliveries 2 through N. The sender is then stuck with an
unsettled delivery in the first slot of its fixed sized array and cannot
send another delivery until that first delivery is settled.
Given this, it's certainly true an outgoing-window of 0 is kind of strange
and useless. It's probably also true that it is never super useful for the
incoming window of the receiver to be larger than the outgoing window of
the sender (or vice versa) since one can't ever exceed the other, so I'd
say your largest-possible-int default and max-frame-like treatment are
fairly appropriate.
--Rafael
On Fri, Jul 3, 2015 at 8:57 AM, Robbie Gemmell <ro...@gmail.com>
wrote:
> Rob, Rafi, as authors of the spec and the related code in proton, do
> you have any thoughts to add here?
>
> Barring any discussion otherwise I will be looking to change proton to
> at least optionally allow controlling the outgoing window along the
> lines I mentioned near the end of my original mail.
>
> Robbie
>
> On 2 July 2015 at 00:15, Robbie Gemmell <ro...@gmail.com> wrote:
> > Thanks James. Some expansion which may be useful to add.
> >
> > When comparing the older JMS client, proton-c via the Messenger API,
> > and the new JMS client using proton-j, its important to note that they
> > aren't all doing the same thing even where their underlying
> > implementations do seem to share the same behaviour in the cases of
> > proton-c and proton-j.
> >
> > The older JMS client initializes its outgoing window to a fixed number
> > in the session Begin frame and then doesnt seem to ever change it for
> > subsequent Flow frames, and simply manages whether its session can
> > later send transfer frames based on the current value of the remote
> > incoming window. Proton-J and Proton-C similarly only base their
> > session level decision to send transfers on the remote incoming window
> > and not their own outgoing window (which as noted below means they
> > violate their advertised outgoing window, which is often going to be
> > 0).
> >
> > Proton-C and Proton-J both currently look to set the outgoing window
> > at any given time to a calculated value based on either the number of
> > buffered messages or the buffered bytes divided by frame size. If
> > there are no buffered messages at the point the Begin and Flow frames
> > are generated, then the outgoing-window will be set to 0. This appears
> > to function the same for both proton-c and proton-j. A key point
> > though is that I think much of the historic usage of proton-c against
> > Service Bus has been via the Messenger API, which works somewhat
> > differently than many others in that it looks to create a session and
> > a sender and sends the messages in one pipelined sequence of transport
> > output, which means that by the point the Begin frame actually gets
> > generated there are indeed buffered messages to send which means the
> > outgoing-window is initialised to a value greater than zero. Other
> > APIs which create the session as a distinct step thus wont ever have
> > buffered messages when the Begin frame gets created and so the
> > outgoing-window is initialised to 0, which is the behaviour observed
> > with the new JMS client using proton-j and also what I saw when trying
> > proton-c via the Qpid Messaging C++ client (against qpidd).
> >
> > Robbie
> >
> >
> > On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
> >> FYI, I have forwarded this and important bits of the preceding
> discussion to our AMQP stack dev within the ServiceBus team.
> >>
> >> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been
> working fine with Azure SB for years now. Proton-J, however, is not
> something we have explored previously, and obviously there is something
> different about its behavior compared to the other clients.
> >>
> >> The Qpid JMS client is our recommended JMS client for interop with
> ServiceBus, and we would like to keep up with the times and not have to
> direct customers to the legacy client, so we are very interested in
> figuring out the correct resolution to this issue.
> >>
> >> -----Original Message-----
> >> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
> >> Sent: Wednesday, July 1, 2015 7:48 AM
> >> To: users@qpid.apache.org; proton@qpid.apache.org
> >> Subject: AMQP 1.0 session outgoing-window usage / meaning
> >>
> >> Hi all,
> >>
> >> Short intro:
> >>
> >> The way we use the outgoing-window feels wrong, and seems to violate at
> least one bit of the related [and unclear overall] description in the spec.
> The way we use it means we currently can't send messages to ServiceBus in
> many cases (likely anything-but-messenger).
> >>
> >>
> >> Full version:
> >>
> >> A user reported being unable to send messages to Service Bus (Azure or
> Windows variants) using the new JMS client. Investigating that lead me to
> the handling of session outgoing-window in proton (and incoming-window in
> ServiceBus). I'd like to discuss how proton uses this, what the spec
> actually says on how it should be used since its not clear to me these are
> currently in alignment, and possibly changing how we utilise it in proton.
> >>
> >> In Proton the outgoing-window is set based on the amount of outstanding
> outgoing bytes for the session (from buffered sends) and the frame size,
> which leads to many situations where the advertised value is 0,
> particularly in a client that creates sessions entirely independently from
> sending messages (such as a JMS client, or the Qpid Messaging C++ client).
> We then send transfer frames if a link has credit, regardless of the
> session outgoing window, only withholding them if the
> remote-incoming-window hits 0.
> >>
> >> This causes problems against ServiceBus because it seems to base its
> advertised incoming window on the clients [initial] advertised outgoing
> window. If it is 0, then it means we can never send any messages.
> >>
> >> The intent/definition of the outgoing-window seems a bit unclear as a
> whole in the spec to me, to the point you could almost remove it without
> issue given the [remote-]incoming-window seems to govern overall behaviour.
> However, it seems wrong to me that we often set the outgoing-window to 0.
> Doing this means we violate the outgoing window (local and at peer)
> whenever we send a message if the remote-incoming-window allowed it. If we
> tried not to violate it, we would have to flow the new session window
> before every send, which again seems odd.
> >>
> >> It feels to me like it would be better to define initial values for the
> windows, perhaps allowing that to be configurable in the same manner as e.g
> max frame size is. Those that want to impose some limit then could, and
> those that don't can leave/set it to the max values to achieve that effect.
> >>
> >> Thoughts?
> >>
> >> Robbie
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Rafael Schloming <rh...@alum.mit.edu>.
IIRC, the definition of the outgoing-window was largely motivated by the
need to express to receivers certain conditions under which they may be
required to settle deliveries in order to receive more. For example if an
implementation uses a fixed sized array to store deliveries, and this array
is keyed by the offset of the delivery-id from the smallest unsettled
delivery, then although the sender may have sufficient credit to send more
transfers, it may not actually be capable of doing this because the next
delivery-id would land outside the range of deliveries that are currently
represented within its fixed size array. This could happen for example if
the receiver issues N credits (where N is the size of the sender's fixed
array) and settles deliveries 2 through N. The sender is then stuck with an
unsettled delivery in the first slot of its fixed sized array and cannot
send another delivery until that first delivery is settled.
Given this, it's certainly true an outgoing-window of 0 is kind of strange
and useless. It's probably also true that it is never super useful for the
incoming window of the receiver to be larger than the outgoing window of
the sender (or vice versa) since one can't ever exceed the other, so I'd
say your largest-possible-int default and max-frame-like treatment are
fairly appropriate.
--Rafael
On Fri, Jul 3, 2015 at 8:57 AM, Robbie Gemmell <ro...@gmail.com>
wrote:
> Rob, Rafi, as authors of the spec and the related code in proton, do
> you have any thoughts to add here?
>
> Barring any discussion otherwise I will be looking to change proton to
> at least optionally allow controlling the outgoing window along the
> lines I mentioned near the end of my original mail.
>
> Robbie
>
> On 2 July 2015 at 00:15, Robbie Gemmell <ro...@gmail.com> wrote:
> > Thanks James. Some expansion which may be useful to add.
> >
> > When comparing the older JMS client, proton-c via the Messenger API,
> > and the new JMS client using proton-j, its important to note that they
> > aren't all doing the same thing even where their underlying
> > implementations do seem to share the same behaviour in the cases of
> > proton-c and proton-j.
> >
> > The older JMS client initializes its outgoing window to a fixed number
> > in the session Begin frame and then doesnt seem to ever change it for
> > subsequent Flow frames, and simply manages whether its session can
> > later send transfer frames based on the current value of the remote
> > incoming window. Proton-J and Proton-C similarly only base their
> > session level decision to send transfers on the remote incoming window
> > and not their own outgoing window (which as noted below means they
> > violate their advertised outgoing window, which is often going to be
> > 0).
> >
> > Proton-C and Proton-J both currently look to set the outgoing window
> > at any given time to a calculated value based on either the number of
> > buffered messages or the buffered bytes divided by frame size. If
> > there are no buffered messages at the point the Begin and Flow frames
> > are generated, then the outgoing-window will be set to 0. This appears
> > to function the same for both proton-c and proton-j. A key point
> > though is that I think much of the historic usage of proton-c against
> > Service Bus has been via the Messenger API, which works somewhat
> > differently than many others in that it looks to create a session and
> > a sender and sends the messages in one pipelined sequence of transport
> > output, which means that by the point the Begin frame actually gets
> > generated there are indeed buffered messages to send which means the
> > outgoing-window is initialised to a value greater than zero. Other
> > APIs which create the session as a distinct step thus wont ever have
> > buffered messages when the Begin frame gets created and so the
> > outgoing-window is initialised to 0, which is the behaviour observed
> > with the new JMS client using proton-j and also what I saw when trying
> > proton-c via the Qpid Messaging C++ client (against qpidd).
> >
> > Robbie
> >
> >
> > On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
> >> FYI, I have forwarded this and important bits of the preceding
> discussion to our AMQP stack dev within the ServiceBus team.
> >>
> >> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been
> working fine with Azure SB for years now. Proton-J, however, is not
> something we have explored previously, and obviously there is something
> different about its behavior compared to the other clients.
> >>
> >> The Qpid JMS client is our recommended JMS client for interop with
> ServiceBus, and we would like to keep up with the times and not have to
> direct customers to the legacy client, so we are very interested in
> figuring out the correct resolution to this issue.
> >>
> >> -----Original Message-----
> >> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
> >> Sent: Wednesday, July 1, 2015 7:48 AM
> >> To: users@qpid.apache.org; proton@qpid.apache.org
> >> Subject: AMQP 1.0 session outgoing-window usage / meaning
> >>
> >> Hi all,
> >>
> >> Short intro:
> >>
> >> The way we use the outgoing-window feels wrong, and seems to violate at
> least one bit of the related [and unclear overall] description in the spec.
> The way we use it means we currently can't send messages to ServiceBus in
> many cases (likely anything-but-messenger).
> >>
> >>
> >> Full version:
> >>
> >> A user reported being unable to send messages to Service Bus (Azure or
> Windows variants) using the new JMS client. Investigating that lead me to
> the handling of session outgoing-window in proton (and incoming-window in
> ServiceBus). I'd like to discuss how proton uses this, what the spec
> actually says on how it should be used since its not clear to me these are
> currently in alignment, and possibly changing how we utilise it in proton.
> >>
> >> In Proton the outgoing-window is set based on the amount of outstanding
> outgoing bytes for the session (from buffered sends) and the frame size,
> which leads to many situations where the advertised value is 0,
> particularly in a client that creates sessions entirely independently from
> sending messages (such as a JMS client, or the Qpid Messaging C++ client).
> We then send transfer frames if a link has credit, regardless of the
> session outgoing window, only withholding them if the
> remote-incoming-window hits 0.
> >>
> >> This causes problems against ServiceBus because it seems to base its
> advertised incoming window on the clients [initial] advertised outgoing
> window. If it is 0, then it means we can never send any messages.
> >>
> >> The intent/definition of the outgoing-window seems a bit unclear as a
> whole in the spec to me, to the point you could almost remove it without
> issue given the [remote-]incoming-window seems to govern overall behaviour.
> However, it seems wrong to me that we often set the outgoing-window to 0.
> Doing this means we violate the outgoing window (local and at peer)
> whenever we send a message if the remote-incoming-window allowed it. If we
> tried not to violate it, we would have to flow the new session window
> before every send, which again seems odd.
> >>
> >> It feels to me like it would be better to define initial values for the
> windows, perhaps allowing that to be configurable in the same manner as e.g
> max frame size is. Those that want to impose some limit then could, and
> those that don't can leave/set it to the max values to achieve that effect.
> >>
> >> Thoughts?
> >>
> >> Robbie
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
Rob, Rafi, as authors of the spec and the related code in proton, do
you have any thoughts to add here?
Barring any discussion otherwise I will be looking to change proton to
at least optionally allow controlling the outgoing window along the
lines I mentioned near the end of my original mail.
Robbie
On 2 July 2015 at 00:15, Robbie Gemmell <ro...@gmail.com> wrote:
> Thanks James. Some expansion which may be useful to add.
>
> When comparing the older JMS client, proton-c via the Messenger API,
> and the new JMS client using proton-j, its important to note that they
> aren't all doing the same thing even where their underlying
> implementations do seem to share the same behaviour in the cases of
> proton-c and proton-j.
>
> The older JMS client initializes its outgoing window to a fixed number
> in the session Begin frame and then doesnt seem to ever change it for
> subsequent Flow frames, and simply manages whether its session can
> later send transfer frames based on the current value of the remote
> incoming window. Proton-J and Proton-C similarly only base their
> session level decision to send transfers on the remote incoming window
> and not their own outgoing window (which as noted below means they
> violate their advertised outgoing window, which is often going to be
> 0).
>
> Proton-C and Proton-J both currently look to set the outgoing window
> at any given time to a calculated value based on either the number of
> buffered messages or the buffered bytes divided by frame size. If
> there are no buffered messages at the point the Begin and Flow frames
> are generated, then the outgoing-window will be set to 0. This appears
> to function the same for both proton-c and proton-j. A key point
> though is that I think much of the historic usage of proton-c against
> Service Bus has been via the Messenger API, which works somewhat
> differently than many others in that it looks to create a session and
> a sender and sends the messages in one pipelined sequence of transport
> output, which means that by the point the Begin frame actually gets
> generated there are indeed buffered messages to send which means the
> outgoing-window is initialised to a value greater than zero. Other
> APIs which create the session as a distinct step thus wont ever have
> buffered messages when the Begin frame gets created and so the
> outgoing-window is initialised to 0, which is the behaviour observed
> with the new JMS client using proton-j and also what I saw when trying
> proton-c via the Qpid Messaging C++ client (against qpidd).
>
> Robbie
>
>
> On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
>> FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
>>
>> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
>>
>> The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
>>
>> -----Original Message-----
>> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
>> Sent: Wednesday, July 1, 2015 7:48 AM
>> To: users@qpid.apache.org; proton@qpid.apache.org
>> Subject: AMQP 1.0 session outgoing-window usage / meaning
>>
>> Hi all,
>>
>> Short intro:
>>
>> The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
>>
>>
>> Full version:
>>
>> A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
>>
>> In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
>>
>> This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
>>
>> The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
>>
>> It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
>>
>> Thoughts?
>>
>> Robbie
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
Rob, Rafi, as authors of the spec and the related code in proton, do
you have any thoughts to add here?
Barring any discussion otherwise I will be looking to change proton to
at least optionally allow controlling the outgoing window along the
lines I mentioned near the end of my original mail.
Robbie
On 2 July 2015 at 00:15, Robbie Gemmell <ro...@gmail.com> wrote:
> Thanks James. Some expansion which may be useful to add.
>
> When comparing the older JMS client, proton-c via the Messenger API,
> and the new JMS client using proton-j, its important to note that they
> aren't all doing the same thing even where their underlying
> implementations do seem to share the same behaviour in the cases of
> proton-c and proton-j.
>
> The older JMS client initializes its outgoing window to a fixed number
> in the session Begin frame and then doesnt seem to ever change it for
> subsequent Flow frames, and simply manages whether its session can
> later send transfer frames based on the current value of the remote
> incoming window. Proton-J and Proton-C similarly only base their
> session level decision to send transfers on the remote incoming window
> and not their own outgoing window (which as noted below means they
> violate their advertised outgoing window, which is often going to be
> 0).
>
> Proton-C and Proton-J both currently look to set the outgoing window
> at any given time to a calculated value based on either the number of
> buffered messages or the buffered bytes divided by frame size. If
> there are no buffered messages at the point the Begin and Flow frames
> are generated, then the outgoing-window will be set to 0. This appears
> to function the same for both proton-c and proton-j. A key point
> though is that I think much of the historic usage of proton-c against
> Service Bus has been via the Messenger API, which works somewhat
> differently than many others in that it looks to create a session and
> a sender and sends the messages in one pipelined sequence of transport
> output, which means that by the point the Begin frame actually gets
> generated there are indeed buffered messages to send which means the
> outgoing-window is initialised to a value greater than zero. Other
> APIs which create the session as a distinct step thus wont ever have
> buffered messages when the Begin frame gets created and so the
> outgoing-window is initialised to 0, which is the behaviour observed
> with the new JMS client using proton-j and also what I saw when trying
> proton-c via the Qpid Messaging C++ client (against qpidd).
>
> Robbie
>
>
> On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
>> FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
>>
>> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
>>
>> The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
>>
>> -----Original Message-----
>> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
>> Sent: Wednesday, July 1, 2015 7:48 AM
>> To: users@qpid.apache.org; proton@qpid.apache.org
>> Subject: AMQP 1.0 session outgoing-window usage / meaning
>>
>> Hi all,
>>
>> Short intro:
>>
>> The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
>>
>>
>> Full version:
>>
>> A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
>>
>> In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
>>
>> This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
>>
>> The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
>>
>> It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
>>
>> Thoughts?
>>
>> Robbie
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
Thanks James. Some expansion which may be useful to add.
When comparing the older JMS client, proton-c via the Messenger API,
and the new JMS client using proton-j, its important to note that they
aren't all doing the same thing even where their underlying
implementations do seem to share the same behaviour in the cases of
proton-c and proton-j.
The older JMS client initializes its outgoing window to a fixed number
in the session Begin frame and then doesnt seem to ever change it for
subsequent Flow frames, and simply manages whether its session can
later send transfer frames based on the current value of the remote
incoming window. Proton-J and Proton-C similarly only base their
session level decision to send transfers on the remote incoming window
and not their own outgoing window (which as noted below means they
violate their advertised outgoing window, which is often going to be
0).
Proton-C and Proton-J both currently look to set the outgoing window
at any given time to a calculated value based on either the number of
buffered messages or the buffered bytes divided by frame size. If
there are no buffered messages at the point the Begin and Flow frames
are generated, then the outgoing-window will be set to 0. This appears
to function the same for both proton-c and proton-j. A key point
though is that I think much of the historic usage of proton-c against
Service Bus has been via the Messenger API, which works somewhat
differently than many others in that it looks to create a session and
a sender and sends the messages in one pipelined sequence of transport
output, which means that by the point the Begin frame actually gets
generated there are indeed buffered messages to send which means the
outgoing-window is initialised to a value greater than zero. Other
APIs which create the session as a distinct step thus wont ever have
buffered messages when the Begin frame gets created and so the
outgoing-window is initialised to 0, which is the behaviour observed
with the new JMS client using proton-j and also what I saw when trying
proton-c via the Qpid Messaging C++ client (against qpidd).
Robbie
On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
> FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
>
> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
>
> The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
>
> -----Original Message-----
> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
> Sent: Wednesday, July 1, 2015 7:48 AM
> To: users@qpid.apache.org; proton@qpid.apache.org
> Subject: AMQP 1.0 session outgoing-window usage / meaning
>
> Hi all,
>
> Short intro:
>
> The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
>
>
> Full version:
>
> A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
>
> In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
>
> This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
>
> The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
>
> It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
>
> Thoughts?
>
> Robbie
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 session outgoing-window usage / meaning
Posted by Robbie Gemmell <ro...@gmail.com>.
Thanks James. Some expansion which may be useful to add.
When comparing the older JMS client, proton-c via the Messenger API,
and the new JMS client using proton-j, its important to note that they
aren't all doing the same thing even where their underlying
implementations do seem to share the same behaviour in the cases of
proton-c and proton-j.
The older JMS client initializes its outgoing window to a fixed number
in the session Begin frame and then doesnt seem to ever change it for
subsequent Flow frames, and simply manages whether its session can
later send transfer frames based on the current value of the remote
incoming window. Proton-J and Proton-C similarly only base their
session level decision to send transfers on the remote incoming window
and not their own outgoing window (which as noted below means they
violate their advertised outgoing window, which is often going to be
0).
Proton-C and Proton-J both currently look to set the outgoing window
at any given time to a calculated value based on either the number of
buffered messages or the buffered bytes divided by frame size. If
there are no buffered messages at the point the Begin and Flow frames
are generated, then the outgoing-window will be set to 0. This appears
to function the same for both proton-c and proton-j. A key point
though is that I think much of the historic usage of proton-c against
Service Bus has been via the Messenger API, which works somewhat
differently than many others in that it looks to create a session and
a sender and sends the messages in one pipelined sequence of transport
output, which means that by the point the Begin frame actually gets
generated there are indeed buffered messages to send which means the
outgoing-window is initialised to a value greater than zero. Other
APIs which create the session as a distinct step thus wont ever have
buffered messages when the Begin frame gets created and so the
outgoing-window is initialised to 0, which is the behaviour observed
with the new JMS client using proton-j and also what I saw when trying
proton-c via the Qpid Messaging C++ client (against qpidd).
Robbie
On 1 July 2015 at 20:54, James Birdsall <jb...@microsoft.com> wrote:
> FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
>
> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
>
> The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
>
> -----Original Message-----
> From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
> Sent: Wednesday, July 1, 2015 7:48 AM
> To: users@qpid.apache.org; proton@qpid.apache.org
> Subject: AMQP 1.0 session outgoing-window usage / meaning
>
> Hi all,
>
> Short intro:
>
> The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
>
>
> Full version:
>
> A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
>
> In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
>
> This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
>
> The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
>
> It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
>
> Thoughts?
>
> Robbie
RE: AMQP 1.0 session outgoing-window usage / meaning
Posted by James Birdsall <jb...@microsoft.com>.
FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
-----Original Message-----
From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
Sent: Wednesday, July 1, 2015 7:48 AM
To: users@qpid.apache.org; proton@qpid.apache.org
Subject: AMQP 1.0 session outgoing-window usage / meaning
Hi all,
Short intro:
The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
Full version:
A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
Thoughts?
Robbie
RE: AMQP 1.0 session outgoing-window usage / meaning
Posted by James Birdsall <jb...@microsoft.com>.
FYI, I have forwarded this and important bits of the preceding discussion to our AMQP stack dev within the ServiceBus team.
Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been working fine with Azure SB for years now. Proton-J, however, is not something we have explored previously, and obviously there is something different about its behavior compared to the other clients.
The Qpid JMS client is our recommended JMS client for interop with ServiceBus, and we would like to keep up with the times and not have to direct customers to the legacy client, so we are very interested in figuring out the correct resolution to this issue.
-----Original Message-----
From: Robbie Gemmell [mailto:robbie.gemmell@gmail.com]
Sent: Wednesday, July 1, 2015 7:48 AM
To: users@qpid.apache.org; proton@qpid.apache.org
Subject: AMQP 1.0 session outgoing-window usage / meaning
Hi all,
Short intro:
The way we use the outgoing-window feels wrong, and seems to violate at least one bit of the related [and unclear overall] description in the spec. The way we use it means we currently can't send messages to ServiceBus in many cases (likely anything-but-messenger).
Full version:
A user reported being unable to send messages to Service Bus (Azure or Windows variants) using the new JMS client. Investigating that lead me to the handling of session outgoing-window in proton (and incoming-window in ServiceBus). I'd like to discuss how proton uses this, what the spec actually says on how it should be used since its not clear to me these are currently in alignment, and possibly changing how we utilise it in proton.
In Proton the outgoing-window is set based on the amount of outstanding outgoing bytes for the session (from buffered sends) and the frame size, which leads to many situations where the advertised value is 0, particularly in a client that creates sessions entirely independently from sending messages (such as a JMS client, or the Qpid Messaging C++ client). We then send transfer frames if a link has credit, regardless of the session outgoing window, only withholding them if the remote-incoming-window hits 0.
This causes problems against ServiceBus because it seems to base its advertised incoming window on the clients [initial] advertised outgoing window. If it is 0, then it means we can never send any messages.
The intent/definition of the outgoing-window seems a bit unclear as a whole in the spec to me, to the point you could almost remove it without issue given the [remote-]incoming-window seems to govern overall behaviour. However, it seems wrong to me that we often set the outgoing-window to 0. Doing this means we violate the outgoing window (local and at peer) whenever we send a message if the remote-incoming-window allowed it. If we tried not to violate it, we would have to flow the new session window before every send, which again seems odd.
It feels to me like it would be better to define initial values for the windows, perhaps allowing that to be configurable in the same manner as e.g max frame size is. Those that want to impose some limit then could, and those that don't can leave/set it to the max values to achieve that effect.
Thoughts?
Robbie
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org