You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@qpid.apache.org by Pankaj Bhagra <pa...@gmail.com> on 2017/08/08 01:05:44 UTC

proton server (azure SB) limit the incoming_window=5000

I am trying to extract bulk messages from azure SB.

As per their documentation the azure SDK doesn't support bulk read message
and recommends using the native amqp for the Azure Service Bus. White
trying to negotiate a session with the azure SB, i noticed that independent
of what client is requesting, the SB dials down the incoming_window=5000.
This limits a max of 5000B send per bulk read, thus my consumer runs dry
till the RTT (which is large for inter cloud) to fetch a new packet.

Is this a restriction of the azure SB or am i not setting any/some of the
parameters correctly from the client side to achieve the negotiated window
size  > 5000B.

I am using python proton MessagingHandler Class and clearly see that
on_message is called for few pkts composing of buffer size ~5000B and then
have to wait for the RTT delay to get the next batch.

Any suggestion to work around this problem and get larger bulked message ?
I can't reduce the RTT between the server and consumer. have some
workaround for parallel consumers but would like to solve the bulk problem
as that is most efficient way of achieving the high throughput.

class Recv(MessagingHandler):

    def __init__(self):

        super(Recv, self).__init__(prefetch=100, auto_accept=True,
auto_settle=True, peer_close_is_error=False)


    def on_start(self, event):

        conn = event.container.connect(connString)

        event.container.create_receiver(conn, subscription)


    def on_message(self, event):

        print(event.message.body)

        print datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3],
self.count, event.receiver.queued


[0xace380]:  -> SASL

[0xace380]:  <- SASL

[0xace380]:0 <- @sasl-mechanisms(64)
[sasl-server-mechanisms=@PN_SYMBOL[:MSSBCBS, :PLAIN, :ANONYMOUS, :EXTERNAL]]

[0xace380]:0 -> @sasl-init(65) [mechanism=:PLAIN,
initial-response=b"\x00iothubroutes_XXXXX\XXXXX="]

[0xace380]:0 <- @sasl-outcome(68) [code=0, additional-data=b"Welcome!"]

[0xace380]:  -> AMQP

[0xace380]:0 -> @open(16)
[container-id="0ad171ca-cefa-4a27-a7dc-0520e5393fa5", hostname="
nebhubsb.servicebus.windows.net", channel-max=32767]

[0xace380]:0 -> @begin(17) [next-outgoing-id=0, incoming-window=2147483647,
outgoing-window=2147483647]

[0xace380]:0 -> @attach(18)
[name="0ad171ca-cefa-4a27-a7dc-0520e5393fa5-kukatopic/Subscriptions/kukasub",
handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0,
source=@source(40) [address="kukatopic/Subscriptions/kukasub", durable=0,
timeout=0, dynamic=false], target=@target(41) [durable=0, timeout=0,
dynamic=false], initial-delivery-count=0, max-message-size=0]

[0xace380]:0 -> @flow(19) [incoming-window=2147483647, next-outgoing-id=0,
outgoing-window=2147483647, handle=0, delivery-count=0, link-credit=100,
drain=false]

[0xace380]:  <- AMQP

[0xace380]:0 <- @open(16)
[container-id="b970f07881334c658eb80ff336f2a683_G16", max-frame-size=65536,
channel-max=4999, idle-time-out=240000]

[0xace380]:0 <- @begin(17) [remote-channel=0, next-outgoing-id=1,
incoming-window=5000, outgoing-window=2147483647, handle-max=255]

[0xace380]:0 <- @attach(18)
[name="0ad171ca-cefa-4a27-a7dc-0520e5393fa5-kukatopic/Subscriptions/kukasub",
handle=0, role=false, rcv-settle-mode=1, source=@source(40)
[address="topic/Subscriptions/sub", durable=0, timeout=0, dynamic=false],
target=@target(41) [durable=0, timeout=0, dynamic=false],
initial-delivery-count=0, max-message-size=266240]

Re: proton server (azure SB) limit the incoming_window=5000

Posted by Gordon Sim <gs...@redhat.com>.

On 08/08/17 02:05, Pankaj Bhagra wrote:
> I am trying to extract bulk messages from azure SB.
> 
> As per their documentation the azure SDK doesn't support bulk read message
> and recommends using the native amqp for the Azure Service Bus. White
> trying to negotiate a session with the azure SB, i noticed that independent
> of what client is requesting, the SB dials down the incoming_window=5000.
> This limits a max of 5000B send per bulk read, thus my consumer runs dry
> till the RTT (which is large for inter cloud) to fetch a new packet.
> 
> Is this a restriction of the azure SB or am i not setting any/some of the
> parameters correctly from the client side to achieve the negotiated window
> size  > 5000B.
> 
> I am using python proton MessagingHandler Class and clearly see that
> on_message is called for few pkts composing of buffer size ~5000B and then
> have to wait for the RTT delay to get the next batch.
> 
> Any suggestion to work around this problem and get larger bulked message ?
> I can't reduce the RTT between the server and consumer. have some
> workaround for parallel consumers but would like to solve the bulk problem
> as that is most efficient way of achieving the high throughput.

The incoming window is set by each peer independently (i.e. it is not 
negotiated) and only covers the incoming transfers. For a link receiving 
from service bus, the incoming window set by service bus isn't relevant. 
It is the clients incoming window that would affect transfers from 
service-bus to client.

How large are the messages? From the protocol trace it looks like you 
are issuing 100 link credits. If the messages were all 50 bytes, that 
might explain the limited batches you are seeing? You could try 
increasing that link credit window.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org