You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Francesco Raviglione <fr...@gmail.com> on 2020/02/08 10:36:39 UTC
Avoiding aggregation of "transfer" messages in Qpid Proton
Hello,
I'm sorry about the several questions I'm posting on the mailing list
during these days, but, unfortunately, I'm struggling a bit in the usage of
Qpid Proton (in particular, on the C version).
One issue I noticed, and that I was not able to solve, is related to the
fact that, when following and adapting the send.c example (available inside
the C API reference:
https://qpid.apache.org/releases/qpid-proton-0.30.0/proton/c/api/send_8c-example.html),
all the AMQP "transfer" messages are aggregated and if, for instance, I
send 200 messages, one every 100 ms (using an additional Linux timer), I
have to wait for 20 seconds before seeing them on the receiving side (an
ActiveMQ broker, in my case), as all the 200 transfers are aggregated into
a single, very big, AMQP message, which is sent only at the end of the
process.
This is something I would like to avoid, as it makes end-to-end message
delivery (from producer, to broker, to consumer) too slow.
Is there a way in which I can avoid this behaviour and make messages being
sent immediately?
Thank you very much in advance.
Re: Avoiding aggregation of "transfer" messages in Qpid Proton
Posted by Francesco Raviglione <fr...@gmail.com>.
Dear Chuck, dear Gordon,
Thank you very much for your replies.
Now it is clear that I have to return from the handler before being able to
perform any I/O operation.
Returning from it after sending each message with "pn_message_send()", as
you suggested, and handling all the successive PN_DELIVERY callbacks, I was
able to obtain the desired result, sending each message almost immediately.
Gordon, what I was noticing was actually a big TCP packet (which was also
fragmented if needed), containing inside a certain number of different AMQP
"transfer" messages (each with the "transfer" performative, one for each
call to "pn_message_send()"). It was then followed by a series of different
TCP messages, each containing a single "disposition" from the receiver.
Thank you again for all your assistance!
Il giorno lun 10 feb 2020 alle ore 12:38 Gordon Sim <gs...@redhat.com> ha
scritto:
> On 10/02/2020 1:12 am, Chuck Rolke wrote:
> > ----- Original Message -----
> >> From: "Francesco Raviglione" <fr...@gmail.com>
> >> To: users@qpid.apache.org
> >> Sent: Saturday, February 8, 2020 5:36:39 AM
> >> Subject: Avoiding aggregation of "transfer" messages in Qpid Proton
> >>
> >> Hello,
> >> I'm sorry about the several questions I'm posting on the mailing list
> >> during these days, but, unfortunately, I'm struggling a bit in the
> usage of
> >> Qpid Proton (in particular, on the C version).
> >>
> >> One issue I noticed, and that I was not able to solve, is related to the
> >> fact that, when following and adapting the send.c example (available
> inside
> >> the C API reference:
> >>
> https://qpid.apache.org/releases/qpid-proton-0.30.0/proton/c/api/send_8c-example.html
> ),
> >> all the AMQP "transfer" messages are aggregated and if, for instance, I
> >> send 200 messages, one every 100 ms (using an additional Linux timer), I
> >> have to wait for 20 seconds before seeing them on the receiving side (an
> >> ActiveMQ broker, in my case), as all the 200 transfers are aggregated
> into
> >> a single, very big, AMQP message, which is sent only at the end of the
> >> process.
> >>
> >> This is something I would like to avoid, as it makes end-to-end message
> >> delivery (from producer, to broker, to consumer) too slow.
> >>
> >> Is there a way in which I can avoid this behaviour and make messages
> being
> >> sent immediately?
> >>
> >> Thank you very much in advance.
> >>
> >
> > While you are in the PN_LINK_FLOW loop the calls to send_message are
> only queueing
> > messages to proton. Proton can't start sending any of them until you
> return from the
> > flow handler.
> >
> > What you could do is send only so many (N) messages in the flow handler
> and then exit.
> > Later the PN_DELIVERY callback will be activated and you can send more
> messages there
> > to keep the N messages in flight.
>
> While it is certainly true that you need to return control to the event
> loop to allow any IO to happen, multiple calls to pn_message_send should
> never be coalesced into a single large message as that method should
> call pn_link_advance which creates a separate distinct message.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Avoiding aggregation of "transfer" messages in Qpid Proton
Posted by Gordon Sim <gs...@redhat.com>.
On 10/02/2020 1:12 am, Chuck Rolke wrote:
> ----- Original Message -----
>> From: "Francesco Raviglione" <fr...@gmail.com>
>> To: users@qpid.apache.org
>> Sent: Saturday, February 8, 2020 5:36:39 AM
>> Subject: Avoiding aggregation of "transfer" messages in Qpid Proton
>>
>> Hello,
>> I'm sorry about the several questions I'm posting on the mailing list
>> during these days, but, unfortunately, I'm struggling a bit in the usage of
>> Qpid Proton (in particular, on the C version).
>>
>> One issue I noticed, and that I was not able to solve, is related to the
>> fact that, when following and adapting the send.c example (available inside
>> the C API reference:
>> https://qpid.apache.org/releases/qpid-proton-0.30.0/proton/c/api/send_8c-example.html),
>> all the AMQP "transfer" messages are aggregated and if, for instance, I
>> send 200 messages, one every 100 ms (using an additional Linux timer), I
>> have to wait for 20 seconds before seeing them on the receiving side (an
>> ActiveMQ broker, in my case), as all the 200 transfers are aggregated into
>> a single, very big, AMQP message, which is sent only at the end of the
>> process.
>>
>> This is something I would like to avoid, as it makes end-to-end message
>> delivery (from producer, to broker, to consumer) too slow.
>>
>> Is there a way in which I can avoid this behaviour and make messages being
>> sent immediately?
>>
>> Thank you very much in advance.
>>
>
> While you are in the PN_LINK_FLOW loop the calls to send_message are only queueing
> messages to proton. Proton can't start sending any of them until you return from the
> flow handler.
>
> What you could do is send only so many (N) messages in the flow handler and then exit.
> Later the PN_DELIVERY callback will be activated and you can send more messages there
> to keep the N messages in flight.
While it is certainly true that you need to return control to the event
loop to allow any IO to happen, multiple calls to pn_message_send should
never be coalesced into a single large message as that method should
call pn_link_advance which creates a separate distinct message.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Avoiding aggregation of "transfer" messages in Qpid Proton
Posted by Chuck Rolke <cr...@redhat.com>.
----- Original Message -----
> From: "Francesco Raviglione" <fr...@gmail.com>
> To: users@qpid.apache.org
> Sent: Saturday, February 8, 2020 5:36:39 AM
> Subject: Avoiding aggregation of "transfer" messages in Qpid Proton
>
> Hello,
> I'm sorry about the several questions I'm posting on the mailing list
> during these days, but, unfortunately, I'm struggling a bit in the usage of
> Qpid Proton (in particular, on the C version).
>
> One issue I noticed, and that I was not able to solve, is related to the
> fact that, when following and adapting the send.c example (available inside
> the C API reference:
> https://qpid.apache.org/releases/qpid-proton-0.30.0/proton/c/api/send_8c-example.html),
> all the AMQP "transfer" messages are aggregated and if, for instance, I
> send 200 messages, one every 100 ms (using an additional Linux timer), I
> have to wait for 20 seconds before seeing them on the receiving side (an
> ActiveMQ broker, in my case), as all the 200 transfers are aggregated into
> a single, very big, AMQP message, which is sent only at the end of the
> process.
>
> This is something I would like to avoid, as it makes end-to-end message
> delivery (from producer, to broker, to consumer) too slow.
>
> Is there a way in which I can avoid this behaviour and make messages being
> sent immediately?
>
> Thank you very much in advance.
>
While you are in the PN_LINK_FLOW loop the calls to send_message are only queueing
messages to proton. Proton can't start sending any of them until you return from the
flow handler.
What you could do is send only so many (N) messages in the flow handler and then exit.
Later the PN_DELIVERY callback will be activated and you can send more messages there
to keep the N messages in flight.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org