You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Ken Giusti (Jira)" <ji...@apache.org> on 2020/04/04 15:54:00 UTC
[jira] [Created] (PROTON-2189) proactor C client has abnormally
long pauses during message send
Ken Giusti created PROTON-2189:
----------------------------------
Summary: proactor C client has abnormally long pauses during message send
Key: PROTON-2189
URL: https://issues.apache.org/jira/browse/PROTON-2189
Project: Qpid Proton
Issue Type: Bug
Components: proton-c
Affects Versions: proton-c-0.30.0
Environment: To compile the clients install qpid-proton-c-devel and simply compile:
gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c
To reproduce my test, build qdrouterd and run it in the background.
You need to have a consumer attached. There is a test receiver client in the qdrouterd build in <build-dir>/tests/test-receiver. This receiver is designed to handle streaming messages (by default sent to 'test-address')
Run the consumer in the background then run each clogger (default params are fine).
You should observe that clogger-reactor runs smoothly (use PN_TRACE_FRM=1 on qdrouterd as well).
You'll see clogger-reactor send the message header, then nothing for awhile, then send the entire message.
Use "-D" for debug output to see how many bytes have been written to pn_link_send()
Reporter: Ken Giusti
Attachments: clogger-proactor.c, clogger-reactor.c
I have a proactor-based C test client that has the ability to slowly send extremely large messages slowly. This is done by sending 'chunks' of body data with pauses in between.
This client was designed to test large streaming messages against qdrouterd.
The behavior of this client is unexpected - I would expect the message data to appear "on the wire" in bursts relatively quickly. In reality the data is buffered - in some cases over 1 GB is buffered - before it is written (as indicated by the lack @transfer frames dumped by the client AND the qdrouterd). In some cases it takes up to 30 seconds before the client's data starts being written to the client.
I've refactored the client to use reactor instead and the data flows as expected. There is minimal buffering and no abnormally long pauses.
The clients are attached.
It is quite likely the proactor client is incorrectly implemented, but I used the qdrouterd I/O loop as the model and cannot see what may be wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: [jira] [Created] (PROTON-2189) proactor C client has abnormally
long pauses during message send
Posted by Michael Goulish <mg...@redhat.com>.
>
>
> *Hey Mick, *
> *That did the trick - color me impressed.*
>
OK.
Ken, impressed:
[image: image.png]
On Mon, Apr 6, 2020 at 9:18 AM Ken Giusti <kg...@redhat.com> wrote:
> Hey Mick,
>
> That did the trick - color me impressed.
>
> If that's the intended behavior for sending streaming messages it wasn't
> clear to me. I'll update the JIRA with your recommendations, thanks!
>
>
> On Sat, Apr 4, 2020 at 12:32 PM Michael Goulish <mg...@redhat.com>
> wrote:
>
>> Ken --
>>
>> I have a proactor client in Mercury and it handles timeouts differently
>> than your code.
>> I don't remember now exactly why this was necessary, but I do vaguely
>> recall weird
>> behavior before I fixed it, at aconway's suggestion.
>>
>> I think that, in the PN_PROACTOR_TIMEOUT case in the event handler, in
>> my code
>> it was not correct to send at that time and then reset the timer. (As you
>> are doing.)
>>
>> Instead, in that case all I did was to wake the cnx:
>> pn_connection_wake ( context->connection );
>>
>> You can then send, and reset the timer only when you get an event of type:
>> PN_CONNECTION_WAKE:
>> send_message ( context );
>> pn_proactor_set_timeout ( context->proactor, context->throttle );
>>
>> In my case, I am sending entire messages -- but this is how I get it to
>> 'throttle' the send-rate, i.e. send 1 message per N msec.
>>
>>
>> I don't remember exactly what Bad Thing was happening to me before I
>> started doing it this way, but I have a Bad Feeling that it may have been
>> similar to what you describe.
>>
>>
>>
>>
>>
>> On Sat, Apr 4, 2020 at 12:00 PM Ken Giusti (Jira) <ji...@apache.org>
>> wrote:
>>
>>> Ken Giusti created PROTON-2189:
>>> ----------------------------------
>>>
>>> Summary: proactor C client has abnormally long pauses
>>> during message send
>>> Key: PROTON-2189
>>> URL: https://issues.apache.org/jira/browse/PROTON-2189
>>> Project: Qpid Proton
>>> Issue Type: Bug
>>> Components: proton-c
>>> Affects Versions: proton-c-0.30.0
>>> Environment: To compile the clients install qpid-proton-c-devel
>>> and simply compile:
>>>
>>> gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c
>>>
>>> To reproduce my test, build qdrouterd and run it in the background.
>>> You need to have a consumer attached. There is a test receiver client
>>> in the qdrouterd build in <build-dir>/tests/test-receiver. This receiver
>>> is designed to handle streaming messages (by default sent to 'test-address')
>>>
>>> Run the consumer in the background then run each clogger (default params
>>> are fine).
>>>
>>> You should observe that clogger-reactor runs smoothly (use
>>> PN_TRACE_FRM=1 on qdrouterd as well).
>>>
>>> You'll see clogger-reactor send the message header, then nothing for
>>> awhile, then send the entire message.
>>>
>>> Use "-D" for debug output to see how many bytes have been written to
>>> pn_link_send()
>>>
>>>
>>> Reporter: Ken Giusti
>>> Attachments: clogger-proactor.c, clogger-reactor.c
>>>
>>> I have a proactor-based C test client that has the ability to slowly
>>> send extremely large messages slowly. This is done by sending 'chunks' of
>>> body data with pauses in between.
>>>
>>> This client was designed to test large streaming messages against
>>> qdrouterd.
>>>
>>> The behavior of this client is unexpected - I would expect the message
>>> data to appear "on the wire" in bursts relatively quickly. In reality the
>>> data is buffered - in some cases over 1 GB is buffered - before it is
>>> written (as indicated by the lack @transfer frames dumped by the client AND
>>> the qdrouterd). In some cases it takes up to 30 seconds before the
>>> client's data starts being written to the client.
>>>
>>> I've refactored the client to use reactor instead and the data flows as
>>> expected. There is minimal buffering and no abnormally long pauses.
>>>
>>> The clients are attached.
>>>
>>> It is quite likely the proactor client is incorrectly implemented, but I
>>> used the qdrouterd I/O loop as the model and cannot see what may be wrong.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> This message was sent by Atlassian Jira
>>> (v8.3.4#803005)
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: dev-help@qpid.apache.org
>>>
>>>
>
> --
> -K
>
Re: [jira] [Created] (PROTON-2189) proactor C client has abnormally
long pauses during message send
Posted by Ken Giusti <kg...@redhat.com>.
Hey Mick,
That did the trick - color me impressed.
If that's the intended behavior for sending streaming messages it wasn't
clear to me. I'll update the JIRA with your recommendations, thanks!
On Sat, Apr 4, 2020 at 12:32 PM Michael Goulish <mg...@redhat.com> wrote:
> Ken --
>
> I have a proactor client in Mercury and it handles timeouts differently
> than your code.
> I don't remember now exactly why this was necessary, but I do vaguely
> recall weird
> behavior before I fixed it, at aconway's suggestion.
>
> I think that, in the PN_PROACTOR_TIMEOUT case in the event handler, in my
> code
> it was not correct to send at that time and then reset the timer. (As you
> are doing.)
>
> Instead, in that case all I did was to wake the cnx:
> pn_connection_wake ( context->connection );
>
> You can then send, and reset the timer only when you get an event of type:
> PN_CONNECTION_WAKE:
> send_message ( context );
> pn_proactor_set_timeout ( context->proactor, context->throttle );
>
> In my case, I am sending entire messages -- but this is how I get it to
> 'throttle' the send-rate, i.e. send 1 message per N msec.
>
>
> I don't remember exactly what Bad Thing was happening to me before I
> started doing it this way, but I have a Bad Feeling that it may have been
> similar to what you describe.
>
>
>
>
>
> On Sat, Apr 4, 2020 at 12:00 PM Ken Giusti (Jira) <ji...@apache.org> wrote:
>
>> Ken Giusti created PROTON-2189:
>> ----------------------------------
>>
>> Summary: proactor C client has abnormally long pauses during
>> message send
>> Key: PROTON-2189
>> URL: https://issues.apache.org/jira/browse/PROTON-2189
>> Project: Qpid Proton
>> Issue Type: Bug
>> Components: proton-c
>> Affects Versions: proton-c-0.30.0
>> Environment: To compile the clients install qpid-proton-c-devel
>> and simply compile:
>>
>> gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c
>>
>> To reproduce my test, build qdrouterd and run it in the background.
>> You need to have a consumer attached. There is a test receiver client in
>> the qdrouterd build in <build-dir>/tests/test-receiver. This receiver is
>> designed to handle streaming messages (by default sent to 'test-address')
>>
>> Run the consumer in the background then run each clogger (default params
>> are fine).
>>
>> You should observe that clogger-reactor runs smoothly (use PN_TRACE_FRM=1
>> on qdrouterd as well).
>>
>> You'll see clogger-reactor send the message header, then nothing for
>> awhile, then send the entire message.
>>
>> Use "-D" for debug output to see how many bytes have been written to
>> pn_link_send()
>>
>>
>> Reporter: Ken Giusti
>> Attachments: clogger-proactor.c, clogger-reactor.c
>>
>> I have a proactor-based C test client that has the ability to slowly send
>> extremely large messages slowly. This is done by sending 'chunks' of body
>> data with pauses in between.
>>
>> This client was designed to test large streaming messages against
>> qdrouterd.
>>
>> The behavior of this client is unexpected - I would expect the message
>> data to appear "on the wire" in bursts relatively quickly. In reality the
>> data is buffered - in some cases over 1 GB is buffered - before it is
>> written (as indicated by the lack @transfer frames dumped by the client AND
>> the qdrouterd). In some cases it takes up to 30 seconds before the
>> client's data starts being written to the client.
>>
>> I've refactored the client to use reactor instead and the data flows as
>> expected. There is minimal buffering and no abnormally long pauses.
>>
>> The clients are attached.
>>
>> It is quite likely the proactor client is incorrectly implemented, but I
>> used the qdrouterd I/O loop as the model and cannot see what may be wrong.
>>
>>
>>
>>
>>
>> --
>> This message was sent by Atlassian Jira
>> (v8.3.4#803005)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: dev-help@qpid.apache.org
>>
>>
--
-K
Re: [jira] [Created] (PROTON-2189) proactor C client has abnormally
long pauses during message send
Posted by Michael Goulish <mg...@redhat.com>.
Ken --
I have a proactor client in Mercury and it handles timeouts differently
than your code.
I don't remember now exactly why this was necessary, but I do vaguely
recall weird
behavior before I fixed it, at aconway's suggestion.
I think that, in the PN_PROACTOR_TIMEOUT case in the event handler, in my
code
it was not correct to send at that time and then reset the timer. (As you
are doing.)
Instead, in that case all I did was to wake the cnx:
pn_connection_wake ( context->connection );
You can then send, and reset the timer only when you get an event of type:
PN_CONNECTION_WAKE:
send_message ( context );
pn_proactor_set_timeout ( context->proactor, context->throttle );
In my case, I am sending entire messages -- but this is how I get it to
'throttle' the send-rate, i.e. send 1 message per N msec.
I don't remember exactly what Bad Thing was happening to me before I
started doing it this way, but I have a Bad Feeling that it may have been
similar to what you describe.
On Sat, Apr 4, 2020 at 12:00 PM Ken Giusti (Jira) <ji...@apache.org> wrote:
> Ken Giusti created PROTON-2189:
> ----------------------------------
>
> Summary: proactor C client has abnormally long pauses during
> message send
> Key: PROTON-2189
> URL: https://issues.apache.org/jira/browse/PROTON-2189
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
> Affects Versions: proton-c-0.30.0
> Environment: To compile the clients install qpid-proton-c-devel
> and simply compile:
>
> gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c
>
> To reproduce my test, build qdrouterd and run it in the background.
> You need to have a consumer attached. There is a test receiver client in
> the qdrouterd build in <build-dir>/tests/test-receiver. This receiver is
> designed to handle streaming messages (by default sent to 'test-address')
>
> Run the consumer in the background then run each clogger (default params
> are fine).
>
> You should observe that clogger-reactor runs smoothly (use PN_TRACE_FRM=1
> on qdrouterd as well).
>
> You'll see clogger-reactor send the message header, then nothing for
> awhile, then send the entire message.
>
> Use "-D" for debug output to see how many bytes have been written to
> pn_link_send()
>
>
> Reporter: Ken Giusti
> Attachments: clogger-proactor.c, clogger-reactor.c
>
> I have a proactor-based C test client that has the ability to slowly send
> extremely large messages slowly. This is done by sending 'chunks' of body
> data with pauses in between.
>
> This client was designed to test large streaming messages against
> qdrouterd.
>
> The behavior of this client is unexpected - I would expect the message
> data to appear "on the wire" in bursts relatively quickly. In reality the
> data is buffered - in some cases over 1 GB is buffered - before it is
> written (as indicated by the lack @transfer frames dumped by the client AND
> the qdrouterd). In some cases it takes up to 30 seconds before the
> client's data starts being written to the client.
>
> I've refactored the client to use reactor instead and the data flows as
> expected. There is minimal buffering and no abnormally long pauses.
>
> The clients are attached.
>
> It is quite likely the proactor client is incorrectly implemented, but I
> used the qdrouterd I/O loop as the model and cannot see what may be wrong.
>
>
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>