Re: event MPM: async write completion?

[Greg Ames <gr...@remulak.net>]

Brian Pane wrote:
> Looking at the pattern of calls to ap_core_output_filter() in the event 
> MPM, it occurred to me that it may be straightforward to hand off the  
> writing of the request to an async completion thread in a lot of useful 
 > real-world  cases.
> 
> In function check_pipeline_flush() in http_request.c, a flush bucket  is 
> passed to the output filter stack if there's no pipelined request ready 
> to be read (or if keep-alives aren't enabled for the connection).

good observation.  these cases are common and simpler to deal with in event than 
the pipelined input data present case.

> When this occurs, two things are true:
>   * One or more complete responses--and no partial responses--have been
>     sent to the output filter stack for this connection.
>   * If these responses have not been sent to the client, they will now 
>      be sent in their entirety before the httpd attempts to do any 
>      further reads from this connection.
> 
> Instead of sending a flush bucket to the output filter stack in this  
> scenario, though, perhaps we could send a new metadata bucket that denotes 
> "end of output until everything currently in the pipeline has been  
> written."  

<nitpick>  what's wrong with using the flush bucket?  I think you're suggesting 
a new improved way to process it. </nitpick>

> Upon receiving this bucket, core_output_filter would register the  connection 
> with a pollset to watch for writability.
> 
> A write completion thread would poll in an endless loop, writing data  from
> each set-aside brigade whenever its corresponding connection became  
> writable.
> Upon sending the last bucket of a set-aside brigade to the network,  the 
> completion
> thread would put the connection in either CONN_STATE_LINGER or
> CONN_STATE_CHECK_REQUEST_LINE_READABLE state (depending on
> whether keepalives were enabled) and re-register the connection with the
> event MPM's main pollset to wait for readability or timeout.
> 
> The pollset used to wait for writability could be the same pollset that's 
> currently used to watch for readability and timeouts.  Similarly, the write  
> completion thread could be combined with the current listener thread.

I would combine the pollsets/threads right away to save later work.  I think 
Paul Q. did a fair amount of work to merge the separate event thread from my + 
Bill Stoddard's event patch into the listener thread.

> I'm eager to hear some feedback on this idea:
> * Will it work?  Or am I overlooking some design flaw?

it should work as long as everything important that happens after the 
check_pipeline_flush call still gets done somehow.  a quick glance at the code 
shows that this is mainly logging, scoreboard + event state changes, and getting 
back into the request loop if appropriate.

> * Will it help?  I.e., will it make a noticeable improvement in the
>   connections-to-threads ratio?

it depends on the workload, of course.  it will help when worker threads are 
frequently tied up serving big static files.

here is a server-status from the mini-specweb99 workload I use frequently - 
http://people.apache.org/~gregames/event-server-status.html

looking at the non-idle, non-server-status threads, there were 15 in "W" state 
serving largish static files (>50K) which are less than 15% of the total 
requests.  there were 14 threads in "C" state (lingering close) which IMO is 
lower hanging fruit because we have unwound back to the MPM so there is less 
state info hanging around and we don't have to worry about logging, reading the 
next request, etc.

> * What about the access logger?  If the writing of responses were
>   completed asynchronously, a request could be logged before its
>   response was sent--or, worse yet, before the sending of its
>   response failed due to, say, the client closing the connection early.
>   One solution that comes to mind is to have the write completion
>   thread invoke the access logger--perhaps triggered by a metadata
>   bucket in the output brigade.

or something like CONN_STATE_FLUSH_OUTPUT followed by CONN_STATE_LOG followed by 
CONN_STATE_CHECK_REQUEST_LINE_READABLE and back into the keepalive request loop.

Greg

Re: event MPM: async write completion?

[Greg Ames <gr...@remulak.net>]

Greg Ames wrote:
> Brian Pane wrote:

>> I'm eager to hear some feedback on this idea:
>> * Will it work?  Or am I overlooking some design flaw?

> it should work as long as everything important that happens after the 
> check_pipeline_flush call still gets done somehow.  a quick glance at 
> the code shows that this is mainly logging, scoreboard + event state 
> changes, and getting back into the request loop if appropriate.

and of course the rest of the output needs to be written.  perhaps the piece of 
core_output_filter that controls the writing could be broken out into a separate 
function and called based on a new CONN_STATE.  it wouldn't hurt 
core_output_filter to slim down anyway.

Greg

Re: event MPM: async write completion?

[Paul Querna <ch...@force-elite.com>]

Greg Ames wrote:
> Brian Pane wrote:
> 
>> Looking at the pattern of calls to ap_core_output_filter() in the
>> event MPM, it occurred to me that it may be straightforward to hand
>> off the  writing of the request to an async completion thread in a lot
>> of useful 
> 
>> real-world  cases.
> 
>>
>> In function check_pipeline_flush() in http_request.c, a flush bucket 
>> is passed to the output filter stack if there's no pipelined request
>> ready to be read (or if keep-alives aren't enabled for the connection).
> 
> 
> good observation.  these cases are common and simpler to deal with in
> event than the pipelined input data present case.
> 
>> When this occurs, two things are true:
>>   * One or more complete responses--and no partial responses--have been
>>     sent to the output filter stack for this connection.
>>   * If these responses have not been sent to the client, they will now
>>      be sent in their entirety before the httpd attempts to do any
>>      further reads from this connection.
>>
>> Instead of sending a flush bucket to the output filter stack in this 
>> scenario, though, perhaps we could send a new metadata bucket that
>> denotes "end of output until everything currently in the pipeline has
>> been  written."  
> 
> 
> <nitpick>  what's wrong with using the flush bucket?  I think you're
> suggesting a new improved way to process it. </nitpick>
> 
>> Upon receiving this bucket, core_output_filter would register the 
>> connection with a pollset to watch for writability.
>>
>> A write completion thread would poll in an endless loop, writing data 
>> from
>> each set-aside brigade whenever its corresponding connection became 
>> writable.
>> Upon sending the last bucket of a set-aside brigade to the network, 
>> the completion
>> thread would put the connection in either CONN_STATE_LINGER or
>> CONN_STATE_CHECK_REQUEST_LINE_READABLE state (depending on
>> whether keepalives were enabled) and re-register the connection with the
>> event MPM's main pollset to wait for readability or timeout.
>>
>> The pollset used to wait for writability could be the same pollset
>> that's currently used to watch for readability and timeouts. 
>> Similarly, the write  completion thread could be combined with the
>> current listener thread.
> 
> 
> I would combine the pollsets/threads right away to save later work.  I
> think Paul Q. did a fair amount of work to merge the separate event
> thread from my + Bill Stoddard's event patch into the listener thread.

FWIW, trunk has been using a combined listener/readability thread for a
long time now...

I have been thinking about adding the writ ability stuff, maybe ill have
a chance to play with it this weekend on some more airplanes.


-Paul