Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Jim Jagielski <ji...@jaguNET.com>]

I was wondering: how much of what we want to do would be easier
if we decided to make serf and/or mod_lua and/or libmill as
dependencies for 2.6/3.0? We could leverage Lua (or libmill)
as libs implementing some level of coroutines and serf for
a more organic bucket impl.

Of course, this could also result in the issue we currently
have w/ APR in which APR itself lags behind what httpd itself
needs and we end up add stuff to httpd anyway...

Hmmmm...

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Graham Leggett <mi...@sharp.fm>]

On 05 Oct 2015, at 5:13 PM, Eric Covener <co...@gmail.com> wrote:

> Is it that common for so much of the server to be tied up in write
> completion, or just a very big problem for some systems? Most of my
> experience is the opposite  problem --  slow (java!) backends rather
> than clients not keeping up with the firehose.

It is not common enough for the server to be in write completion. Until now mod_ssl couldn’t enter write completion because it had no way to yield without writing the entire response successfully to the network. As soon as you added SSL the server got synchronous.

The key problem to solve was the “mod_cache problem”. How does a backend write into mod_cache at full speed and then go away as soon as possible, while a slow client eventually reads the response? We can now (soon, when mod_cache is taught to be async like mod_ssl is) do that, and that is very cool.

The slow clients that become a killer are those on a dodgy mobile phone connection that ends up using a slot for far longer than they should. If that slot is tied through via the proxy to a slow java app, that sucks even more.

>  It seems like
> extending that to handlers (and implicitly, request filters) is more
> the  "can't be done" part.

I’ve been working from right to left, from the network filter backwards to the handlers, and the solution I’ve explored is “if it can’t be done in a handler, make it possible to be done in a filter instead”.

The idea is that the handler sets up the filter stack, and then sends down some data to kick things off (or no data like mod_cache does). At that point the handler exits and we’re now in write completion. The filter now (soon) has the power to suspend the connection if we’re waiting for someone else (like a connection from the proxy, a timer callback, whatever), and also has the power to switch the “sense” of a connection from a write to a read. So the output filter might switch the sense to read and then leave, and then when the read event is triggered and we’re kicked, try do a read on the input filter stack.

I suspect that a handler can do the same thing right now - switch the sense to read and then suspend itself. On the next read, the handler will be woken up, and off we go, ready to read once. Rinse repeat until we’re done reading.

Regards,
Graham
—

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Graham Leggett <mi...@sharp.fm>]

On 05 Oct 2015, at 5:13 PM, Eric Covener <co...@gmail.com> wrote:

> It seems like
> extending that to handlers (and implicitly, request filters) is more
> the  "can't be done" part.

Having had a chance to look at the handlers in more detail, I suspect the problem is that async support wasn’t extended far enough. You can return SUSPENDED from a handler, but only on the main request. If you’re in an internal redirect or a subrequest, SUSPENDED is treated as an error.

Request filters are already asynchronous, they aren’t a problem. A caller can only take advantage of the async nature of the input filters if the caller itself is asynchronous.

It should be possible to use an async input filter from an async output filter simply by changing the sense of the connection and letting the core deal with it. This is next for me, I plan to fix any bugs that stops this being possible.

Regards,
Graham
—

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Eric Covener <co...@gmail.com>]

On Mon, Oct 5, 2015 at 10:57 AM, Graham Leggett <mi...@sharp.fm> wrote:
>
> What exactly constituted a moon on a stick? I wanted all of the following:
>
> - I wanted to make httpd asynchronous, and decisively allow httpd to join the list of servers that can support hundreds of thousands of concurrent requests across the board, not just as a special case for specific content (files).
>
> - I didn’t want to wait. This means I wanted whatever change I made to be backport-able to httpd v2.4. I don’t care to wait 5 years for httpd v2.6 to finally to appear as the default version in stable OS distros.
>
> - I wanted to work with what was already there. That meant making the existing ap_filter API and the existing buckets work with backwards compatible changes to httpd and no changes to apr/apr-util.
>
> - I wanted a mechanism that was not brittle. This is probably the hardest part of async programming, how do you prevent something spinning or hanging? When the new mechanism is used wrong the thread just becomes synchronous until the data is written and eventually works anyway, which is existing behaviour and not the end of the world.
>
> I am aware that some people have said that it couldn’t be done in httpd’s design. I figured let me test that hypothesis by trying anyway, the worst that could happen was that they were right but the server could at least be improved as best it could. I believe however that there is a good chance the moon on a stick has been achieved.

Is it that common for so much of the server to be tied up in write
completion, or just a very big problem for some systems? Most of my
experience is the opposite  problem --  slow (java!) backends rather
than clients not keeping up with the firehose.  It seems like
extending that to handlers (and implicitly, request filters) is more
the  "can't be done" part.

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Jim Jagielski <ji...@jaguNET.com>]

Thx (somewhat behind) :)

> On Oct 5, 2015, at 12:24 PM, Graham Leggett <mi...@sharp.fm> wrote:
> 
> On 05 Oct 2015, at 6:10 PM, Jim Jagielski <ji...@jaguNET.com> wrote:
> 
>> Is there any reason to not fold this into trunk and start playing
>> around?
> 
> I committed it yesterday:
> 
> http://svn.apache.org/r1706669
> http://svn.apache.org/r1706670
> 
> Regards,
> Graham
> —
> 

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Graham Leggett <mi...@sharp.fm>]

On 05 Oct 2015, at 6:10 PM, Jim Jagielski <ji...@jaguNET.com> wrote:

> Is there any reason to not fold this into trunk and start playing
> around?

I committed it yesterday:

http://svn.apache.org/r1706669
http://svn.apache.org/r1706670

Regards,
Graham
—

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Jim Jagielski <ji...@jaguNET.com>]

Is there any reason to not fold this into trunk and start playing
around?

> On Oct 5, 2015, at 11:52 AM, Jim Jagielski <ji...@jaguNET.com> wrote:
> 
> Achieved!!
> 
> Once again: this is some super cool mojo!!!
> 
>> On Oct 5, 2015, at 10:57 AM, Graham Leggett <mi...@sharp.fm> wrote:
>> 
>> On 05 Oct 2015, at 3:41 PM, Jim Jagielski <ji...@jaguNET.com> wrote:
>> 
>>> Of course, this could also result in the issue we currently
>>> have w/ APR in which APR itself lags behind what httpd itself
>>> needs and we end up add stuff to httpd anyway...
>>> 
>>> Hmmmm...
>> 
>> This was my approach:
>> 
>> I wanted the moon on a stick.
>> 
>> What exactly constituted a moon on a stick? I wanted all of the following:
>> 
>> - I wanted to make httpd asynchronous, and decisively allow httpd to join the list of servers that can support hundreds of thousands of concurrent requests across the board, not just as a special case for specific content (files).
>> 
>> - I didn’t want to wait. This means I wanted whatever change I made to be backport-able to httpd v2.4. I don’t care to wait 5 years for httpd v2.6 to finally to appear as the default version in stable OS distros.
>> 
>> - I wanted to work with what was already there. That meant making the existing ap_filter API and the existing buckets work with backwards compatible changes to httpd and no changes to apr/apr-util.
>> 
>> - I wanted a mechanism that was not brittle. This is probably the hardest part of async programming, how do you prevent something spinning or hanging? When the new mechanism is used wrong the thread just becomes synchronous until the data is written and eventually works anyway, which is existing behaviour and not the end of the world.
>> 
>> I am aware that some people have said that it couldn’t be done in httpd’s design. I figured let me test that hypothesis by trying anyway, the worst that could happen was that they were right but the server could at least be improved as best it could. I believe however that there is a good chance the moon on a stick has been achieved.
>> 
>> Regards,
>> Graham
>> —
>> 
> 

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Jim Jagielski <ji...@jaguNET.com>]

Achieved!!

Once again: this is some super cool mojo!!!

> On Oct 5, 2015, at 10:57 AM, Graham Leggett <mi...@sharp.fm> wrote:
> 
> On 05 Oct 2015, at 3:41 PM, Jim Jagielski <ji...@jaguNET.com> wrote:
> 
>> Of course, this could also result in the issue we currently
>> have w/ APR in which APR itself lags behind what httpd itself
>> needs and we end up add stuff to httpd anyway...
>> 
>> Hmmmm...
> 
> This was my approach:
> 
> I wanted the moon on a stick.
> 
> What exactly constituted a moon on a stick? I wanted all of the following:
> 
> - I wanted to make httpd asynchronous, and decisively allow httpd to join the list of servers that can support hundreds of thousands of concurrent requests across the board, not just as a special case for specific content (files).
> 
> - I didn’t want to wait. This means I wanted whatever change I made to be backport-able to httpd v2.4. I don’t care to wait 5 years for httpd v2.6 to finally to appear as the default version in stable OS distros.
> 
> - I wanted to work with what was already there. That meant making the existing ap_filter API and the existing buckets work with backwards compatible changes to httpd and no changes to apr/apr-util.
> 
> - I wanted a mechanism that was not brittle. This is probably the hardest part of async programming, how do you prevent something spinning or hanging? When the new mechanism is used wrong the thread just becomes synchronous until the data is written and eventually works anyway, which is existing behaviour and not the end of the world.
> 
> I am aware that some people have said that it couldn’t be done in httpd’s design. I figured let me test that hypothesis by trying anyway, the worst that could happen was that they were right but the server could at least be improved as best it could. I believe however that there is a good chance the moon on a stick has been achieved.
> 
> Regards,
> Graham
> —
> 

Re: Serf and mod_lua and libmill (Was: Re: [Patch] Async write completion for the full connection filter stack)

[Graham Leggett <mi...@sharp.fm>]

On 05 Oct 2015, at 3:41 PM, Jim Jagielski <ji...@jaguNET.com> wrote:

> Of course, this could also result in the issue we currently
> have w/ APR in which APR itself lags behind what httpd itself
> needs and we end up add stuff to httpd anyway...
> 
> Hmmmm...

This was my approach:

I wanted the moon on a stick.

What exactly constituted a moon on a stick? I wanted all of the following:

- I wanted to make httpd asynchronous, and decisively allow httpd to join the list of servers that can support hundreds of thousands of concurrent requests across the board, not just as a special case for specific content (files).

- I didn’t want to wait. This means I wanted whatever change I made to be backport-able to httpd v2.4. I don’t care to wait 5 years for httpd v2.6 to finally to appear as the default version in stable OS distros.

- I wanted to work with what was already there. That meant making the existing ap_filter API and the existing buckets work with backwards compatible changes to httpd and no changes to apr/apr-util.

- I wanted a mechanism that was not brittle. This is probably the hardest part of async programming, how do you prevent something spinning or hanging? When the new mechanism is used wrong the thread just becomes synchronous until the data is written and eventually works anyway, which is existing behaviour and not the end of the world.

I am aware that some people have said that it couldn’t be done in httpd’s design. I figured let me test that hypothesis by trying anyway, the worst that could happen was that they were right but the server could at least be improved as best it could. I believe however that there is a good chance the moon on a stick has been achieved.

Regards,
Graham
—