More musings about asynchronous MPMs Re: Event MPM

[Brian Pane <br...@apache.org>]

Paul Querna wrote:

> Paul Querna wrote:
>  > A thread per-connection that is currently being processed.
>
>>
>> Note that this is not the traditional 'event' model that people write 
>> huge papers about and thttpd raves about, but rather a hybrid that 
>> uses a Worker Thread todo the processing, and a single 'event' thread 
>> to handle places where we are waiting for IO. (Currently accept() and 
>> Keep Alive Requests).  Perhaps it needs a different name? (eworker?)
>>
>> A future direction to investigate would be to make all of the initial 
>> Header parsing be done Async, and then switch to a Worker thread to 
>> preform all the post_read hooks, handlers, and filters.  I believe 
>> this could be done without breaking many 3rd party modules. (SSL 
>> could be a bigger challenge here...)
>
>
> Some academics have played with this model of a event thread + worker 
> threads.  Best I can find is the Java 'SEDA: An Architecture for 
> Highly Concurrent Server Applications'. [1]


Yeah, SEDA's model of processing "stages"--basically a succession of 
thread pools through
which a request passes, each with a queue in front--looks like a 
promising way of mixing
event-based and non-event-based processing.

Another approach I've looked at is to use Schmidt's "Reactor" design 
pattern, in which a central
listener thread dispatches each received event to one of a pool of 
worker threads.  The worker
is then allowed to do arbitrarily complex processing (provided you have 
enough worker threads)
before deciding to tell the listener thread that it wants to wait for 
another event.  I did some
prototyping of a Leader/Followers variant of this: have a pool of worker 
threads that take turns
grabbing the next event from a queue of received events.  If there are 
no events left in the queue,
the next worker thread does a select/poll/etc (and any other idle 
workers block on a condition
variable).  This seemed to work reasonably well in a toy test app (it 
was *almost* capable of
implementing HTTP/0.9, but I never had time to try 1.0 or 1.1; oh, and 
it depended on the
java.nio classes to do the select efficiently, so I didn't even try to 
commit it to
server/mpm/experimental. :-)

On a more general topic...and at the risk of igniting a distracting 
debate...does anybody else
out there have an interest in doing an async httpd in Java?

There are a lot of reasons *not* to do so, mostly related to all the 
existing httpd-2.0 modules
that wouldn't work.  The things that seem appealing about trying a 
Java-based httpd, though,
are:

- The pool memory model at the core of httpd-1.x and -2.x isn't well 
suited to MPM
designs where multiple threads need to handle the same 
connection--possibly at the same
time, for example when a handler that's generating content needs to push 
output buckets
to an I/O completion thread to avoid having to block the (probably 
heavyweight) handler
thread or make it event-based.  Garbage collection on a per-object basis 
would be a lot
easier.
- Modern Java implementations seem to be doing smart things from a 
scalability perspective,
like using kqueue/epoll/etc.
- And (minor issue) it's a lot easier to refactor things in Java than in 
C, and I expect that
building a good async MPM that handles dynamic content and proxying 
effectively will
require a lot of iterations of design trial and error.

Brian

Re: More musings about asynchronous MPMs Re: Event MPM

["Wayne S. Frazee" <wf...@wynweb.net>]

I dont claim to be a java expert but I use apache extensively and figure i 
might as well toss in my $.02   

Frankly, there are two reasons I hate java.  

1) Its a resource hog.  Running on a JRE or Virtual Machine setup provides an 
extra layer of resource requirements that a similar C, C++, etc program would 
not really have.

2) Its relatively slow in comparison to many modern languages.  Mostly due to 
item 1 and the way that the JRE does cleanup, etc.

I recognize that there are also several positive effects that moving to a java 
based MPM would have.   For me, I have recently been subjected repeatedly to 
the mantra of one of my co-workers: "Use the best tool for the job at hand."  
Frankly, for MPMs whose main requirement is speed, efficiency, and a large 
level of scalability, I really dont think that Java is going to be a more 
useful tool in executing highly scalable MPM designs than C, C++, or some 
other languages would be.

Just my humble opinion, take it or leave it for what its worth.

-- 
--------------------
Wayne S. Frazee
"Any sufficiently developed bug is indistinguishable from a feature."On Monday

 25 October 2004 15:51, Brian Pane wrote:
> There are a lot of reasons *not* to do so, mostly related to all the
> existing httpd-2.0 modules
> that wouldn't work.  The things that seem appealing about trying a
> Java-based httpd, though,
> are:
>
> - The pool memory model at the core of httpd-1.x and -2.x isn't well
> suited to MPM
> designs where multiple threads need to handle the same
> connection--possibly at the same
> time, for example when a handler that's generating content needs to push
> output buckets
> to an I/O completion thread to avoid having to block the (probably
> heavyweight) handler
> thread or make it event-based.  Garbage collection on a per-object basis
> would be a lot
> easier.
> - Modern Java implementations seem to be doing smart things from a
> scalability perspective,
> like using kqueue/epoll/etc.
> - And (minor issue) it's a lot easier to refactor things in Java than in
> C, and I expect that
> building a good async MPM that handles dynamic content and proxying
> effectively will
> require a lot of iterations of design trial and error.
>
> Brian

Re: More musings about asynchronous MPMs Re: Event MPM

[Ian Holsman <li...@holsman.net>]

Bill Stoddard wrote:
> Brian Pane wrote:
> 
>> Paul Querna wrote:
>>
>>> Paul Querna wrote:
>>>  > A thread per-connection that is currently being processed.
>>>
>>
>> Yeah, SEDA's model of processing "stages"--basically a succession of 
>> thread pools through
>> which a request passes, each with a queue in front--looks like a 
>> promising way of mixing
>> event-based and non-event-based processing.
> 
> 
> Thread context switches will suck performance out of any SEDA 
> implementation.
> 

If I remember correctly SEDA got around this by passing the thread 
through if the queue was empty, and batching up requests so that instead
of processing one request in a stage you would handle X at a time.


> For sure.
> 
>>
>> Brian
>>
> 

Re: More musings about asynchronous MPMs Re: Event MPM

[Bill Stoddard <bi...@wstoddard.com>]

Brian Pane wrote:

> Paul Querna wrote:
> 
>> Paul Querna wrote:
>>  > A thread per-connection that is currently being processed.
>>
>>>
>>> Note that this is not the traditional 'event' model that people write 
>>> huge papers about and thttpd raves about, but rather a hybrid that 
>>> uses a Worker Thread todo the processing, and a single 'event' thread 
>>> to handle places where we are waiting for IO. (Currently accept() and 
>>> Keep Alive Requests).  Perhaps it needs a different name? (eworker?)
>>>
>>> A future direction to investigate would be to make all of the initial 
>>> Header parsing be done Async, and then switch to a Worker thread to 
>>> preform all the post_read hooks, handlers, and filters.  I believe 
>>> this could be done without breaking many 3rd party modules. (SSL 
>>> could be a bigger challenge here...)
>>
>>
>>
>> Some academics have played with this model of a event thread + worker 
>> threads.  Best I can find is the Java 'SEDA: An Architecture for 
>> Highly Concurrent Server Applications'. [1]
> 
> 
> 
> Yeah, SEDA's model of processing "stages"--basically a succession of 
> thread pools through
> which a request passes, each with a queue in front--looks like a 
> promising way of mixing
> event-based and non-event-based processing.

Thread context switches will suck performance out of any SEDA implementation.

> On a more general topic...and at the risk of igniting a distracting 
> debate...does anybody else
> out there have an interest in doing an async httpd in Java?

Sounds interesting to me though I doubt I would have much time to spend on it.

> 
> There are a lot of reasons *not* to do so, mostly related to all the 
> existing httpd-2.0 modules
> that wouldn't work.  The things that seem appealing about trying a 
> Java-based httpd, though,
> are:
> 
> - The pool memory model at the core of httpd-1.x and -2.x isn't well 
> suited to MPM
> designs where multiple threads need to handle the same 
> connection--possibly at the same
> time, for example when a handler that's generating content needs to push 
> output buckets
> to an I/O completion thread to avoid having to block the (probably 
> heavyweight) handler
> thread or make it event-based.  Garbage collection on a per-object basis 
> would be a lot
> easier.

In general you'll get better performance avoiding async/event driven i/o for i/o that can be completed quickly 
  . And the httpd pools work perfectly for handling GET requests for smallish static files. GC per object is 
more difficult to get right imo. That said, I do understand your point; the httpd pool model falls down in the 
cases where you do need to pass a request to different threads.

> - Modern Java implementations seem to be doing smart things from a 
> scalability perspective,
> like using kqueue/epoll/etc.
They are getting better.

> - And (minor issue) it's a lot easier to refactor things in Java than in 
> C,   and I expect that
> building a good async MPM that handles dynamic content and proxying 
> effectively will
> require a lot of iterations of design trial and error.

For sure.

> 
> Brian
>