You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Manoj Kasichainula <ma...@io.com> on 2000/07/23 11:02:02 UTC
Anywhere to go for a summmary of I/O filtering?
I'm attempting to get caught up on 2.0 development. Is there an
existing summary of the Greg vs. Ryan deathmatch in the archives
anywhere? There are 299 messages from the past month in my mailbox
mentioning buckets or filtering in the subject (mutt rocks!) and I'd
really like to reclaim the hours it will take to digest them.
My two cents on filtering/layering/buckets while I'm still completely
uninformed: it'd be nice if the 2.0 release wasn't delayed until 2001.
I have a bad feeling that the non-schedule is being pushed out and
out, though. I'm leaning towards just not doing any filtering work in
2.0; async and nonblocking I/O will change how things work anyway.
- Manoj
*still* trying to get DSL. :(
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Jul 23, 2000 at 01:26:43PM +0100, David Reid wrote:
> I have to admit that I agree that Manoj's sentiment on this one. While the
> I/O fltering may be nice, we haven't had an alpha in a very lng time now and
> a lot of things have been fixed/improved since the last one. We had a
> volunteer for rolling A5 so maybe we should at least do that next week to
> try and keep some momentum going?
I volunteered at one point to do the A5, saw that I needed to do PGP and
generate some keys, and punted :-)
The filtering stuff is not holding up A5 at all. Why should/would anybody
wait?
Apache is there; if somebody is inclined, then they should whip up a
release. Waiting on particular features before making a release isn't a good
way to get "release often". But that's just IMO... doesn't count for much
unless I go generate releases :-)
And the other side of the question?
If somebody wants a release, then they should be quiet and simply go and
make a release. :-)
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Jul 23, 2000 at 08:22:13AM -0700, rbb@covalent.net wrote:
> On Sun, 23 Jul 2000, David Reid wrote:
>
> > I have to admit that I agree that Manoj's sentiment on this one. While the
> > I/O fltering may be nice, we haven't had an alpha in a very lng time now and
> > a lot of things have been fixed/improved since the last one. We had a
> > volunteer for rolling A5 so maybe we should at least do that next week to
> > try and keep some momentum going?
>
> I would mention here that the filtering didn't hold up the alpha at
> all. What held it up, was nobody ever did it. I would prefer to do one
> of two things.
>
> 1) Get filtering in this week and have an alpha next week.
> 2) Remove all of the filtering stuff from CVS and have a beta this week.
Rather than removing the stuff from CVS, it is probably best to:
1) do the regular tagging of the repository
2) delete the new tag from src/lib/apr/buckets
Thus, when you pull the source (based on the tag), you won't get any of the
filtering stuff.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Anywhere to go for a summmary of I/O filtering?
Posted by David Reid <dr...@jetnet.co.uk>.
Sounds good.
+1
----- Original Message -----
From: "William A. Rowe, Jr." <wr...@lnd.com>
To: <ne...@apache.org>
Sent: Sunday, July 23, 2000 11:07 PM
Subject: RE: Anywhere to go for a summmary of I/O filtering?
> > From: David Reid [mailto:dreid@jetnet.co.uk]
> > Sent: Sunday, July 23, 2000 7:27 AM
> >
> > I have to admit that I agree that Manoj's sentiment on this one. While
the
> > I/O fltering may be nice, we haven't had an alpha in a very lng time now
and
> > a lot of things have been fixed/improved since the last one. We had a
> > volunteer for rolling A5 so maybe we should at least do that
> > next week to try and keep some momentum going?
>
> ++1 on this and 1.3.13 this week... it would be nice to announce
> (two messages, of course) that we are working to produce a new
> product, but while you are waiting here is progress in making the
> existing server just a bit more stable under a number of OS's.
>
> Bill
>
RE: Anywhere to go for a summmary of I/O filtering?
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: David Reid [mailto:dreid@jetnet.co.uk]
> Sent: Sunday, July 23, 2000 7:27 AM
>
> I have to admit that I agree that Manoj's sentiment on this one. While the
> I/O fltering may be nice, we haven't had an alpha in a very lng time now and
> a lot of things have been fixed/improved since the last one. We had a
> volunteer for rolling A5 so maybe we should at least do that
> next week to try and keep some momentum going?
++1 on this and 1.3.13 this week... it would be nice to announce
(two messages, of course) that we are working to produce a new
product, but while you are waiting here is progress in making the
existing server just a bit more stable under a number of OS's.
Bill
Re: Anywhere to go for a summmary of I/O filtering?
Posted by rb...@covalent.net.
On Sun, 23 Jul 2000, David Reid wrote:
> I have to admit that I agree that Manoj's sentiment on this one. While the
> I/O fltering may be nice, we haven't had an alpha in a very lng time now and
> a lot of things have been fixed/improved since the last one. We had a
> volunteer for rolling A5 so maybe we should at least do that next week to
> try and keep some momentum going?
I would mention here that the filtering didn't hold up the alpha at
all. What held it up, was nobody ever did it. I would prefer to do one
of two things.
1) Get filtering in this week and have an alpha next week.
2) Remove all of the filtering stuff from CVS and have a beta this week.
This code is most definately stable enough for a beta release, we just
need to do it finally.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Anywhere to go for a summmary of I/O filtering?
Posted by David Reid <dr...@jetnet.co.uk>.
I have to admit that I agree that Manoj's sentiment on this one. While the
I/O fltering may be nice, we haven't had an alpha in a very lng time now and
a lot of things have been fixed/improved since the last one. We had a
volunteer for rolling A5 so maybe we should at least do that next week to
try and keep some momentum going?
david
----- Original Message -----
From: "Manoj Kasichainula" <ma...@io.com>
To: "Apache Developers" <ne...@apache.org>
Sent: Sunday, July 23, 2000 10:02 AM
Subject: Anywhere to go for a summmary of I/O filtering?
> I'm attempting to get caught up on 2.0 development. Is there an
> existing summary of the Greg vs. Ryan deathmatch in the archives
> anywhere? There are 299 messages from the past month in my mailbox
> mentioning buckets or filtering in the subject (mutt rocks!) and I'd
> really like to reclaim the hours it will take to digest them.
>
> My two cents on filtering/layering/buckets while I'm still completely
> uninformed: it'd be nice if the 2.0 release wasn't delayed until 2001.
> I have a bad feeling that the non-schedule is being pushed out and
> out, though. I'm leaning towards just not doing any filtering work in
> 2.0; async and nonblocking I/O will change how things work anyway.
>
> - Manoj
> *still* trying to get DSL. :(
>
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Jeff Trawick <tr...@bellsouth.net>.
Manoj Kasichainula <ma...@io.com> writes:
> On Mon, Jul 24, 2000 at 12:17:09PM -0500, William A. Rowe, Jr. wrote:
> > Does anyone have the description here? And do unix threads allow
> > thread-based user/group impersonation?
>
> Standard pthreads don't. Linux clones do, I believe. I don't quite see
> the benefit, though. If threads can clobber each other's resources,
> giving them seperate uids doesn't seem to be very helpful. OS/390 does
> it though, so they must have figured this out.
I think it is a matter of that (clobber each other's resources) being
a problem that they didn't try to solve. It is "merely" used so that
access checking done by system services invoked on the thread uses the
identify of a secondary (e.g., logged-in) user. If your server
performs actions based on a logged-in user, once the user presents
their credentials you can create a thread with the appropriate
identity. When that thread invokes system services, the system
services use the thread-specific identity for access checking. Unlike
with separate address spaces, bugs in the server code can allow the
work done by one login to affect/break work done by another login;
that is a drawback as you point out. You probably shouldn't use this
mechanism for a quick hack where you declare success as soon as it
runs without segfaulting :) Thread-specific identity is very useful
on OS/390 because creating a new address space for a logged-in user is
much more expensive than creating a new thread.
In an application where the dispatchable unit
(process/thread/whatever) with the appropriate identity is not
created/destroyed frequently, the benefits aren't nearly so great.
Isn't this the Apache scenario?
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Manoj Kasichainula <ma...@io.com>.
On Mon, Jul 24, 2000 at 12:17:09PM -0500, William A. Rowe, Jr. wrote:
> Does anyone have the description here? And do unix threads allow
> thread-based user/group impersonation?
Standard pthreads don't. Linux clones do, I believe. I don't quite see
the benefit, though. If threads can clobber each other's resources,
giving them seperate uids doesn't seem to be very helpful. OS/390 does
it though, so they must have figured this out.
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Manoj Kasichainula <ma...@io.com>.
On Tue, Jul 25, 2000 at 02:07:15AM -0700, Manoj Kasichainula wrote:
> I'm actually thinking supporting async and non-blocking I/O won't be
> too hard.
OK, I realized that I left an important thought out of this
sentence. I think doing so while not breaking most of our platforms
won't add that much extra complexity.
Re: Anywhere to go for a summmary of I/O filtering?
Posted by rb...@covalent.net.
On Tue, 25 Jul 2000, Manoj Kasichainula wrote:
> On Mon, Jul 24, 2000 at 09:18:21AM -0700, rbb@covalent.net wrote:
> > Filtering does a lot more than just SSL or mod_include processing
> > CGIs. Filters allow us to simplify the entire server, remove some hacks,
> > and get a really cool proxy.
>
> Do we plan to introduce these benefits for 2.0?
We don't have to, if we get the API right for filtering. If the API is
right, those are features that can be added with 2.1 or 2.2, without
affecting the rest of the server. If the filtering API is wrong, those
features are impossible to add later because they require us to break
every module again.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Manoj Kasichainula <ma...@io.com>.
On Mon, Jul 24, 2000 at 09:18:21AM -0700, rbb@covalent.net wrote:
> Filtering does a lot more than just SSL or mod_include processing
> CGIs. Filters allow us to simplify the entire server, remove some hacks,
> and get a really cool proxy.
Do we plan to introduce these benefits for 2.0?
> This will be tough to do. How many modules does www.apache.org use that
> haven't been ported to 2.0? Remember, you have to look at more than just
> www.apache.org, you have to look at *.apache.org, because those are all
> just virtual hosts.
Once 2.0 mod_proxy is ready :), we could ProxyPass a 1.3 server for
all the modules that haven't been ported. We could also run all images
off of a 2.0 server running off of a seperate port or something.
> On Mon, 24 Jul 2000, Bill Stoddard wrote:
> > Apache 3.0 requirements....
> > filtering
> > async i/o
> > per process User/Group assignment
> > integrated mod_ssl (perhaps even in 2.0 if the legal climate is agreeable)
>
> Ever notice that we just keep pushing those same requirements down to the
> next release? These are the exact same things that we said were going to
> be in 2.0 a couple of years ago.
Well, a few have been knocked off the list. I don't think anyone would
advocate doing any of the tough parts above for 2.0.
> > async i/o and filtering will have a significant anount of design
> > interdependence, so doing one without the other is not wise, IMHO.
>
> While I agree that async I/O would be cool, it isn't portable
> currently. If we really want to write an async server, we will need to be
> very careful about how it is done. This will not be a fast
> process. Delaying filtering until we can write an async server is not a
> good idea IMNSHO. That would give us filtering in two or three years.
I'm actually thinking supporting async and non-blocking I/O won't be
too hard. I'm thinking a threaded server, each thread running a
two-layer event-driven state machine. The non-blocking case does both
layers of state machine, the async case does only the
read->write->read->write state machine layer, and the
thread-per-connection case just follows the states in order. This will
require seperate MPMs, and probably can't be APRized so easily, but I
think it can be done in Apache.
I'm being purposefully terse and obtuse because we're not doing async
yet; it's not time to slow down 2.0 development for this. I'll try to
write something up in a few months when 2.0 is stabilizing more.
Anyway, my only point when talking about the relationship between
async and filtering is that it would be nice if 2.0 went out the door.
Filtering is not in 2.0 yet. Lots of code and design will end up being
trashed for 3.0 anyway. Greg did raise a reasonable point about
getting the experience from filtering to use in future versions, but
that can be said for any new big, complicated feature.
One of the reasons that people want to throw features to 2.0 is that
releases take so long. This is a constant problem on the linux-kernel
list too, so maybe this is futile, but after 2.0, it'd be nice to try
to peg a short list of showstopper features that should go into the
next release. Other features wouldn't be rejected, but they wouldn't
become release showstoppers without a vote.
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Ben Laurie <be...@algroup.co.uk>.
"William A. Rowe, Jr." wrote:
> > integrated mod_ssl (perhaps even in 2.0 if the legal climate
> > is agreeable)
>
> There should be no reason, provided rollout is no earlier than Oct,
> that this won't happen. I believe the patent expires in Aug-Sept.
> The only critical aspect is export restrictions, and I don't know
> how the Apache group wants to address this. mod_ssl is an imported
> module that we can't reexport, so the question may fall on out-of-US
> developers that want to host/support the product and build distribs.
> I suppose we can then use locus.apache.org to distribute to the US
> only, and point out the host site for out of US interests (provided
> we are only 'borrowing' a foreign distribution... even linking could
> get us in trouble, I suppose.)
We are taking advise on this whole question (and several related ones)
at the moment. I hope to have something to say on the subject in weeks
rather than months.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html
Coming to ApacheCon Europe 2000? http://apachecon.com/
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Bill Stoddard <st...@raleigh.ibm.com>.
>
> > per process User/Group assignment
>
> Does anyone have the description here? And do unix threads allow
> thread-based user/group impersonation?
My understanding...
If I am hosting multiple virtual hosts and each wants to run PHP scripts, all the scripts
run under the same user/group id. It would be nice to be able to enable the scripts to run
under their own unique id and still run in-process (i.e., we do not want to suffer the
overhead of forking a suexec'ed CGI process).
Thread impersonation sounds interesting.... OS/390 supports thread impersonation I am told
(it is called something else though). I don't think this is a standard Unix thing though.
Bill
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Jul 24, 2000 at 10:44:21AM -0700, rbb@covalent.net wrote:
> On Mon, 24 Jul 2000, William A. Rowe, Jr. wrote:
>...
> > > filtering
> >
> > This could happen in 2.0, but I think we are still missing some things.
> > Has anyone detailed/documented the 'user' side (httpd.conf) of using
> > filters yet? I'm curious, since I don't want the entire server to pipe
> > through the same superset of filters, and would expect we want to 'focus'
> > filtering to specific classes (e.g. file extention) or for specific
> > paths (e.g. /webdocs/joesweb.com/).
>
> None of the current patches have all requests going through the same
> filters. Basically, all of the current patches allow people to specify in
> the http.conf file a set of filters that a file or directory should go
> through, and they all allow the server to be configured to do filtering
> based on Mime type. For example:
>
> <Directory cgi-bin>
> SetFilters php perl include
> </Directory>
>
> Is how the http.conf file is used, or if the mime type is set at the end
> of this CGI, then the server can just figure it out.
>
> None of these directives are written yet, because they are relatively easy
> to write, and we wanted to get filtering in before we add directives to
> use it.
To clarify:
The underlying filtering design is what Ryan, Roy and I have been
concentrating on. Once we have the capability, then we can concern ourselves
with the details of getting them inserted into the output filter chain.
If we tried to do it all at once, then we would have an unreviewable mass of
code.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Jeff Trawick <tr...@bellsouth.net>.
Tony Finch <do...@dotat.at> writes:
> Bill Stoddard <st...@raleigh.ibm.com> wrote:
>
> Um, do you mean event-driven IO (select/poll/kevent) rather than async
> IO (aio_*,SIGIO)?
>
> I agree that offloading keepalive connections to a holding thread
> would be a win, and the bonus is it's something that can be done with
> minimal change outside the mpm.
Another relatively easy task would be to manage lingering close in a
similar manner. I don't know how to assess the value of such a
change. I would guess that not tying up any extra processes/threads
in lingering close would be of some benefit, though not nearly as much
as not tying up extra processes/threads waiting for the next request
(keepalive). Once the infrastructure for select/read-for-keepalive is
in the MPM, lingering close is easy to handle with multiplexed I/O.
Drastically reducing the number of processes/threads in keepalive (and
lingering close to a much lesser extent) seems to have a very high
benefit-to-cost-ratio. It shouldn't require any *serious* modifications
to Apache other than in the MPM. Apache would of course need to be
able to return back to the MPM in keepalive state so that the MPM can
manage it.
The MPM work is somewhat interesting...
One thread within one process needs to poll for new connections in
addition to readability on connections that process owns and which are
in keepalive. One thread in every other process needs to pool for
readability on connections that process owns and which are in
keepalive. (And don't forget periodic time out processing for
connections in keepalive.) This is a little different than today
where only one thread among all MPM threads/processes is in poll. The
existing thread mutex decides which thread within a given process
proceeds towards poll. A new problem to solve is how to arbitrate
which process waits on new connections in addition to work on
connections owned by that process. One way to arbitrate this is to to
get the interprocess mutex in blocking mode if the process has no
other work and to attempt to get the interprocess mutex in
non-blocking mode if the process has other work to wait for. (Greg
Ames and I discussed this enough a month or so ago to get some feeling
for the types of problems we would run into if starting from
mpmt_pthread or dexter. Any stupid ideas above are definitely mine,
not Greg's.)
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Bill Stoddard <st...@raleigh.ibm.com>.
>
> Um, do you mean event-driven IO (select/poll/kevent) rather than async
> IO (aio_*,SIGIO)?
>
Actually, I mean async i/o on Windows. This is the problem pointed out by Ryan, that every
platform implements async i/o differently. Doing non-blocking event driven i/o is probably
more efficient on Unix.
Bill
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Tony Finch <do...@dotat.at>.
Bill Stoddard <st...@raleigh.ibm.com> wrote:
>
>I agree that async i/o isn't a 2.0 candidate. But it is not
>necessarily a huge change (keep reading Ryan, before you go off ...
>:-). It really depends on how far we want to take the async model.
>IIS enables ISAPI modules to access async i/o. I'm not sure we need
>to go that far. Just doing async reads (async i) would free
>processes/threads blocked on keep-alive sessions which is a major
>scalability problem we have now and this would be relatively easy to
>accomplish. And YES, before you say it, I completely agree that
>making Apache a fully async server and exposing the async APIs to
>modules is a HUGE change. Just food for thought. I am not advocating
>one approach or another. Maybe I'll start a fork and see how far I
>can take it.
Um, do you mean event-driven IO (select/poll/kevent) rather than async
IO (aio_*,SIGIO)?
I agree that offloading keepalive connections to a holding thread
would be a win, and the bonus is it's something that can be done with
minimal change outside the mpm.
Tony.
--
f.a.n.finch fanf@covalent.net dot@dotat.at
367 it's not just awful...it's god-awful
Re: Anywhere to go for a summmary of I/O filtering?
Posted by rb...@covalent.net.
> How do you handle a server running, say, 100,000 virtual hosts? Rasmus, what is the
> requirement here?
For each new user id, there must be a child process. If the configuration
doesn't work, then we fail to start with an error message. If we have
100,000 virtual hosts with different user ids, then we need to have
100,000 child processes. If we have 100,000 virtual hosts, and only 10
distinct user ids, then we only need 10 child processes.
> As I mentioned earlier, it is already possible to make SSL a pure module with the existing
> iol (this one reason I implemented ap_pop_iol).
Except that you had to patch around the sendfile stuff in the core. :-)
> I agree that async i/o isn't a 2.0 candidate. But it is not necessarily a huge change
> (keep reading Ryan, before you go off ... :-). It really depends on how far we want to
> take the async model. IIS enables ISAPI modules to access async i/o. I'm not sure we need
> to go that far. Just doing async reads (async i) would free processes/threads blocked on
> keep-alive sessions which is a major scalability problem we have now and this would be
> relatively easy to accomplish. And YES, before you say it, I completely agree that making
> Apache a fully async server and exposing the async APIs to modules is a HUGE change. Just
> food for thought. I am not advocating one approach or another. Maybe I'll start a fork and
> see how far I can take it.
It's bigger than that. I have looked at some different async models, and
they just aren't portable. Windows, Linux, and OS/390 all do async
differently. Because of that, we pretty much have to simulate async with
non-blocking reads/writes on most platforms.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Bill Stoddard <st...@raleigh.ibm.com>.
>
> > > per process User/Group assignment
> >
> > Does anyone have the description here? And do unix threads allow
> > thread-based user/group impersonation? It seems like we may have
> > another MPM split here, win32 does thread/user impersonation, and
> > other platforms may require proc/user impersonation.
>
> I should have an new MPM to implement this some time in the next few
> weeks. I have a lot of projects that I am working on right now, for
> Apache and for Covalent and myself. Basically, all we need to do, is
> re-write Dexter to have a specified user per process.
> When a child
> process gets a request that is destined for another virtual host, it looks
> that up in a shared table, and forwards off the request and socket. The
> threads themselves do not change ownership at all, the server just
> redirects the requests to the right process.
>
How do you handle a server running, say, 100,000 virtual hosts? Rasmus, what is the
requirement here?
>
> I have no great feelings about an ssl module. I personally think that if
> 2.0 is written correctly, then SSL can be a pure module, and we can avoid
> the legal issues all together, because anybody will be able to download a
> single file and get SSL.
As I mentioned earlier, it is already possible to make SSL a pure module with the existing
iol (this one reason I implemented ap_pop_iol).
> > Wouldn't Async be the more logical starting point, minimally in terms
> > of the specs, so we can build filters to suite sync, async or both?
>
> NO! Async is a huge change. It is not going to go into 2.0, because it
> will require a complete re-write. I really think we need to learn a bit
> about filtering in 2.0, and apply that knowledge to a 3.0 async server.
I agree that async i/o isn't a 2.0 candidate. But it is not necessarily a huge change
(keep reading Ryan, before you go off ... :-). It really depends on how far we want to
take the async model. IIS enables ISAPI modules to access async i/o. I'm not sure we need
to go that far. Just doing async reads (async i) would free processes/threads blocked on
keep-alive sessions which is a major scalability problem we have now and this would be
relatively easy to accomplish. And YES, before you say it, I completely agree that making
Apache a fully async server and exposing the async APIs to modules is a HUGE change. Just
food for thought. I am not advocating one approach or another. Maybe I'll start a fork and
see how far I can take it.
Bill
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Tony Finch <do...@dotat.at>.
Greg Stein <gs...@lyra.org> wrote:
>
>It is a complete red herring to focus on how async will impact the
>filter design. Async will impact how modules are written, how filters
>are written, and how the overall server is designed.
Actually, I don't think that is necessarily true. I've been thinking
along the lines of creating a clearer response abstraction that
bundles up the metadata (essentially the headers) and the content (a
bucket brigade). In the complicated case the content will be consumed
as it is constructed, but for event-driven IO we're more interested in
the simple case.
One observation is that sometimes we want some data from the previous
request to hang around after the handler has finished in order to put
it into the same packet as the beginning of the response to the next
pipelined connection. For event-driven IO *most* of the response is
held back so that it can be passed to the select() thread.
So: The response handler's job is to put together a bucket brigade
which it does mostly by adding buckets to the end of an existing
brigade and passing them down to the filters which will mangle them in
interesting ways, eventually pulling buckets off the front of the
brigade and sending the contents to the network. At the end of request
handling there will be a brigade containing just the tail-end of the
response. The handler can return the brigade to the core which can
decide whether to flush it now or keep it (hanging off the connection
rec?) so that it can be sent with the start of the responce to the
next pipelined request.
In the simple case the bottom-level filter code can notice that the
brigade is just FILE, EOS and decide that it'll do nothing (except
perhaps mark it as being simple) and let the whole thing be returned
to the core by the handler. The core can then spot the simple brigade
and pass it aside to the select() thread.
Thus you can get event-driven IO in many cases without having to write
event-driven response handlers. The crucial thing is to have a
reasonably powerful abstraction for responses, which does *not* mean
it is really complicated.
(... Further in the future you can allow response handlers to return
a brigade to the core that doesn't have an EOS marker, and the core
will know that it should call the handler again to get more data. This
gives you a simple migration towards fully event-driven response
generation. ...)
Tony.
--
f.a.n.finch fanf@covalent.net dot@dotat.at
478 fruit-flavored floor wax
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Greg Stein <gs...@lyra.org>.
It is a complete red herring to focus on how async will impact the filter
design. Async will impact how modules are written, how filters are written,
and how the overall server is designed. "we can't add filters because async
will change them" is wrong-headed... using that approach, we would say
"the module/Apache API is wrong and we should not implement any new modules
because async will change them."
A possible, future async model is definitely not a reason to be concerned
about today's filtering model.
Cheers,
-g
On Mon, Jul 24, 2000 at 01:16:15PM -0500, William A. Rowe, Jr. wrote:
>...
> That's fine. Forewarn and forearm module authors that filtering is subject
> to overhaul when async is implemented in 3.0 ...
>
> What 3.0 major feature did you have in mind when you suggested that async
> is a feature for 4.0?
>
> I have only two concerns:
>
> 1) async may become a major consideration for authors/administrators when
> it comes to choosing their web server in 2002, if it's not already.
>
> 2) async may blow our filters design out of the water if we don't bear it
> some consideration (not _implementation_, only _design_) as we create
> the filter api. I don't want to give the authors in 2002 the choice
> between rewriting their module for Apache and rewriting for another
> platform. If we bear it in mind, we can avoid big rewrites (unless
> they want to fully exploit async, which is a different issue.)
>
> If we design async, and then stub it for filters, we can keep moving and
> no code will be harmed in the production. When we are ready to expose it,
> then we can revamp the server. As it is, I'm reviewing mod_isapi to fully
> implement the API, but emulate async for authors so they aren't coding
> twice, and for admins who aren't interested in arguing with the coder.
>
> Bill
--
Greg Stein, http://www.lyra.org/
Re: ap_connect - part 2
Posted by Gregory Nicholls <gn...@level8.com>.
OK I'll give it a whirl. For the record I'm CVS ignorant and barely Unix literate (old MF guy I'm
afraid). I'll take a stab at it and we can all have a good laugh at the results.
G.
(who's now off to see if he can get CVS working ....)
Jeff Trawick wrote:
> Gregory Nicholls <gn...@level8.com> writes:
>
> > While you're in there could you check something else for me plz ?? I was attempting a
> > bind to port 0 and then trying to use get_local_port to retrieve the port that is
> > (theoretically) assigned. Now this (again Win32 only) didn't work. It returned port 0. The
> > code for this function is:
> >
> > *port = ntohs(sock->local_addr->sin_port);
> > return APR_SUCCESS;
> >
> > in sockaddr.c.
> >
> > Now I don't know if I'm interpreting the function's purpose correctly however I think it
> > needs a getsockname() call somewhere along the line . . .
> >
> > G.
>
> Yes sir, that is correct. A getsockname() call could be added after
> bind() (and connect() for that matter), or
> getsockname-deferral/avoidance logic (like in the
> lib/apr/network_io/unix code) could be implemented in the Win32 path.
> Either method is simple to implement. Do you want to handle this and
> post the patch? It looks like you're in a position to test it. I
> don't have time in the short term.
>
> I'd recommend getting your anonymous cvs access working first so that
> you can use the output of "cvs diff -u filename ..." as the patch to
> post to the list when you get it working.
>
> --
> Jeff Trawick | trawick@ibm.net | PGP public key at web site:
> http://www.geocities.com/SiliconValley/Park/9289/
> Born in Roswell... married an alien...
Re: ap_connect - part 2
Posted by Jeff Trawick <tr...@bellsouth.net>.
Gregory Nicholls <gn...@level8.com> writes:
> While you're in there could you check something else for me plz ?? I was attempting a
> bind to port 0 and then trying to use get_local_port to retrieve the port that is
> (theoretically) assigned. Now this (again Win32 only) didn't work. It returned port 0. The
> code for this function is:
>
> *port = ntohs(sock->local_addr->sin_port);
> return APR_SUCCESS;
>
> in sockaddr.c.
>
> Now I don't know if I'm interpreting the function's purpose correctly however I think it
> needs a getsockname() call somewhere along the line . . .
>
> G.
Yes sir, that is correct. A getsockname() call could be added after
bind() (and connect() for that matter), or
getsockname-deferral/avoidance logic (like in the
lib/apr/network_io/unix code) could be implemented in the Win32 path.
Either method is simple to implement. Do you want to handle this and
post the patch? It looks like you're in a position to test it. I
don't have time in the short term.
I'd recommend getting your anonymous cvs access working first so that
you can use the output of "cvs diff -u filename ..." as the patch to
post to the list when you get it working.
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
ap_connect - part 2
Posted by Gregory Nicholls <gn...@level8.com>.
Jeff Trawick wrote:
> Ahh... Silly me... I was looking in the Unix code. Sorry! I'll put
> in the trivial change to the win32 version so that it works like the Unix code.
>
While you're in there could you check something else for me plz ?? I was attempting a
bind to port 0 and then trying to use get_local_port to retrieve the port that is
(theoretically) assigned. Now this (again Win32 only) didn't work. It returned port 0. The
code for this function is:
*port = ntohs(sock->local_addr->sin_port);
return APR_SUCCESS;
in sockaddr.c.
Now I don't know if I'm interpreting the function's purpose correctly however I think it
needs a getsockname() call somewhere along the line . . .
G.
Re: ap_connect ???
Posted by Jeff Trawick <tr...@bellsouth.net>.
Gregory Nicholls <gn...@level8.com> writes:
> Here's a cutout of the connect code in the version I'm using. It blew up trying to
> check *hostname.
> Obviously since it's NULL this isn't good. This was on line 195 of sockets.c in the
> Win32 version.
Ahh... Silly me... I was looking in the Unix code. Sorry! I'll put
in the trivial change to the win32 version so that it works like the Unix code.
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
Re: ap_connect ???
Posted by Gregory Nicholls <gn...@level8.com>.
Hi,
I'd created the socket, set the remote port and remoteaddr (127.0.0.1) and then
issued the connect a la ap_connect(sock,NULL).
Here's a cutout of the connect code in the version I'm using. It blew up trying to
check *hostname.
Obviously since it's NULL this isn't good. This was on line 195 of sockets.c in the
Win32 version.
I'll keep playing with it until I find the correct incantation.
G.
--------------------------- cut here -----------------------------------------------
if ((sock->sock == INVALID_SOCKET) || (!sock->local_addr)) {
return APR_ENOTSOCK;
}
if (*hostname >= '0' && *hostname <= '9' &&
strspn(hostname, "0123456789.") == strlen(hostname)) {
sock->remote_addr->sin_addr.s_addr = inet_addr(hostname);
}
------------------------------------------------------------------------------------
Jeff Trawick wrote:
> Gregory Nicholls <gn...@level8.com> writes:
>
> > Hiya,
> > The comments in the header for ap_connect seem to permit a hostname parameter
> > of NULL. I tried this and received an abend. Looking at the source for ap_connect,
> > there doesn't seem to be any checks for NULL so I'm wondering if the comments are
> > wrong, the code is wrong, my version is outdated or I'm just thick. I'm using
> > 2.0a4 base with no mods (CVS-challenged I'm afraid).
> > Thanks,
> > G.
>
> What operations had you done on the ap_socket_t prior to ap_connect()?
> One what line did you abend? I don't see a friendly way in the API to
> set the target address other than with ap_connect(), but I don't see
> why ap_connect() should abend either. The current ap_connect() does
> avoid some processing if the 2nd parm is NULL; it assumes that the remote
> address in the ap_socket_t has already been set.
>
> About CVS... Follow the instructions at
> http://dev.apache.org/anoncvs.txt. It is really easy and quite cool.
>
> --
> Jeff Trawick | trawick@ibm.net | PGP public key at web site:
> http://www.geocities.com/SiliconValley/Park/9289/
> Born in Roswell... married an alien...
Re: ap_connect ???
Posted by Jeff Trawick <tr...@bellsouth.net>.
Gregory Nicholls <gn...@level8.com> writes:
> Hiya,
> The comments in the header for ap_connect seem to permit a hostname parameter
> of NULL. I tried this and received an abend. Looking at the source for ap_connect,
> there doesn't seem to be any checks for NULL so I'm wondering if the comments are
> wrong, the code is wrong, my version is outdated or I'm just thick. I'm using
> 2.0a4 base with no mods (CVS-challenged I'm afraid).
> Thanks,
> G.
What operations had you done on the ap_socket_t prior to ap_connect()?
One what line did you abend? I don't see a friendly way in the API to
set the target address other than with ap_connect(), but I don't see
why ap_connect() should abend either. The current ap_connect() does
avoid some processing if the 2nd parm is NULL; it assumes that the remote
address in the ap_socket_t has already been set.
About CVS... Follow the instructions at
http://dev.apache.org/anoncvs.txt. It is really easy and quite cool.
--
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
Re: ap_connect ???
Posted by rb...@covalent.net.
It is likely that the comments are for a previous version. Unfortunately,
the docs have not been keeping up with the code recently. I am trying to
stay on top of the docs, but I'm not succeeding. The best I can suggest
is if you see a problem with the docs, make a patch to fix them, or send a
note here (like you did :-) and we'll try to fix it ASAP.
Ryan
On Tue, 25 Jul 2000, Gregory Nicholls wrote:
> Hiya,
> The comments in the header for ap_connect seem to permit a hostname parameter
> of NULL. I tried this and received an abend. Looking at the source for ap_connect,
> there doesn't seem to be any checks for NULL so I'm wondering if the comments are
> wrong, the code is wrong, my version is outdated or I'm just thick. I'm using
> 2.0a4 base with no mods (CVS-challenged I'm afraid).
> Thanks,
> G.
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
ap_connect ???
Posted by Gregory Nicholls <gn...@level8.com>.
Hiya,
The comments in the header for ap_connect seem to permit a hostname parameter
of NULL. I tried this and received an abend. Looking at the source for ap_connect,
there doesn't seem to be any checks for NULL so I'm wondering if the comments are
wrong, the code is wrong, my version is outdated or I'm just thick. I'm using
2.0a4 base with no mods (CVS-challenged I'm afraid).
Thanks,
G.
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Manoj Kasichainula <ma...@io.com>.
On Tue, Jul 25, 2000 at 03:25:56AM -0700, Greg Stein wrote:
> IMO, we should try to move away from the Directory section. It is filesystem
> specific and (slightly) hinders the move to repository independence.
+1
It can be useful to apply access control to specific files, but this
should be handled by configuration in the repository layer
(eventually).
> That said: using a vhost as the granularity means that any requests arriving
> at a vhost have a single user/group. If you make it per-Loc, then you're
> going to need to start passing request handling between processes/threads.
> Possible? Dunno. But certainly more difficult, I'd think.
As Tony mentioned, name-based vhosts require this as well. This is why
I shudder slightly at the thought of an MPM that does this.
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Tony Finch <do...@dotat.at>.
Greg Stein <gs...@lyra.org> wrote:
>
>IMO, we should try to move away from the Directory section. It is filesystem
>specific and (slightly) hinders the move to repository independence.
+1
>That said: using a vhost as the granularity means that any requests arriving
>at a vhost have a single user/group. If you make it per-Loc, then you're
>going to need to start passing request handling between processes/threads.
>Possible? Dunno. But certainly more difficult, I'd think.
You have to pass requests from one thread to another for vhosts as
well, unless you restrict yourself to IP-based vhosting.
Tony.
--
f.a.n.finch fanf@covalent.net dot@dotat.at
285 sucker punch in the gut bucket
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Jul 25, 2000 at 02:18:35AM -0700, Manoj Kasichainula wrote:
> On Mon, Jul 24, 2000 at 01:16:15PM -0500, William A. Rowe, Jr. wrote:
> > And are we certain that the virtualhost is sufficient? I'm picturing, as
> > a commerce administrator, that I have a group of administration scripts
> > in a common share-bin folder that all should run as user isp_common,
> > while their folders can all use their own id's. The individual web admins
> > have no permissions to the common cgi (they can't hack them), while they
> > still have cgi-bin to play their own games.
>
> I can't think of a reason that the model Ryan proposed can't be
> applied to Location blocks. Directory and File might be harder, but
> Location is plenty.
IMO, we should try to move away from the Directory section. It is filesystem
specific and (slightly) hinders the move to repository independence.
That said: using a vhost as the granularity means that any requests arriving
at a vhost have a single user/group. If you make it per-Loc, then you're
going to need to start passing request handling between processes/threads.
Possible? Dunno. But certainly more difficult, I'd think.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Manoj Kasichainula <ma...@io.com>.
On Mon, Jul 24, 2000 at 01:16:15PM -0500, William A. Rowe, Jr. wrote:
> And are we certain that the virtualhost is sufficient? I'm picturing, as
> a commerce administrator, that I have a group of administration scripts
> in a common share-bin folder that all should run as user isp_common,
> while their folders can all use their own id's. The individual web admins
> have no permissions to the common cgi (they can't hack them), while they
> still have cgi-bin to play their own games.
I can't think of a reason that the model Ryan proposed can't be
applied to Location blocks. Directory and File might be harder, but
Location is plenty.
RE: Anywhere to go for a summmary of I/O filtering?
Posted by rb...@covalent.net.
> Then we simply have VHostAsUser joe and VHostAsGroup webauthors
> properties going? What about passwords, I assume the issue is
> mute under unix? And I'm assuming we aren't discussing running
> as the client/user, only the author.
We don't care about passwords. The superuser always has access to change
to any other user on Unix. I can't parse the last sentance, sorry. :-)
> And are we certain that the virtualhost is sufficient? I'm picturing, as
> a commerce administrator, that I have a group of administration scripts
> in a common share-bin folder that all should run as user isp_common,
> while their folders can all use their own id's. The individual web admins
> have no permissions to the common cgi (they can't hack them), while they
> still have cgi-bin to play their own games.
I said VirtualHost, but it is really on a per-child basis. I can't see
any other way to do this, and with tweaking the user and groups properly,
I think we can accomplish what you are discussing above.
> That's fine. Forewarn and forearm module authors that filtering is subject
> to overhaul when async is implemented in 3.0 ...
I don't think any of us mind breaking all sorts of modules with major
releases, we just don't want to break them with minor releases.
> What 3.0 major feature did you have in mind when you suggested that
> async is a feature for 4.0?
I always expect new things to crop up. Currently, I think 3.0 will be
async, but hey, you never know. :-)
> 1) async may become a major consideration for authors/administrators when
> it comes to choosing their web server in 2002, if it's not already.
>
> 2) async may blow our filters design out of the water if we don't bear it
> some consideration (not _implementation_, only _design_) as we create
> the filter api. I don't want to give the authors in 2002 the choice
> between rewriting their module for Apache and rewriting for another
> platform. If we bear it in mind, we can avoid big rewrites (unless
> they want to fully exploit async, which is a different issue.)
>
> If we design async, and then stub it for filters, we can keep moving and
> no code will be harmed in the production. When we are ready to expose it,
> then we can revamp the server. As it is, I'm reviewing mod_isapi to fully
> implement the API, but emulate async for authors so they aren't coding
> twice, and for admins who aren't interested in arguing with the coder.
Cool.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
RE: Anywhere to go for a summmary of I/O filtering?
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: rbb@covalent.net [mailto:rbb@covalent.net]
> Sent: Monday, July 24, 2000 12:44 PM
>
> > > per process User/Group assignment
> >
> > Does anyone have the description here? And do unix threads allow
> > thread-based user/group impersonation? It seems like we may have
> > another MPM split here, win32 does thread/user impersonation, and
> > other platforms may require proc/user impersonation.
>
> I should have an new MPM to implement this some time in the next few
> weeks. I have a lot of projects that I am working on right now, for
> Apache and for Covalent and myself. Basically, all we need to do, is
> re-write Dexter to have a specified user per process. When a child
> process gets a request that is destined for another virtual host, it looks
> that up in a shared table, and forwards off the request and socket. The
> threads themselves do not change ownership at all, the server just
> redirects the requests to the right process.
Then we simply have VHostAsUser joe and VHostAsGroup webauthors
properties going? What about passwords, I assume the issue is
mute under unix? And I'm assuming we aren't discussing running
as the client/user, only the author.
And are we certain that the virtualhost is sufficient? I'm picturing, as
a commerce administrator, that I have a group of administration scripts
in a common share-bin folder that all should run as user isp_common,
while their folders can all use their own id's. The individual web admins
have no permissions to the common cgi (they can't hack them), while they
still have cgi-bin to play their own games.
> I have no clue how to implement this sort of thing on
> Windows, so I am not even going to try.
Don't worry about it :) I'm picturing toggling the ImpersonateLoggedOnUser
for the given thread, and RevertToSelf at the end of the request. Logged
on users and tokens can be cached, so the hit should be nominal. If we
structure it right, it should be fluid between Unix and Win32, although
implemented quite differently.
> NO! Async is a huge change. It is not going to go into 2.0, because it
> will require a complete re-write. I really think we need to learn a bit
> about filtering in 2.0, and apply that knowledge to a 3.0 async server.
That's fine. Forewarn and forearm module authors that filtering is subject
to overhaul when async is implemented in 3.0 ...
What 3.0 major feature did you have in mind when you suggested that async
is a feature for 4.0?
I have only two concerns:
1) async may become a major consideration for authors/administrators when
it comes to choosing their web server in 2002, if it's not already.
2) async may blow our filters design out of the water if we don't bear it
some consideration (not _implementation_, only _design_) as we create
the filter api. I don't want to give the authors in 2002 the choice
between rewriting their module for Apache and rewriting for another
platform. If we bear it in mind, we can avoid big rewrites (unless
they want to fully exploit async, which is a different issue.)
If we design async, and then stub it for filters, we can keep moving and
no code will be harmed in the production. When we are ready to expose it,
then we can revamp the server. As it is, I'm reviewing mod_isapi to fully
implement the API, but emulate async for authors so they aren't coding
twice, and for admins who aren't interested in arguing with the coder.
Bill
RE: Anywhere to go for a summmary of I/O filtering?
Posted by rb...@covalent.net.
On Mon, 24 Jul 2000, William A. Rowe, Jr. wrote:
> > From: Bill Stoddard [mailto:stoddard@raleigh.ibm.com]
> > Sent: Monday, July 24, 2000 11:03 AM
> >
> > Apache 3.0 requirements....
>
> I'm thinking these are each 2.x generation issues...
Some are 2.x and some are 3.0.
>
> > filtering
>
> This could happen in 2.0, but I think we are still missing some things.
> Has anyone detailed/documented the 'user' side (httpd.conf) of using
> filters yet? I'm curious, since I don't want the entire server to pipe
> through the same superset of filters, and would expect we want to 'focus'
> filtering to specific classes (e.g. file extention) or for specific
> paths (e.g. /webdocs/joesweb.com/).
None of the current patches have all requests going through the same
filters. Basically, all of the current patches allow people to specify in
the http.conf file a set of filters that a file or directory should go
through, and they all allow the server to be configured to do filtering
based on Mime type. For example:
<Directory cgi-bin>
SetFilters php perl include
</Directory>
Is how the http.conf file is used, or if the mime type is set at the end
of this CGI, then the server can just figure it out.
None of these directives are written yet, because they are relatively easy
to write, and we wanted to get filtering in before we add directives to
use it.
> > async i/o
>
> Absolutely, this is a fundemental redesign. While it won't be hard,
> we don't have all of APR or filtering down yet. Althought I wonder
> if filtering wouldn't make more sense to implement <within> the async
> model, since we will probably have to scrap and reimplement good parts
> of it in the transition from sync to async processing.
Async is NOT a 2.x issue. It requires a complete redesign of most of the
server. I do not think anybody wants to re-write the server again for
2.0. Async is most definately a 3.0 or even 4.0 issue.
> > per process User/Group assignment
>
> Does anyone have the description here? And do unix threads allow
> thread-based user/group impersonation? It seems like we may have
> another MPM split here, win32 does thread/user impersonation, and
> other platforms may require proc/user impersonation.
I should have an new MPM to implement this some time in the next few
weeks. I have a lot of projects that I am working on right now, for
Apache and for Covalent and myself. Basically, all we need to do, is
re-write Dexter to have a specified user per process. When a child
process gets a request that is destined for another virtual host, it looks
that up in a shared table, and forwards off the request and socket. The
threads themselves do not change ownership at all, the server just
redirects the requests to the right process.
I have no clue how to implement this sort of thing on Windows, so I am not
even going to try.
>
> > integrated mod_ssl (perhaps even in 2.0 if the legal climate
> > is agreeable)
>
> There should be no reason, provided rollout is no earlier than Oct,
> that this won't happen. I believe the patent expires in Aug-Sept.
> The only critical aspect is export restrictions, and I don't know
> how the Apache group wants to address this. mod_ssl is an imported
> module that we can't reexport, so the question may fall on out-of-US
> developers that want to host/support the product and build distribs.
> I suppose we can then use locus.apache.org to distribute to the US
> only, and point out the host site for out of US interests (provided
> we are only 'borrowing' a foreign distribution... even linking could
> get us in trouble, I suppose.)
I have no great feelings about an ssl module. I personally think that if
2.0 is written correctly, then SSL can be a pure module, and we can avoid
the legal issues all together, because anybody will be able to download a
single file and get SSL. I would prefer to operate on that model until
the legal questions are all answered. I say that, only because I am a
programmer, and I would rather be hacking code than trying to figure out
if I can export it or not. If we just point to a place that people can
download a module, I think we are fine.
> > async i/o and filtering will have a significant anount of design
> > interdependence, so doing one without the other is not wise, IMHO.
>
> Wouldn't Async be the more logical starting point, minimally in terms
> of the specs, so we can build filters to suite sync, async or both?
NO! Async is a huge change. It is not going to go into 2.0, because it
will require a complete re-write. I really think we need to learn a bit
about filtering in 2.0, and apply that knowledge to a 3.0 async server.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
RE: Anywhere to go for a summmary of I/O filtering?
Posted by "William A. Rowe, Jr." <wr...@lnd.com>.
> From: Bill Stoddard [mailto:stoddard@raleigh.ibm.com]
> Sent: Monday, July 24, 2000 11:03 AM
>
> Apache 3.0 requirements....
I'm thinking these are each 2.x generation issues...
> filtering
This could happen in 2.0, but I think we are still missing some things.
Has anyone detailed/documented the 'user' side (httpd.conf) of using
filters yet? I'm curious, since I don't want the entire server to pipe
through the same superset of filters, and would expect we want to 'focus'
filtering to specific classes (e.g. file extention) or for specific
paths (e.g. /webdocs/joesweb.com/).
> async i/o
Absolutely, this is a fundemental redesign. While it won't be hard,
we don't have all of APR or filtering down yet. Althought I wonder
if filtering wouldn't make more sense to implement <within> the async
model, since we will probably have to scrap and reimplement good parts
of it in the transition from sync to async processing.
> per process User/Group assignment
Does anyone have the description here? And do unix threads allow
thread-based user/group impersonation? It seems like we may have
another MPM split here, win32 does thread/user impersonation, and
other platforms may require proc/user impersonation.
> integrated mod_ssl (perhaps even in 2.0 if the legal climate
> is agreeable)
There should be no reason, provided rollout is no earlier than Oct,
that this won't happen. I believe the patent expires in Aug-Sept.
The only critical aspect is export restrictions, and I don't know
how the Apache group wants to address this. mod_ssl is an imported
module that we can't reexport, so the question may fall on out-of-US
developers that want to host/support the product and build distribs.
I suppose we can then use locus.apache.org to distribute to the US
only, and point out the host site for out of US interests (provided
we are only 'borrowing' a foreign distribution... even linking could
get us in trouble, I suppose.)
> async i/o and filtering will have a significant anount of design
> interdependence, so doing one without the other is not wise, IMHO.
Wouldn't Async be the more logical starting point, minimally in terms
of the specs, so we can build filters to suite sync, async or both?
Re: Anywhere to go for a summmary of I/O filtering?
Posted by rb...@covalent.net.
On Mon, 24 Jul 2000, Bill Stoddard wrote:
> > My two cents on filtering/layering/buckets while I'm still completely
> > uninformed: it'd be nice if the 2.0 release wasn't delayed until 2001.
> > I have a bad feeling that the non-schedule is being pushed out and
> > out, though. I'm leaning towards just not doing any filtering work in
> > 2.0; async and nonblocking I/O will change how things work anyway.
> >
>
> I agree with Manoj... lets stabilize Apache 2.0 and release sans filtering. I don't care
> much for the filtering patches I've seen so far and I am still not sure we even have the
> requirements nailed down.
Why don't you like the patches submitted so far?
Filtering does a lot more than just SSL or mod_include processing
CGIs. Filters allow us to simplify the entire server, remove some hacks,
and get a really cool proxy.
> Some thoughts...
> Multithreading will be a big win on most platforms except Linux (and FreeBSD?). We need
> mod_dav, mod_perl, mod_php and mod_ssl working before the release. Lets try running Apache
> 2.0 on www.apache.org, today! IMO, it will not be nearly so stable as some of us would
This will be tough to do. How many modules does www.apache.org use that
haven't been ported to 2.0? Remember, you have to look at more than just
www.apache.org, you have to look at *.apache.org, because those are all
just virtual hosts.
> Apache 3.0 requirements....
> filtering
> async i/o
> per process User/Group assignment
> integrated mod_ssl (perhaps even in 2.0 if the legal climate is agreeable)
Ever notice that we just keep pushing those same requirements down to the
next release? These are the exact same things that we said were going to
be in 2.0 a couple of years ago.
> async i/o and filtering will have a significant anount of design interdependence, so doing
> one without the other is not wise, IMHO.
While I agree that async I/O would be cool, it isn't portable
currently. If we really want to write an async server, we will need to be
very careful about how it is done. This will not be a fast
process. Delaying filtering until we can write an async server is not a
good idea IMNSHO. That would give us filtering in two or three years.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Anywhere to go for a summmary of I/O filtering?
Posted by Bill Stoddard <st...@raleigh.ibm.com>.
> My two cents on filtering/layering/buckets while I'm still completely
> uninformed: it'd be nice if the 2.0 release wasn't delayed until 2001.
> I have a bad feeling that the non-schedule is being pushed out and
> out, though. I'm leaning towards just not doing any filtering work in
> 2.0; async and nonblocking I/O will change how things work anyway.
>
I agree with Manoj... lets stabilize Apache 2.0 and release sans filtering. I don't care
much for the filtering patches I've seen so far and I am still not sure we even have the
requirements nailed down.
Some thoughts...
Multithreading will be a big win on most platforms except Linux (and FreeBSD?). We need
mod_dav, mod_perl, mod_php and mod_ssl working before the release. Lets try running Apache
2.0 on www.apache.org, today! IMO, it will not be nearly so stable as some of us would
like to think. However, we gotta start somewhere so let's light the fuse and get ready to
pick up pieces. Getting a couple of revs of Apache 2.0 out before embarking on Apache 3.0
will help us wring the bugs out of APR.
Apache 3.0 requirements....
filtering
async i/o
per process User/Group assignment
integrated mod_ssl (perhaps even in 2.0 if the legal climate is agreeable)
async i/o and filtering will have a significant anount of design interdependence, so doing
one without the other is not wise, IMHO.
Bill
Re: Anywhere to go for a summmary of I/O filtering?
Posted by rb...@covalent.net.
On Sun, 23 Jul 2000, Manoj Kasichainula wrote:
> I'm attempting to get caught up on 2.0 development. Is there an
> existing summary of the Greg vs. Ryan deathmatch in the archives
> anywhere? There are 299 messages from the past month in my mailbox
The best place to look is src/lib/apr/buckets. That has basically docs on
what everybody wants to see in filters, and the two patches. Greg's is
much better documented than mine is. I really believe my buckets are
pretty self explanatory, and the filter registration methods seem to just
bother everybody. :-)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------