You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Ryan Bloom <rb...@raleigh.ibm.com> on 2000/01/04 22:06:07 UTC
Small direction change for APR's file I/O routines.
The more I get involved in making the current changes for ap_stat and
ap_getfileinfo, the less I like them. I want to change them completely,
but I need the groups reaction first.
Currently, we store basically what accounts to all of the fields from the
unix stat structure in the ap_file_t instance. This makes those fields
unavailable to programs that want access to individual fields. For
example, a programmer cannot get to the uid field of a file directly, they
have to call a function to get that data.
I am thinking of redefining the stat structure within APR, and just having
ap_stat and ap_get_fileinfo fill out a public structure. There is really
no reason for this information to be in an incomplete type. Any field
that does not have a good implementation on a platform, could just have an
error condition. For example, if the uid field didn't map well to
Windows, we could just leave that field with a -999 in it, which would be
invalid.
I think this cleans up the code a lot and reduces the number of functions
that we would need to call. I have had a couple of people suggest this in
the past, but I have been hesitant. Seeing how we are using stat info, I
think this is the correct direction to take. This will also make a stat
cache easier to add if APR decides it is necessary later on. :-)
If I don't hear from anybody, I am going to start this work relatively
soon.
Ryan
_______________________________________________________________________
Ryan Bloom rbb@raleigh.ibm.com
4205 S Miami Blvd
RTP, NC 27709
Come to the first official Apache Software Foundation
Conference! <http://ApacheCon.Com/>
Re: Small direction change for APR's file I/O routines.
Posted by Manoj Kasichainula <ma...@raleigh.ibm.com>.
On Tue, Jan 04, 2000 at 04:06:07PM -0500, Ryan Bloom wrote:
> I am thinking of redefining the stat structure within APR, and just having
> ap_stat and ap_get_fileinfo fill out a public structure.
+1. I think it would be cleaner to have macros accessing the values,
and then just *officially* document the macros. But no big deal.
You are talking about completely decoupling the stat structure from
ap_file_t, right?
--
Manoj Kasichainula - manojk@raleigh.ibm.com
IBM, Apache Development
Re: Small direction change for APR's file I/O routines.
Posted by Sascha Schumann <sa...@schumann.cx>.
On Tue, Jan 04, 2000 at 04:06:07PM -0500, Ryan Bloom wrote:
>
> The more I get involved in making the current changes for ap_stat and
> ap_getfileinfo, the less I like them. I want to change them completely,
> but I need the groups reaction first.
[...]
Sounds good, even if it violates the usual information hiding
rules. +1
--
Regards,
Sascha Schumann
Consultant
Re: Small direction change for APR's file I/O routines.
Posted by David Reid <ab...@dial.pipex.com>.
Sounds like a good idea to me. Will make life easier...
d.
----- Original Message -----
From: "Ryan Bloom" <rb...@raleigh.ibm.com>
To: <ne...@apache.org>
Sent: Tuesday, January 04, 2000 9:06 PM
Subject: Small direction change for APR's file I/O routines.
>
> The more I get involved in making the current changes for ap_stat and
> ap_getfileinfo, the less I like them. I want to change them completely,
> but I need the groups reaction first.
>
> Currently, we store basically what accounts to all of the fields from the
> unix stat structure in the ap_file_t instance. This makes those fields
> unavailable to programs that want access to individual fields. For
> example, a programmer cannot get to the uid field of a file directly, they
> have to call a function to get that data.
>
> I am thinking of redefining the stat structure within APR, and just having
> ap_stat and ap_get_fileinfo fill out a public structure. There is really
> no reason for this information to be in an incomplete type. Any field
> that does not have a good implementation on a platform, could just have an
> error condition. For example, if the uid field didn't map well to
> Windows, we could just leave that field with a -999 in it, which would be
> invalid.
>
> I think this cleans up the code a lot and reduces the number of functions
> that we would need to call. I have had a couple of people suggest this in
> the past, but I have been hesitant. Seeing how we are using stat info, I
> think this is the correct direction to take. This will also make a stat
> cache easier to add if APR decides it is necessary later on. :-)
>
> If I don't hear from anybody, I am going to start this work relatively
> soon.
>
> Ryan
>
> _______________________________________________________________________
> Ryan Bloom rbb@raleigh.ibm.com
> 4205 S Miami Blvd
> RTP, NC 27709
>
> Come to the first official Apache Software Foundation
> Conference! <http://ApacheCon.Com/>
>
>
Re: Small direction change for APR's file I/O routines.
Posted by David Reid <ab...@dial.pipex.com>.
That looks like a good idea. Apologies for earlier misunderstandings...
d.
----- Original Message -----
From: <rb...@apache.org>
To: <ne...@apache.org>
Sent: Tuesday, January 11, 2000 2:18 AM
Subject: Re: Small direction change for APR's file I/O routines.
>
>
>
> > > Or, I have seriously had three or four people tell me that they really
> > > want to have APR provide a stat cache for them. I really don't care,
but
> > > if enough people continue to ask for it, I will put it into APR.
> >
> > I'm with Dean on this one: I don't think a library can always do a great
> > job of it. The application knows best about its behavior -- when/why it
> > calls stat(), when it should purge, how big the cache should be, etc.
> >
>
> Obviously I need to do one of two things. Either never speak about
> possible future enhancements to APR, or explain them in detail whenever I
> do. :-) This wasn't meant as a "I'm doing this tomorrow" kind of thing,
> but I figure I'll quell this now since I brought it up.
>
> I have had a couple of people express the desire for a stat cache in APR,
> this does not mean APR will manage the cache, it just means APR will
> provide the functions necessary to do the cache management for you. It
> doesn't even have to be limited to a stat cache, it could conceivably
> cache anything. For example, why should the base Apache, and mod_proxy,
> and mod_php, and mod_perl all have cache code in them? For the most part,
> a cache is a cache what is being cached changes, how long it should be
> cached for changes, but the basic operations remain the same. Let me give
> you an example of the API I was considering.
>
> ap_initialize_cache(ap_cache_t *thecache, int num_elements, func
> *cleanup_routine, ap_context_t *cont);
> ap_add_element(void *elem, ap_cache_t *thecache, int timeout);
> ap_get_element(void *elem, ap_cache_t *thecache);
> ap_free_cache(ap_cache_t *thecache);
>
> This is not a set API, there isn't even ideathing saying this will ever be
> implemented, but the idea of APR providing cache'ing functions is the same
> as APR providing queue'ing functions. These are portable, and it doesn't
> make any sense for every program to write them again. It would even be
> possible for programs to compile APR without having these functions be
> provided.
>
> I will not be addressing the stat cache or cache'ing issue again, because
> it isn't really important to Apache right now, it has nothing to do with
> Apache 2.0, and I never should have brought it up. But, providing basic
> routines like this is a good thing for APR to do, IMO. I know mod_proxy
> needs a cache, and I know mod_php and mod_perl already use a cache, IMHO,
> it should be the same code, and it isn't currently.
>
> > The library just can't know all this.
>
> And, as I just explained, the library doesn't have to know any of it.
>
> Ryan
>
Re: Small direction change for APR's file I/O routines.
Posted by rb...@apache.org.
> > Or, I have seriously had three or four people tell me that they really
> > want to have APR provide a stat cache for them. I really don't care, but
> > if enough people continue to ask for it, I will put it into APR.
>
> I'm with Dean on this one: I don't think a library can always do a great
> job of it. The application knows best about its behavior -- when/why it
> calls stat(), when it should purge, how big the cache should be, etc.
>
Obviously I need to do one of two things. Either never speak about
possible future enhancements to APR, or explain them in detail whenever I
do. :-) This wasn't meant as a "I'm doing this tomorrow" kind of thing,
but I figure I'll quell this now since I brought it up.
I have had a couple of people express the desire for a stat cache in APR,
this does not mean APR will manage the cache, it just means APR will
provide the functions necessary to do the cache management for you. It
doesn't even have to be limited to a stat cache, it could conceivably
cache anything. For example, why should the base Apache, and mod_proxy,
and mod_php, and mod_perl all have cache code in them? For the most part,
a cache is a cache what is being cached changes, how long it should be
cached for changes, but the basic operations remain the same. Let me give
you an example of the API I was considering.
ap_initialize_cache(ap_cache_t *thecache, int num_elements, func
*cleanup_routine, ap_context_t *cont);
ap_add_element(void *elem, ap_cache_t *thecache, int timeout);
ap_get_element(void *elem, ap_cache_t *thecache);
ap_free_cache(ap_cache_t *thecache);
This is not a set API, there isn't even ideathing saying this will ever be
implemented, but the idea of APR providing cache'ing functions is the same
as APR providing queue'ing functions. These are portable, and it doesn't
make any sense for every program to write them again. It would even be
possible for programs to compile APR without having these functions be
provided.
I will not be addressing the stat cache or cache'ing issue again, because
it isn't really important to Apache right now, it has nothing to do with
Apache 2.0, and I never should have brought it up. But, providing basic
routines like this is a good thing for APR to do, IMO. I know mod_proxy
needs a cache, and I know mod_php and mod_perl already use a cache, IMHO,
it should be the same code, and it isn't currently.
> The library just can't know all this.
And, as I just explained, the library doesn't have to know any of it.
Ryan
Re: Small direction change for APR's file I/O routines.
Posted by Greg Stein <gs...@lyra.org>.
On Mon, 10 Jan 2000 rbb@apache.org wrote:
> > Oh, Ryan's just taunting me, that's all. I've been nagging him on and
> > of about the old system for a while now.
>
> Or, I have seriously had three or four people tell me that they really
> want to have APR provide a stat cache for them. I really don't care, but
> if enough people continue to ask for it, I will put it into APR.
I'm with Dean on this one: I don't think a library can always do a great
job of it. The application knows best about its behavior -- when/why it
calls stat(), when it should purge, how big the cache should be, etc.
The library just can't know all this.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Small direction change for APR's file I/O routines.
Posted by rb...@apache.org.
> Oh, Ryan's just taunting me, that's all. I've been nagging him on and
> of about the old system for a while now.
Or, I have seriously had three or four people tell me that they really
want to have APR provide a stat cache for them. I really don't care, but
if enough people continue to ask for it, I will put it into APR.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@ntrnet.net
6209 H Shanda Dr.
Raleigh, NC 27609 Ryan Bloom -- thinker, adventurer, artist,
writer, but mostly, friend.
-------------------------------------------------------------------------------
Re: Small direction change for APR's file I/O routines.
Posted by Manoj Kasichainula <ma...@io.com>.
On Sun, Jan 09, 2000 at 12:58:56PM -0800, Dean Gaudet wrote:
> > think this is the correct direction to take. This will also make a stat
> > cache easier to add if APR decides it is necessary later on. :-)
>
> i would think a stat cache would be an application-level thing rather than
> in a portability library.
Oh, Ryan's just taunting me, that's all. I've been nagging him on and
of about the old system for a while now.
Re: Small direction change for APR's file I/O routines.
Posted by Bill Stoddard <st...@raleigh.ibm.com>.
> i would think a stat cache would be an application-level thing rather than
> in a portability library.
I agree. A stat cache does not belong in APR. Ryan has know my opinion about
this for ages.
Bill
Re: Small direction change for APR's file I/O routines.
Posted by David Reid <ab...@dial.pipex.com>.
After our brief discussions last year I was convinced that the stat cache
should be at application level by Ryan. The portability layer shouldn't be
doing things like that and should concentrate on being as portable and as
efficient as possible!
Didn't Bill have some code for Apache that would do it at the Apache level
already written?
david
----- Original Message -----
From: "Ryan Bloom" <rb...@raleigh.ibm.com>
To: <ne...@apache.org>
Sent: Sunday, January 09, 2000 9:02 PM
Subject: Re: Small direction change for APR's file I/O routines.
> > > I am thinking of redefining the stat structure within APR, and just
having
> > > ap_stat and ap_get_fileinfo fill out a public structure. There is
really
> > > no reason for this information to be in an incomplete type. Any field
> > > that does not have a good implementation on a platform, could just
have an
> > > error condition. For example, if the uid field didn't map well to
> > > Windows, we could just leave that field with a -999 in it, which would
be
> > > invalid.
> >
> > actually better than an error condition would be a compile-time
> > conditional such as HAS_UNIX_UID, HAS_UNIX_GID, or something like that.
> > this was an area which was painful in NSPR too -- there was no way to
> > implement mod_userdir, for example.
>
> A compile time option would work just as well.
>
> >
> > > think this is the correct direction to take. This will also make a
stat
> > > cache easier to add if APR decides it is necessary later on. :-)
> >
> > i would think a stat cache would be an application-level thing rather
than
> > in a portability library.
>
> That is an opinion not held by everybody in the group. The stat cache was
> not my idea, and it has been suggested to me multiple times. I have
> already said it won't go into aPR 1.0, because I want to stabilize APR
> first, but if enough people ask for it, I will add a stat cache.
>
> Ryan
>
> _______________________________________________________________________
> Ryan Bloom rbb@raleigh.ibm.com
> 4205 S Miami Blvd
> RTP, NC 27709
>
> Come to the first official Apache Software Foundation
> Conference! <http://ApacheCon.Com/>
>
>
Re: what is apache? (Re: Small direction change for APR's file I/Oroutines.)
Posted by David Reid <ab...@dial.pipex.com>.
I'm forced to agree with Dean that a few of the modules would be better off
as OS specific. Making them specific would be useful and would make
rewriting them into the 2.0 world easier, whilst giving a server that was
easier to tailor. The windows port might gain by having a different set of
modules that each has it's own config directives. Certainly platforms such
as OS2 and BeOS probably would, and this fits very nicely with my recent
proposal for configuration files! (Sorry couldn't resist that :-))
david
----- Original Message -----
From: "Dean Gaudet" <dg...@arctic.org>
To: <ne...@apache.org>
Sent: Sunday, January 09, 2000 10:51 PM
Subject: what is apache? (Re: Small direction change for APR's file
I/Oroutines.)
> On Sun, 9 Jan 2000 rbb@apache.org wrote:
>
> >
> > > > actually better than an error condition would be a compile-time
> > > > conditional such as HAS_UNIX_UID, HAS_UNIX_GID, or something like
that.
> > > > this was an area which was painful in NSPR too -- there was no way
to
> > > > implement mod_userdir, for example.
> > >
> > > A compile time option would work just as well.
> >
> > I take that back. I am not sure a compile time option is as good. I am
> > worried about portability in the future, and having people add
> > HAS_WINDOWS_(UID|GID). The same goes for any other platform. This was
> > easy enough to deal with when we were hiding the internals. People
didn't
> > like hiding the internals, so I opened them up. Now, we are going to
> > start putting back all the #ifdef's that APR was designed to take out.
> > Except, that this time, we are changing the conditions from the obvious
> > #ifdef WIN32 to #ifndef HAS_UNIX_UID (which may be true for BeOS or
> > MacOSX). I need to think about this more, but I definately am back to
> > sitting on the fence about compile time options versus defined
constants.
> > :-)
>
> my problem is that i don't know anything at all about access rights,
> users, groups, etc. on other platforms... so i couldn't figure out an
> abstraction which would let things such as mod_userdir become portable.
>
> ok so the truth is, this is an edge of functionality where we get to
> debate on "what is apache?" and "who are the users of apache?"
>
> maybe we just want to declare some modules to be unix specific and be done
> with it... that'd be really nice.
>
> 'cause really something like mod_userdir is a unix thing.
>
> i think the core technology in apache is akin to "HTTP routing", and i
> know some others agree with this. get requests off the wire and delivered
> to the correct module as fast as possible.
>
> the other stuff is "packaging" -- consider our endless iterations on
> mod_autoindex/mod_include. each time we iterate we get a little bit
> closer to a full programming language. but there's other groups of folks
> (for example php) that have already done this far better than we will.
> i've not used php, but i imagine it's really easy to do the equivalent of
> all the mod_autoindex ReadMe, and Header stuff, plus format the directory
> indicies into tables with whatever fancy colours and gifs you want... and
> you don't need to patch the server to change the format.
>
> it might make sense to pick a few modules for 2.0 which are the core
> technology, decide that we'll provide a portability layer which
> encompasses that, and then leave the rest in os-specific modules. spin off
> a side-project which is "apache classic for unix" which includes the
> mod_userdir, mod_autoindex, mod_include, and suexec stuff.
>
> Dean
>
what is apache? (Re: Small direction change for APR's file I/O
routines.)
Posted by Dean Gaudet <dg...@arctic.org>.
On Sun, 9 Jan 2000 rbb@apache.org wrote:
>
> > > actually better than an error condition would be a compile-time
> > > conditional such as HAS_UNIX_UID, HAS_UNIX_GID, or something like that.
> > > this was an area which was painful in NSPR too -- there was no way to
> > > implement mod_userdir, for example.
> >
> > A compile time option would work just as well.
>
> I take that back. I am not sure a compile time option is as good. I am
> worried about portability in the future, and having people add
> HAS_WINDOWS_(UID|GID). The same goes for any other platform. This was
> easy enough to deal with when we were hiding the internals. People didn't
> like hiding the internals, so I opened them up. Now, we are going to
> start putting back all the #ifdef's that APR was designed to take out.
> Except, that this time, we are changing the conditions from the obvious
> #ifdef WIN32 to #ifndef HAS_UNIX_UID (which may be true for BeOS or
> MacOSX). I need to think about this more, but I definately am back to
> sitting on the fence about compile time options versus defined constants.
> :-)
my problem is that i don't know anything at all about access rights,
users, groups, etc. on other platforms... so i couldn't figure out an
abstraction which would let things such as mod_userdir become portable.
ok so the truth is, this is an edge of functionality where we get to
debate on "what is apache?" and "who are the users of apache?"
maybe we just want to declare some modules to be unix specific and be done
with it... that'd be really nice.
'cause really something like mod_userdir is a unix thing.
i think the core technology in apache is akin to "HTTP routing", and i
know some others agree with this. get requests off the wire and delivered
to the correct module as fast as possible.
the other stuff is "packaging" -- consider our endless iterations on
mod_autoindex/mod_include. each time we iterate we get a little bit
closer to a full programming language. but there's other groups of folks
(for example php) that have already done this far better than we will.
i've not used php, but i imagine it's really easy to do the equivalent of
all the mod_autoindex ReadMe, and Header stuff, plus format the directory
indicies into tables with whatever fancy colours and gifs you want... and
you don't need to patch the server to change the format.
it might make sense to pick a few modules for 2.0 which are the core
technology, decide that we'll provide a portability layer which
encompasses that, and then leave the rest in os-specific modules. spin off
a side-project which is "apache classic for unix" which includes the
mod_userdir, mod_autoindex, mod_include, and suexec stuff.
Dean
Re: Small direction change for APR's file I/O routines.
Posted by rb...@apache.org.
> > actually better than an error condition would be a compile-time
> > conditional such as HAS_UNIX_UID, HAS_UNIX_GID, or something like that.
> > this was an area which was painful in NSPR too -- there was no way to
> > implement mod_userdir, for example.
>
> A compile time option would work just as well.
I take that back. I am not sure a compile time option is as good. I am
worried about portability in the future, and having people add
HAS_WINDOWS_(UID|GID). The same goes for any other platform. This was
easy enough to deal with when we were hiding the internals. People didn't
like hiding the internals, so I opened them up. Now, we are going to
start putting back all the #ifdef's that APR was designed to take out.
Except, that this time, we are changing the conditions from the obvious
#ifdef WIN32 to #ifndef HAS_UNIX_UID (which may be true for BeOS or
MacOSX). I need to think about this more, but I definately am back to
sitting on the fence about compile time options versus defined constants.
:-)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@ntrnet.net
6209 H Shanda Dr.
Raleigh, NC 27609 Ryan Bloom -- thinker, adventurer, artist,
writer, but mostly, friend.
-------------------------------------------------------------------------------
Re: Small direction change for APR's file I/O routines.
Posted by Ryan Bloom <rb...@raleigh.ibm.com>.
> > I am thinking of redefining the stat structure within APR, and just having
> > ap_stat and ap_get_fileinfo fill out a public structure. There is really
> > no reason for this information to be in an incomplete type. Any field
> > that does not have a good implementation on a platform, could just have an
> > error condition. For example, if the uid field didn't map well to
> > Windows, we could just leave that field with a -999 in it, which would be
> > invalid.
>
> actually better than an error condition would be a compile-time
> conditional such as HAS_UNIX_UID, HAS_UNIX_GID, or something like that.
> this was an area which was painful in NSPR too -- there was no way to
> implement mod_userdir, for example.
A compile time option would work just as well.
>
> > think this is the correct direction to take. This will also make a stat
> > cache easier to add if APR decides it is necessary later on. :-)
>
> i would think a stat cache would be an application-level thing rather than
> in a portability library.
That is an opinion not held by everybody in the group. The stat cache was
not my idea, and it has been suggested to me multiple times. I have
already said it won't go into aPR 1.0, because I want to stabilize APR
first, but if enough people ask for it, I will add a stat cache.
Ryan
_______________________________________________________________________
Ryan Bloom rbb@raleigh.ibm.com
4205 S Miami Blvd
RTP, NC 27709
Come to the first official Apache Software Foundation
Conference! <http://ApacheCon.Com/>
Re: Small direction change for APR's file I/O routines.
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 4 Jan 2000, Ryan Bloom wrote:
> I am thinking of redefining the stat structure within APR, and just having
> ap_stat and ap_get_fileinfo fill out a public structure. There is really
> no reason for this information to be in an incomplete type. Any field
> that does not have a good implementation on a platform, could just have an
> error condition. For example, if the uid field didn't map well to
> Windows, we could just leave that field with a -999 in it, which would be
> invalid.
actually better than an error condition would be a compile-time
conditional such as HAS_UNIX_UID, HAS_UNIX_GID, or something like that.
this was an area which was painful in NSPR too -- there was no way to
implement mod_userdir, for example.
> think this is the correct direction to take. This will also make a stat
> cache easier to add if APR decides it is necessary later on. :-)
i would think a stat cache would be an application-level thing rather than
in a portability library.
Dean