You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Ryan Bloom <rb...@raleigh.ibm.com> on 1999/08/26 19:30:26 UTC
Why bother with APR?
I am testing the waters here. I don't mind getting flamed, but
constructive answers would be much better. This is not me being upset
about what has happened over the last few days, this is me really not
seeing a future for APR.
The original goal of APR is gone, and APR is floundering out there right
now. I no longer see a need for it. The original goal was a portability
layer, then it became a portable run-time (small difference but important
when you consider what APR will cover), most recently it seems to be a
very very small portable run-time again. Fine. That is easy to
implement. I would argue however, that this has been done already in
Apache, and a separate library is not needed.
The ap_popen* routines easily do what it looks like APR is being asked to
do. The only problem with them, is that they haven't really been ported
fully to windows (I think this is the case, but I could be wrong).
The few functions that we don't have (e.g. ap_open_pipe) could be written
easily enough, and just added to the alloc.c file or whatever file ends up
containing ap_popen*).
Basically, I think the APR and MPM concepts are at odds with each other.
MPM's seem to want to split out function based on what system you are on,
and APR wants to combine function regardless of what system you are on.
There are a few places where having comine calling formats makes a lot of
sense, but they are few and far between, and they do not require their own
library. Create_Process is a function that screams out to be created on
all platforms with a common API across platforms. Great, take those few
calls that are required, and put them in the os/* directory in a file
called common_api.c. The directory functions should also be put in this
area.
Unless I am missing something, APR should be canabalized into the
common_api.c file, and we should go on coding Apache, and ignore APR. If
I am missing something, please let me know.
Personally, I would love to get back to writing part of the web server. I
only moved down to APR, because it seemed like a necessary project, that
need seems to have disappeared now.
Ryan
_______________________________________________________________________
Ryan Bloom rbb@raleigh.ibm.com
4205 S Miami Blvd
RTP, NC 27709 It's a beautiful sight to see good dancers
doing simple steps. It's a painful sight to
see beginners doing complicated patterns.
Re: Why bother with APR?
Posted by Dean Gaudet <dg...@arctic.org>.
dude, I can't respond to this right now, but I think we're still having
problems communicating. I still see a need for APR, in fact you'll see
notes about it in the MPM sources... I'll try to write more later.
Dean
On Thu, 26 Aug 1999, Ryan Bloom wrote:
>
> I am testing the waters here. I don't mind getting flamed, but
> constructive answers would be much better. This is not me being upset
> about what has happened over the last few days, this is me really not
> seeing a future for APR.
>
> The original goal of APR is gone, and APR is floundering out there right
> now. I no longer see a need for it. The original goal was a portability
> layer, then it became a portable run-time (small difference but important
> when you consider what APR will cover), most recently it seems to be a
> very very small portable run-time again. Fine. That is easy to
> implement. I would argue however, that this has been done already in
> Apache, and a separate library is not needed.
>
> The ap_popen* routines easily do what it looks like APR is being asked to
> do. The only problem with them, is that they haven't really been ported
> fully to windows (I think this is the case, but I could be wrong).
>
> The few functions that we don't have (e.g. ap_open_pipe) could be written
> easily enough, and just added to the alloc.c file or whatever file ends up
> containing ap_popen*).
>
> Basically, I think the APR and MPM concepts are at odds with each other.
> MPM's seem to want to split out function based on what system you are on,
> and APR wants to combine function regardless of what system you are on.
>
> There are a few places where having comine calling formats makes a lot of
> sense, but they are few and far between, and they do not require their own
> library. Create_Process is a function that screams out to be created on
> all platforms with a common API across platforms. Great, take those few
> calls that are required, and put them in the os/* directory in a file
> called common_api.c. The directory functions should also be put in this
> area.
>
> Unless I am missing something, APR should be canabalized into the
> common_api.c file, and we should go on coding Apache, and ignore APR. If
> I am missing something, please let me know.
>
> Personally, I would love to get back to writing part of the web server. I
> only moved down to APR, because it seemed like a necessary project, that
> need seems to have disappeared now.
>
> Ryan
>
> _______________________________________________________________________
> Ryan Bloom rbb@raleigh.ibm.com
> 4205 S Miami Blvd
> RTP, NC 27709 It's a beautiful sight to see good dancers
> doing simple steps. It's a painful sight to
> see beginners doing complicated patterns.
>
>
Re: Why bother with APR?
Posted by Ryan Bloom <rb...@ntrnet.net>.
> So instead, how about splitting APR out into two
> libraries such that the feature set of APR ( md5, pools,
> contexts etc) sits on top of a general purpose cross platform
> lib? At this stage it wouldnt be a huge effort since the
> bulk of the apache feature set is in sperate code to the
> actual cp parts. Add to that, it could be done as a low priority
> background task that would be completely transparent to
> current APRizing effort.
What would you suggest putting in the general cross platform lib? I
personnaly think our time would be better spent implementing different
ways to do things in apr. For example, implementing memory allocation
using malloc/free instead of pools. And getting these options into
autoconf. But then, I don't really understand what you are proosing.
Could you please specify what you envision being in each portion of APR?
>
> Anyways, I must credit you sir, you have done a bloddy
> good job of apr. I must appologise if I have caused you in any
> way to feel that there is no future for apr. As it stands, the
> structure of APR does lend itself well to both jobs and it will
> have a firm place in the Apache web server, as well as many
> other projects to come.
Nobody caused me to feel there was no future for APR. I did have some
serious questions about the future of APR in apache. Those have been
resolved for me. Now, it's just a question of doing it the right way. :)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@ntrnet.net
6209 H Shanda Dr.
Raleigh, NC 27609 Ryan Bloom -- thinker, adventurer, artist,
writer, but mostly, friend.
-------------------------------------------------------------------------------
Re: Why bother with APR?
Posted by "Michael H. Voase" <mv...@midcoast.com.au>.
Ryan Bloom wrote:
[snip]
>
> I would also like to get some feedback here. It was always assumed, that
> APR would do most of the allocation. If people would rather leave memory
> allocation/management out of APR, and have the user allocate any memory
> and passs that into APR, and have APR use malloc when necessary, please
> say so this weekend. I would like to have APR back to a usable state on
> Monday.
>
> Ryan
I do have a segestion, but it is relative to
how usefull it may seem to you. When I was confused
about the purpose of APR, I did'nt see why pools would
be included in a portability layer. Of course as I later
found out, it must be there to support existing apache
code.
In light of that fact, it still stirkes me that
there is a lot of excellent work in the cross platform
dept. that would be really nice to have for general
application programming. Hence why me reply after putting
the paper bag on was 'could I fork off another library?'. This
is a rediculous proposition since one would be maintaining
two specific libs instead of one well designed lib that
could be used for both purposes.
So instead, how about splitting APR out into two
libraries such that the feature set of APR ( md5, pools,
contexts etc) sits on top of a general purpose cross platform
lib? At this stage it wouldnt be a huge effort since the
bulk of the apache feature set is in sperate code to the
actual cp parts. Add to that, it could be done as a low priority
background task that would be completely transparent to
current APRizing effort.
Anyways, I must credit you sir, you have done a bloddy
good job of apr. I must appologise if I have caused you in any
way to feel that there is no future for apr. As it stands, the
structure of APR does lend itself well to both jobs and it will
have a firm place in the Apache web server, as well as many
other projects to come.
Cheers Mik.
--
----------------------------------------------------------------------------
/~\ /~\ CASTLE INDUSTRIES PTY. LTD.
| |_____| | Incorporated 1969. in N.S.W., Australia
| | Phone +612 6567 1227 Fax +612 6567 1449
| /~\ | Web http://www.midcoast.com.au/~mvoase
| [ ] | Michael H. Voase. Director.
~~~~~~~~~~~~~~ Castle Industries - Industrial Strength
Solutions.
----------------------------------------------------------------------------
Re: memory allocation (was Re: Why bother with APR?)
Posted by Tony Finch <do...@dotat.at>.
I was speaking to Dan Sheppard on Saturday night about his memory
allocator, and some of his work seems in quite close alignment with
what Apache has done in the past and the direction it is moving in
now. You might want to look at his page at
http://www.ep.cs.nott.ac.uk/~dps/czone/intro1.html
Tony.
--
f.a.n.finch dot@dotat.at fanf@demon.net e pluribus unix
Re: memory allocation (was Re: Why bother with APR?)
Posted by Manoj Kasichainula <ma...@io.com>.
On Tue, Sep 21, 1999 at 02:07:25AM -0700, Dean Gaudet wrote:
> it'd be cool if we could stop calling the memory allocator a pool, it
> confuses things...
Well, we need another name for it, then. 1.3pool?
> - or you could adjust the malloc size by sizeof(void *) and stick a
> back pointer to the cleanup in it... but if you're going to do this,
> you might as well adjust the malloc size by sizeof(struct mem)
> where struct mem { mem *next; mem *prev };. this way free() is
> O(1).
The only (minor) problem with this is that we may tend to malloc in
powers of 2 (I should look, but I'm lazy), and this addition might
cause memory to be wasted. It would in the basic mallocs I learned
about in school.
> - don't use malloc/free to implement ap_malloc/ap_free. instead grab
> some malloc implementation (such as *bsd's) and modify it to
> use ap_palloc underneath instead of going through brk()/sbrk().
Other than having to bundle our own malloc, this sounds very cool.
> - ap_malloc and ap_free are not associated with pools at all, the users
> of them have to do their own resource management. this is exactly
> the route i've taken already if you look around at the few malloc()s
> i put in.
For the shared cache case, this is no big deal. We'd have to maintain
reference counts to ensure that a cached object isn't pulled out from
under anyone anyway. Any automatic cleanup scheme would only serve to
hide bugs in the reference counting algorithms.
--
Manoj Kasichainula - manojk at io dot com - http://www.io.com/~manojk/
Memory allocation (controversial)
Posted by David Reid <ab...@dial.pipex.com>.
Oh learned and guru-type people of the ASF,
I'm sure this will result in some flames, but here goes anyhow...
<CONTROVERY>
OK, so the ongoing discussion is interesting, but it seems to be going round
in circles...
Maybe I'm sticking my nose in where it shouldn't be, but it strikes me we
should at the beginning (or at least where I think the beginning is/was/used
to be/etc)
APR was developed to provide a platform independent library upon which
Apache can sit. APR now incorporates the memory management stuff as it was
brought over with a lot of the other ap_ functions. The question in my mind
is have we paused to consider if this is still the best way of
allocating/cleaning up memory for -2.0? If there is a different way of
doing things, should we look at it now before APR gets too much older? I
don't mean simply should we change the structures like this and then call
them that, I mean we should re-examine and spell out what we want from our
memory management?
BTW I happen to think that a lot of the memory management stuff in Apache
already works well and am not trying to say "let's rip it all out and start
over", just asking if some rational debate on the subject would be a good
idea at this point in time.
It would be nice to have a single set of calls that could use malloc,
pre-allocated memory blocks, mmap whatever as required with no need for the
user to know/care. The CZone stuff that Tony pointed out looks interesting
if unfinished. There must be other people out there with work that's
similar.
Shouldn't we spend a bit of time looking at these now and considering if the
way we do things presently is the best way for -2.0? If we conclude that it
is, great! If not, then at least we've got a new starting point.
I realise that the stuff in -1.3 was developed over a long period, but -2.0
isn't -1.3 and as a lot has already changed maybe we should consider this as
well.
</CONTROVERSY>
Just my thoughts on it, so if you think this is stupid/pointless just ignore
it :-)
david
Re: memory allocation (was Re: Why bother with APR?)
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 21 Sep 1999, Rodent of Unusual Size wrote:
> Dean Gaudet wrote:
> >
> > a pool is just a place to hang cleanups in
> > my opinion
>
> Well, my opinion, at least, differs. That happens to be a
> feature of pools to my mind, but not their raison d'être.
ok, would you agree that memory and child subprocesses are examples
of resources? these happen to be the only two resources (1.x) which
require a little extra state information on a per-pool basis.
would you agree that pools are used when resources need to be tracked,
and destroyed as a group?
would you agree that palloc is a memory allocator with the property
that allocations can only be freed as one large unit? this is a common
abstraction by the way -- it's present in other programs, under different
names. most frequently present for fixed-size allocations, but i digress.
you guys notice a new abstraction which is useful -- you notice that
it's useful to abstract per-pool state. that's great, it is. in fact,
if you study the 1.x pools you'll see we already have two examples of
per-pool state: palloc and child subprocesses. but you didn't notice
that, so you went about creating a structure which looks like this:
struct context {
abstract per-context state information;
struct pool {
abstract resource cleanup information;
not abstracted palloc allocator information;
not abstracted subprocess information;
};
};
at this point it should be obvious that those two resource trackers should
use the abstract per-context state information. and you're left with:
struct context {
abstract per-context state information;
struct pool {
abstract resource cleanup information;
};
};
ok, maybe pools are going to be used on their own somewhere without the
context. but, that isn't the case. ryan has just gone through the
code and done a search and replace changing context to pool. there's
absolutely no reason for the pool to exist on its own. so in fact
perhaps you'll all realise that a logical next step is this:
struct context {
abstract per-context state information;
abstract resource cleanup information;
};
bye bye pools. in retrospect you might realise now that you've essentially
forced every single apache author to learn a new concept... when really,
only 1% of them (us mainly) needed to care about the per-context state
information. furthermore, you've changed the name from something with
a lot of overloaded meaning, pool, but with a lot of history within apache,
to something which has even more overloaded meanings: context. this is
a step backwards in readability.
*and furthermore* the code now has bloody confusing crap like this
segment spread all over the place:
ap_context_t *tmp;
ap_create_context(p, NULL, &tmp);
...
ap_destroy_pool(p);
you guys get 10 points for figuring out the per-pool state abstraction.
but you lose about 100 points for the boneheaded moves which followed
that.
Dean
Re: memory allocation (was Re: Why bother with APR?)
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Dean Gaudet wrote:
>
> a pool is just a place to hang cleanups in
> my opinion
Well, my opinion, at least, differs. That happens to be a
feature of pools to my mind, but not their raison d'être.
--
#ken P-)}
Ken Coar <http://Web.Golux.Com/coar/>
Apache Software Foundation <http://www.apache.org/>
"Apache Server for Dummies" <http://ASFD.MeepZor.Com/>
Re: malloc/free ...
Posted by Ben Hyde <bh...@pobox.com>.
I think I'm happy. I've been using subpools to fill in for
malloc/free. It's very sweet and addictive. I rarely build data
structures that are just contiguous memory, in fact I'm embaressed
when I do.
It might be better to be fixated on enhancing the performance of or
creating variants of these two operations in place of adding another
memory abstraction.
I have accumulated a large amount of code where when ever I want to
create a "object" or an "abstract data type" or a "data structure"
I give each instance it's own pool. This tends to look like this,
#define MAKE_INSTANCE(pool, type) ((type*)ap_palloc(pool,sizeof(type)))
typedef struct foo_s foo;
struct foo_s { ap_pool *pool;... };
void destroy_foo(foo *)
{
ap_destroy_pool(foo->pool);
}
foo *create_foo(ap_pool p, ...p1 ...pn)
{
ap_pool foo_pool = ap_make_sub_pool(p);
foo *result = MAKE_INSTANCE(p, foo_pool);
result->pool = foo_pool;
result->p1 = p1;
...
result->pn = pn;
... further initialize foo ...
... cleanup for foo in foo_pool ...
}
foo operationX_foo(foo *f ...)
{
foo_element *e = MAKE_INSTANCE(f->pool, foo_element);
e->next = f->whatever;
f->whatever = z;
... more what ever ...
}
I particularly like the transparent semantics of the destroy
operation, and that the enclosing pool/object folds my cleaning
into his destruction. That is _so_ much better than the
ad hoc cleanup strategies so commonly found in systems.
This example glosses over some details. For example my create_foo
rarely takes a pool explicitly, but instead takes an "activity"
object and a "parent" object and then use the activity's pool to
create the child and then it will stow the child into the parent
object. Then cleaning is a little subtle since the termination
of the parent child or activity can triger it. Getting the
various unhooking and ordering right reminds one of why a
free operation is such an incredible thorn the foot.
- ben
Re: memory allocation (was Re: Why bother with APR?)
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 21 Sep 1999, Manoj Kasichainula wrote:
> On Tue, Sep 14, 1999 at 09:43:27AM -0700, Dean Gaudet wrote:
> > note: when i was suggesting adding cleanups for malloc'd memory i really
> > wasn't suggesting that a cleanup be added for each piece of malloc'd
> > memory. that will consume a fair bit more memory per allocation... i was
> > actually thinking the solution would be to prefix the malloc allocation
> > with a struct mem { mem *next; mem *prev }; ... and having a single
> > cleanup which walks the entire list of malloc allocations.
>
> Sounds like a pool to me. :) What's the advantage of this? If the list
> is short, we're reverting closer to the one-malloc-per-cleanup case.
> If the list is long, pools start looking really good. Or am I missing
> something?
it'd be cool if we could stop calling the memory allocator a pool, it
confuses things... a pool is just a place to hang cleanups in my opinion
(which gets us back to the "why create contexts" debate, because contexts
appear to just be pools with an extra void *, but i digress).
i'm not sure what piece you're missing... but here's the main points:
- we're debating on adding an ap_malloc() and a companion ap_free() --
malloc returns something which lives until a free or the pool is
cleared, whichever comes first. we want this for various reasons i've
already mentioned (shared caches being the main case).
- cleanups are a singly linked list. if you have a cleanup for each
malloced bit of ram, then you get wonderful O(n^2) behaviours depending
on the free() patterns.
- if you make cleanups a doubly-linked list then you're still stuck with
O(n^2) behaviour for free because you still need to find which
cleanup is associated with the piece of memory.
- or you could adjust the malloc size by sizeof(void *) and stick a
back pointer to the cleanup in it... but if you're going to do this,
you might as well adjust the malloc size by sizeof(struct mem)
where struct mem { mem *next; mem *prev };. this way free() is
O(1).
- this last case also has the advantage that an ap_malloc() only
requires one call to malloc(), not two (one for the result, one
for the cleanup).
Or, another alternative:
- don't use malloc/free to implement ap_malloc/ap_free. instead grab
some malloc implementation (such as *bsd's) and modify it to
use ap_palloc underneath instead of going through brk()/sbrk().
- this way you don't need any cleanups for the ap_malloc -- you assume
that the caller will always call ap_free() with the right pool.
Or, another alternative:
- ap_malloc and ap_free are not associated with pools at all, the users
of them have to do their own resource management. this is exactly
the route i've taken already if you look around at the few malloc()s
i put in.
Dean
Re: memory allocation (was Re: Why bother with APR?)
Posted by Manoj Kasichainula <ma...@io.com>.
On Tue, Sep 14, 1999 at 09:43:27AM -0700, Dean Gaudet wrote:
> note: when i was suggesting adding cleanups for malloc'd memory i really
> wasn't suggesting that a cleanup be added for each piece of malloc'd
> memory. that will consume a fair bit more memory per allocation... i was
> actually thinking the solution would be to prefix the malloc allocation
> with a struct mem { mem *next; mem *prev }; ... and having a single
> cleanup which walks the entire list of malloc allocations.
Sounds like a pool to me. :) What's the advantage of this? If the list
is short, we're reverting closer to the one-malloc-per-cleanup case.
If the list is long, pools start looking really good. Or am I missing
something?
--
Manoj Kasichainula - manojk at io dot com - http://www.io.com/~manojk/
Re: memory allocation (was Re: Why bother with APR?)
Posted by Dean Gaudet <dg...@arctic.org>.
On Mon, 13 Sep 1999, Manoj Kasichainula wrote:
> On Tue, Sep 07, 1999 at 04:00:19PM -0700, Dean Gaudet wrote:
> > so i'm thinking that it may not be possible to decide ahead of time what
> > type of memory allocation should be associated with a particular pool.
> > that is to say -- some users of a pool may want palloc-style allocation,
> > which isn't freed until pool destruction, and other users may want
> > malloc/free style which can be freed at will.
> >
> > there's two reasons i'm thinking along these lines -- the first is that
> > some users of a pool may be a cache, and could be hidden behind a library
> > interface. the second reason is that i don't like lots of conditionals ;)
>
> If pools will support malloc/free, I'd like to add another function to
> the pool interface:
>
> ap_run_cleanups(struct context_t *p, void *data);
>
> It would work like ap_run_cleanup, but it would execute all the
> cleanups for a given pointer. With this addition, it becomes possible
> to support malloc/free-mode and the current system without the app
> actually having to care which was used.
>
> If an app wants to support freeing a pointer's memory on-demand, it
> simply calls ap_run_all_cleanups on the pointer. If the pool used its
> own memory chunk to allocate the memory, it won't do anything because
> no actual cleanups were registered. But if malloc was used, the memory
> will get freed. And anything missed by the application will get freed
> when the pool is cleaned out in both cases.
note: when i was suggesting adding cleanups for malloc'd memory i really
wasn't suggesting that a cleanup be added for each piece of malloc'd
memory. that will consume a fair bit more memory per allocation... i was
actually thinking the solution would be to prefix the malloc allocation
with a struct mem { mem *next; mem *prev }; ... and having a single
cleanup which walks the entire list of malloc allocations.
although at some point here we start arguing about how to implement
generic linked list support ;)
Dean
Re: memory allocation (was Re: Why bother with APR?)
Posted by Manoj Kasichainula <ma...@io.com>.
On Tue, Sep 07, 1999 at 04:00:19PM -0700, Dean Gaudet wrote:
> so i'm thinking that it may not be possible to decide ahead of time what
> type of memory allocation should be associated with a particular pool.
> that is to say -- some users of a pool may want palloc-style allocation,
> which isn't freed until pool destruction, and other users may want
> malloc/free style which can be freed at will.
>
> there's two reasons i'm thinking along these lines -- the first is that
> some users of a pool may be a cache, and could be hidden behind a library
> interface. the second reason is that i don't like lots of conditionals ;)
If pools will support malloc/free, I'd like to add another function to
the pool interface:
ap_run_cleanups(struct context_t *p, void *data);
It would work like ap_run_cleanup, but it would execute all the
cleanups for a given pointer. With this addition, it becomes possible
to support malloc/free-mode and the current system without the app
actually having to care which was used.
If an app wants to support freeing a pointer's memory on-demand, it
simply calls ap_run_all_cleanups on the pointer. If the pool used its
own memory chunk to allocate the memory, it won't do anything because
no actual cleanups were registered. But if malloc was used, the memory
will get freed. And anything missed by the application will get freed
when the pool is cleaned out in both cases.
> i think of a pool as really a place to store cleanups. in essence,
> a pool could just be a list of cleanups.
That's how I've always thought of it. The memory allocation stuff is
just a nice performance tweak.
> and ap_shmalloc() could register a cleanup for the shared mem
> allocations...
Apparently, supporting pool-like operations in shared memory isn't
portable once you aren't in Unixland anymore.
--
Manoj Kasichainula - manojk at io dot com - http://www.io.com/~manojk/
memory allocation (was Re: Why bother with APR?)
Posted by Dean Gaudet <dg...@arctic.org>.
so i'm thinking that it may not be possible to decide ahead of time what
type of memory allocation should be associated with a particular pool.
that is to say -- some users of a pool may want palloc-style allocation,
which isn't freed until pool destruction, and other users may want
malloc/free style which can be freed at will.
there's two reasons i'm thinking along these lines -- the first is that
some users of a pool may be a cache, and could be hidden behind a library
interface. the second reason is that i don't like lots of conditionals ;)
i think of a pool as really a place to store cleanups. in essence, a pool
could just be a list of cleanups.
then ap_palloc() would register a cleanup the first time it is used.
and ap_malloc() could register a cleanup for the list of malloc/free
allocations.
and ap_shmalloc() could register a cleanup for the shared mem
allocations...
this way the same pool can be used for any combination of the three types
of allocations (and other allocations) without knowing ahead of time what
style of pool it is.
Dean
Re: Why bother with APR?
Posted by Ryan Bloom <rb...@raleigh.ibm.com>.
Ok, I am mostly convinced that there is still room for APR in Apache.
This is what I was really looking for, a road map so that I knew what to
code. Here is what I would like to do with APR to start with:
context becomes:
{
ap_pool_t *pool
void *user_data
}
In the future this may expand to have more memory allocation schemes in
it, or perhaps other data, but right now, this is the important stuff. I
still like the contet idea, especially after Dean's message yesterday that
we may want multiple allocation schemes in the same server.
I would like some ffedback to this design. I like the user data. It lets
us put all the user data in a common place in all of our structures. I
think the very fact that most of the Apache structures have a place for
user_Data, even if it isn't called that, is proof that having this
attached to the memory allocator is a good thing.
Once APR is re-worked/re-designed, I will re-create the context change if
it is required.
I would also like to get some feedback here. It was always assumed, that
APR would do most of the allocation. If people would rather leave memory
allocation/management out of APR, and have the user allocate any memory
and passs that into APR, and have APR use malloc when necessary, please
say so this weekend. I would like to have APR back to a usable state on
Monday.
Ryan
Re: Why bother with APR?
Posted by Dean Gaudet <dg...@arctic.org>.
On Thu, 26 Aug 1999, Ryan Bloom wrote:
> The original goal of APR is gone
I would say that the original goal of APR wasn't quite right... it was
more aligned with NSPR ... and that wasn't quite right either. The most
important area of the server performance-wise is the mapping of incoming
connections to threads... and the most efficient way to do that varies
from platform to platform (and on each platform there may be several
choices). This is the key abstraction I sought out in MPM. It wasn't
clear to us a year ago when we started.
This abstraction is essentially the guts of any server... a mapping from
incoming requests to "working units", threads/fibers, whatever you want to
call them. And we've made steps to generalise things so that we can
support other protocols.
Also, I wanted to get the main abstraction done without fuss, and when I
did it, I created two new things which probably belong in APR -- ap_iol,
and the thread mutex stuff. This was just a shortcut so that I could get
to working code fast... they should be replaced with APR equivalents.
With APR we originally set out to make all platforms look the same -- so
that we'd have one accept loop implementation shared across everything...
NSPR is even worse in this area because they hide so many details in their
big select loops that we'd have trouble making the changes we want.
I think the modified APR goal should be to supplement the above method.
In theory the MPM stuff could be part of a library... but let's get to
that later.
Specifically, APR can help us in these ways:
- io layering
- TCP socket functions, sockopts, listen, accept, bind
- pipes
- named pipes
- shared memory
- thread creation, synchronization
- process spawning
- unified interface to times, time parsing, time formatting, interval
timers
One key difference between this and NSPR is that NSPR requires all i/o to
go through its functions, so that on platforms which don't have kernel
threads it can provide a userland threads library. However, this is
totally at odds with 3rd party libraries that need to do i/o... and by
forcing all platforms to go through the NSPR select loops I think they
lose performance on the modern platforms which all have native threads. A
better approach, in my opinion, is to use native methods under the covers
-- so that 3rd party libraries work without trouble. That penalises only
those platforms which don't have native threads... instead of penalising
everything.
This ideal also lets us investigate asynchronous possibilities in the
future ... and the async stuff gives us the benefits of a hybrid
userland/kernel threading approach, without the overhead of making it
appear seamless. We just do asynchronous stuff for the parts where it
matters -- less overhead.
> The ap_popen* routines easily do what it looks like APR is being asked to
> do. The only problem with them, is that they haven't really been ported
> fully to windows (I think this is the case, but I could be wrong).
they're also a pain because they're laden with #ifdefs... and the spawning
stuff is a total mess. And I really think this is the stuff that should
be moved into APR ...
> Basically, I think the APR and MPM concepts are at odds with each other.
> MPM's seem to want to split out function based on what system you are on,
> and APR wants to combine function regardless of what system you are on.
Nope, MPM just wants to split out the distribution of requests to threads.
We still want to provide a uniform cross-platform interface to things such
as mutexes, rw locks... otherwise it'll be a royal pain when we start
implementing caches and such.
> Personally, I would love to get back to writing part of the web server. I
> only moved down to APR, because it seemed like a necessary project, that
> need seems to have disappeared now.
I really think you're on the right path -- you should start removing code
from apache and calling into APR instead...
I'm only badgering you about APR details because I want to make sure we
don't end up with bloat. It really is best to start out simple and add
features later... that's how we got apache here, and that's how we should
proceed. I always question the need for a feature -- I always ask for
sample applications, I'm not just picking on you. I annoyed everyone last
year asking why they thought layered i/o was needed until we came up with
lists of things we'd use it for... that's good, that lets us make sure our
design fits expected uses.
Dean