Re: SMS stuff [long]

[David Reid <dr...@jetnet.co.uk>]

Well I did say I wouldn't get back into this "debate" for a while, but...

> On Fri, Jun 08, 2001 at 11:09:42AM +0200, Sander Striker wrote:
> > > I am well beyond 100% in support of the sms schema.  The only
> > > question is what is which, do we wrap the pool schema in sms,
> > > or do 'pools' become sms?

It's a question of layers really.  I'd rather we just had one layer, Greg
wants to see 2, or at least that's how I view his postings.  I always like
the KISS principle :)

> >
> > I'll try and write up a comparison between the apis in the weekend,
> > together with how it could be handled _if_ pools are replaced with
> > sms. If after that people are still not convinced *), I'm starting
> > to think 'we've tried. no deal. move on.'.
>
> A short description/comparison may help.
>
> However, I believe that no matter what you may end up describing, the only
> thing that people will be comfortable with is an absolute retention of
> apr_pool_t and SMS growing silently underneath that API. Certain
> applications may choose to use SMS directly (fine!). At the end point,
> apr_pool_t will be a slim cover, and we can choose what to do at that
point.

OK.  I'm going to try and explain what was in our (myself and Sanders) mind
to do.  Maybe this hasn't been clear before??  If so then most humble
apologies.  BTW, this wasn't so much discussed as it seemed obvious
following some discussions about why they had coded some parts of sms as
they had.

At present we have a set of logic for managing memory that we call pools.
Essentially it allocates a large block and assigns smaller chunks from it
using lists to keep track of what it's done, cleaning up after it's
finished.  Locking and cleanups are provided in the code and a number of
ways of getting memory are provided.  However, the pools code is just that,
an implementation of a particular set of logic for dealing with memory.
It's well proven and is fast/stable and has all the other things we think a
good memory system should have, but it's logic is fixed in the code.  Can we
agree with that?

Now, imagine, and until the code is written you'll have to just imagine,
that there was an sms module that implemented the same logic.  The actual
details don't matter to a degree here because what we're talking about is
the same logic.  The sms module takes a larger block of memory and allocates
smaller blocks from within it, cleaning up when it's no longer needed. All
the locking and cleanups are handled within the sms and we have a number of
ways of getting at the memory.  So, the question becomes what is the
difference between this sms and pools? Do we still need to have the pool
logic (all the tracking, cleanups and so on) living on top of code that
already has all those things?  This is what I refer to as replacing pools -
i.e. I mean replacing the implementation not the logic.

If we had such a module then we could say that

apr_sms_t == apr_pool_t

apr_palloc == apr_sms_malloc
apr_pcalloc == apr_sms_calloc

These could be, as Luke keeps pointing out, in apr_compat.h or in
apr_pools.h or wherever, but they should provide the same abilities we have
at present.  If this is what Greg means by a "slim" wrapper then fair
enough, we have been talking the same language all along, but I always had
the impression he was talking about rewriting the pool code to use sms for
the allocation of memory in the pools.

Now, by having apr_pool_t == apr_sms_t we gain another big advantage - the
ability to vary the sms module to suit the situation using the same API.  If
we choose to simply create a pool (which would become an sms under the
wrapper) then we get the good old pool logic we're used to with all the
cleanups and so on we'd need.  However, should we want a different logic for
our memory we'd create a different type of sms, but it would still be a pool
as far as the code was concerned.  Why would we want different memory?
Well, it might be that we knew the memory would be used in a single thread,
so we wanted an sms that was optimised for speed.

Now, such a thing is a big step and so progress will be slow.  Having pools
logic sit on top of sms, given the above scenario seems to be restrictive to
my mind and limits what we can accomplish using the sms framework.  I mean
what would the pools code become. If they are simply to become wrappers then
we're talking the same thing, but if that's the case then we should over
time start phasing the apr_pool_t out of our code.  Maybe it was me seeing
this in my mind that casued me to jump a bit too far ahead in some of my
postings on this topic.

Hopefully there will be the start of such sn sms module in the codebase
before too long.  the basic aim is that it should implement the same logic
as pools, have the same if not more functionality (cleanups are a big part
of that as people have kept saying) but as it's a rewrite and based on the
sms framework the hope is that it will be faster while keeping or enhancing
the stability that the pools code has.  This is an ambitious target, but IF
we can achieve it (and given how many people we have here to
review/comment/improve the code I think we can), then I don't see why we'd
need the pool code anymore, just some simple typedefs and wrappers.

>
> > > Greg's argument (and I'm leaning that way) says 'pools are now
> > > widely deployed'.

I'd say "the apr_pool_t and associated API are widely deployed and in large
scale usage".  Assuming that whetever we do people can still do

apr_pool_t *pool;
apr_create_pool(pool, NULL);
apr_palloc(pool, sizeof(blah));

then we won't have any problems with supporting these folks.  Of course,
we'd also be able to offer people the ability to do

apr_sms_t *sms;
apr_sms_blah_create(&sms);
apr_sms_malloc(sms, sizeof(blah));

The end result would be identical if blah was the same "standard pool type"
sms we end up with.  The fact is that for a user of APR to get the most
flexability from the new memory system they probably need to use the sms
API.

> >
> > I tend to disagree on that. It is 'only' deployed in httpd and
> > subversion. [not counting apr and apr-util]. If we want to change
> > things like this, now (read this as _after_ the release of
> > httpd-2.0) is the time.
>
> Um. Hello? Have you counted the number of third party modules in
existence?
> Tossing the pools means tossing the basic framework for a couple hundred
> modules. apr_pool_t and apr_p* will remain (effectively) forever.

Correct.  However they don't have to point to the current pool code do they?

>
> It is entirely reasonable to assume that apr_pool_t could be a different
> name for a particular type of apr_sms_t, but there is no effective way to
> eliminate apr_pool_t and its associated functions.

Correct, but this hasn't been our intention.  I think we may have been
talking cross purposes.

When I wrote

apr_pstrdup(apr_sms_t *sms, char *str)

I was assuming that somewhere we'd have (and maybe I should have pointed
this out but it was about 1am ISTR when I wrote it)

#typedef apr_pool_t    apr_sms_t

So

apr_pstrdup(apr_sms_t *sms, char *str) == apr_pstrdup(apr_pool_t *p, char
*str)

The only thing I would say is that internally we should change to using the
apr_sms_t instead of apr_pool_t to emphasise that we are using sms.  Which
form outside users decide to use will either be decided by legacy (they've
got so they'll keep it) or the one they feel more comfortable with.

>
> > > Grow a pool into an sms, don't break anyone's code along the way
> > > (by making the default sms our beloved pool schema) and we are in
> > > strong shape.
> >
> > There is a way to do everything _without_ breaking code along the way.
> > I'll include that in the thingy I had planned for the weekend.

Exactly.

>
> Right. I can see this, and it follows what some of us have been saying:
grow
> the SMS stuff *under* the pool abstraction. Keep pools as the top-level
API.

This is the point I don't see I guess.  We should support the pool API but
we should start to move away from it wherever we can as sms offers more than
pools and keeping to a more restrictive API than we need to doesn't make a
lot of sense to me.

>
> > *) convinced is not the right word here. I'm more looking for open
> >    to change, but that is also not what I'm trying to say. Damn,
> >    it is hard to express yourself sometimes when you kind find the
> >    words... :-)
>
> Don't worry about it. We're all convinced of particular things. But we are
> all, also open to change. If we were obstinate mother fuckers, then we
would
> never have received commit privileges. (that is the hope, at least :-)

It's just the rate of change that varies.  I mean I still think the earth is
flat!

:)

david