stackable memory system (sms)

[Sander Striker <s....@striker.nl>]

hi this is luke, sitting at sander's.

we've looked over the apr pool memory system: we wanted to make
it one of several stackable memory systems.  we think /note it
works like this:

1) the memory allocation inside apr pool uses a static malloc function
(malloc_block - or more specifically new_block), free
and free-block list (block_free_list).

all pools and all sub-pools use the same malloc and free functions
and the same free-block-list.

all "accounting" procedures use the same malloc and free functions.

2) apr pool is like talloc but with clean-up systems, free-block
list reuse/optimisation and thread-safety, basically.

3) sub-pools are effectively... allocated from the samba global
memory scope.  the management or "apparent" sub-pool nature is
done through a separate list structure, which makes a pool
"responsible" for destroying some other pools when it too is
destroyed.


we got very confused by 3) when thinking that sub-pool really
IS sub-pooling!  i.e. you use the parent pool to allocate
memory _for_ the sub-pools [palloc(parent_pool, sizeof(pool))],
_not_ new_block(sizeof(pool)).


so, we were thinking, hey, simple job, right?  when creating
a sub-pool, use palloc(parent_block) instead of new_block()?

wrong.  apr_join_pool makes that... well... almost impossible.

pool1 = apr_make_pool()
pool2 = apr_make_pool()
sub_pool1 = apr_make_sub_pool(pool1)
apr_pool_join(pool2, sub_pool1)

looks fine, right?

now what if you [conceptually!] do this:

pool1 = apr_sms_make_pool(SMS_GPG_LOCKED_MEMORY_ALLOCATOR)
pool2 = apr_sms_make_pool(SMS_DEFAULT_APR_POOL_MEMORY_ALLOCATOR)
sub_pool1 = apr_make_sub_pool(pool1)
apr_pool_join(pool2, sub_pool1)

... whoops!

... *bing*  sander's just pointed out that apr_pool_join
already does a parent-check.  so, in the first example,
an abort() will be thrown because sub_pool1 is not a
child of pool2.

not only is it a parent check, it's a _direct_ parent
check.  so joining to a grand-parent will also abort().

... hang on: apr_pool_join is only used in APR_POOL_DEBUG mode.

okay, this is getting weird.  okay. grep -r "pool_join" http2.0
shows only three places where it's used:
modules/filters/mod_include.c:
	apr_pool_join(r->main->pool, r->pool);
modules/mappers/mod_dir.c:
	apr_pool_join(r->pool, rr->pool);
modules/mappers/mod_negotiation.c:
	apr_pool_join(r->pool, sub_req->pool);

header file shows that in release mode (ndef APR_POOL_DEBUG),
apr_pool_join is defined to do nothing.

aaarrgh!  :)

so.

the comment in the source file, about the sub-pool being
guaranteed not to be destroyed before its parent is, is
completely invalid?

... would anyone mind if i made a suggestion that apr_pool_join
is removed?

1) the guarantees are not fulfilled at release time, unless
by guarantees you mean that the registered cleanup functions
are called before the destroy.

2) it's very limited (direct parent only)

3) it's very limiting (makes turning apr_pool into an instance
of a stackable memory system quite hard to do)

4) it's only used in three places in httpd

5) it's only _relevant_ in those three places in compile-time
debug mode!  and when you look at 1), above, then well...
using apr_pool_join is really dangerous, as bugs present
in release-mode can *disappear* in debug mode!


once apr_pool_join goes, we can turn all of the "statics" etc.:

- malloc() (not block_malloc() because that _uses_ malloc())
- global_block_list
- block_freelist
- free() (not free_block_list() because that _uses_ free())

into private-members of an apr_sms_pool_t instance [btw
i'm still thinking out loud, here :)]

then, add to apr_pool_a pointer to its parent SMS.

then, an extra function is provided that allows for the
creation of a pool using a specific SMS, with the
default being the malloc/free one - the system's SMS
(awlk!  don't expand that out from its abbreviation,
it'll only get _really_ confusing!)

then, you're pretty much done!

apr_pool_join has to diiie :)

luke

 ----- Luke Kenneth Casson Leighton <lk...@samba-tng.org> -----

"i want a world of dreams, run by near-sighted visionaries"
"good.  that's them sorted out.  now, on _this_ world..."