Re: pool stacking

[Greg Stein <gs...@lyra.org>]

On Fri, Mar 02, 2001 at 09:03:59PM +1100, Luke Kenneth Casson Leighton wrote:
>...
> in xmlvl, i investigated splitting out the pool code into stackable pools.
> sander and i came up with sma_mem_sys as a result.
> 
> imagine that you want to do secure memory stacking, using the gpg
> memory-locking method, but you don't want to have to rewrite entire
> sections of apr and apache to do it.
> 
> passing in a sma_mem_sys (which is an array of parent-pool functions)
> which does gpg-style memory-page-locking into apr_pool_creeate(), you can
> do this.
> 
> the "default" parent-pool sma_mem_sys of course should be one that wraps
> malloc and free.
> 
> you can then define sub-pools to use an apache-pool sma_mem_sys, and in
> fact, the only difference then between a sub-pool and the global pool is
> that the sub-pool uses an apr-sma_mem_sys instance whilst the global pool
> uses a malloc/free-sma_mem_sys.
> 
> i'm going over to sander's at the w/e, we'll see if we can thrash it out.

Please let us know your result. We've talked about types of pools before.
One that keeps the current semantics, one that maps straight to malloc/free,
and one that handled shared memory.

I didn't know about a GPG memory type :-) Sounds way cool, and I definitely
bet that we could use that within Apache.


We have had some problems with the pool types, however. Consider that you
pass a function a malloc-based pool. Who calls the free? The function
doesn't know whether it is a standard pool, or a malloc pool. And inserting
a bunch of apr_pool_free(ptr) calls (which are noops for some pools) defeats
the purpose of using pools to avoid carefully tracking free().

For shared memory, I think we were having some issues with grabbing the
right amounts of shared memory, then doing the suballocations. I think this
can be solved, though.


I do like the idea of pool types, and I know others do, too. At least from
me, there is a big +1 on integrating GPG pools into APR.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Re: pool stacking

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Fri, 2 Mar 2001, Elrond wrote:

> 
> I've some further notes about pools.
> 
> The reason, why I asked for refcounting:
> 
> An object might want to live in two pools.

under the current impl. of apr_pool.c, it may be possible to modify it to
do that.

but it's currently not set up like that and i would not recommend getting
apr_pool.c to do it.


> Of course it
> should only die, if both pools do not anymore reference
> that object.
> Of course, the object can do the refcounting itself,
> pointing the pools destroy-fn-ptr to its unref-fn.
> 
> Is there a way to remove a object from a pool, without
> knowing the pool it was added to?

no.
 
if you wanted reference counting, then perhaps...


perhaps we could add in some way to extend sms?  or to provide sms-ref as
a wrapper around sms?

sander?

 ----- Luke Kenneth Casson Leighton <lk...@samba-tng.org> -----

"i want a world of dreams, run by near-sighted visionaries"
"good.  that's them sorted out.  now, on _this_ world..."

Re: pool stacking

[Elrond <el...@samba-tng.org>]

I've some further notes about pools.

The reason, why I asked for refcounting:

An object might want to live in two pools. Of course it
should only die, if both pools do not anymore reference
that object.
Of course, the object can do the refcounting itself,
pointing the pools destroy-fn-ptr to its unref-fn.

Is there a way to remove a object from a pool, without
knowing the pool it was added to?

Pools somewhat resemble, what I once called "rmm -
referencing memory manager", it was somewhat even more
general, apr_pool would have been just another object in
the whole thing and the like... I'm just curious, if most
is there, that I came up with. (it was never implemented,
and I doubt it will ever, nor do I know, if it is actually
necessary)


    Elrond

Re: pool stacking

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

> > i'm going over to sander's at the w/e, we'll see if we can thrash it out.
> 
> Please let us know your result. We've talked about types of pools before.
> One that keeps the current semantics, one that maps straight to malloc/free,
> and one that handled shared memory.
> 
> I didn't know about a GPG memory type :-) Sounds way cool, and I definitely
> bet that we could use that within Apache.

well, it's not so much a GPG memory type as that they have their own
wrappers on memory allocation to make sure that it gets locked (i forget
the system call to do it) and stays in-memory and never swapped to disk.

you don't want your unencrypted private key in a swap file, do you? :)

> We have had some problems with the pool types, however. Consider that you
> pass a function a malloc-based pool. Who calls the free?

ah ha :)

we worked this out.

the sma_mem_sys has pointers to free, destroy, reset, and alloc (which
have corresponding functions - except for free - in apr_pool-ing)

the _users_ of an sma_mem_sys (apache pools, for example), must check if
the parent-sma_mem_sys has either a free or a destroy, and call them if
they exist.



> The function
> doesn't know whether it is a standard pool, or a malloc pool. And inserting
> a bunch of apr_pool_free(ptr) calls (which are noops for some pools) defeats
> the purpose of using pools to avoid carefully tracking free().

...  you still have to call free() in the existing apr_pool code, right???
:) :)

so you call pool->sma_parent->free() instead().

well, actually - implementation of apr_pool_destroy() something like this:

foreach pool alloc'd block:
	if (pool->sma_parent->free != NULL)
		pool->sma_parent->free(pool->sma_parent_data,
		                       pool_alloc'd_block)

followed up by:
if (pool->sma_parent->free_all != NULL)
	pool->sma_parent->free_all(pool->sma_parent_data)

in this way, you cover _both_ cases where the sma_parent instance can
either be capable of doing freeing _or_ is in fact _yet_ another "pool"
allocation system and doesn't have a free at all.

then, you can have an sma_parent which is in fact another apr_pool.

uhm... a little tricky, i know.


> For shared memory, I think we were having some issues with grabbing the
> right amounts of shared memory, then doing the suballocations. I think this
> can be solved, though.

*scared* :)
 
l

 ----- Luke Kenneth Casson Leighton <lk...@samba-tng.org> -----

"i want a world of dreams, run by near-sighted visionaries"
"good.  that's them sorted out.  now, on _this_ world..."