MaxMemFree implementation

[Jean-Jacques Clar <JJ...@novell.com>]

I have been trying to understand why MaxMemFree is not working as 
I am expecting it to in our MPM (NetWare).
I am getting to the conclusion that either the memory allocation in our
MPM
is missing a piece or there is a missing function in the apr_bucket
set.
I think this is relevant to other MPMs.

Each thread is initialized using the following calls:

    apr_allocator_create(&allocator);
    apr_allocator_max_free_set(allocator, ap_max_mem_free);
    apr_pool_create_ex(&ptrans, NULL, NULL, allocator);
    apr_allocator_owner_set(allocator, ptrans);
    apr_pool_tag(ptrans, "transaction");

    bucket_alloc = apr_bucket_alloc_create(pmain);

the transaction pool is then cleared after processing every
connection.

Most of the memory storage allocated is used by buckets while
caching/refreshing large 
files for the memory caching modules (output filter).
Buckets memory is correctly returned using:
apr_brigade_destroy() -> apr_brigade_cleanup() then calling
apr_bucket_delete() 
for each bucket in the brigade.

In apr_bucket_free, because the size of the nodes used are mostly
bigger than SMALL_NODE_SIZE,
apr_allocator_free is called passing as a parameter 
the allocator field from the bucket_alloc structure created using:

    bucket_alloc = apr_bucket_alloc_create(pmain);

struct apr_bucket_alloc_t {
    apr_pool_t *pool;
    apr_allocator_t *allocator;
    node_header_t *freelist;
    apr_memnode_t *blocks;
};

The problem is right there: the allocator field of the
apr_bucket_alloc_t struct is used to
alloc and free buckets. I did not find any way to set the
max_free_index on that field.
Am I missing something?

Using the following new function in apr_buckets_alloc.c, it includes a
size parameter and a call
to apr_allocator_free(), works a lot better for me. 
Using a small MaxMemFree directive,  the memory allocation for APRLIB
is down around 50 MB 
instead of the previous 250MB.

    bucket_alloc = apr_bucket_alloc_create(pmain, ap_max_mem_free);

APU_DECLARE_NONSTD(apr_bucket_alloc_t *)
apr_bucket_alloc_create_set_max_free(apr_pool_t *p, apr_size_t size)
{
    apr_allocator_t *allocator;
    apr_bucket_alloc_t *list;
    apr_memnode_t *block;

    apr_allocator_create(&allocator);
    block = apr_allocator_alloc(allocator, ALLOC_AMT);
    list = (apr_bucket_alloc_t *)block->first_avail;
    list->pool = p;
    list->allocator = allocator;
    list->freelist = NULL;
    list->blocks = block;
    block->first_avail += APR_ALIGN_DEFAULT(sizeof(*list));

    apr_allocator_max_free_set(list->allocator, size);

    apr_pool_cleanup_register(list->pool, list, alloc_cleanup,
                              apr_pool_cleanup_null);

    return list;
}


Am I following a wrong direction or this should be implemented to limit
memory usage
in case like mine?

Thanks,

JJ

Re: MaxMemFree implementation

[Cliff Woolley <jw...@virginia.edu>]

On Mon, 7 Oct 2002, Jean-Jacques Clar wrote:

>
> Am I following a wrong direction or this should be implemented to limit
> memory usage
> in case like mine?


Yes, that's on my list of things to work on soonish.

--Cliff