Declaring large stack variables...

[Brad Nicholes <BN...@novell.com>]

   It appears that we are running into the same problem with Apache 2.0
that we had with Apache 1.3.  The problem is recursive functions that
declare large stack variables such as:

static int read_type_map(apr_file_t **map, negotiation_state *neg,
request_rec *rr)
{
    request_rec *r = neg->r;
    apr_file_t *map_ = NULL;
    apr_status_t status;
    char buffer[MAX_STRING_LEN];           <------------ resolves to
8192
    enum header_state hstate;
    struct var_rec mime_info;
    int has_content;

    if (!map)
        map = &map_;


Now I know that the above function is not directly recursive, but it
could still be called several times within the same code path chewing up
8k each time.  

    Unlike many other platforms, NetWare does not have growable stacks.
 When a thread is started, it is given a stack size.  If that stack size
is exceeded, bad things happen.  Declaring large stack variables aren't
normally a problem except when used in recursive functions.  It doesn't
take many recursive calls with each call allocating an 8k stack variable
before you blow a 64k stack (which is our default for apache).  This
isn't a problem in APR where I can just do things the "NetWare" way, but
HTTPD is a different story.  Is there any way that we can avoid
allocating large stack variables or at least minimize it.  Allocating
buffers as in the above code seems to be a common occurrance in Apache. 
I haven't yet had the time to indentify all of the potential problem
spots in the code, but I will as we get into more stress testing.  Until
then I just wanted to point out this problem and I don't see a good way
to fix it given the HTTPD architecture.

Any ideas?

Brad

Re: Declaring large stack variables...

[Brian Pane <br...@cnet.com>]

[dev@apr.apache.org added to the distribution list because one of
my proposed solutions is an APR enhancement]

Brad Nicholes wrote:

>   It appears that we are running into the same problem with Apache 2.0
>that we had with Apache 1.3.  The problem is recursive functions that
>declare large stack variables such as:
>
>static int read_type_map(apr_file_t **map, negotiation_state *neg,
>request_rec *rr)
>{
>    request_rec *r = neg->r;
>    apr_file_t *map_ = NULL;
>    apr_status_t status;
>    char buffer[MAX_STRING_LEN];           <------------ resolves to
>819
>

That's definitely bad.  I can think of three solutions, given that this
buffer is used to read a line at a time with apr_file_gets():

1. Use a smaller buffer size
2. Allocate the buffer from a pool (but that solution has its own problems,
   because allocating an 8KB block from a pool will often necessitate
   an additional malloc)
3. Create a variant of apr_file_gets() that allocates its own space
   from a pool supplied by the caller.  (It could use a power-of-two
   allocator.)

I think I like the third option the best.  Anybody else have an
opinion on this?

--Brian

Re: Declaring large stack variables...

[Brian Pane <br...@cnet.com>]

[dev@apr.apache.org added to the distribution list because one of
my proposed solutions is an APR enhancement]

Brad Nicholes wrote:

>   It appears that we are running into the same problem with Apache 2.0
>that we had with Apache 1.3.  The problem is recursive functions that
>declare large stack variables such as:
>
>static int read_type_map(apr_file_t **map, negotiation_state *neg,
>request_rec *rr)
>{
>    request_rec *r = neg->r;
>    apr_file_t *map_ = NULL;
>    apr_status_t status;
>    char buffer[MAX_STRING_LEN];           <------------ resolves to
>819
>

That's definitely bad.  I can think of three solutions, given that this
buffer is used to read a line at a time with apr_file_gets():

1. Use a smaller buffer size
2. Allocate the buffer from a pool (but that solution has its own problems,
   because allocating an 8KB block from a pool will often necessitate
   an additional malloc)
3. Create a variant of apr_file_gets() that allocates its own space
   from a pool supplied by the caller.  (It could use a power-of-two
   allocator.)

I think I like the third option the best.  Anybody else have an
opinion on this?

--Brian