Re: Apache.xs patch for get_client_block

[Doug MacEachern <do...@covalent.net>]

On 31 Aug 2000 joe@sunstarsys.com wrote:

> Doug,
> 
> Sorry to belabor a dull issue, but I'm not sure I'm getting my point across.

no problem, this is important stuff to understand.
 
> Most of the troubles I've run across in the mod_perl and libapreq code
> have to do with attempts to allocate memory resources for buffers 
> at runtime.  That's exactly what the BUFF API does, right?

not really, BUFF uses some stack allocation, it's generally up to the
caller to manage i/o buffers.
 
> What I'm seeing is that you are also resizing/reallocating these buffers
> each time the code is called. For instance, the original 
> get_client_block had this line in it:
> 
> > -    buffer = (char*)palloc(r->pool, bufsiz);
> 
> The one you just sent me has this line instead:
> 
> > +    SvGROW(buffer, bufsiz+1);
> 
> I'm not sure what SvGROW does, but if it involves a malloc
> without freeing the old buffer, you've got the same problem as before.  
> Each time the perl version of get_client_block is called, 
> SvGROW will work it's majic on RAM, and perl won't give that RAM
> back to the OS. 

but that "problem" still existed with your patch in this line:
	sv_setpvn((SV*)ST(1), buffer, nrd);

same thing happens underneath (see sv.c).  we can't just point Perl
variables at buffers allocated on the stack.  SvGROW() will only malloc()
(or realloc) if the buffer isn't already large enough to the given length.
Perl hangs onto that allocation as an optimization, so it only has to
malloc() once for a given variable, and maybe realloc() if a later string
assignment is larger.  consider this example:

sub foo {
   ...
    my $buff;
    while (my $len = $r->get_client_block($buff, 1024)) {
        print "$buff\n";
    }
   ...
}

the SvGROW() underneath will only trigger malloc() for $buff _once_, for
the entire lifetime of the process.
Perl will only release the string buffer allocation if you explicity
undef() the variable.  this is true for all variables, including those
scoped with my().  clearly, you would not want to undef($buff) inside the
while loop, otherwise malloc() will be called each time through the loop.
outside the loop would not be so bad, then malloc() will only happen for
$buff each time sub foo is called.  if it's called often, like for every
request, it might be better to let Perl hang onto the allocation, that's
up to you.

now, what happens when that buffer is released depends on your Perl and
perhaps os.  if your Perl is built with Perl's malloc, the free()
triggered by undef() will give the memory back to Perl's pool of memory.
otherwise, free() will probably resolve to libc's free().  on some
platforms that means it is "given back to the os", on others it just means
that chunk of memory is available for reuse in that process the next time
somebody malloc's.

Re: Apache.xs patch for get_client_block

[jo...@sunstarsys.com]

Doug,

The get_client_block part of the patch works just fine - haven't had a chance
to confirm the rest of it.

Thanks again.

-- 
Joe Schaefer
joe@sunstarsys.com

SunStar Systems, Inc.

Re: Apache.xs patch for get_client_block

[Doug MacEachern <do...@covalent.net>]

On 31 Aug 2000 joe@sunstarsys.com wrote:
 
> Btw: I've been playing with the keep-alive stuff you left lying around
> in Connection.xs.  The naive implementation I made seems to work fine,
> once the headers are sent to the client.  Are you planning to add them 
> in the next mod_perl release? 

you mean this?
#  int keepalive;		/* Are we using HTTP Keep-Alive? */
#  int keptalive;		/* Did we use HTTP Keep-Alive? */
#  int keepalives;		/* How many times have we used it? */

i have no plans for adding hooks to those unless they're useful in
someway.  have you found a use?

Re: Apache.xs patch for get_client_block

[jo...@sunstarsys.com]

Doug,

Thanks for clearing this up for me - great explanation!  
I'll let you know how your patch works out.

Btw: I've been playing with the keep-alive stuff you left lying around
in Connection.xs.  The naive implementation I made seems to work fine,
once the headers are sent to the client.  Are you planning to add them 
in the next mod_perl release? 

-- 
Joe Schaefer
joe@sunstarsys.com

SunStar Systems, Inc.