You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Antonios Christofides <A....@hydro.ntua.gr> on 2001/05/28 18:12:36 UTC
Memory leak
Hi all. I have a huge memory leak with mod_perl.
I'm using a RedHat 6.2 Linux 2.2.14 system with Perl 5.00503, Apache
1.3.14, and mod_perl 1.23. Both Apache and mod_perl come from RedHat's
packages, and mod_perl runs as DSO.
My script uses CGI, DBI, Apache::DBI, and HTML::Template. No errors or
warnings (I have 'use strict' and '-w'). Among other things, the
script offers database search facilities on the web. If a search is
performed which results in many (namely 400) rows being returned, then
the httpd child that serves the request grows by 2 MB. Have a child
serve that request ten times, and its size will grow from an initial 8
MB to 28 MB. Another ten times and it will go to around 50 MB.
I understand that my script may be expensive in memory, but it
should reuse the space, shouldn't it? What can I have done so wrong?
Or do you think it could be mod_perl leaking (I've heard that it has a
leak when DSO)?
Solved: Memory leak
Posted by Antonios Christofides <A....@hydro.ntua.gr>.
In case anyone is interested: the memory leak I described a week ago
turns out to be a problem of the combination of HTML::Template with
Perl 5.005_03 (probably a bug in Perl 5.005_03 memory management
triggered by HTML::Template). I upgraded to Perl 5.6.1 and now not a
single byte leaks.
My original exposition of the problem:
> Hi all. I have a huge memory leak with mod_perl.
>
> I'm using a RedHat 6.2 Linux 2.2.14 system with Perl 5.00503, Apache
> 1.3.14, and mod_perl 1.23. Both Apache and mod_perl come from RedHat's
> packages, and mod_perl runs as DSO.
>
> My script uses CGI, DBI, Apache::DBI, and HTML::Template. No errors or
> warnings (I have 'use strict' and '-w'). Among other things, the
> script offers database search facilities on the web. If a search is
> performed which results in many (namely 400) rows being returned, then
> the httpd child that serves the request grows by 2 MB. Have a child
> serve that request ten times, and its size will grow from an initial 8
> MB to 28 MB. Another ten times and it will go to around 50 MB.
>
> I understand that my script may be expensive in memory, but it
> should reuse the space, shouldn't it? What can I have done so wrong?
> Or do you think it could be mod_perl leaking (I've heard that it has a
> leak when DSO)?
Re: Memory leak
Posted by Steven Lembark <le...@wrkhors.com>.
> > script offers database search facilities on the web. If a search is
> > performed which results in many (namely 400) rows being returned, then
> > the httpd child that serves the request grows by 2 MB. Have a child
> > serve that request ten times, and its size will grow from an initial 8
> > MB to 28 MB. Another ten times and it will go to around 50 MB.
Depending on how the return is processed this looks more like
an unused reference left behind. You might want to try, e.g.,
calling the handler from a harness and see where the memory is
going in the debugger.
It also depends on how the rows are being retrieved. If you
are in a while( nextrow ) loop vs. a for( fetchall_arrayref )
type the memory use gets pretty large.
Depending on the size of your queries this could also just be
perl's buffering the input. Remember that perl doesn't free
the space it uses but keeps it internally for later use. If you
are running large enough queries over time the memory use will
track the high-water mark of the rows returned (which obviuosly
pertaint to fetchall-type returns). If the queries get large
enough you can suck up lotsaMB.
--
Steven Lembark 2930 W. Palmer St.
Chicago, IL 60647
lembark@wrkhors.com 800-762-1582
Re: AW: Memory leak
Posted by Stas Bekman <st...@stason.org>.
On Mon, 28 May 2001, Matthias Hanns wrote:
> Hi,
>
> I had the same prob and lowering maxrequestsperchild helped.
>
> Are there possibilities to clean up the used mem on end of the
> mod_perl-script manually? Or have I always set maxrequestsperchild to lower
> values? If it?s possible to do it by myself, what means the overhead of
> creating a new apache-thread in opposite of it in the manual clean-up?
http://perl.apache.org/guide/performance.html#Limiting_the_Size_of_the_Process
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
AW: Memory leak
Posted by Matthias Hanns <mh...@ncsc.de>.
Hi,
I had the same prob and lowering maxrequestsperchild helped.
Are there possibilities to clean up the used mem on end of the
mod_perl-script manually? Or have I always set maxrequestsperchild to lower
values? If it?s possible to do it by myself, what means the overhead of
creating a new apache-thread in opposite of it in the manual clean-up?
Matthias
-----Ursprungliche Nachricht-----
Von: Ged Haywood [mailto:ged@www2.jubileegroup.co.uk]
Gesendet: Montag, 28. Mai 2001 20:08
An: Antonios Christofides
Cc: modperl@apache.org
Betreff: Re: Memory leak
Hi there,
On Mon, 28 May 2001, Antonios Christofides wrote:
> script offers database search facilities on the web. If a search is
> performed which results in many (namely 400) rows being returned, then
> the httpd child that serves the request grows by 2 MB. Have a child
> serve that request ten times, and its size will grow from an initial 8
> MB to 28 MB. Another ten times and it will go to around 50 MB.
If a row is around 5kbytes then I don't think it's a Perl/mod_perl
problem, I think you're keeping your variables hanging around in memory.
Don't forget that when you use mod_perl the interpreter doesn't exit,
so a lot of memory cleanup which happens at exit doesn't happen under
mod_perl. What happens if you set maxrequestsperchild to ten?
73,
Ged.
Re: Memory leak
Posted by Ged Haywood <ge...@www2.jubileegroup.co.uk>.
Hi there,
On Mon, 28 May 2001, Antonios Christofides wrote:
> script offers database search facilities on the web. If a search is
> performed which results in many (namely 400) rows being returned, then
> the httpd child that serves the request grows by 2 MB. Have a child
> serve that request ten times, and its size will grow from an initial 8
> MB to 28 MB. Another ten times and it will go to around 50 MB.
If a row is around 5kbytes then I don't think it's a Perl/mod_perl
problem, I think you're keeping your variables hanging around in memory.
Don't forget that when you use mod_perl the interpreter doesn't exit,
so a lot of memory cleanup which happens at exit doesn't happen under
mod_perl. What happens if you set maxrequestsperchild to ten?
73,
Ged.