You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Assen Tchorbadjiev <bo...@tchorbadjiev.com> on 2006/09/29 18:31:17 UTC
possible memory leaks, FreeBSD,mod_perl/1.29
Hi all,
I've read through similar postings to this list before, but none of them
describe what I am seeing, and its not about that `apache restart` is leaking
memory.
Here's the case:
FreeBSD `myhost` 5.4-STABLE.
Apache 1.3.37, perl 5.8.8, mod_perl 1.29 - all built via the ports
(by default mod_perl builds dynamic into the server)
I start the server in single mode and make lets say 30 requests. The process
size at this point seems normal, a bit more than the original startup size,
but this is since I do not preload anything inside the httpd.conf file.
After these 30 (sometimes 40) requests, the process starts to grow by an avg.
1.4K per request , which I think is rather huge, since there are no other
modules than the default loaded by the Apache::* / mod_perl stuff .
I've tried to re-compile Perl (and mod_perl) outside the ports tree - with and
w/o the settings: -Dusemymalloc , and -Dbinmode5005, but none of the combos
seem to fix the problem.
The same thing described above applies to FreeBSD 6.0-STABLE, perl 5.8.7 -
same apache/mod_perl versions, same build procedures.
I guess it could be an external library leaking, but has anybody experienced
this on recent configurations like mine ?
This is the dummy handler package
====================
package My::X;
sub handler {
my $r = shift;
$r->content_type( 'text/html' );
$r->send_http_header();
$r->print( $$ );
200;
}
1;
====== httpd.conf =======
PerlModule My::Dummy
<Location /my>
SetHandler perl-script
PerlHandler My::Dummy
</Location>
Here is the most recent build I've tried (again, via the bsd ports):
Embedded Perl version v5.8.8 for Apache/1.3.37 (Unix) mod_perl/1.29 process
====
osname=freebsd, osvers=5.4-stable, archname=i386-freebsd-64int
[ .. ]
-Duseshrplib -Doptimize=-O -Ui_gdbm -Dusethreads=n -Dusemymalloc=y
-Duse64bitint
[ .. ]
usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
[ .. ]
usemymalloc=y, bincompat5005=undef
========================
Regards,
Assen
--
--------------------------------------------------
Assen Tchorbadjiev
box2tchorbadjiev.com
8D6D 1FCF 96DB F37E E421 1BD2 D04E A856 20A4 AA20 <pgp.mit.edu>
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by Assen Tchorbadjiev <bo...@tchorbadjiev.com>.
Yeah, obviously I was wrong assuming that it should not grow in case there are
only the core modules loaded.
If I put Devel::LeakTrace at the beginning of the Apache module, it gives
several warnings about leaks being found in Apache::Constants::Exports,
Exporter ?!, mod_perl and the Apache module itself.
Anyway, I haven't dug deeper into it yet so I cannot tell anything more.
Regards,
Assen
On Monday 02 October 2006 20:00, Perrin Harkins wrote:
> On Sat, 2006-09-30 at 13:19 +0300, Assen Tchorbadjiev wrote:
> > (top output: SIZE/RES)
> > 6276K 4652K # at startup
> > 6312K 4676K # after first request
> > 7624K 5992K # after 1000 requests
> > 8984K 7356K # after 2000 requests
>
> That's not what I would expect to see on my Linux setup, but it's also
> not bad. You can easily control such slow growth with
> Apache::SizeLimit.
>
> - Perrin
--
---------------------------------------------------------------
Assen Tchorbadjiev
box2tchorbadjiev.com
8D6D 1FCF 96DB F37E E421 1BD2 D04E A856 20A4 AA20 <pgp.mit.edu>
---------------------------------------------------------------
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Perrin Harkins wrote:
> Philip M. Gollucci wrote:
>> What do expect/see on your Linux setup ?
>
> No growth at all, if it's just hitting the same simple handler over and
> over.
Does/Try t/lib/TestCommon/MemoryLeak.pm help ?
--
------------------------------------------------------------------------
Philip M. Gollucci (pgollucci@p6m7g8.com) 323.219.4708
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Software Engineer - TicketMaster - http://ticketmaster.com
1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F
When I call your name, Girl, it starts to flame
Burning in my heart, Tearing it all apart..
No matter how I try My love I cannot hide....
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by Perrin Harkins <pe...@elem.com>.
Philip M. Gollucci wrote:
> What do expect/see on your Linux setup ?
No growth at all, if it's just hitting the same simple handler over and
over.
- Perrin
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Perrin Harkins wrote:
> On Sat, 2006-09-30 at 13:19 +0300, Assen Tchorbadjiev wrote:
>> (top output: SIZE/RES)
>> 6276K 4652K # at startup
>> 6312K 4676K # after first request
>> 7624K 5992K # after 1000 requests
>> 8984K 7356K # after 2000 requests
>
> That's not what I would expect to see on my Linux setup, but it's also
> not bad. You can easily control such slow growth with
> Apache::SizeLimit.
What do expect/see on your Linux setup ?
--
------------------------------------------------------------------------
Philip M. Gollucci (pgollucci@p6m7g8.com) 323.219.4708
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Software Engineer - TicketMaster - http://ticketmaster.com
1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F
When I call your name, Girl, it starts to flame
Burning in my heart, Tearing it all apart..
No matter how I try My love I cannot hide....
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by Perrin Harkins <pe...@elem.com>.
On Sat, 2006-09-30 at 13:19 +0300, Assen Tchorbadjiev wrote:
> (top output: SIZE/RES)
> 6276K 4652K # at startup
> 6312K 4676K # after first request
> 7624K 5992K # after 1000 requests
> 8984K 7356K # after 2000 requests
That's not what I would expect to see on my Linux setup, but it's also
not bad. You can easily control such slow growth with
Apache::SizeLimit.
- Perrin
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by Assen Tchorbadjiev <bo...@tchorbadjiev.com>.
Thanks both to Jonathan and Perrin,
> Jonathan
> That's probably just normal apache growth-- possibly nothing to do
w/o mod_perl loaded the httpd process does not grow a bit even for 3000+
requests
> i'd suggest using:... Devel::GC::Helper ...
Devel::LeakTrace shows a leak right after the return from my handler routine.
If I remove the `$r = shift` [and further $r usage], I does not show anything
in the log. I guess it goes deeper inside the Apache::* stuff.
The procedure I am following is:
1. use Devel::LeakTrace # inside the My::Dummy module
2. make 2000 requests to the url
3. killall -HUP httpd (httpd started in single mode)
4. see the httpd error log
> Perrin
> That's all you have loaded?
yes, only this module. No DBI, nothing.
> This code does have a bug, which is the 200 at the end. ...
You're right, I was trying to get rid of any module use from within my module,
even the Constants package.
> I don't use FreeBSD, but I would second Jonathan's advice about ...
(top output: SIZE/RES)
6276K 4652K # at startup
6312K 4676K # after first request
7624K 5992K # after 1000 requests
8984K 7356K # after 2000 requests
I've removed event the `$r = shift; $r->* stuff`.
The package now looks like:
====
package My::Dummy;
use strict;
use warnings;
sub handler {
print <<EF;
Content-type: text/html
boo?!
EF
return 0;
}
1;
====
.. and the httpd.conf:
====
PerlModule My::Dummy
<Location /mem>
SetHandler perl-script
PerlHandler My::Dummy
PerlSendHeader On
</Location>
====
Assen
--
---------------------------------------------------------------
Assen Tchorbadjiev
box@tchorbadjiev.com
8D6D 1FCF 96DB F37E E421 1BD2 D04E A856 20A4 AA20 <pgp.mit.edu>
---------------------------------------------------------------
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by Perrin Harkins <pe...@elem.com>.
On Fri, 2006-09-29 at 19:31 +0300, Assen Tchorbadjiev wrote:
> This is the dummy handler package
> ====================
> package My::X;
> sub handler {
> my $r = shift;
> $r->content_type( 'text/html' );
> $r->send_http_header();
> $r->print( $$ );
> 200;
> }
> 1;
That's all you have loaded?
This code does have a bug, which is the 200 at the end. You should
return OK from Apache::Const instead. These are not HTTP response
codes.
I don't use FreeBSD, but I would second Jonathan's advice about
measuring size every 100-1000 requests rather than every request. It
can take a while to hit equilibrium because of the way perl allocates
memory.
- Perrin
Re: possible memory leaks, FreeBSD,mod_perl/1.29
Posted by Jonathan <mo...@2xlp.com>.
That's probably just normal apache growth-- possibly nothing to do
with modperl
i'd suggest using:
Devel::GC::Helper
and
Devel::Leak
to check refcounts... see what's going on
There's a lot of crazy ways you can have process growth within mod-
perl that are just perl issues:
a- there's a slight leak in string evals , its fixed in 5.9, but
5.8.8 has it
b- DBI keeps an internal cache of 120 statement handles and
connection handles. so you'll see it grow quite a bit, then reset
later on. if you use dbi (or apache::session mysql/postgres/etc ),
expect it to consume some internal memory for the cache.
there are more, those are just off the top of my head.
i'd suggest measuring the process size between 1 and 1000 requests.
often you'll see stuff taper off as perl caches more data or uses
larger variables. You can also use sizelimit to force a reset, or
use maxrequests to spawn a new child in a sweet-spot around 200-750
requests.