You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by snacktime <sn...@gmail.com> on 2005/07/20 18:04:21 UTC
memory leak with perl5.8/freebsd/mod_perl 1.29
Before I spend a whole lot of time on this I'm curious if someone else
has run across this. This is mod perl 1.29.
We upgraded perl on one of our freebsd 5.4 servers from 5.6.1 to
5.8.7. This included reinstalling around 50 perl modules (some to new
versions) from the freebsd ports. Ever since the upgrade, apache
increases in memory by about a third of it's original size whenever by
it restarts via SIGHUP.
Does this ring a bell for anyone, or do I start debugging from scratch?
Chris
Re: memory leak with perl5.8/freebsd/mod_perl 1.29
Posted by Stas Bekman <st...@stason.org>.
snacktime wrote:
> Before I spend a whole lot of time on this I'm curious if someone else
> has run across this. This is mod perl 1.29.
>
> We upgraded perl on one of our freebsd 5.4 servers from 5.6.1 to
> 5.8.7. This included reinstalling around 50 perl modules (some to new
> versions) from the freebsd ports. Ever since the upgrade, apache
> increases in memory by about a third of it's original size whenever by
> it restarts via SIGHUP.
>
> Does this ring a bell for anyone, or do I start debugging from scratch?
http://perl.apache.org/docs/1.0/guide/troubleshooting.html#httpd_keeps_on_growing_after_each_restart
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: memory leak with perl5.8/freebsd/mod_perl 1.29
Posted by snacktime <sn...@gmail.com>.
> > rev 1.61
>
> "Switch to perlmalloc by
> default, unless threaded perl is built, to improve performance."
>
> > rev 1.49
>
> "WITH_PERL_MALLOC - to compile with perl's own malloc, as opposed to
> the freebsd system malloc. Some might find this useful, since perl's
> malloc is marginally faster in typical cases, and orders of
> magnitude faster in degenerate cases. Use with care."
>
> Guess this does show the main reason for this is a performance problem with
> Perl's usage of FreeBSD's malloc(). But I'd be curious to see if this
> performance problem is still a problem. And if it still is a problem, maybe
> there is something Perl isn't doing quite right.
>
> In any case, getting to the bottom of this issue sure sounds interesting.
>
> In the meantime, FreeBSD'ers that have growing problems on FreeBSD should
> consider building their ports Perl with WITH_PERL_MALLOC=no
>
That's a requirement btw if you are using perl 5.6.x on Freebsd and
build mod perl from ports. The default build of perl 5.6.x on freebsd
is incompatible with the mod perl port because the perl malloc causes
a segfault when mod perl is built as a dso. The mod perl port gives a
warning about it though.
Chris
freebsd + threads + httpd(worker)/mod_perl2
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Speaking of FreeBSD,
I confirm the following combo passes 100% of the test suites and the
ModPerl::Registry ones too.
FreeBSD 5.4-RELEASE
gcc 3.4.2
perl 5.8.7 WITH ithreads
httpd 2.0.54 apr IS THREADED (worker mpm)
mod_perl2 svn trunk
mod_apreq2 2.0.6 (just released)
As I didn't use the ports for this, I'll list the exact configure lines
****************************************************
before you start, edit your /etc/libmap.conf:
pgollucci@smgellar /etc rv=0 16 >cat libmap.conf
libc_r.so.5 libpthread.so.1
libc_r.so libpthread.so
on 6-current change this appropriately
*****************************************************
perl:
./Configure -sde \
-Dprefix=$PREFIX/perl \
-Darchlib=$PREFIX/perl/lib \
-Dsitearch=$PREFIX/perl/lib \
-Dsitelib=$PREFIX/perl/lib \
-Dprivlib=$PREFIX/perl/lib \
-Dman3dir=$PREFIX/perl/man/man3 \
-Dman1dir=$PREFIX/perl/man/man1 \
-Dsiteman3dir=$PREFIX/perl/man/man3 \
-Dsiteman1dir=$PREFIX/perl/man/man1 \
-Dscriptdir=$PREFIX/perl/bin \
-Uinstallusrbinperl \
-Dcc=cc \
-Doptimize="-ggdb3 -O0" \
-Duseshrplib \
-Dusedevel \
-Dusethreads=y \
-Dusemymalloc=y \ **** note this one well ****
-Duse64bitint \
-Dccflags=-pipe \
-Ud_dosuid \
-Ui_gdbm \
-Ui_malloc \
-Ui_iconv \
-Ubincompat5005
httpd:
CFLAGS="-ggdb3 -O0" \
./configure --prefix=$PREFIX/httpd \
--with-perl=$PREFIX/perl/bin/perl \
--enable-v4-mapped \
--with-dbm=sdbm \
--with-ssl=/usr \
--enable-ssl \
--enable-so \
--with-port=80 \
--with-mpm=worker \
--with-expat=/usr/local \
--enable-maintainer-mode \
--enable-threads \
--enable-debug \
--enable-mods-shared=all \
--enable-modules=all \
--enable-deflate=shared \
--enable-proxy=shared \
--enable-threads \
--disable-log-forensic
mod_perl2:
$PREFIX/perl/bin/perl Makefile.PL \
MP_APXS=$PREFIX/httpd/bin/apxs \
MP_MAINTAINER=1 \
MP_DEBUG=1 \
MP_TRACE=1
libapreq2:
./configure \
--enable-perl-glue \
--enable-maintainer-mode \
--with-perl=$PREFIX/perl/bin/perl \
--with-apache2-apxs=$PREFIX/httpd/bin/apxs \
--with-expat=/usr/local
(don't forget to use gmake for libapreq2)
--
END
------------------------------------------------------------
What doesn't kill us can only make us stronger.
Nothing is impossible.
Philip M. Gollucci (pgollucci@p6m7g8.com) 301.254.5198
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Developer / Liquidity Services, Inc.
http://www.liquidityservicesinc.com
Re: memory leak with perl5.8/freebsd/mod_perl 1.29
Posted by "Philippe M. Chiasson" <go...@ectoplasm.org>.
Philip M. Gollucci wrote:
> Philippe M. Chiasson wrote:
>> Philip M. Gollucci wrote:
>>> Philippe M. Chiasson wrote:
>>>
>> Poking ports@freebsd.org about this issue could possibly help,
>> especially if
>> the reason this was done can be shown not to apply anymore.
>>
> http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/perl5.8/Makefile
>
> rev 1.61
"Switch to perlmalloc by
default, unless threaded perl is built, to improve performance."
> rev 1.49
"WITH_PERL_MALLOC - to compile with perl's own malloc, as opposed to
the freebsd system malloc. Some might find this useful, since perl's
malloc is marginally faster in typical cases, and orders of
magnitude faster in degenerate cases. Use with care."
Guess this does show the main reason for this is a performance problem with
Perl's usage of FreeBSD's malloc(). But I'd be curious to see if this
performance problem is still a problem. And if it still is a problem, maybe
there is something Perl isn't doing quite right.
In any case, getting to the bottom of this issue sure sounds interesting.
In the meantime, FreeBSD'ers that have growing problems on FreeBSD should
consider building their ports Perl with WITH_PERL_MALLOC=no
--------------------------------------------------------------------------------
Philippe M. Chiasson m/gozer\@(apache|cpan|ectoplasm)\.org/ GPG KeyID : 88C3A5A5
http://gozer.ectoplasm.org/ F9BF E0C2 480E 7680 1AE5 3631 CB32 A107 88C3A5A5
Re: memory leak with perl5.8/freebsd/mod_perl 1.29
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Philippe M. Chiasson wrote:
>Philip M. Gollucci wrote:
>
>
>>Philippe M. Chiasson wrote:
>>
>>
>Poking ports@freebsd.org about this issue could possibly help, especially if
>the reason this was done can be shown not to apply anymore.
>
>
http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/perl5.8/Makefile
rev 1.61
rev 1.49
rev 1.40
END
---------------------------------------------------------
What doesn't kill us can only make us stronger.
Nothing is impossible.
Philip M. Gollucci (pgollucci@p6m7g8.com) 301.254.5198
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Developer / Liquidity Services, Inc.
http://www.liquidityservicesinc.com
http://www.liquidation.com
http://www.uksurplus.com
http://www.govliquidation.com
http://www.gowholesale.com
Re: memory leak with perl5.8/freebsd/mod_perl 1.29
Posted by "Philippe M. Chiasson" <go...@ectoplasm.org>.
Philip M. Gollucci wrote:
> Philippe M. Chiasson wrote:
>
>> I've encouontered this problem before on FreeBSD and have had some
>> success
>> working around it. One of the main problems seems to be with FreeBSD's
>> Perl
>> build picking Perl's malloc instead of the native FreeBSD malloc().
>> You can see
>> how your perl is configured like this:
>>
>> $> perl -V:usemymalloc
>> usemymalloc='y';
>>
> The default /usr/port/lang/perl5.8
> does build perl with this flag. At least on 5.4 and 6.0-current.
>
> pgollucci@smgellar /home/pgollucci rv=0 5 >uname
> FreeBSD smgellar 5.4-RELEASE i386
>
> pgollucci@smgellar /home/pgollucci rv=0 4 >/usr/local/bin/perl
> -V:usemymalloc
> usemymalloc='y';
>
> I may be able to nudge ports@ to change add a flag to Makefile.
I remember reading a thread on ports@ about the reason why that flag was
added in the first ( can't find the thread right now ), but I seem to recall
there was some problem with Perl's usage of FreeBSD's malloc() causing some
performance problem. I believe that problem went away at some point but the
workaround remained.
Poking ports@freebsd.org about this issue could possibly help, especially if
the reason this was done can be shown not to apply anymore.
--------------------------------------------------------------------------------
Philippe M. Chiasson m/gozer\@(apache|cpan|ectoplasm)\.org/ GPG KeyID : 88C3A5A5
http://gozer.ectoplasm.org/ F9BF E0C2 480E 7680 1AE5 3631 CB32 A107 88C3A5A5
Re: memory leak with perl5.8/freebsd/mod_perl 1.29
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Philippe M. Chiasson wrote:
>I've encouontered this problem before on FreeBSD and have had some success
>working around it. One of the main problems seems to be with FreeBSD's Perl
>build picking Perl's malloc instead of the native FreeBSD malloc(). You can see
>how your perl is configured like this:
>
>$> perl -V:usemymalloc
>usemymalloc='y';
>
>
The default /usr/port/lang/perl5.8
does build perl with this flag. At least on 5.4 and 6.0-current.
pgollucci@smgellar /home/pgollucci rv=0 5 >uname
FreeBSD smgellar 5.4-RELEASE i386
pgollucci@smgellar /home/pgollucci rv=0 4 >/usr/local/bin/perl
-V:usemymalloc
usemymalloc='y';
I may be able to nudge ports@ to change add a flag to Makefile.
--
END
---------------------------------------------------------
What doesn't kill us can only make us stronger.
Nothing is impossible.
Philip M. Gollucci (pgollucci@p6m7g8.com) 301.254.5198
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Developer / Liquidity Services, Inc.
http://www.liquidityservicesinc.com
http://www.liquidation.com
http://www.uksurplus.com
http://www.govliquidation.com
http://www.gowholesale.com
Re: memory leak with perl5.8/freebsd/mod_perl 1.29
Posted by "Philippe M. Chiasson" <go...@ectoplasm.org>.
snacktime wrote:
> Before I spend a whole lot of time on this I'm curious if someone else
> has run across this. This is mod perl 1.29.
>
> We upgraded perl on one of our freebsd 5.4 servers from 5.6.1 to
> 5.8.7. This included reinstalling around 50 perl modules (some to new
> versions) from the freebsd ports. Ever since the upgrade, apache
> increases in memory by about a third of it's original size whenever by
> it restarts via SIGHUP.
>
> Does this ring a bell for anyone, or do I start debugging from scratch?
I've encouontered this problem before on FreeBSD and have had some success
working around it. One of the main problems seems to be with FreeBSD's Perl
build picking Perl's malloc instead of the native FreeBSD malloc(). You can see
how your perl is configured like this:
$> perl -V:usemymalloc
usemymalloc='y';
And if it's the case, my recommendation would be to rebuild your perl with the
system malloc() and see if this problem goes away.
--------------------------------------------------------------------------------
Philippe M. Chiasson m/gozer\@(apache|cpan|ectoplasm)\.org/ GPG KeyID : 88C3A5A5
http://gozer.ectoplasm.org/ F9BF E0C2 480E 7680 1AE5 3631 CB32 A107 88C3A5A5