You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Max Kanat-Alexander <mk...@bugzilla.org> on 2006/06/30 01:23:16 UTC
[QUESTION] pnotes, Template-Toolkit, and a memory leak
Hi. I'm working on making Bugzilla support mod_perl.
I have a very strange memory leak.
Bugzilla uses the Template Toolkit (TT2).
We're using MP2. In particular, 1.999022 (aka 2.0.0-RC5). I've also
tested the below on 2.0.1, with the same results.
I have httpd-2.0.52-22 from RHEL4 (Red Hat Enterprise Linux 4) for my
Apache. My perl is also 5.8.5, from RHEL4. (The server is a RHEL4
server.)
All my testing was in a single-process prefork server, running as
"httpd -X".
I store a Template object inside of $r->pnotes().
I call ->process() on that Template object, with a particular template,
and throw away the output.
The script ends.
I reload the script, it uses an additional 512K of RAM.
Every time I reload the script, the process uses an *additional* 512K
of RAM, forever, on and on, until my server runs out of memory.
HOWEVER: If I "delete $r->pnotes->{template}" before the script ends,
there's no memory leak.
So clearly, mod_perl's RequestRec cleanup is doing something different
than delete is doing, and mod_perl is missing cleaning up certain
references.
We have a bug on this in our system, you can see it at:
https://bugzilla.mozilla.org/show_bug.cgi?id=343166
Any ideas on how I could fix this in Bugzilla?
-Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla Services. And Everything Else, too.
Re: [QUESTION] pnotes, Template-Toolkit, and a memory leak
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Max Kanat-Alexander wrote:
> But one way or another, this does seem like a bug. That DESTROY handler
> ought to get called.
By saying go out of scope, or holding a reference, he is implying that the reference count is not = 0
ever. When a "object"'s ref count reaches 0, DESTROY is called.
pnotes() is special in that it doesn't create a copy of the variable (depending on how you use it) it just does an
SvREF_inc() on the HV. So when you delete it from the pnotes, you might be trigger more then you think (agreeing with
Johnathna's though that Bugzilla is doing something strange with a pnotes key)
--
------------------------------------------------------------------------
Philip M. Gollucci (pgollucci@p6m7g8.com) 323.219.4708
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Software Engineer - TicketMaster - http://ticketmaster.com
1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F
"In all that I've done wrong I know I must have done something right to
deserve a hug every morning and butterfly kisses at night."
Re: [QUESTION] pnotes, Template-Toolkit, and a memory leak
Posted by Max Kanat-Alexander <mk...@bugzilla.org>.
On Fri, 2006-06-30 at 18:49 -0400, Jonathan Vanasco wrote:
> I think its more likely that the bug is in the way Bugzilla uses TT
> -- a some reference to the template object is getting stored
> persistently ( i think everyone has made a similar mistake ). I've
> never had a problem with a pnote living on after a request, and your
> issue is either the pnote is still around or the object the pnote
> references is linked elsewhere.
Yeah, I suspected that too, but it couldn't be--because the cleanup
happens correctly if I delete that *one* reference, and it doesn't
happen correctly if I don't delete that *one* reference. So it doesn't
have anything to do with anything else holding on to a reference.
> I think the easiest fix, for now, would be to toss in a cleanup
> handler that just undefs the pnote if its defined. if you're only
> using the pnote within a single handler stage, i'd just undef it at
> the end of the handler after all the output is generated. i do tons
> of 'just in case' cleanup things.
Okay. Yeah, that's probably what I'll do, is just call delete on all
the pnote keys.
But one way or another, this does seem like a bug. That DESTROY handler
ought to get called.
-Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla Services. And Everything Else, too.
Re: [QUESTION] pnotes, Template-Toolkit, and a memory leak
Posted by Jonathan Vanasco <mo...@2xlp.com>.
On Jun 30, 2006, at 6:20 PM, Max Kanat-Alexander wrote:
> On Fri, 2006-06-30 at 00:37 -0400, Perrin Harkins wrote:
>>> HOWEVER: If I "delete $r->pnotes->{template}" before the script
>>> ends,
>>> there's no memory leak.
>>
>> It sounds like a problem with the DESTROY method in
>> Template::Provider.
>> Can you add some logging to that method to see if it gets called?
>
> Hey Perrin. You are indeed exactly correct. I tested it, and the
> DESTROY method in Template::Provider never gets called. However,
> when I
> manually do the delete on the hash key, DESTROY does get called.
>
> So, is there anything I can do about this in my code? Would this be a
> bug in mod_perl or in the Template Toolkit?
>
> This is TT 2.15, FWIW.
I had the same inkling as perrin... my guess is that somewhere in the
code someone did something that is not letting the object go out of
scope.
I think its more likely that the bug is in the way Bugzilla uses TT
-- a some reference to the template object is getting stored
persistently ( i think everyone has made a similar mistake ). I've
never had a problem with a pnote living on after a request, and your
issue is either the pnote is still around or the object the pnote
references is linked elsewhere.
I think the easiest fix, for now, would be to toss in a cleanup
handler that just undefs the pnote if its defined. if you're only
using the pnote within a single handler stage, i'd just undef it at
the end of the handler after all the output is generated. i do tons
of 'just in case' cleanup things.
| - - - - - - - - - - - - - - - - - - - -
| RoadSound.com / Indie-Rock.net
| Collaborative Online Management And Syndication Tools
| - - - - - - - - - - - - - - - - - - - -
| - - - - - - - - - - - - - - - - - - - -
| RoadSound.com / Indie-Rock.net
| Collaborative Online Management And Syndication Tools
| - - - - - - - - - - - - - - - - - - - -
Re: [QUESTION] pnotes, Template-Toolkit, and a memory leak
Posted by Max Kanat-Alexander <mk...@bugzilla.org>.
On Fri, 2006-06-30 at 00:37 -0400, Perrin Harkins wrote:
> > HOWEVER: If I "delete $r->pnotes->{template}" before the script ends,
> > there's no memory leak.
>
> It sounds like a problem with the DESTROY method in Template::Provider.
> Can you add some logging to that method to see if it gets called?
Hey Perrin. You are indeed exactly correct. I tested it, and the
DESTROY method in Template::Provider never gets called. However, when I
manually do the delete on the hash key, DESTROY does get called.
So, is there anything I can do about this in my code? Would this be a
bug in mod_perl or in the Template Toolkit?
This is TT 2.15, FWIW.
-Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla Services. And Everything Else, too.
Re: [QUESTION] pnotes, Template-Toolkit, and a memory leak
Posted by Perrin Harkins <pe...@elem.com>.
Max Kanat-Alexander wrote:
> I store a Template object inside of $r->pnotes().
[...]
> Every time I reload the script, the process uses an *additional* 512K
> of RAM, forever, on and on, until my server runs out of memory.
>
> HOWEVER: If I "delete $r->pnotes->{template}" before the script ends,
> there's no memory leak.
It sounds like a problem with the DESTROY method in Template::Provider.
Can you add some logging to that method to see if it gets called?
- Perrin