You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by John ORourke <jo...@o-rourke.org> on 2007/05/11 13:36:02 UTC
Apache::DProf missing some subroutines
Hi folks,
Any idea why Apache::DProf would be failing to list some subroutines
that are definitely being called, in the tmon.out file?
I grepped all sub references from tmon.out and it's listing my method
handlers, it's listing some of my constructors, and its listing some
object methods, but there are loads missing. I know they should be
there because some of the listed subs are only called from non-listed ones!
The only things I'm doing which might affect it are:
- a fair bit of subclassing, but that doesn't explain many of the
missing ones
- using Error.pm - could this be interfering?
cheers
John
Re: Apache::DProf missing some subroutines
Posted by Geoffrey Young <ge...@modperlcookbook.org>.
John ORourke wrote:
> Hi folks,
>
> Any idea why Apache::DProf would be failing to list some subroutines
> that are definitely being called, in the tmon.out file?
>
> I grepped all sub references from tmon.out and it's listing my method
> handlers, it's listing some of my constructors, and its listing some
> object methods, but there are loads missing. I know they should be
> there because some of the listed subs are only called from non-listed ones!
>
> The only things I'm doing which might affect it are:
> - a fair bit of subclassing, but that doesn't explain many of the
> missing ones
> - using Error.pm - could this be interfering?
I have no idea, but I'd suggest trying Devel::Profiler, as it tends to
handle mod_perl applications more gracefully. it also has an interface
for skipping over unfriendly packages and subs, which Error.pm may
indeed be.
it also will shortly have a TT plugin for adding [% PROCESS %] timings
to tmon.out, if you find that kind of thing useful ;)
--Geff
Re: Apache::DProf missing some subroutines
Posted by John ORourke <jo...@versatilia.com>.
Perrin Harkins wrote:
> On 5/11/07, John ORourke <jo...@o-rourke.org> wrote:
>> Any idea why Apache::DProf would be failing to list some subroutines
>> that are definitely being called, in the tmon.out file?
>
> Yes. You're probably loading some of your code before initializing
> the debugger. Put something like this in httpd.conf before any
> PerlModule or PerlRequire statements:
> <Perl>
> require Apache::DB;
> Apache::DB->init;
> </Perl>
Brilliant, that did the trick. I was using PerlModule and
PerlLoadModule to load some stuff in during server init, and of course
that was defining things before any children were initialised.
thanks for the various tips folks, I've got something which works now :)
John
Re: Apache::DProf missing some subroutines
Posted by Perrin Harkins <pe...@elem.com>.
On 5/11/07, John ORourke <jo...@o-rourke.org> wrote:
> Any idea why Apache::DProf would be failing to list some subroutines
> that are definitely being called, in the tmon.out file?
Yes. You're probably loading some of your code before initializing
the debugger. Put something like this in httpd.conf before any
PerlModule or PerlRequire statements:
<Perl>
require Apache::DB;
Apache::DB->init;
</Perl>
Note that this is exactly the opposite of the advice you're getting
about Devel::Profiler. DProf has to be loaded before any of the code
you want to profile, not after.
- Perrin
Re: Apache::DProf missing some subroutines
Posted by Geoffrey Young <ge...@modperlcookbook.org>.
Jeff wrote:
> -------- Original Message --------
> Subject: Re:Apache::DProf missing some subroutines
> From: Geoffrey Young <ge...@modperlcookbook.org>
> CC: modperl <mo...@perl.apache.org>
> Date: 11 May 2007 14:11:14
>
>>
>>> Also, Devel::Profiler can't handle many of the DBI implementations (you
>>> have to bad_pkgs=>[] them.
>>
>>
>> I don't have any trouble with core DBI or DBD::mysql or DBD::Oracle.
>> but there are lots of DBI-esque packages out there so I'll believe you.
>> in general Devel::Profiler doesn't like overloaded things.
>>
>
> interesting - DBD::mysql is the one that I reported a problem with a few
> years ago:
>
> http://groups.google.co.uk/group/perl.perl5.porters/browse_thread/thread/29602d15e7586b79/a11948e8b7854a4b
current svn given me this on our codebase:
0.67 0.056 0.056 780 0.0001 0.0001 DBD::mysql::st::fetch
0.65 0.054 0.117 480 0.0001 0.0002 DBD::_::db::prepare_cached
0.46 0.038 0.038 480 0.0001 0.0001 DBI::db::prepare_cached
0.44 0.037 0.037 480 0.0001 0.0001 DBI::st::execute
0.34 0.028 0.028 480 0.0001 0.0001 DBI::st::bind_columns
0.23 0.019 0.019 220 0.0001 0.0001 DBD::mysql::db::ping
0.23 0.019 0.019 220 0.0001 0.0001 DBI::db::do
0.23 0.019 0.019 480 0.0000 0.0000 DBD::mysql::st::execute
0.22 0.018 0.018 220 0.0001 0.0001 DBI::db::ping
0.22 0.018 0.018 700 0.0000 0.0000 DBI::db::FETCH
0.22 0.018 0.018 500 0.0000 0.0000 DBI::st::FETCH
0.22 0.018 0.018 480 0.0000 0.0000 DBD::mysql::st::FETCH
0.22 0.018 0.018 440 0.0000 0.0000 DBD::mysql::st::finish
0.11 0.009 0.009 700 0.0000 0.0000 DBD::mysql::db::FETCH
0.11 0.009 0.009 480 0.0000 0.0000 DBD::_::st::bind_columns
0.11 0.009 0.009 780 0.0000 0.0000 DBI::st::fetch
0.11 0.009 0.009 440 0.0000 0.0000 DBI::st::finish
which looks reasonable enough to me :) you might want to give svn a
try. also, our bad_pkgs looks like
bad_pkgs => [qw( UNIVERSAL Time::HiRes B Carp
Exporter Cwd Config CORE DynaLoader XSLoader
AutoLoader utf8 )],
in case any of those are the ones causing trouble for you.
>
>> you might also try the latest from svn to see if that works better for
>> you:
>> http://sourceforge.net/projects/devel-profiler/
>
>
> I have the last released => v0.04, June 9 2002, but I see that there was
> a not-yet-released change in 2003 for autoloaded. Might just wait until
> an 0.05 is floated - we can always blame the database for performance
> issues in the mean time :)
there's no timeframe for 0.05 at this point, but I suspect svn looks
like 0.05 will with the exception of some additional documentation updates.
HTH
--Geoff
Re: Apache::DProf missing some subroutines
Posted by Jeff <mo...@aquabolt.com>.
-------- Original Message --------
Subject: Re:Apache::DProf missing some subroutines
From: Geoffrey Young <ge...@modperlcookbook.org>
CC: modperl <mo...@perl.apache.org>
Date: 11 May 2007 14:11:14
>
>> Also, Devel::Profiler can't handle many of the DBI implementations (you
>> have to bad_pkgs=>[] them.
>
> I don't have any trouble with core DBI or DBD::mysql or DBD::Oracle.
> but there are lots of DBI-esque packages out there so I'll believe you.
> in general Devel::Profiler doesn't like overloaded things.
>
interesting - DBD::mysql is the one that I reported a problem with a few
years ago:
http://groups.google.co.uk/group/perl.perl5.porters/browse_thread/thread/29602d15e7586b79/a11948e8b7854a4b
> you might also try the latest from svn to see if that works better for you:
> http://sourceforge.net/projects/devel-profiler/
I have the last released => v0.04, June 9 2002, but I see that there was
a not-yet-released change in 2003 for autoloaded. Might just wait until
an 0.05 is floated - we can always blame the database for performance
issues in the mean time :)
Jeff
Re: Apache::DProf missing some subroutines
Posted by Geoffrey Young <ge...@modperlcookbook.org>.
John ORourke wrote:
> Geoffrey Young wrote:
>
>> Jeff wrote:
>>
>>
>>> Not sure if its relevant, but from general Perl usage, Devel::Profiler
>>> often fails to properly recognise function names if its USE statement
>>> ends up being executed before your classes have been USEd.
>>>
>>
>> yes, because Devel::Profiler scans for then during it's import routine.
>> the behavior is slightly different under mod_perl (the scan is delayed)
>>
>
>
> No joy with Devel::Profiler::Apache sadly - I couldn't even get it to
> write anything out. I tried log dir permissions, tried creating the
> 'profiler' directory like you have to do for DProf, I even tried
> updating Profiler/Apache.pm to use Apache2::ServerUtil instead of
> Apache->, and I tried pulling it in with either PerlModule or 'use' -
> the only time it even appeared to load was when I tried it with a 'use'
> statement on the original code, and of course it complained about the
> 'Apache' package not existing.
sorry, I didnt' realize this was mp2. Devel::Profiler in svn has been
ported to mp2 already, so you shouldn't need to worry about that.
as for setup, you can see an exact mod_perl config in the (new) tests
under t/conf in the svn sources. or you can browse them here
http://devel-profiler.cvs.sourceforge.net/devel-profiler/Devel-Profiler/t/conf/extra.conf.in?revision=1.5
http://devel-profiler.cvs.sourceforge.net/devel-profiler/Devel-Profiler/t/conf/modperl_extra.pl?revision=1.3
in short, you need to
use Devel::Profiler::Apache;
from your startup.pl, since PerlModule doesn't actually call import().
see the documentation comments in those two files.
fwiw, I'm sorry it's all in svn - I got interested in Devel::Profiler
after version 0.04, and worked hard for a while to improve things on the
mod_perl side, but it's simply been a lack of tuits on everyone's part
getting 0.05 out.
HTH
--Geoff
Re: Apache::DProf missing some subroutines
Posted by John ORourke <jo...@o-rourke.org>.
Geoffrey Young wrote:
> Jeff wrote:
>
>> Not sure if its relevant, but from general Perl usage, Devel::Profiler
>> often fails to properly recognise function names if its USE statement
>> ends up being executed before your classes have been USEd.
>>
> yes, because Devel::Profiler scans for then during it's import routine.
> the behavior is slightly different under mod_perl (the scan is delayed)
>
No joy with Devel::Profiler::Apache sadly - I couldn't even get it to
write anything out. I tried log dir permissions, tried creating the
'profiler' directory like you have to do for DProf, I even tried
updating Profiler/Apache.pm to use Apache2::ServerUtil instead of
Apache->, and I tried pulling it in with either PerlModule or 'use' -
the only time it even appeared to load was when I tried it with a 'use'
statement on the original code, and of course it complained about the
'Apache' package not existing.
I'll try using DProf again but with a 'use' after all my other stuff is
loaded.
cheers
John
Re: Apache::DProf missing some subroutines
Posted by Geoffrey Young <ge...@modperlcookbook.org>.
Jeff wrote:
>
> Not sure if its relevant, but from general Perl usage, Devel::Profiler
> often fails to properly recognise function names if its USE statement
> ends up being executed before your classes have been USEd.
>
> If you use factories to instantiate objects from calculated class names,
> those methods won't be recognised unless you explicitly USE them (i.e.
> rather than run-time REQUIRE them).
yes, because Devel::Profiler scans for then during it's import routine.
the behavior is slightly different under mod_perl (the scan is delayed)
so it probably doesn't have the same effect - in mod_perl we don't want
to generate things the way you're describing since it prevents us from
maximizing shared memory (you want to use() everything before the
children are forked).
>
> To test if this is your issue, you might find a class with methods not
> being recognised, and add a USE statement for this class just before you
> call in the profiler, and then check the results.
he's not using Devel::Profiler (yet) but maybe Devel::DProf has the same
limitation. it never hurts to try :)
>
> Also, Devel::Profiler can't handle many of the DBI implementations (you
> have to bad_pkgs=>[] them.
I don't have any trouble with core DBI or DBD::mysql or DBD::Oracle.
but there are lots of DBI-esque packages out there so I'll believe you.
in general Devel::Profiler doesn't like overloaded things.
you might also try the latest from svn to see if that works better for you:
http://sourceforge.net/projects/devel-profiler/
HTH
--Geoff
>
> Regards
> Jeff
>
>
> -------- Original Message --------
> Subject: Apache::DProf missing some subroutines
> From: John ORourke <jo...@o-rourke.org>
> To: modperl <mo...@perl.apache.org>
> Date: 11 May 2007 12:36:02
>
>> Hi folks,
>>
>> Any idea why Apache::DProf would be failing to list some subroutines
>> that are definitely being called, in the tmon.out file?
>>
>> I grepped all sub references from tmon.out and it's listing my method
>> handlers, it's listing some of my constructors, and its listing some
>> object methods, but there are loads missing. I know they should be
>> there because some of the listed subs are only called from non-listed
>> ones!
>>
>> The only things I'm doing which might affect it are:
>> - a fair bit of subclassing, but that doesn't explain many of the
>> missing ones
>> - using Error.pm - could this be interfering?
>>
>> cheers
>> John
>>
Re: Apache::DProf missing some subroutines
Posted by Jeff <mo...@aquabolt.com>.
Not sure if its relevant, but from general Perl usage, Devel::Profiler
often fails to properly recognise function names if its USE statement
ends up being executed before your classes have been USEd.
If you use factories to instantiate objects from calculated class names,
those methods won't be recognised unless you explicitly USE them (i.e.
rather than run-time REQUIRE them).
To test if this is your issue, you might find a class with methods not
being recognised, and add a USE statement for this class just before you
call in the profiler, and then check the results.
Also, Devel::Profiler can't handle many of the DBI implementations (you
have to bad_pkgs=>[] them.
Regards
Jeff
-------- Original Message --------
Subject: Apache::DProf missing some subroutines
From: John ORourke <jo...@o-rourke.org>
To: modperl <mo...@perl.apache.org>
Date: 11 May 2007 12:36:02
> Hi folks,
>
> Any idea why Apache::DProf would be failing to list some subroutines
> that are definitely being called, in the tmon.out file?
>
> I grepped all sub references from tmon.out and it's listing my method
> handlers, it's listing some of my constructors, and its listing some
> object methods, but there are loads missing. I know they should be
> there because some of the listed subs are only called from non-listed ones!
>
> The only things I'm doing which might affect it are:
> - a fair bit of subclassing, but that doesn't explain many of the
> missing ones
> - using Error.pm - could this be interfering?
>
> cheers
> John
>