Profiling rules with DProf problems

[Cedric Knight <ce...@gn.apc.org>]

Hello

I'm trying to get some performance data on a customised ruleset using
the instructions at
http://wiki.apache.org/spamassassin/ProfilingRulesWithDprof
and have two problems.

Firstly, I'm not actually getting any *_body_test or *_head_test data in
tmon.out.  Instead, after running dprofpp, all the body tests with
priority 0 (amounting to up to 80% of time) appear as a single item
"Mail::SpamAssassin::Plugin::Check::_body_tests_0"

I've tried it with 4000 spam and 1300 ham; and with just 90 or 3 spam
with similar results.  I do get body_test and head_data using just the
3.1 branch from svn, but not with 3.2 or 3.3 (all using Perl 5.10).

Is this method still supposed to work in SA >= 3.2 ?   Or is there
something else that can be done to give each rule its own subroutine?

Secondly, and more minor, on an existing installation running 3.2.5
(Debian lenny), if I load the "zoom" plugin with
  loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody
through my usual config "-c /etc/spamassassin"

I get:
plugin: eval failed: panic: Devel::DProf inconsistent subroutine return
at
/home/mimo/spamassassin-3.2/masses/../lib/Mail/SpamAssassin/Plugin/Rule2XSBody.pm
line 87.

Thanks for any help.

CK

Re: Profiling rules with DProf problems

[Mark Martinec <Ma...@ijs.si>]

On Sunday 24 October 2010 16:09:08 Cedric Knight wrote:
> I'm trying to get some performance data on a customised ruleset using
> the instructions at
> http://wiki.apache.org/spamassassin/ProfilingRulesWithDprof
> and have two problems.
> 
> Firstly, I'm not actually getting any *_body_test or *_head_test data in
> tmon.out.  Instead, after running dprofpp, all the body tests with
> priority 0 (amounting to up to 80% of time) appear as a single item
> "Mail::SpamAssassin::Plugin::Check::_body_tests_0"
> 
> I've tried it with 4000 spam and 1300 ham; and with just 90 or 3 spam
> with similar results.  I do get body_test and head_data using just the
> 3.1 branch from svn, but not with 3.2 or 3.3 (all using Perl 5.10).

The rules are no longer wrapped as individual subroutines, so the
Devel::DProf approach in that wiki page can no longer supply the
fine-grained (per-rule) detail. The change was made with 3.2 I believe
and brought some speedup, at the expense of fine granularity.

> Is this method still supposed to work in SA >= 3.2 ?   Or is there
> something else that can be done to give each rule its own subroutine?

Yes, there is a plugin HitFreqsRuleTiming.pm that comes with the
distribution, but is not enabled or installed by default.

Please find the HitFreqsRuleTiming.pm file and make it available
somewhere, then let SpamAssassin load it by a 'loadplugin' in one
of your .pre files, such as a local.pre :

loadplugin HitFreqsRuleTiming /usr/local/etc/mail/spamassassin/HitFreqsRuleTiming.pm

At the end of a spamassassin run it should write its report in
a file 'timing.log' in a current directory. Make sure the directory
is writable, or at least that an empty file 'timing.log' exists and
is writable. The report is a list of rules, sorted by their run times.
 
I'm not sure how well this method works with mass-checks, but
it should work there too I suppose.

> Secondly, and more minor, on an existing installation running 3.2.5
> (Debian lenny), if I load the "zoom" plugin with
>   loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody
> through my usual config "-c /etc/spamassassin"
> 
> I get:
> plugin: eval failed: panic: Devel::DProf inconsistent subroutine
> return at
> /home/mimo/spamassassin-3.2/masses/../lib/Mail/SpamAssassin/Plugin/
>   Rule2XSBody.pm line 87.

Fixed in today's trunk (3.4) I believe. Thanks for pointing it out.

  Mark