Very long scan times - Finding the culprit rule

["Paul J. Smith" <pj...@mtgsy.net>]

 Hi,

We are currently seeing scan times of 60-90 seconds on a P4 3Ghz box
after adding some new rules emporium rules to try to increase the
effectiveness of spamassassin.

Is there a way to list the timing for each test rather that the total
scan time so I can see which parts are taking significant time and drop
them?

Thanks.

Re: Very long scan times - Finding the culprit rule

[Matt Kettler <mk...@comcast.net>]

At 04:12 AM 8/15/2005, Paul J. Smith wrote:
>We are currently seeing scan times of 60-90 seconds on a P4 3Ghz box
>after adding some new rules emporium rules to try to increase the
>effectiveness of spamassassin.
>
>Is there a way to list the timing for each test rather that the total
>scan time so I can see which parts are taking significant time and drop
>them?


No, but you can narrow it down a bit.

If you run a message through spamassassin -D you can see it tell you when 
it is running header rules, body rules, etc. You can use this information 
to a least know what type of rule you're looking for.

Then you can start pulling SARE files out one at a time till the scan time 
drops.

I would also double-check your memory footprint. If your spamds are all 
really large (>60m) then you should look at the SARE rulesets and see which 
one is large on disk and remove it. In general be a little warry of a rules 
file that's over 256k or so. (bigevil in particular)

Re: Very long scan times - Finding the culprit rule

[jdow <jd...@earthlink.net>]

Candidate rules right off the bat are DNS based if you are seeing
long delays. You probably have a half dozen or more DNS based rules
setup and DNS is not working.

{^_^}
----- Original Message ----- 
From: "Paul J. Smith" <pj...@mtgsy.net>


Hi,

We are currently seeing scan times of 60-90 seconds on a P4 3Ghz box
after adding some new rules emporium rules to try to increase the
effectiveness of spamassassin.

Is there a way to list the timing for each test rather that the total
scan time so I can see which parts are taking significant time and drop
them?

Thanks.

Re: Very long scan times - Finding the culprit rule

[Loren Wilton <lw...@earthlink.net>]

You can run DProf manually on SA and see what it says about rule timings.
Or at least you are supposed to be able to; the last time I tried it I
couldn't get it to work.

However, there may be a simpler answer.  You didn't mention the amount of
ram you have nor the number of children you are running.

My bet is that the extra rules have increased the size of your spamd
children, and you have enough of them that you are now thrashing.

If you added every SARE rule file (or at least every one you are supposed to
add for 3.x) then you have probably doubled the size of your spamd children.
This means that they have gone from 30-40MB each to probably 40-60MB each,
or possibly a tad larger.  If you had 5 children and 512MB, you probably
broke the bank.

Answer would be to FIRST make sure that you only have the rules files that
you should have for whichever version of SA you are running.  (Far too many
people just grab everything, seemingly without noticing that some files are
only for certain versions of SA.)

Next, check the child sizes and available memory.  Consider cutting back on
some rules files, or the number of children; or adding a stick of memory.
:-)

        Loren