A few questions

[Adam Moffett <ad...@plexicomm.net>]

These issues came up when I was trying to address performance problems, 
I hope they aren't major RTFM items.

1) I used sa-compile as suggested by the FAQ and the CPU load dropped 
*dramatically*.  The question is do I have to run that every time I 
sa-update or will it happen automatically?

2) I disabled the auto whitelist module, and got scan times down from 
200+ secs to ~40 secs.  The AWL db file was over 2.5Gig.  The FAQ 
implies that I don't really need AWL, is this the general concensus?  If 
I keep using it, is there an easy automatic way to prune the AWL db for 
old or seldom used entries.

3) I disabled Bayes and now scan times are down to 1 or 2 secs.  That's 
great, but I think bayes really helps so I'd rather keep it.  The 
bayes_toks db is 162MB...that seems like a pretty big db to scan for 
every message.  I know it does auto expire because I have a multitude of 
bayes_toks.expire files ranging from 40-80MB in size.  Can I tune what 
gets expired to reduce the size of the db?  Is there another solution?  
We are definitely I/O bound when bayes is enabled because we have long 
scan times but CPU usage stays in the 8-10% range.

-Adam

Re: A few questions

[Jari Fredriksson <ja...@iki.fi>]

On 10.6.2010 19:10, Adam Moffett wrote:
> These issues came up when I was trying to address performance problems,
> I hope they aren't major RTFM items.
> 
> 1) I used sa-compile as suggested by the FAQ and the CPU load dropped
> *dramatically*.  The question is do I have to run that every time I
> sa-update or will it happen automatically?

Yes, every time.

> 
> 2) I disabled the auto whitelist module, and got scan times down from
> 200+ secs to ~40 secs.  The AWL db file was over 2.5Gig.  The FAQ
> implies that I don't really need AWL, is this the general concensus?  If
> I keep using it, is there an easy automatic way to prune the AWL db for
> old or seldom used entries.
> 

You can add a timestamp into the awl table, if using SQL back end. I
think the description to that is somewhere in SQL howto in wiki, or
someone will post that later...

> 3) I disabled Bayes and now scan times are down to 1 or 2 secs.  That's
> great, but I think bayes really helps so I'd rather keep it.  The
> bayes_toks db is 162MB...that seems like a pretty big db to scan for
> every message.  I know it does auto expire because I have a multitude of
> bayes_toks.expire files ranging from 40-80MB in size.  Can I tune what
> gets expired to reduce the size of the db?  Is there another solution? 
> We are definitely I/O bound when bayes is enabled because we have long
> scan times but CPU usage stays in the 8-10% range.
> 

If you have more than one spamd instance, a separate SQL db would be
good. I use MySQL, while this still is basically a one user system.

-- 
http://www.iki.fi/jarif/
I use PGP. If there is an incompatibility problem with your mail
client, please contact me.

You own a dog, but you can only feed a cat.

Re: A few questions

[Andy Dills <an...@xecu.net>]

On Thu, 10 Jun 2010, Adam Moffett wrote:

> These issues came up when I was trying to address performance problems, I hope
> they aren't major RTFM items.
> 
> 1) I used sa-compile as suggested by the FAQ and the CPU load dropped
> *dramatically*.  The question is do I have to run that every time I sa-update
> or will it happen automatically?

I run this very simple script from cron nightly:

#!/usr/local/bin/perl

$update = '/usr/local/bin/sa-update';
$compile = '/usr/local/bin/sa-compile';
$amavis = '/usr/local/sbin/amavisd reload';

$sysret = system("$update");

if (!$sysret) {
        print "New rules!\n";
        $compret = system("$compile");
        if (!$compret) {
                print "Compiled Correctly!\n";
                system("$amavis");
        }
}


Andy

---
Andy Dills
Xecunet, Inc.
www.xecu.net
301-682-9972
---

Re: A few questions

[RW <rw...@googlemail.com>]

On Thu, 10 Jun 2010 12:10:54 -0400
Adam Moffett <ad...@plexicomm.net> wrote:


>                    I know it does auto expire because I have a
> multitude of bayes_toks.expire files ranging from 40-80MB in size.
> Can I tune what gets expired to reduce the size of the db? 

Expire it from cron and turn-off autoexpire. The extra files are a
sign that autoexpire is not working properly - it's killed by a timeout
before it completes