You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Payal Rathod <pa...@scriptkitchen.com> on 2006/09/21 16:36:46 UTC
SA increasing load average a lot and spams getting through
Hi,
A friend of mine is maintaining a mail server for a school and I am
helpig her out. There are just 150-200 accounts and the machine is
having 512 Mb RAM and 1Gb swap running Cent OS with processor being AMD
Athlon. spamd and spamc pair is used with qmail as mailserver and
processing done via .qmail files.
spamd is started as,
/usr/bin/spamd -m 10 -x -u spamd -H /home/spamd
Sometimes when there is heavy load we get in logs,
Cannot open bayes databases /var/bayes/bayes_* R/W: lock failed: File
exists
(Bayes works properly though).
The load average always hovers around 20.0 and top shows spamd is
culprit. Also, I have trained over 10,000 each of spam and ham and also
use all sare rules yet many spams are passing thru' last few days.
Now 2 questions, what is that error for and what more can I do for spam?
The teachers are sick of spam since last few days.
With warm regards,
-Payal
Re: SA increasing load average a lot and spams getting through
Posted by Payal Rathod <pa...@scriptkitchen.com>.
On Fri, Sep 22, 2006 at 10:44:11AM -0700, John Goubeaux wrote:
> you might also want to check:
>
> http://spamassassin.apache.org/full/3.1.x/dist/doc/Mail_SpamAssassin_Conf.html#miscellaneous_options
>
> eg lock_method I had the same bayes files errors and set this to
> lock_method flock and think i have alleviated the problems.
This has really helped a LOT.
With warm regards,
-Payal
Re: SA increasing load average a lot and spams getting through
Posted by John Goubeaux <go...@education.ucsb.edu>.
you might also want to check:
http://spamassassin.apache.org/full/3.1.x/dist/doc/Mail_SpamAssassin_Conf.html#miscellaneous_options
eg lock_method I had the same bayes files errors and set this to
lock_method flock and think i have alleviated the problems.
-john
sokka wrote:
> 1. Increase your RAM
> 2. Increase your child spam process.
>
> regards
>
>
> On 9/21/06, *Payal Rathod* <payal-sa@scriptkitchen.com
> <ma...@scriptkitchen.com>> wrote:
>
> Hi,
> A friend of mine is maintaining a mail server for a school and I am
> helpig her out. There are just 150-200 accounts and the machine is
> having 512 Mb RAM and 1Gb swap running Cent OS with processor
> being AMD
> Athlon. spamd and spamc pair is used with qmail as mailserver and
> processing done via .qmail files.
> spamd is started as,
> /usr/bin/spamd -m 10 -x -u spamd -H /home/spamd
> Sometimes when there is heavy load we get in logs,
> Cannot open bayes databases /var/bayes/bayes_* R/W: lock failed: File
> exists
> (Bayes works properly though).
> The load average always hovers around 20.0 and top shows spamd is
> culprit. Also, I have trained over 10,000 each of spam and ham
> and also
> use all sare rules yet many spams are passing thru' last few days.
>
> Now 2 questions, what is that error for and what more can I do for
> spam?
> The teachers are sick of spam since last few days.
>
> With warm regards,
> -Payal
>
>
>
>
--
John Goubeaux
Systems Administrator
Gevirtz Graduate School of Education
UC Santa Barbara
Phelps Hall 3534
805 893-8190
Re: SA increasing load average a lot and spams getting through
Posted by sokka <so...@gmail.com>.
1. Increase your RAM
2. Increase your child spam process.
regards
On 9/21/06, Payal Rathod <pa...@scriptkitchen.com> wrote:
>
> Hi,
> A friend of mine is maintaining a mail server for a school and I am
> helpig her out. There are just 150-200 accounts and the machine is
> having 512 Mb RAM and 1Gb swap running Cent OS with processor being AMD
> Athlon. spamd and spamc pair is used with qmail as mailserver and
> processing done via .qmail files.
> spamd is started as,
> /usr/bin/spamd -m 10 -x -u spamd -H /home/spamd
> Sometimes when there is heavy load we get in logs,
> Cannot open bayes databases /var/bayes/bayes_* R/W: lock failed: File
> exists
> (Bayes works properly though).
> The load average always hovers around 20.0 and top shows spamd is
> culprit. Also, I have trained over 10,000 each of spam and ham and also
> use all sare rules yet many spams are passing thru' last few days.
>
> Now 2 questions, what is that error for and what more can I do for spam?
> The teachers are sick of spam since last few days.
>
> With warm regards,
> -Payal
>
>
>
>
Re: SA increasing load average a lot and spams getting through
Posted by Matt Kettler <mk...@comcast.net>.
Payal,
Is the machine making substantial use of the swapfile?
You should be able to use "free" or "top" to find out, if there's more
swap in use than free+cache then you're over-loaded. A little swap usage
is normal, as your kernel might push pages that haven't been used in a
long time into the swap to make room for extra disk cache. However, if
there's more swap in use than free+cache you can prove this isn't the case.
On a box with only 512mb of ram, -m 10 seems a bit large, I'd cut that
back if you're swapping. If you have too many spamd children, your
system will grind to a halt as it spends all its time swapping to disk.
Also, if you've added any add-on rulesets I'd check the size of the
spamd processes in memory If your spamds are over 50mb each, consider
removing some of your rulesets.
In particular, if you're using either the "sa-blacklist.cf" or
"sa-blacklist-uri.cf" files, remove them first. These files are really
only suitable for system with lots of ram (>3gig), or very low usage
(<300 emails a day) with substantial amounts of ram (>1gig).
Payal Rathod wrote:
> Hi,
> A friend of mine is maintaining a mail server for a school and I am
> helpig her out. There are just 150-200 accounts and the machine is
> having 512 Mb RAM and 1Gb swap running Cent OS with processor being AMD
> Athlon. spamd and spamc pair is used with qmail as mailserver and
> processing done via .qmail files.
> spamd is started as,
> /usr/bin/spamd -m 10 -x -u spamd -H /home/spamd
> Sometimes when there is heavy load we get in logs,
> Cannot open bayes databases /var/bayes/bayes_* R/W: lock failed: File
> exists
> (Bayes works properly though).
> The load average always hovers around 20.0 and top shows spamd is
> culprit. Also, I have trained over 10,000 each of spam and ham and also
> use all sare rules yet many spams are passing thru' last few days.
>
> Now 2 questions, what is that error for and what more can I do for spam?
> The teachers are sick of spam since last few days.
>
> With warm regards,
> -Payal
>
>
>
>
>
Re: SA increasing load average a lot and spams getting through
Posted by Loren Wilton <lw...@earthlink.net>.
> Will reducing it to -m 5 make the mail delivery slow?
If you were thrashing and reducing the number of children gets you out of
thrashing it will probably speed things up, not slow them down.
I don't know what your mail volume is. If you get a lot of mail you may
need more children, or if mail comes in very large bursts. If you are
getting maybe 10.000 a day or less mail, I suspect 5 children should be
fine.
You can check your logs. If SA starts running out of children it will
output some message about "suggest increasing the number of children". I
don't remember the exact wording.
If you do increase your number of children, keep an eye on swap usage and
the load average. I think most people run a fairly low load average,
certainly under 5.
Loren
Re: SA increasing load average a lot and spams getting through
Posted by Payal Rathod <pa...@scriptkitchen.com>.
On Thu, Sep 21, 2006 at 07:55:40AM -0700, Loren Wilton wrote:
> >The load average always hovers around 20.0 and top shows spamd is
> >culprit. Also, I have trained over 10,000 each of spam and ham and also
>
> You probably have too many spamd children running. With only 512M you
> probably want 5 children or less.
We both have added 512 Mb more immediately and also reduced -m 10 to -m
5
/var/bayes has,
-rwxrwxrwx 1 spamd spamd 17M Sep 21 12:07 bayes_journal
-rwxrwxrwx 1 root root 21M Sep 20 22:25 bayes_seen
-rwxrwxrwx 1 spamd spamd 5.2M Sep 20 22:25 bayes_toks
Will reducing it to -m 5 make the mail delivery slow?
With warm regards,
-Payal
>
> >Sometimes when there is heavy load we get in logs,
> >Cannot open bayes databases /var/bayes/bayes_* R/W: lock failed: File
> >exists
>
> Lock timeout, some message didn't get a bayes score. This is a side effect
> of the high load average.
>
> Loren
>
Re: SA increasing load average a lot and spams getting through
Posted by Loren Wilton <lw...@earthlink.net>.
> The load average always hovers around 20.0 and top shows spamd is
> culprit. Also, I have trained over 10,000 each of spam and ham and also
You probably have too many spamd children running. With only 512M you
probably want 5 children or less.
> Sometimes when there is heavy load we get in logs,
> Cannot open bayes databases /var/bayes/bayes_* R/W: lock failed: File
> exists
Lock timeout, some message didn't get a bayes score. This is a side effect
of the high load average.
Loren
Re: SA increasing load average a lot and spams getting through
Posted by Payal Rathod <pa...@scriptkitchen.com>.
On Thu, Sep 21, 2006 at 03:59:00PM +0100, Martin Hepworth wrote:
> How many emails per day are you processing.....?? and of what average
> size.
Many these days around 5000-7000 daily (including spam). Average mai
size is 4k approximately. Many kids are subscribed to kids forums and
newletters, so we get log of mails daily. Also, there are some who have
joined mailing lists. The exact no. of mail we are not certain.
Also, it has clamav.
This server also has website for school internal use.
With warm regards,
-Payal
Re: SA increasing load average a lot and spams getting through
Posted by Martin Hepworth <ma...@solidstatelogic.com>.
Payal Rathod wrote:
> On Thu, Sep 21, 2006 at 03:39:26PM +0100, Martin Hepworth wrote:
>> try dropping non-valid email addresses in the MTA inbound....this
>> will drop over 66% of traffic I find....
>>
>
> I have already done that.
> Any more ideas?
>
> With warm regards,
> -Payal
OK
let keep this on list for a while ;-)
How many emails per day are you processing.....?? and of what average size.
Also your first email implies this is all happening on 1 box...
I'd get a dedicated gateway to handle the load - 1GB per CPU core....
also check what DCC/razor etc you are using, what extra SARE rules (and
others)...
--
Martin Hepworth
Senior Systems Administrator
Solid State Logic
Tel: +44 (0)1865 842300
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.
This footnote confirms that this email message has been swept
for the presence of computer viruses and is believed to be clean.
**********************************************************************