You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Technical Department <te...@netcom.co.uk> on 2005/07/14 12:15:19 UTC
Long Scanning Delays
Hi
We are running Qmail Scanner 1.25 + Spamassassin 3.04 + Clamd 1.86.1 on
our dual 1.4Ghz P3 Linux mail server. We've been experiencing some
problems with the server taking a long time to scan messages (In most
cases they are taking between 60 and 80 seconds). I have the following
rulesets installed:
70_sare_adult.cf 70_sare_unsub.cf init.pre
70_sare_bayes_poison_nxm.cf 72_sare_bml_post25x.cf last-update
70_sare_genlsubj0.cf 72_sare_redirect_post3.0.0.cf local.cf*
70_sare_header0.cf 88_FVGT_subject.cf mr_wiggly.cf
70_sare_html0.cf 90_FVGT.cf netcom.cf
70_sare_obfu1.cf 99_sare_fraud_post25x.cf nov2rules.cf
70_sare_oem.cf 99_sober.cf rolex.cf
70_sare_ratware.cf DomainDigits1.cf surbl.cf
70_sare_spoof.cf evilnumbers.cf weeds.cf
At the moment I have disabled Razor, SPF and RBL checks from within the
Spamassassin configuration.
Spamd is started from Daemontools (/service/spamd/run):
#!/bin/sh
exec /usr/local/bin/softlimit -m 50000000 /usr/bin/spamd -m 8
--username=qscand
Qmail-smtpd is also started from Daemontools (/service/qmail-smtpd/run):
#!/bin/sh
QMAILDUID=`id -u qmaild`
NOFILESGID=`id -g qmaild`
MAXSMTPD=`cat /var/qmail/control/concurrencyincoming`
LOCAL=`head -1 /var/qmail/control/me`
if [ -z "$QMAILDUID" -o -z "$NOFILESGID" -o -z "$MAXSMTPD" -o -z
"$LOCAL" ]; then
echo QMAILDUID, NOFILESGID, MAXSMTPD, or LOCAL is unset in
echo /var/qmail/supervise/qmail-smtpd/run
exit 1
fi
if [ ! -f /var/qmail/control/rcpthosts ]; then
echo "No /var/qmail/control/rcpthosts!"
echo "Refusing to start SMTP listener because it'll create an open
relay"
exit 1
fi
exec /usr/local/bin/softlimit -m 50000000 \
/usr/local/bin/tcpserver -v -R -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" -u
"$QMAILDUID" -g "$NOFILESGID" 0 smtp /usr/local/bin/rblsmtpd -r
spam.dnsbl.sorbs.net -r smtp.dnsbl.sorbs.net -r sbl-xbl.spamhaus.org -r
bl.spamcop.net sh -c '/var/qmail/bin/qmail-smtpd' 2>&1
The server has 1.5GB of memory so I was wondering if the softlimit needs
to be changed? Has anyone else had a problem with this sort of thing?
Thanks,
John
Re: Long Scanning Delays
Posted by Robert Menschel <Ro...@Menschel.net>.
Hello Technical,
Thursday, July 14, 2005, 3:15:19 AM, you wrote:
TD> Hi
TD> We are running Qmail Scanner 1.25 + Spamassassin 3.04 + Clamd 1.86.1 on
TD> our dual 1.4Ghz P3 Linux mail server. We've been experiencing some
TD> problems with the server taking a long time to scan messages (In most
TD> cases they are taking between 60 and 80 seconds). I have the following
TD> rulesets installed:
TD> 70_sare_adult.cf 70_sare_unsub.cf init.pre
TD> 70_sare_bayes_poison_nxm.cf 72_sare_bml_post25x.cf last-update
TD> 70_sare_genlsubj0.cf 72_sare_redirect_post3.0.0.cf local.cf*
TD> 70_sare_header0.cf 88_FVGT_subject.cf mr_wiggly.cf
TD> 70_sare_html0.cf 90_FVGT.cf netcom.cf
TD> 70_sare_obfu1.cf 99_sare_fraud_post25x.cf nov2rules.cf
TD> 70_sare_oem.cf 99_sober.cf rolex.cf
TD> 70_sare_ratware.cf DomainDigits1.cf surbl.cf
TD> 70_sare_spoof.cf evilnumbers.cf weeds.cf
If you use 70_sare_obfu1.cf, you should also be using
70_sare_obfu0.cf, or perhaps instead 70_sare_obfu.cf, which combines
those two files into one.
70_sare_ratware.cf is empty/discontinued.
Replace evilnumbers.cf with 70_sare_evilnum0.cf and possibly
70_sare_evilnum1.cf -- they're the current maintenance versions.
mr_wiggly.cf is obsolete. Use 70_sare_specific.cf instead. Likewise,
70_sare_specific.cf should catch anything flagged by rolex.cf
nov2rules.cf is nov 2003 and should probably be dropped. Any rules
there of significant value have migrated to other rules files.
Don't recognize the non-SARE rules files, but the ones you have
shouldn't be causing the performance problem.
Check your network connection. If SURBL tests are failing and timing
out, that could cause the delays you're seeing.
> spamassassin -D <message
should give you output indicating what does and doesn't work. That
might help you figure this out.
Bob Menschel
Re: Long Scanning Delays
Posted by Technical Department <te...@netcom.co.uk>.
Hi
Thank you for yours and everyones advice, I now have a decent set of
rules and found that SURBL checks were being done twice (Or possibly
conflicting) and it seems to be running better since I upped the
softlimit so hopefully that should be sorted :)
Cheers,
John
Daryl C. W. O'Shea wrote:
> jdow wrote:
>
>>>> From: "Technical Department" <te...@netcom.co.uk>
>
>
>>>> We are running Qmail Scanner 1.25 + Spamassassin 3.04 + Clamd 1.86.1 on
>>>> our dual 1.4Ghz P3 Linux mail server. We've been experiencing some
>>>> problems with the server taking a long time to scan messages (In most
>>>> cases they are taking between 60 and 80 seconds). I have the following
>>>> rulesets installed:
>
>
>> If softlimit does what I think it does the memory expansion won't help.
>> Spamd is taking about 63 megabytes of memory. So he may be starving it.
>> And he may be driving it into swapping. On the other paw 60 to 80
>> seconds end to end for one message is about right for 1.6 GHz. It's
>> maybe a bit low for so few rule sets. There is also a potential for
>> a bad DNS based rule in is batch.
>> {^_^}
>
>
> 60 to 80 seconds is normal? Maybe a 15th of that would be considered
> normal.
>
> Try increasing the softlimit to 75MB and if that doesn't work definitely
> check in to things like slow/broken DNS or failing bayes expiry.
>
>
> Daryl
Re: Long Scanning Delays
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
jdow wrote:
>>>From: "Technical Department" <te...@netcom.co.uk>
>>>We are running Qmail Scanner 1.25 + Spamassassin 3.04 + Clamd 1.86.1 on
>>>our dual 1.4Ghz P3 Linux mail server. We've been experiencing some
>>>problems with the server taking a long time to scan messages (In most
>>>cases they are taking between 60 and 80 seconds). I have the following
>>>rulesets installed:
> If softlimit does what I think it does the memory expansion won't help.
> Spamd is taking about 63 megabytes of memory. So he may be starving it.
> And he may be driving it into swapping. On the other paw 60 to 80
> seconds end to end for one message is about right for 1.6 GHz. It's
> maybe a bit low for so few rule sets. There is also a potential for
> a bad DNS based rule in is batch.
> {^_^}
60 to 80 seconds is normal? Maybe a 15th of that would be considered
normal.
Try increasing the softlimit to 75MB and if that doesn't work definitely
check in to things like slow/broken DNS or failing bayes expiry.
Daryl
Re: Long Scanning Delays
Posted by jdow <jd...@earthlink.net>.
From: "Loren Wilton" <lw...@earthlink.net>
> I'd look at the load average on the system and see if you are running out
of
> memory. I don't know qmail, so can't suggest how to cure it if it is the
> case. However, that is about the most common reason for slow scanning.
You
> don't seem to say how much memory you have. If it is less than 512M, go
get
> some more, SA is memory hungry.
>
> Loren
>
> ----- Original Message -----
> From: "Technical Department" <te...@netcom.co.uk>
>
> > Hi
> >
> > We are running Qmail Scanner 1.25 + Spamassassin 3.04 + Clamd 1.86.1 on
> > our dual 1.4Ghz P3 Linux mail server. We've been experiencing some
> > problems with the server taking a long time to scan messages (In most
> > cases they are taking between 60 and 80 seconds). I have the following
> > rulesets installed:
> > 70_sare_adult.cf 70_sare_unsub.cf init.pre
> > 70_sare_bayes_poison_nxm.cf 72_sare_bml_post25x.cf last-update
> > 70_sare_genlsubj0.cf 72_sare_redirect_post3.0.0.cf local.cf*
> > 70_sare_header0.cf 88_FVGT_subject.cf mr_wiggly.cf
> > 70_sare_html0.cf 90_FVGT.cf netcom.cf
> > 70_sare_obfu1.cf 99_sare_fraud_post25x.cf nov2rules.cf
> > 70_sare_oem.cf 99_sober.cf rolex.cf
> > 70_sare_ratware.cf DomainDigits1.cf surbl.cf
> > 70_sare_spoof.cf evilnumbers.cf weeds.cf
> >
> > At the moment I have disabled Razor, SPF and RBL checks from within the
> > Spamassassin configuration.
> >
> > Spamd is started from Daemontools (/service/spamd/run):
> > #!/bin/sh
> > exec /usr/local/bin/softlimit -m 50000000 /usr/bin/spamd -m 8
If softlimit does what I think it does the memory expansion won't help.
Spamd is taking about 63 megabytes of memory. So he may be starving it.
And he may be driving it into swapping. On the other paw 60 to 80
seconds end to end for one message is about right for 1.6 GHz. It's
maybe a bit low for so few rule sets. There is also a potential for
a bad DNS based rule in is batch.
{^_^}
Re: Long Scanning Delays
Posted by Loren Wilton <lw...@earthlink.net>.
I don't see anything much wrong with the rule selections, other than I think
mr_wiggly and maybe nov2rules are pretty old, and at least mr wiggly is in
sare_specific (I believe it is) these days. I think perhaps also
domaindigits has been depreciated; but I could be wrong about that.
You might also consider adding the chickenpox ruleset. It works for most
people, although there are a few people that find it hits where it
shouldn't.
I'd look at the load average on the system and see if you are running out of
memory. I don't know qmail, so can't suggest how to cure it if it is the
case. However, that is about the most common reason for slow scanning. You
don't seem to say how much memory you have. If it is less than 512M, go get
some more, SA is memory hungry.
Loren
----- Original Message -----
From: "Technical Department" <te...@netcom.co.uk>
To: <us...@spamassassin.apache.org>
Cc: "technical" <te...@netcom.co.uk>
Sent: Thursday, July 14, 2005 3:15 AM
Subject: Long Scanning Delays
> Hi
>
> We are running Qmail Scanner 1.25 + Spamassassin 3.04 + Clamd 1.86.1 on
> our dual 1.4Ghz P3 Linux mail server. We've been experiencing some
> problems with the server taking a long time to scan messages (In most
> cases they are taking between 60 and 80 seconds). I have the following
> rulesets installed:
> 70_sare_adult.cf 70_sare_unsub.cf init.pre
> 70_sare_bayes_poison_nxm.cf 72_sare_bml_post25x.cf last-update
> 70_sare_genlsubj0.cf 72_sare_redirect_post3.0.0.cf local.cf*
> 70_sare_header0.cf 88_FVGT_subject.cf mr_wiggly.cf
> 70_sare_html0.cf 90_FVGT.cf netcom.cf
> 70_sare_obfu1.cf 99_sare_fraud_post25x.cf nov2rules.cf
> 70_sare_oem.cf 99_sober.cf rolex.cf
> 70_sare_ratware.cf DomainDigits1.cf surbl.cf
> 70_sare_spoof.cf evilnumbers.cf weeds.cf
>
> At the moment I have disabled Razor, SPF and RBL checks from within the
> Spamassassin configuration.
>
> Spamd is started from Daemontools (/service/spamd/run):
> #!/bin/sh
> exec /usr/local/bin/softlimit -m 50000000 /usr/bin/spamd -m 8
> --username=qscand
>
> Qmail-smtpd is also started from Daemontools (/service/qmail-smtpd/run):
> #!/bin/sh
> QMAILDUID=`id -u qmaild`
> NOFILESGID=`id -g qmaild`
> MAXSMTPD=`cat /var/qmail/control/concurrencyincoming`
> LOCAL=`head -1 /var/qmail/control/me`
> if [ -z "$QMAILDUID" -o -z "$NOFILESGID" -o -z "$MAXSMTPD" -o -z
> "$LOCAL" ]; then
> echo QMAILDUID, NOFILESGID, MAXSMTPD, or LOCAL is unset in
> echo /var/qmail/supervise/qmail-smtpd/run
> exit 1
> fi
> if [ ! -f /var/qmail/control/rcpthosts ]; then
> echo "No /var/qmail/control/rcpthosts!"
> echo "Refusing to start SMTP listener because it'll create an open
> relay"
> exit 1
> fi
> exec /usr/local/bin/softlimit -m 50000000 \
> /usr/local/bin/tcpserver -v -R -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" -u
> "$QMAILDUID" -g "$NOFILESGID" 0 smtp /usr/local/bin/rblsmtpd -r
> spam.dnsbl.sorbs.net -r smtp.dnsbl.sorbs.net -r sbl-xbl.spamhaus.org -r
> bl.spamcop.net sh -c '/var/qmail/bin/qmail-smtpd' 2>&1
>
>
> The server has 1.5GB of memory so I was wondering if the softlimit needs
> to be changed? Has anyone else had a problem with this sort of thing?
>
> Thanks,
> John