You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Josh Trutwin <jo...@trutwins.homeip.net> on 2006/08/18 16:38:15 UTC
timeout help
I've recently had a server experience some really slow spam
processing - I'm not sure what's going on but I notice a lot of
timeouts in the mail log:
Aug 18 09:20:21 www spamd[27673]: timeout with empty $@
at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm line
182, <GEN16> line 1126. Aug 18 09:22:02 www spamd[27674]: timeout
with empty $@
at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm line
182. Aug 18 09:22:02 www spamd[27675]: timeout with empty $@
at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm line
182. Aug 18 09:23:20 www spamd[27746]: bayes: _put_tokens: SQL
error: Lock wait timeout exceeded; try restarting transaction Aug
18 09:23:27 www spamd[27675]: spamd: copy_config timeout,
respawning child process after 3 messages at /usr/bin/spamd line
967. Aug 18 09:23:29 www spamd[27745]: spamd: copy_config timeout,
respawning child process after 1 messages at /usr/bin/spamd line
967. Aug 18 09:24:31 www spamd[27674]: spamd: copy_config timeout,
respawning child process after 3 messages at /usr/bin/spamd line
967. Aug 18 09:24:49 www spamd[27732]: child processing timeout
at /usr/bin/spamd line 1084. Aug 18 09:25:02 www spamd[27730]:
spamd: copy_config timeout, respawning child process after 1
messages at /usr/bin/spamd line 967. Aug 18 09:25:19 www spamd
[27746]: bayes: child processing timeout at /usr/bin/spamd line
1084. Aug 18 09:26:21 www spamd[27672]: bayes: child processing
timeout at /usr/bin/spamd line 1084. Aug 18 09:27:19 www spamd
[27732]: spamd: copy_config timeout, respawning child process after
1 messages at /usr/bin/spamd line 967. Aug 18 09:27:19 www spamd
[27746]: spamd: copy_config timeout, respawning child process after
1 messages at /usr/bin/spamd line 967. Aug 18 09:27:32 www spamd
[27672]: spamd: copy_config timeout, respawning child process after
2 messages at /usr/bin/spamd line 967. Aug 18 09:27:38 www spamd
[27743]: child processing timeout at /usr/bin/spamd line 1084. Aug
18 09:27:39 www spamd[27743]: child processing timeout
at /usr/bin/spamd line 1084.
Any suggestions?
Debian linux - spamd 3.0.4 with pyzor/dcc/razor
spamd running with:
/usr/bin/spamd -d -D -q -x -H /etc/razor --max-children=12
--socketpath=/var/spool/spamassassin/spamd.sock -u spamd
Thanks,
Josh
Re: Do I have too many rules? [WAS: timeout help]
Posted by jdow <jd...@earthlink.net>.
From: "Josh Trutwin" <jo...@trutwins.homeip.net>
> On Sat, 19 Aug 2006 18:06:59 -0400
> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
>
>> Josh Trutwin wrote:
>> > On Fri, 18 Aug 2006 12:16:51 -0400
>> > "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
>> >
>> >> Josh Trutwin wrote:
>> >>> I've recently had a server experience some really slow spam
>> >>> processing - I'm not sure what's going on but I notice a lot
>> >>> of timeouts in the mail log:
>> >>>
>> >>> Aug 18 09:20:21 www spamd[27673]: timeout with empty $@
>> >>> at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm
>> >>> line 182, <GEN16> line 1126. Aug 18 09:22:02 www spamd[27674]:
>> >>> timeout with empty $@
>> >>> Any suggestions?
>> >>>
>> >>> Debian linux - spamd 3.0.4 with pyzor/dcc/razor
>> >>>
>> >>> spamd running with:
>> >>>
>> >>> /usr/bin/spamd -d -D -q -x -H /etc/razor --max-children=12
>> >>> --socketpath=/var/spool/spamassassin/spamd.sock -u spamd
>> >> Unless you've got at least 600 or more MB of free RAM just for
>> >> spamd's use, you've got too many children and are swap
>> >> thrashing. Back of the --max-children number.
>> >
>> > I was getting the same results with less values - the box has 1
>> > GB
>> > - more is on the way though. I disabled network tests with -L
>> > and things work great again so something along that line is the
>> > culprit.
>>
>> Using -L processes messages faster, thus requiring less children
>> to handle the load. This error is caused by a system not being
>> able to restore the child's config fast enough after it's done
>> processing a message. It's always due to one of two things...
>> high load, or high load caused by swap thrashing.
>>
>> If you're using a lot of add-on rulesets your children may be
>> taking up even more memory than budgeted above. See what they're
>> using and confirm that you're not going to hit swap. If the
>> problem still persists I'd like to hear about it.
>
> Still having problems - even with -L. Server has 1 GB of memory,
> more is on the way I hope. Anyway - I have the following rules:
>
> fastconcepts:/etc/mail/spamassassin# ls
> 10_misc.cf 88_FVGT_headers.cf
> 70_sare_adult.cf 88_FVGT_rawbody.cf
> 70_sare_bayes_poison_nxm.cf 88_FVGT_subject.cf
> 70_sare_evilnum0.cf 88_FVGT_uri.cf
> 70_sare_genlsubj.cf 99_FVGT_DomainDigits.cf
> 70_sare_header.cf 99_FVGT_meta.cf
> 70_sare_highrisk.cf 99_sare_fraud_post25x.cf
> 70_sare_html.cf RulesDuJour
> 70_sare_obfu.cf antidrug.cf
> 70_sare_oem.cf backhair.cf
> 70_sare_random.cf bogus-virus-warnings.cf
> 70_sare_ratware.cf chickenpox.cf
> 70_sare_specific.cf coaching.cf
> 70_sare_spoof.cf init.pre
> 70_sare_stocks.cf local.cf
> 70_sare_unsub.cf mr_wiggly.cf
> 70_sare_uri.cf random.cf
> 70_sare_uri0.cf tripwire.cf
> 70_sare_whitelist_rcvd.cf tsa-list.cf
> 70_sare_whitelist_spf.cf useless.cf
> 72_sare_bml_post25x.cf v310.pre
> 72_sare_redirect_post3.0.0.cf v312.pre
> 88_FVGT_body.cf weeds.cf
>
> I created coaching.cf and tsa-list.cf - they are basically one
> (well two technically) liners.
>
> If I need to cut back I'm not sure where to start - the SARE rules
> are the ones listed on rulesemporium.com - the FVGT are on
> http://www.exit0.us/index.php?pagename=RulesDuJourRuleSets. Of the
> rest, only bogus-virus-warnings.cf is of any substantial size (and
> still 1/3 the size of the largest SARE rule).
>
> Should v310.pre be removed if v312.pre is also there?
No. Cut down on the number of children. 12 is ridiculous.
{^_^}
Re: Do I have too many rules? [WAS: timeout help]
Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Sat, 19 Aug 2006 18:06:59 -0400
"Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> Josh Trutwin wrote:
> > On Fri, 18 Aug 2006 12:16:51 -0400
> > "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> >
> >> Josh Trutwin wrote:
> >>> I've recently had a server experience some really slow spam
> >>> processing - I'm not sure what's going on but I notice a lot
> >>> of timeouts in the mail log:
> >>>
> >>> Aug 18 09:20:21 www spamd[27673]: timeout with empty $@
> >>> at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm
> >>> line 182, <GEN16> line 1126. Aug 18 09:22:02 www spamd[27674]:
> >>> timeout with empty $@
> >>> Any suggestions?
> >>>
> >>> Debian linux - spamd 3.0.4 with pyzor/dcc/razor
> >>>
> >>> spamd running with:
> >>>
> >>> /usr/bin/spamd -d -D -q -x -H /etc/razor --max-children=12
> >>> --socketpath=/var/spool/spamassassin/spamd.sock -u spamd
> >> Unless you've got at least 600 or more MB of free RAM just for
> >> spamd's use, you've got too many children and are swap
> >> thrashing. Back of the --max-children number.
> >
> > I was getting the same results with less values - the box has 1
> > GB
> > - more is on the way though. I disabled network tests with -L
> > and things work great again so something along that line is the
> > culprit.
>
> Using -L processes messages faster, thus requiring less children
> to handle the load. This error is caused by a system not being
> able to restore the child's config fast enough after it's done
> processing a message. It's always due to one of two things...
> high load, or high load caused by swap thrashing.
>
> If you're using a lot of add-on rulesets your children may be
> taking up even more memory than budgeted above. See what they're
> using and confirm that you're not going to hit swap. If the
> problem still persists I'd like to hear about it.
Still having problems - even with -L. Server has 1 GB of memory,
more is on the way I hope. Anyway - I have the following rules:
fastconcepts:/etc/mail/spamassassin# ls
10_misc.cf 88_FVGT_headers.cf
70_sare_adult.cf 88_FVGT_rawbody.cf
70_sare_bayes_poison_nxm.cf 88_FVGT_subject.cf
70_sare_evilnum0.cf 88_FVGT_uri.cf
70_sare_genlsubj.cf 99_FVGT_DomainDigits.cf
70_sare_header.cf 99_FVGT_meta.cf
70_sare_highrisk.cf 99_sare_fraud_post25x.cf
70_sare_html.cf RulesDuJour
70_sare_obfu.cf antidrug.cf
70_sare_oem.cf backhair.cf
70_sare_random.cf bogus-virus-warnings.cf
70_sare_ratware.cf chickenpox.cf
70_sare_specific.cf coaching.cf
70_sare_spoof.cf init.pre
70_sare_stocks.cf local.cf
70_sare_unsub.cf mr_wiggly.cf
70_sare_uri.cf random.cf
70_sare_uri0.cf tripwire.cf
70_sare_whitelist_rcvd.cf tsa-list.cf
70_sare_whitelist_spf.cf useless.cf
72_sare_bml_post25x.cf v310.pre
72_sare_redirect_post3.0.0.cf v312.pre
88_FVGT_body.cf weeds.cf
I created coaching.cf and tsa-list.cf - they are basically one
(well two technically) liners.
If I need to cut back I'm not sure where to start - the SARE rules
are the ones listed on rulesemporium.com - the FVGT are on
http://www.exit0.us/index.php?pagename=RulesDuJourRuleSets. Of the
rest, only bogus-virus-warnings.cf is of any substantial size (and
still 1/3 the size of the largest SARE rule).
Should v310.pre be removed if v312.pre is also there?
Thanks,
Josh
Re: timeout help
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Josh Trutwin wrote:
> On Fri, 18 Aug 2006 12:16:51 -0400
> "Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
>
>> Josh Trutwin wrote:
>>> I've recently had a server experience some really slow spam
>>> processing - I'm not sure what's going on but I notice a lot of
>>> timeouts in the mail log:
>>>
>>> Aug 18 09:20:21 www spamd[27673]: timeout with empty $@
>>> at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm line
>>> 182, <GEN16> line 1126. Aug 18 09:22:02 www spamd[27674]:
>>> timeout with empty $@
>>> Any suggestions?
>>>
>>> Debian linux - spamd 3.0.4 with pyzor/dcc/razor
>>>
>>> spamd running with:
>>>
>>> /usr/bin/spamd -d -D -q -x -H /etc/razor --max-children=12
>>> --socketpath=/var/spool/spamassassin/spamd.sock -u spamd
>> Unless you've got at least 600 or more MB of free RAM just for
>> spamd's use, you've got too many children and are swap
>> thrashing. Back of the --max-children number.
>
> I was getting the same results with less values - the box has 1 GB
> - more is on the way though. I disabled network tests with -L and
> things work great again so something along that line is the culprit.
Using -L processes messages faster, thus requiring less children to
handle the load. This error is caused by a system not being able to
restore the child's config fast enough after it's done processing a
message. It's always due to one of two things... high load, or high
load caused by swap thrashing.
If you're using a lot of add-on rulesets your children may be taking up
even more memory than budgeted above. See what they're using and
confirm that you're not going to hit swap. If the problem still
persists I'd like to hear about it.
Daryl
Re: timeout help
Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Fri, 18 Aug 2006 12:16:51 -0400
"Daryl C. W. O'Shea" <sp...@dostech.ca> wrote:
> Josh Trutwin wrote:
> > I've recently had a server experience some really slow spam
> > processing - I'm not sure what's going on but I notice a lot of
> > timeouts in the mail log:
> >
> > Aug 18 09:20:21 www spamd[27673]: timeout with empty $@
> > at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm line
> > 182, <GEN16> line 1126. Aug 18 09:22:02 www spamd[27674]:
> > timeout with empty $@
>
> > Any suggestions?
> >
> > Debian linux - spamd 3.0.4 with pyzor/dcc/razor
> >
> > spamd running with:
> >
> > /usr/bin/spamd -d -D -q -x -H /etc/razor --max-children=12
> > --socketpath=/var/spool/spamassassin/spamd.sock -u spamd
>
> Unless you've got at least 600 or more MB of free RAM just for
> spamd's use, you've got too many children and are swap
> thrashing. Back of the --max-children number.
I was getting the same results with less values - the box has 1 GB
- more is on the way though. I disabled network tests with -L and
things work great again so something along that line is the culprit.
Josh
Re: timeout help
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Josh Trutwin wrote:
> I've recently had a server experience some really slow spam
> processing - I'm not sure what's going on but I notice a lot of
> timeouts in the mail log:
>
> Aug 18 09:20:21 www spamd[27673]: timeout with empty $@
> at /usr/local/share/perl/5.8.4/Mail/SpamAssassin/Timeout.pm line
> 182, <GEN16> line 1126. Aug 18 09:22:02 www spamd[27674]: timeout
> with empty $@
> Any suggestions?
>
> Debian linux - spamd 3.0.4 with pyzor/dcc/razor
>
> spamd running with:
>
> /usr/bin/spamd -d -D -q -x -H /etc/razor --max-children=12
> --socketpath=/var/spool/spamassassin/spamd.sock -u spamd
Unless you've got at least 600 or more MB of free RAM just for spamd's
use, you've got too many children and are swap thrashing. Back of the
--max-children number.
Daryl