You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Curtis MacDuff <cu...@pspinc.com> on 2010/03/09 19:24:19 UTC
problem with the Bayesian filter
My Bayesian filter keeps getting screwed up and causing mail flow to
stop. The problem seems to be expiring tokens out of the database. My
expiry setting is set to 200,000. I've tried many different settings for
this but they all seem to behave about the same. Auto learn is also on.
When this happens the mysqld service eats up loads of CPU and stops
responding to requests from Amavisd-new. Since Amavisd-new is expecting
information from mysqld it stops the mail flow. I attached the output
from the 'show processlist' command, one query seems to have been ran
for 12hours without stopping.
Is there other useful information I should get from the server? I have a
script set up to fix this state, collecting other information would not
be difficult. Using default MySQL settings (probably going to start
playing with that today).
Any suggestions would be greatly appreciated, my experience with MySQL
is very limited.
Thanks,
Curtis
Re: problem with the Bayesian filter
Posted by Kai Schaetzl <ma...@conactive.com>.
Curtis MacDuff wrote on Tue, 09 Mar 2010 10:24:19 -0800:
> My Bayesian filter keeps getting screwed up and causing mail flow to
> stop. The problem seems to be expiring tokens out of the database. My
> expiry setting is set to 200,000. I've tried many different settings for
> this but they all seem to behave about the same. Auto learn is also on.
200.000 is rather low for my taste.
Stop the automatic expiration and run it manually, then you will see if
there is a problem. After that, keep doing it manually overnight.
Those queries are abbreviated, so it's not clear what they are doing. Is
this copied from PHPMyAdmin?
Kai
--
Get your web at Conactive Internet Services: http://www.conactive.com
Re: problem with the Bayesian filter
Posted by Jari Fredriksson <ja...@iki.fi>.
On 9.3.2010 20:24, Curtis MacDuff wrote:
> My Bayesian filter keeps getting screwed up and causing mail flow to
> stop. The problem seems to be expiring tokens out of the database. My
> expiry setting is set to 200,000. I've tried many different settings for
> this but they all seem to behave about the same. Auto learn is also on.
>
> When this happens the mysqld service eats up loads of CPU and stops
> responding to requests from Amavisd-new. Since Amavisd-new is expecting
> information from mysqld it stops the mail flow. I attached the output
> from the 'show processlist' command, one query seems to have been ran
> for 12hours without stopping.
>
> Is there other useful information I should get from the server? I have a
> script set up to fix this state, collecting other information would not
> be difficult. Using default MySQL settings (probably going to start
> playing with that today).
>
> Any suggestions would be greatly appreciated, my experience with MySQL
> is very limited.
>
If you are using MyISAM (the default) engine in MySQL tables, try to
change it to InnoDB (much better in table locking, InnoDB has per row
locks while MyISAM always locks the whole table making concurrent access
a pain).
SQL> alter table table_name Engine = InnoDB ;
Do this for all SpamAssassin tables.
--
http://www.iki.fi/jarif/
I've touch'd the highest point of all my greatness;
And from that full meridian of my glory
I haste now to my setting. I shall fall,
Like a bright exhalation in the evening
And no man see me more.
-- Shakespeare
Re: problem with the Bayesian filter
Posted by Curtis MacDuff <cu...@pspinc.com>.
I've tried the manual idea with the --force-expire before. Had the same
problem doing it this way, unless its required to stop Amavis during
this process?
You seemed to have hit the nail on the head though with the Sql module bit:
bayes_store_module Mail::SpamAssassin::BayesStore::SQL
I'll be changing that to MySQL instead ...
Kai, I'd agree 200,000 is low. Now that I've found the above I'll
probably change it to 500,000.
On 3/9/2010 10:45 AM, Michael Scheidell wrote:
>
>
> On 3/9/10 1:24 PM, Curtis MacDuff wrote:
>> My Bayesian filter keeps getting screwed up and causing mail flow to
>> stop. The problem seems to be expiring tokens out of the database. My
>> expiry setting is set to 200,000. I've tried many different settings
>> for this but they all seem to behave about the same. Auto learn is
>> also on.
> don't use auto expire? turn it off? then in nightly cronjob, stop
> amavis, manually expire it?
>
> and for sql, you are using the MySql.pm, not the generic Sql.pm for
> bayes, right?
>
>
--
Curtis MacDuff curtis.macduff@pspinc.com
PSP: Focus on your business, not your IT
1404 140th Place NE, Bellevue, WA 98007 USA
http://www.pspinc.com
DISCLAIMER: This information is confidential and is intended
only for the use of the individual or entity named above.
If the reader of this message is not the intended recipient,
please disregard and destroy this email and its content.
Thank you.
Re: problem with the Bayesian filter
Posted by Michael Scheidell <sc...@secnap.net>.
On 3/9/10 1:24 PM, Curtis MacDuff wrote:
> My Bayesian filter keeps getting screwed up and causing mail flow to
> stop. The problem seems to be expiring tokens out of the database. My
> expiry setting is set to 200,000. I've tried many different settings
> for this but they all seem to behave about the same. Auto learn is
> also on.
don't use auto expire? turn it off? then in nightly cronjob, stop
amavis, manually expire it?
and for sql, you are using the MySql.pm, not the generic Sql.pm for
bayes, right?
--
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
> *| *SECNAP Network Security Corporation
* Certified SNORT Integrator
* 2008-9 Hot Company Award Winner, World Executive Alliance
* Five-Star Partner Program 2009, VARBusiness
* Best Anti-Spam Product 2008, Network Products Guide
* King of Spam Filters, SC Magazine 2008
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: problem with the Bayesian filter
Posted by John Hardin <jh...@impsec.org>.
On Tue, 9 Mar 2010, Curtis MacDuff wrote:
> When this happens the mysqld service eats up loads of CPU and stops
> responding to requests from Amavisd-new.
Verify your schema, and that you're not missing any indexes.
And, as others have suggested, turn off auto-expiry and expire from a cron
job scheduled at a time when your imbound email volume is low.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Failure to plan ahead on someone else's part does not constitute
an emergency on my part. -- David W. Barts in a.s.r
-----------------------------------------------------------------------
5 days until Daylight Saving Time begins in U.S. - Spring Forward
Re: problem with the Bayesian filter
Posted by RW <rw...@googlemail.com>.
On Tue, 09 Mar 2010 10:24:19 -0800
Curtis MacDuff <cu...@pspinc.com> wrote:
> My Bayesian filter keeps getting screwed up and causing mail flow to
> stop. The problem seems to be expiring tokens out of the database. My
> expiry setting is set to 200,000. I've tried many different settings
> for this but they all seem to behave about the same. Auto learn is
> also on.
>
> When this happens the mysqld service eats up loads of CPU and stops
> responding to requests from Amavisd-new. Since Amavisd-new is
> expecting information from mysqld it stops the mail flow. I attached
> the output from the 'show processlist' command, one query seems to
> have been ran for 12hours without stopping.
Try running sa-learn --force-expire from cron and turn-off auto-expiry