Re: spamassassin at 100 percent CPU

[Joe Quinn <jq...@pccc.com>]

Keep replies on list.

Do you remember making any changes, or are you using spamassassin as it 
comes? What kind of email is going through your server? Very large 
emails can cause trouble with poorly written rules. If you can, perhaps 
systematically turn off things that are pushing email to that server 
could narrow it down to a particular type of email.

On 8/9/2014 4:41 PM, Noah wrote:
> Hi there,
>
> thanks for your response.  I am not handling much email its a new 
> server and currently the MX points to another server.
>
> How do I check the SA configuration?  How do I check if I am using 
> additional rules?
>
>
> Cheers,
> Noah
>
>
> On 7/31/14 12:27 PM, Joe Quinn wrote:
>> On 7/31/2014 3:19 PM, Noah wrote:
>>> Hi there,
>>>
>>> what are some things to check with spamassassin commonly running at
>>> 100 percent?  I used apt-get to reinstall of spamassassin
>>> 3.3.2-2ubuntu1 and no cure.  nothing in the syslog that seems relevant.
>>>
>>> Ubuntu 12.04
>>> Linux 3.15.4-x86_64
>>>
>>> Cheers,
>> It depends on a lot of things. What are the specs of the machine? How
>> much email are you handling? How have you configured SA? Are you using
>> any rules in addition to stock?

Re: spamassassin at 100 percent CPU

[Karsten Bräckelmann <gu...@rudersport.de>]

On Wed, 2014-08-13 at 11:20 -0700, Noah wrote:
> This is a new machine with rules copied over from another machine.  How 
> about this?  I just start new.  Is there a good page out that explains 
> setting up spamassassin from scratch and getting the sa rules set up 
> well and cleaned up nicely?  I am happy to start from the beginning with 
> best practices.

If you cannot answer our rather specific questions, you're in for a much
steeper learning curve than you seem to expect...


What the best way of setting up SA on a new machine is? Just install the
distro provided SA packages.

Getting the SA rules set up well? Same. Cleaned up? Do not copy over
configuration and rules from $ome other system, unless you know what you
are copying. IOW, don't. That's clean by definition.

What I really don't get from your reply is this, though:

A new machine, with "rules copied over". Yet, you seem to be unable to
answer our questions regarding custom rules and configuration you put
there. Which equals everything you "copied over" to begin with. If you
did, why can't you answer our question?

Or revert that "copying over", which results in the "cleaned up" state
you asked for.


Regardless of continuing with the current system, or setting up the
whole system from scratch again -- there are important questions raised,
you just didn't answer. Which, frankly, are likely to have a *much* more
severe impact than removing bad, copied rules.

What mail is that system handling, if it is not an MX? How large are
those messages, and what's your size limit? How is SA integrated, what
software is passing mail to SA?

What is the actual process's name, and for how long does it run at CPU
max?


Without answering these (basically, get back to my previous post and
actually answer all my very specific questions), there is absolutely no
point in you posing more or other questions. It won't help.


Reference:

> On 8/11/14 4:31 PM, Karsten Bräckelmann wrote:
> > On Mon, 2014-08-11 at 09:18 -0400, Joe Quinn wrote:
> >> Keep replies on list.
> >>
> >> Do you remember making any changes, or are you using spamassassin as it
> >> comes? What kind of email is going through your server? Very large
> >> emails can cause trouble with poorly written rules. If you can, perhaps
> >> systematically turn off things that are pushing email to that server
> >> could narrow it down to a particular type of email.
> >>
> >> On 8/9/2014 4:41 PM, Noah wrote:
> >>> thanks for your response.  I am not handling much email its a new
> >>> server and currently the MX points to another server.
> >
> > What mail is it handling?
> >
> > Not MX, so I assume it does not receive externally generated mail at
> > all. Which pretty much leaves us with locally generated -- cron noise
> > and other report types.
> >
> > How is SA integrated? What's your message size limit (see config of the
> > service passing mail to SA)? Are you per chance scanning multi MB text
> > reports?
> >
> > A sane size limit is about 500 kB. Besides, local generated mail isn't
> > worth processing with SA, and in the case of cron mail often harmful
> > (think virus scanner report).
> >
> >
> >>> How do I check the SA configuration?  How do I check if I am using
> >>> additional rules?
> >
> > By additional rules, we mean any rules or configuration that is not
> > stock SA. Anything other than the debian package or running sa-update.
> > Generally, anything *you* added.
> >
> >
> >>>> On 7/31/2014 3:19 PM, Noah wrote:
> >>>>> what are some things to check with spamassassin commonly running at
> >>>>> 100 percent?
> >
> > For how long does it run at CPU max? What is the actual process name?
> >
> > It would be rather common for the plain 'spamassassin' script to consume
> > a couple wall-clock seconds of CPU, since it has to read and compile the
> > full rule-set at each invocation.
> >
> > Unlike the 'spamd' daemon, which has that considerable overhead only
> > once during service start. In both cases may the actual scan time with
> > high CPU load be lower than the start-up overhead.
> >
> >
-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: spamassassin at 100 percent CPU

[Noah <no...@enabled.com>]

Hi there,

This is a new machine with rules copied over from another machine.  How 
about this?  I just start new.  Is there a good page out that explains 
setting up spamassassin from scratch and getting the sa rules set up 
well and cleaned up nicely?  I am happy to start from the beginning with 
best practices.

Cheers,
Noah


On 8/11/14 4:31 PM, Karsten Bräckelmann wrote:
> On Mon, 2014-08-11 at 09:18 -0400, Joe Quinn wrote:
>> Keep replies on list.
>>
>> Do you remember making any changes, or are you using spamassassin as it
>> comes? What kind of email is going through your server? Very large
>> emails can cause trouble with poorly written rules. If you can, perhaps
>> systematically turn off things that are pushing email to that server
>> could narrow it down to a particular type of email.
>>
>> On 8/9/2014 4:41 PM, Noah wrote:
>>> thanks for your response.  I am not handling much email its a new
>>> server and currently the MX points to another server.
>
> What mail is it handling?
>
> Not MX, so I assume it does not receive externally generated mail at
> all. Which pretty much leaves us with locally generated -- cron noise
> and other report types.
>
> How is SA integrated? What's your message size limit (see config of the
> service passing mail to SA)? Are you per chance scanning multi MB text
> reports?
>
> A sane size limit is about 500 kB. Besides, local generated mail isn't
> worth processing with SA, and in the case of cron mail often harmful
> (think virus scanner report).
>
>
>>> How do I check the SA configuration?  How do I check if I am using
>>> additional rules?
>
> By additional rules, we mean any rules or configuration that is not
> stock SA. Anything other than the debian package or running sa-update.
> Generally, anything *you* added.
>
>
>>>> On 7/31/2014 3:19 PM, Noah wrote:
>>>>> what are some things to check with spamassassin commonly running at
>>>>> 100 percent?
>
> For how long does it run at CPU max? What is the actual process name?
>
> It would be rather common for the plain 'spamassassin' script to consume
> a couple wall-clock seconds of CPU, since it has to read and compile the
> full rule-set at each invocation.
>
> Unlike the 'spamd' daemon, which has that considerable overhead only
> once during service start. In both cases may the actual scan time with
> high CPU load be lower than the start-up overhead.
>
>

Re: spamassassin at 100 percent CPU

[Karsten Bräckelmann <gu...@rudersport.de>]

On Mon, 2014-08-11 at 09:18 -0400, Joe Quinn wrote:
> Keep replies on list.
> 
> Do you remember making any changes, or are you using spamassassin as it 
> comes? What kind of email is going through your server? Very large 
> emails can cause trouble with poorly written rules. If you can, perhaps 
> systematically turn off things that are pushing email to that server 
> could narrow it down to a particular type of email.
> 
> On 8/9/2014 4:41 PM, Noah wrote:
> > thanks for your response.  I am not handling much email its a new 
> > server and currently the MX points to another server.

What mail is it handling?

Not MX, so I assume it does not receive externally generated mail at
all. Which pretty much leaves us with locally generated -- cron noise
and other report types.

How is SA integrated? What's your message size limit (see config of the
service passing mail to SA)? Are you per chance scanning multi MB text
reports?

A sane size limit is about 500 kB. Besides, local generated mail isn't
worth processing with SA, and in the case of cron mail often harmful
(think virus scanner report).


> > How do I check the SA configuration?  How do I check if I am using 
> > additional rules?

By additional rules, we mean any rules or configuration that is not
stock SA. Anything other than the debian package or running sa-update.
Generally, anything *you* added.


> > > On 7/31/2014 3:19 PM, Noah wrote:
> > > > what are some things to check with spamassassin commonly running at
> > > > 100 percent?

For how long does it run at CPU max? What is the actual process name?

It would be rather common for the plain 'spamassassin' script to consume
a couple wall-clock seconds of CPU, since it has to read and compile the
full rule-set at each invocation.

Unlike the 'spamd' daemon, which has that considerable overhead only
once during service start. In both cases may the actual scan time with
high CPU load be lower than the start-up overhead.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}