You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Randy Ramsdell <rr...@activedg.com> on 2010/08/06 17:43:56 UTC
Local rules trigger bug
I found an bug in spamassassin that can be reliably reproduced when
using our local rules. What would be interesting is to track down where
this bug is exactly.
1. The process runs @ 100% cpu and hangs there. Has t o be kill -9 'ed
2. I see no errors in spamassassin -D
For the time being I have removed our rules until this problem is resolved.
My question is is what would be the best way to determine what bug I am
hitting when the process simply hangs?
Re: Local rules trigger bug
Posted by Michael Scheidell <mi...@secnap.com>.
On 8/6/10 11:43 AM, Randy Ramsdell wrote:
> I found an bug in spamassassin that can be reliably reproduced when
> using our local rules. What would be interesting is to track down
> where this bug is exactly.
>
> 1. The process runs @ 100% cpu and hangs there. Has t o be kill -9 'ed
> 2. I see no errors in spamassassin -D
>
> For the time being I have removed our rules until this problem is
> resolved.
>
> My question is is what would be the best way to determine what bug I
> am hitting when the process simply hangs?
local rules?
eliminate the local rules, one at a time till you are left with the one
that causes the problem
post results on bugzilla.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
ISN: 1259*1300
> *| *SECNAP Network Security Corporation
* Certified SNORT Integrator
* 2008-9 Hot Company Award Winner, World Executive Alliance
* Five-Star Partner Program 2009, VARBusiness
* Best in Email Security,2010: Network Products Guide
* King of Spam Filters, SC Magazine 2008
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Local rules trigger bug
Posted by Bowie Bailey <Bo...@BUC.com>.
On 8/6/2010 12:18 PM, Randy Ramsdell wrote:
> Ralf Hildebrandt wrote:
>> * Randy Ramsdell <rr...@activedg.com>:
>>
>>> I found an bug in spamassassin that can be reliably reproduced when
>>> using our local rules. What would be interesting is to track down
>>> where this bug is exactly.
>>>
>>> 1. The process runs @ 100% cpu and hangs there. Has t o be kill -9 'ed
>>> 2. I see no errors in spamassassin -D
>>>
>>> For the time being I have removed our rules until this problem is
>>> resolved.
>>>
>>> My question is is what would be the best way to determine what bug I
>>> am hitting when the process simply hangs?
>>>
>>
>> Add first halve of your rules, test
>> if it exposes the error, split in two halves and test each halve.
>> etc.
>>
>>
> Yeah that is the fastest way. :) I used a little diff formula and
> found the issue. My I think this may not be the rule we were going for
> but ...
> body __RCR_MEGADK /.*(M.*E.*G.*A.*D.*K).*/
That looks like a VERY good candidate! It is always a bad idea to use
.* in a body rule. Try limiting the match.
body __RCR_MEGADK /M.{0,100}E.{0,100}G.{0,100}A.{0,100}D.{0,100}K/
I also removed the unneeded parenthesis and the .* at the beginning and end.
--
Bowie
Re: Local rules trigger bug
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2010-08-06 at 13:51 -0400, Randy Ramsdell wrote:
> > > body __RCR_MEGADK /.*(M.*E.*G.*A.*D.*K).*/
> >
> > There are a few things that strike me as peculiar about that rule. Not
> > least of which is that it would appear to match the following -
> > hypothetical, but plausible - message. The presence of seven
> > unrestricted greedy specifiers makes it perfectly plausible to me that
> > it would take quite a long time to process any moderately long message.
> It does take a long time to process a message and a very short message
> to boot. In fact, it never finishes and runs the cpus to 100% so the
> rule has been removed. I still wonder if this is a bug.
It is a bug with the RE. It leads straight to backtracking hell.
OK, what happened there? First, the greedy /.*/ at the beginning slurps
in the entire string. Then the Perl RE engine removes one char at a
time, in an effort to match an M -- the last occurrence.
If it found one, the second greedy /.*/ slurps in the entire remainder
of the string, and the backtracking game begins again, this time
matching an E. And so on.
However, each time this process does not result in a match for the
entire RE, backtracking will cause the above process to go back a step
(a char in that RE), and start from there.
It is not a bug with SA, neither with Perl. It is the pathological RE
that causes an insane amount of backtracking with long strings, in
particular if there are many candidates -- that means upper-case chars.
The worst problem is the use of multiple, unbound, greedy /.*/ patterns.
As has been mentioned before, it would be much better to use a bound
variant like /.{0,10}/ instead. Also, using non-greedy variants likely
would help, too. As would using a char-class, rather than accepting any
char -- at the very least, that will help accuracy and prevent matches
on innocent text.
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Local rules trigger bug
Posted by Randy Ramsdell <rr...@activedg.com>.
Dominic Benson wrote:
> On 06/08/10 17:18, Randy Ramsdell wrote:
>> Yeah that is the fastest way. :) I used a little diff formula and
>> found the issue. My I think this may not be the rule we were going
>> for but ...
>> body __RCR_MEGADK /.*(M.*E.*G.*A.*D.*K).*/
>>
>>
> There are a few things that strike me as peculiar about that rule. Not
> least of which is that it would appear to match the following -
> hypothetical, but plausible - message. The presence of seven
> unrestricted greedy specifiers makes it perfectly plausible to me that
> it would take quite a long time to process any moderately long message.
>
>
> Dear Mr. Edwards,
>
> Gary passed your suggestion to me, and I believe that the AMD system
> would be best.
>
> Kind Regards,
>
Matches Uppercase
It does take a long time to process a message and a very short message
to boot. In fact, it never finishes and runs the cpus to 100% so the
rule has been removed. I still wonder if this is a bug.
Re: Local rules trigger bug
Posted by Dominic Benson <do...@lenny.cus.org>.
On 06/08/10 17:18, Randy Ramsdell wrote:
> Yeah that is the fastest way. :) I used a little diff formula and
> found the issue. My I think this may not be the rule we were going for
> but ...
> body __RCR_MEGADK /.*(M.*E.*G.*A.*D.*K).*/
>
>
There are a few things that strike me as peculiar about that rule. Not
least of which is that it would appear to match the following -
hypothetical, but plausible - message. The presence of seven
unrestricted greedy specifiers makes it perfectly plausible to me that
it would take quite a long time to process any moderately long message.
Dear Mr. Edwards,
Gary passed your suggestion to me, and I believe that the AMD system
would be best.
Kind Regards,
Re: Local rules trigger bug
Posted by Randy Ramsdell <rr...@activedg.com>.
Ralf Hildebrandt wrote:
> * Randy Ramsdell <rr...@activedg.com>:
>
>> I found an bug in spamassassin that can be reliably reproduced when
>> using our local rules. What would be interesting is to track down
>> where this bug is exactly.
>>
>> 1. The process runs @ 100% cpu and hangs there. Has t o be kill -9 'ed
>> 2. I see no errors in spamassassin -D
>>
>> For the time being I have removed our rules until this problem is resolved.
>>
>> My question is is what would be the best way to determine what bug I
>> am hitting when the process simply hangs?
>>
>
> Add first halve of your rules, test
> if it exposes the error, split in two halves and test each halve.
> etc.
>
>
Yeah that is the fastest way. :) I used a little diff formula and found
the issue. My I think this may not be the rule we were going for but ...
body __RCR_MEGADK /.*(M.*E.*G.*A.*D.*K).*/
Re: Local rules trigger bug
Posted by Ralf Hildebrandt <Ra...@charite.de>.
* Randy Ramsdell <rr...@activedg.com>:
> I found an bug in spamassassin that can be reliably reproduced when
> using our local rules. What would be interesting is to track down
> where this bug is exactly.
>
> 1. The process runs @ 100% cpu and hangs there. Has t o be kill -9 'ed
> 2. I see no errors in spamassassin -D
>
> For the time being I have removed our rules until this problem is resolved.
>
> My question is is what would be the best way to determine what bug I
> am hitting when the process simply hangs?
Add first halve of your rules, test
if it exposes the error, split in two halves and test each halve.
etc.
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebrandt@charite.de | http://www.charite.de