You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Karsten Bräckelmann <gu...@rudersport.de> on 2011/03/21 01:57:00 UTC
Reproducing Bug 6559 (was: Re: __PILL_PRICE Problems)
There are now reports, that this bug is not strictly related to 32 bit
architecture (though always with compiled rules).
Since there have been offers for further testing: One data point is to
collect details about systems, CPU architecture, instruction set used
for compiling, versions (OS, kernel, compiler, re2c, Perl) and patch-
level.
Another might be to reproduce the issue, and get a minimal test-case.
For that, can you reproduce the problem with trivial REs for the three
__PILL_PRICE_x sub-rules?
Can you reproduce the problem by keeping (a renamed copy of) the
original sub-rules and tflags, using a simple meta rule? Are two of them
sufficient? Or even one?
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Reproducing Bug 6559
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
> > Another might be to reproduce the issue, and get a minimal test-case.
>
> Just received a reply privately, containing like heaps of information
> and debugging. Two very noteworthy points it appears to surface.
>
> (a) The actual rule that triggers this is the __PILL_PRICE_3 variant.
Scratch that, probably a red herring. Focus on the usage of space,
instead of \s in each of these rules.
> (b) It might be due to the usage of \s.
>
> Which reminded me -- something I thought about seeing the original
> rules, but now it might be key to the re2c compilation problem.
>
>
> These sub-rules are body tests. That is a *normalized* version of the
> original body. Within a paragraph (in the old-school double newline
> sense), there is only one kind of whitespace. A space. There are no
> tabs, nor newlines -- and no multiple, consecutive occurrences of
> whitespace.
>
> IF the \s turns out to be part of the problem, a simple SPACE instead
> would do just as fine for body rules.
>
> If you change any occurrence of \s in the original sub-rules
> (specifically the third one) to a simple space, does that fix the issue?
Most helpful users on the US east coast going to hit the sleeves soon.
What the fuck am I still doing here!? ;-)
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Reproducing Bug 6559
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2011-03-21 at 01:57 +0100, Karsten Bräckelmann wrote:
> Another might be to reproduce the issue, and get a minimal test-case.
Just received a reply privately, containing like heaps of information
and debugging. Two very noteworthy points it appears to surface.
(a) The actual rule that triggers this is the __PILL_PRICE_3 variant.
(b) It might be due to the usage of \s.
Which reminded me -- something I thought about seeing the original
rules, but now it might be key to the re2c compilation problem.
These sub-rules are body tests. That is a *normalized* version of the
original body. Within a paragraph (in the old-school double newline
sense), there is only one kind of whitespace. A space. There are no
tabs, nor newlines -- and no multiple, consecutive occurrences of
whitespace.
IF the \s turns out to be part of the problem, a simple SPACE instead
would do just as fine for body rules.
If you change any occurrence of \s in the original sub-rules
(specifically the third one) to a simple space, does that fix the issue?
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Reproducing Bug 6559
Posted by Michael Scheidell <mi...@secnap.com>.
On 3/23/11 5:10 PM, Karsten Bräckelmann wrote:
> Michael, I don't think I could follow you. Did you say that these
> "identical" systems do have different rules?
>
there might be some slight differences in local.cf. thats it.
this one is very strange.
offlist if you want more details...
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
ISN: 1259*1300
>*| *SECNAP Network Security Corporation
* Best Intrusion Prevention Product, Networks Product Guide
* Certified SNORT Integrator
* Hot Company Award, World Executive Alliance
* Best in Email Security, 2010 Network Products Guide
* King of Spam Filters, SC Magazine
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Reproducing Bug 6559
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2011-03-21 at 05:33 -0400, Michael Scheidell wrote:
> 32 systems, exactly the same cpu, step software. only minor differences
> would be.. well, not even the exact set of rules. but can re2c randomly
> compile something different depending on internal cpu cache?
>
> only two of them had a problem.
Michael, I don't think I could follow you. Did you say that these
"identical" systems do have different rules?
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Reproducing Bug 6559
Posted by Michael Scheidell <mi...@secnap.com>.
On 3/20/11 11:33 PM, Karsten Bräckelmann wrote:
> [1] CPU version or rather stepping?
>
not in my instance.
freebsd jails are like ibm pseries 'lpars'. not exactly visualization,
but chrooted . super chrooted. chrooted users also, root uid is
chrooted as well.
32 systems, exactly the same cpu, step software. only minor differences
would be.. well, not even the exact set of rules. but can re2c randomly
compile something different depending on internal cpu cache?
only two of them had a problem.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
ISN: 1259*1300
>*| *SECNAP Network Security Corporation
* Best Intrusion Prevention Product, Networks Product Guide
* Certified SNORT Integrator
* Hot Company Award, World Executive Alliance
* Best in Email Security, 2010 Network Products Guide
* King of Spam Filters, SC Magazine
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Reproducing Bug 6559
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Sun, 2011-03-20 at 23:13 -0400, Michael Scheidell wrote:
> On 3/20/11 8:57 PM, Karsten Bräckelmann wrote:
> > There are now reports, that this bug is not strictly related to 32 bit
> > architecture (though always with compiled rules).
> >
> > Since there have been offers for further testing: One data point is to
> > collect details about systems, CPU architecture, instruction set used
> > for compiling, versions (OS, kernel, compiler, re2c, Perl) and patch-
> > level.
>
> I had it happen on two out of 32 jailed freebsd clients.
> guess what: all the same hardware, os level, software, software level.
> all amd64, freebsd 7.3, perl 5.10.0, sa 3.3.1 running through
> amavisd-new 2.6.4, running compiled rules.
Uhm, you do realize that is NOT helpful, don't you? ;)
Seriously, thanks Michael! Supports the previous conclusion that this is
not strictly based on any system or environment -- but likely a very
obscure bug with re2c compilation, triggering in some highly specific
circumstances only [1].
Does the "use space instead of \s" theory fix it for you?
[1] CPU version or rather stepping?
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Reproducing Bug 6559
Posted by Michael Scheidell <mi...@secnap.com>.
On 3/20/11 8:57 PM, Karsten Bräckelmann wrote:
> There are now reports, that this bug is not strictly related to 32 bit
> architecture (though always with compiled rules).
>
> Since there have been offers for further testing: One data point is to
> collect details about systems, CPU architecture, instruction set used
> for compiling, versions (OS, kernel, compiler, re2c, Perl) and patch-
> level.
>
I had it happen on two out of 32 jailed freebsd clients.
guess what: all the same hardware, os level, software, software level.
all amd64, freebsd 7.3, perl 5.10.0, sa 3.3.1 running through
amavisd-new 2.6.4, running compiled rules.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
ISN: 1259*1300
>*| *SECNAP Network Security Corporation
* Best Intrusion Prevention Product, Networks Product Guide
* Certified SNORT Integrator
* Hot Company Award, World Executive Alliance
* Best in Email Security, 2010 Network Products Guide
* King of Spam Filters, SC Magazine
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Reproducing Bug 6559 (was: Re: __PILL_PRICE Problems)
Posted by jp <jp...@saucer.midcoast.com>.
Here's an interesting graph of the affect it had on load:
http://mc4.midcoast.com/mrtg/load.html
Took a while for load to subside after fixing it due to the backlog of email
to process.
On Mon, Mar 21, 2011 at 04:12:45PM -0400, jp wrote:
> I've had the problem happen starting Sunday morning with an automatic sa-update.
> (I have until Sunday had it automatically sa-compile and restart after an update)
>
> It affected various 64 bit machines; we only use 64 bit OSs on AMD64 quad and
> hex core machines. I use the Suse factory provided desktop kernels.
>
> Opensuse 10.3, perl 5.8.8, SA 3.3.1, re2c 0.13.5
> 4x Opensuse 11.3, perl v5.12.1, SA 3.3.1, re2c-0.13.5-29.1.x86_64
>
> My computers ran out of ram as a few of the spamd processes were using up 2GB of
> ram each and not stopping, even with timeouts specified on the spamd startup. I
> could watch the processes keep growing and not dying in top.
>
> I turned off the automatic updating, restored from a recent backup
> /var/lib/spamassassin/, sa-compiled, and it's working great.
>
> I'll wait for word that the stuff distributed via sa-update is fixed.
>
> -Jason
>
> On Mon, Mar 21, 2011 at 01:57:00AM +0100, Karsten Bräckelmann wrote:
> > There are now reports, that this bug is not strictly related to 32 bit
> > architecture (though always with compiled rules).
> >
> > Since there have been offers for further testing: One data point is to
> > collect details about systems, CPU architecture, instruction set used
> > for compiling, versions (OS, kernel, compiler, re2c, Perl) and patch-
> > level.
> >
> > Another might be to reproduce the issue, and get a minimal test-case.
> >
> > For that, can you reproduce the problem with trivial REs for the three
> > __PILL_PRICE_x sub-rules?
> >
> > Can you reproduce the problem by keeping (a renamed copy of) the
> > original sub-rules and tflags, using a simple meta rule? Are two of them
> > sufficient? Or even one?
> >
> >
> > --
> > char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
> > main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
> > (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
>
> --
> /*
> Jason Philbrook | Midcoast Internet Solutions - Wireless and DSL
> KB1IOJ | Broadband Internet Access, Dialup, and Hosting
> http://f64.nu/ | for Midcoast Maine http://www.midcoast.com/
> */
--
/*
Jason Philbrook | Midcoast Internet Solutions - Wireless and DSL
KB1IOJ | Broadband Internet Access, Dialup, and Hosting
http://f64.nu/ | for Midcoast Maine http://www.midcoast.com/
*/
Re: Reproducing Bug 6559 (was: Re: __PILL_PRICE Problems)
Posted by John Hardin <jh...@impsec.org>.
On Mon, 21 Mar 2011, jp wrote:
> I turned off the automatic updating, restored from a recent backup
> /var/lib/spamassassin/, sa-compiled, and it's working great.
>
> I'll wait for word that the stuff distributed via sa-update is fixed.
I'd suggest you disable the PILL_PRICE rules as outlined upthread and take
the update; it includes a lot of good stuff that's been added over the
past few months since the last update that you'd probably benefit from.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Watch... Wallet... Gun... Knee... -- Denny Crane
-----------------------------------------------------------------------
8 days until the M1911 is 100 years old - and still going strong!
Re: Reproducing Bug 6559 (was: Re: __PILL_PRICE Problems)
Posted by jp <jp...@saucer.midcoast.com>.
I've had the problem happen starting Sunday morning with an automatic sa-update.
(I have until Sunday had it automatically sa-compile and restart after an update)
It affected various 64 bit machines; we only use 64 bit OSs on AMD64 quad and
hex core machines. I use the Suse factory provided desktop kernels.
Opensuse 10.3, perl 5.8.8, SA 3.3.1, re2c 0.13.5
4x Opensuse 11.3, perl v5.12.1, SA 3.3.1, re2c-0.13.5-29.1.x86_64
My computers ran out of ram as a few of the spamd processes were using up 2GB of
ram each and not stopping, even with timeouts specified on the spamd startup. I
could watch the processes keep growing and not dying in top.
I turned off the automatic updating, restored from a recent backup
/var/lib/spamassassin/, sa-compiled, and it's working great.
I'll wait for word that the stuff distributed via sa-update is fixed.
-Jason
On Mon, Mar 21, 2011 at 01:57:00AM +0100, Karsten Bräckelmann wrote:
> There are now reports, that this bug is not strictly related to 32 bit
> architecture (though always with compiled rules).
>
> Since there have been offers for further testing: One data point is to
> collect details about systems, CPU architecture, instruction set used
> for compiling, versions (OS, kernel, compiler, re2c, Perl) and patch-
> level.
>
> Another might be to reproduce the issue, and get a minimal test-case.
>
> For that, can you reproduce the problem with trivial REs for the three
> __PILL_PRICE_x sub-rules?
>
> Can you reproduce the problem by keeping (a renamed copy of) the
> original sub-rules and tflags, using a simple meta rule? Are two of them
> sufficient? Or even one?
>
>
> --
> char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
> main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
> (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
--
/*
Jason Philbrook | Midcoast Internet Solutions - Wireless and DSL
KB1IOJ | Broadband Internet Access, Dialup, and Hosting
http://f64.nu/ | for Midcoast Maine http://www.midcoast.com/
*/