You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Matt Elson <me...@fastmail.net> on 2009/06/30 19:43:34 UTC
Weird Problem w/ Rule2XSBody + Sought Rule
Hey all,
I stumbled upon an odd issue the other day that I'm having trouble
tracking down. Namely, a certain rule in the sought rule set, when
compiled for use with Rule2XSBody is causing the processing of *some*
emails to, well, never really end. Piping the mail through spamassassin
or into spamd just results in the process hanging and the memory usage
going higher and higher (2+ gigs, easily) and seemingly ignoring any
sort of timeouts. The process finally gets killed only when the OS
notices it's out of memory and starts killing processes or when I'm able
to sneak in and kill -9 it. There's nothing in the debug of SA whatsoever.
I was wondering if anyone else has seen this or if it's some quirk of my
environment. I admit that I'm no expert in this sort of thing, but
(hopefully) some useful information is below the dotted line.
-----
This happened on four of my machines which have the following configuration:
RHEL5.2 / SA 3.2.5 / Perl 5.8.8 / gcc 4.1.2
RHEl5.2 / SA 3.2.4 / Perl 5.8.8 / gcc 4.1.2
RHELAS 4 (Update 6) / SA 3.2.4 / Perl 5.8.5 / gcc 3.4.6
RHELAS 4 (Update 6) / SA 3.2.4 / Perl 5.8.5 / gcc 3.4.6
The SA is built from source off the main website, and the perl is just
stock redhat.
If I copy down all my rules/configuration to my Debian desktop using its
packaging, the problem doesn't emerge (sa 3.2.5/perl 5.10.0/gcc 4.3.3 there)
Removing the compiled rulesets works around the issue fairly handily.
I'm stubborn though, so after I did so, I dug around a bit and it seems
one specific body rule was causing the issue, namely:
body __SEEK_1R0JFS /\x{ff}\x{fe} \x{00} \x{00} \x{00}
\x{00}<\x{00}m\x{00}e\x{00}t\x{00}a\x{00}
\x{00}h\x{00}t\x{00}t\x{00}p\x{00}-\x{00}e\x{00}q\x{00}u\x{00}i\x{00}v\x{00}=\x{00}\'\x{00}R\x{00}e\x{00}f\x{00}r\x{00}e\x{00}s\x{00}h\x{00}\'\x{00}
\x{00}c\x{00}o\x{00}n\x{00}t\x{00}e\x{00}n\x{00}t\x{00}=\x{00}\'\x{00}0\x{00};\x{00}
\x{00}u\x{00}r\x{00}l\x{00}=\x{00}h\x{00}t\x{00}t\x{00}p\x{00}:\x{00}\/\x{00}\/\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}/
Once I comment out the rule, compiled rulesets work fine again. I don't
know enough to know what the heck that regex even is, or why it would be
causing problems (I basically found which rule was causing a problem by
commenting out anything that looked scary to me, running sa-compile, and
testing to see if I the "hanging" behavior went away)
I'm not sure the best way to post up a sample of the mail that was
choking the system without it getting mangled (though I'll gladly post
it if someone can show me where), but fooling around, it seemed to come
down to the message containing this as one of its parts:
-
Content-Type: text/html;
Content-Transfer-Encoding: quoted-printable
(Any content could go here)
=00
-
Removing =00 OR Content-Transfer-Encoding: quoted-printable causes the
mail to pass through without a problem. It seems to only be both
combined that resulted in the behavior I saw.
Anyhoo, any thoughts? This a legitimate bug or something wrong with my
setup?
Matt
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Don Drake <do...@gmail.com>.
On Wed, Jul 1, 2009 at 6:37 AM, Sean Cardus <sc...@zebrahosts.net> wrote:
> > I've been seeing exactly the same behaviour off and on since Friday
> > last week. I'd not yet managed to narrow it down to a specific rule or
> > email, but your example triggers it every time on my i386 boxes.
>
> Here's a copy of an email that I've modified with the extra section which
> I'm able to reproduce the problem with...
>
> http://pastebin.com/m2bd8546b
>
> Sean
>
>
I am having the same problem, started a few days ago. I have since disabled
sought rules and my segmentation faults have stopped. The above pastebin
segfaults for me, and I have other examples if anyone wants them.
I am running CentOS 5.2 x86_64, SA 3.2.5, perl 5.8.8 and re2c 0.13.5.
Besides disabling the rule, is there a real fix for this?
-Don
--
Donald Drake
Drake Consulting
http://www.drakeconsulting.com/
http://www.MailLaunder.com/
http://www.DrudgeSiren.com/
http://plu.gd/
800-733-2143
RE: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Sean Cardus <sc...@zebrahosts.net>.
> I've been seeing exactly the same behaviour off and on since Friday
> last week. I'd not yet managed to narrow it down to a specific rule or
> email, but your example triggers it every time on my i386 boxes.
Here's a copy of an email that I've modified with the extra section which I'm able to reproduce the problem with...
http://pastebin.com/m2bd8546b
Sean
RE: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Sean Cardus <sc...@zebrahosts.net>.
> > I stumbled upon an odd issue the other day that I'm having trouble
> > tracking down. Namely, a certain rule in the sought rule set, when
> > compiled for use with Rule2XSBody is causing the processing of *some*
> > emails to, well, never really end. Piping the mail through
> > spamassassin or into spamd just results in the process hanging
> > and the memory usage
> > going higher and higher (2+ gigs, easily) and seemingly ignoring any
> > sort of timeouts. The process finally gets killed only when the OS
> > notices it's out of memory and starts killing processes or when I'm
> > able to sneak in and kill -9 it. There's nothing in the debug of SA
> > whatsoever.
I've been seeing exactly the same behaviour off and on since Friday last week. I'd not yet managed to narrow it down to a specific rule or email, but your example triggers it every time on my i386 boxes.
> hey Matt -- what version of re2c is installed?
I'm currently using re2c v0.12.1 on both i386 and x64. However, I can only reproduce the problem on i386, spamd processes & returns the email immediately on x64.
Sean
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Steve Freegard <st...@fsl.com>.
Matthew Elson wrote:
> Justin Mason wrote:
>> hey Matt -- what version of re2c is installed?
>
> Knew I forgot something :P.
>
> re2c 0.13.2 was what was on all of the machines that had the issue -
> when I ran into the issue, the first thing I did was upgrade it to
> 0.13.5 on one of them; the problem still occurred. The Debian box that
> seems to handle things fine is running 0.13.5.
>
> Everywhere I've tested is x86, 32-bit - even the one where I can't seem
> to trigger the problem.
>
> Dunno if it helps, but in some cases the email piped through
> spamassassin actually gives me a segmentation fault. I've not traced
> down why, exactly, but I got the segfault initially until I noticed I
> had my SARE rulesets in both /etc/mail/spamassassin/ *AND* in
> /var/lib/spamassassin/3.002004. Once I removed the rulesets from
> /etc/mail/spamassassin/, it went to the never-ending process behavior I
> first mentioned.
>
IIRC - I had this problem on a couple of machines (not using the SOUGHT
rules though); I installed 3.3.0 from SVN and that cured the issue.
Regards,
Steve.
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Matthew Elson <me...@fastmail.net>.
Justin Mason wrote:
> hey Matt -- what version of re2c is installed?
Knew I forgot something :P.
re2c 0.13.2 was what was on all of the machines that had the issue -
when I ran into the issue, the first thing I did was upgrade it to
0.13.5 on one of them; the problem still occurred. The Debian box that
seems to handle things fine is running 0.13.5.
Everywhere I've tested is x86, 32-bit - even the one where I can't seem
to trigger the problem.
Dunno if it helps, but in some cases the email piped through
spamassassin actually gives me a segmentation fault. I've not traced
down why, exactly, but I got the segfault initially until I noticed I
had my SARE rulesets in both /etc/mail/spamassassin/ *AND* in
/var/lib/spamassassin/3.002004. Once I removed the rulesets from
/etc/mail/spamassassin/, it went to the never-ending process behavior I
first mentioned.
Matt
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Justin Mason <jm...@jmason.org>.
hey Matt -- what version of re2c is installed?
On Tue, Jun 30, 2009 at 18:43, Matt Elson<me...@fastmail.net> wrote:
> Hey all,
>
> I stumbled upon an odd issue the other day that I'm having trouble
> tracking down. Namely, a certain rule in the sought rule set, when
> compiled for use with Rule2XSBody is causing the processing of *some*
> emails to, well, never really end. Piping the mail through spamassassin
> or into spamd just results in the process hanging and the memory usage
> going higher and higher (2+ gigs, easily) and seemingly ignoring any
> sort of timeouts. The process finally gets killed only when the OS
> notices it's out of memory and starts killing processes or when I'm able
> to sneak in and kill -9 it. There's nothing in the debug of SA whatsoever.
>
> I was wondering if anyone else has seen this or if it's some quirk of my
> environment. I admit that I'm no expert in this sort of thing, but
> (hopefully) some useful information is below the dotted line.
>
> -----
> This happened on four of my machines which have the following configuration:
>
>
> RHEL5.2 / SA 3.2.5 / Perl 5.8.8 / gcc 4.1.2
> RHEl5.2 / SA 3.2.4 / Perl 5.8.8 / gcc 4.1.2
> RHELAS 4 (Update 6) / SA 3.2.4 / Perl 5.8.5 / gcc 3.4.6
> RHELAS 4 (Update 6) / SA 3.2.4 / Perl 5.8.5 / gcc 3.4.6
>
>
> The SA is built from source off the main website, and the perl is just
> stock redhat.
>
> If I copy down all my rules/configuration to my Debian desktop using its
> packaging, the problem doesn't emerge (sa 3.2.5/perl 5.10.0/gcc 4.3.3 there)
>
> Removing the compiled rulesets works around the issue fairly handily.
> I'm stubborn though, so after I did so, I dug around a bit and it seems
> one specific body rule was causing the issue, namely:
>
> body __SEEK_1R0JFS /\x{ff}\x{fe} \x{00} \x{00} \x{00}
> \x{00}<\x{00}m\x{00}e\x{00}t\x{00}a\x{00}
> \x{00}h\x{00}t\x{00}t\x{00}p\x{00}-\x{00}e\x{00}q\x{00}u\x{00}i\x{00}v\x{00}=\x{00}\'\x{00}R\x{00}e\x{00}f\x{00}r\x{00}e\x{00}s\x{00}h\x{00}\'\x{00}
> \x{00}c\x{00}o\x{00}n\x{00}t\x{00}e\x{00}n\x{00}t\x{00}=\x{00}\'\x{00}0\x{00};\x{00}
> \x{00}u\x{00}r\x{00}l\x{00}=\x{00}h\x{00}t\x{00}t\x{00}p\x{00}:\x{00}\/\x{00}\/\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}/
>
> Once I comment out the rule, compiled rulesets work fine again. I don't
> know enough to know what the heck that regex even is, or why it would be
> causing problems (I basically found which rule was causing a problem by
> commenting out anything that looked scary to me, running sa-compile, and
> testing to see if I the "hanging" behavior went away)
>
> I'm not sure the best way to post up a sample of the mail that was
> choking the system without it getting mangled (though I'll gladly post
> it if someone can show me where), but fooling around, it seemed to come
> down to the message containing this as one of its parts:
>
>
> -
> Content-Type: text/html;
> Content-Transfer-Encoding: quoted-printable
>
> (Any content could go here)
> =00
> -
>
> Removing =00 OR Content-Transfer-Encoding: quoted-printable causes the
> mail to pass through without a problem. It seems to only be both
> combined that resulted in the behavior I saw.
>
> Anyhoo, any thoughts? This a legitimate bug or something wrong with my
> setup?
>
> Matt
>
>
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Justin Mason <jm...@jmason.org>.
On Thu, Jul 2, 2009 at 15:28, Sean Cardus<sc...@zebrahosts.net> wrote:
>> > An re2c bug, presumably? Is anyone having problems without using sa-
>> > compile?
>>
>> If I removed the compiled rule sets, everything works fine again...
>
> I've noticed that sa-update pulled in a new set of Sought rules this morning (version 320790507). I've run sa-compile over them again, re-tried the mail that previously failed and I'm glad to say I'm no longer seeing the memory/loop problem.
I stopped it publishing rules containing that pattern.
We could still do with reproducing the bug though ;)
--j.
RE: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Sean Cardus <sc...@zebrahosts.net>.
> > An re2c bug, presumably? Is anyone having problems without using sa-
> > compile?
>
> If I removed the compiled rule sets, everything works fine again...
I've noticed that sa-update pulled in a new set of Sought rules this morning (version 320790507). I've run sa-compile over them again, re-tried the mail that previously failed and I'm glad to say I'm no longer seeing the memory/loop problem.
Thanks,
Sean
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Noah Meyerhans <no...@csail.mit.edu>.
On Wed, Jul 01, 2009 at 01:31:25PM +0100, Sean Cardus wrote:
> > An re2c bug, presumably? Is anyone having problems without using sa-
> > compile?
>
> If I removed the compiled rule sets, everything works fine again...
I was just about to report a similar problem when I came across this
thread.
I'm using the sought rules, SARE, and updates.spamassassin.org on a 64
bit Debian etch system with the spamassassin 3.2.4 packages from
backports.org. (I'm the Debian SA maintainer.) We update our rulesets
nightly using sa-update. The updates that we pulled in at Mon Jun 29
10:27:30 UTC 2009 introduced periodic segfaults. I suspect that the
problem is being triggered in the sought rules, as their removal made
the segfaults go away. Beyond that I haven't narrowed things down any
further.
I have a couple of 32 bit etch systems running an otherwise identical
setup that have not seen any segfaults, though their mail volume is
quite a bit lighter.
We're compiling our rules with re2c 0.9.12, FWIW.
noah
RE: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Sean Cardus <sc...@zebrahosts.net>.
> An re2c bug, presumably? Is anyone having problems without using sa-
> compile?
If I removed the compiled rule sets, everything works fine again...
Sean
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by "McDonald, Dan" <Da...@austinenergy.com>.
On Wed, 2009-07-01 at 13:20 +0100, Adam Stephens wrote:
> __SEEK_1R0JFS
I can confirm that removing that test and recompiling eliminates my
segfaults. running re2c 0.12.0
--
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
www.austinenergy.com
Re: Weird Problem w/ Rule2XSBody + Sought Rule
Posted by Adam Stephens <ad...@bristol.ac.uk>.
Matt Elson wrote:
> I dug around a bit and it seems
> one specific body rule was causing the issue, namely:
>
> body __SEEK_1R0JFS /\x{ff}\x{fe} \x{00} \x{00} \x{00}
> \x{00}<\x{00}m\x{00}e\x{00}t\x{00}a\x{00}
> \x{00}h\x{00}t\x{00}t\x{00}p\x{00}-\x{00}e\x{00}q\x{00}u\x{00}i\x{00}v\x{00}=\x{00}\'\x{00}R\x{00}e\x{00}f\x{00}r\x{00}e\x{00}s\x{00}h\x{00}\'\x{00}
> \x{00}c\x{00}o\x{00}n\x{00}t\x{00}e\x{00}n\x{00}t\x{00}=\x{00}\'\x{00}0\x{00};\x{00}
> \x{00}u\x{00}r\x{00}l\x{00}=\x{00}h\x{00}t\x{00}t\x{00}p\x{00}:\x{00}\/\x{00}\/\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}.\x{00}/
>
>
> I'm not sure the best way to post up a sample of the mail that was
> choking the system without it getting mangled (though I'll gladly post
> it if someone can show me where), but fooling around, it seemed to come
> down to the message containing this as one of its parts:
>
>
> -
> Content-Type: text/html;
> Content-Transfer-Encoding: quoted-printable
>
> (Any content could go here)
> =00
> -
>
I've been seeing frequent segfaults and coredumps on my systems since
yesterday morning (SPARC, Solaris 9, SA 3.2.5, perl 5.8.8, re2c was
0.12.0, now 0.13.5) . I can reproduce it with your example, and fix it
by removing the __SEEK_1R0JFS rule.
An re2c bug, presumably? Is anyone having problems without using sa-compile?
Adam.
--
--------------------------------
Adam Stephens
Network Specialist - Email & DNS
adam.stephens@bristol.ac.uk