You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2017/10/31 14:52:45 UTC
[Bug 7490] New: Match for any UTF-8 character stopped working in
3.4.1
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7490
Bug ID: 7490
Summary: Match for any UTF-8 character stopped working in 3.4.1
Product: Spamassassin
Version: 3.4.1
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: Libraries
Assignee: dev@spamassassin.apache.org
Reporter: jaroslav@thinline.cz
Target Milestone: Undefined
Hello,
few weeks ago we noticed that some custom body rules are not matching anymore
on some of our company's mail servers. After some investigation we found out
that the cause is a regression in the new SpamAssassin version - rules stopped
working on servers with 3.4.1 while working correctly on servers with 3.4.0.
(I verified this by upgrading one of the 3.4.0 servers - Debian Jessie - to
3.4.1 from jessie-backports. Rules stopped working, which was fixed by
downgrading back to 3.4.0.)
Rules in question are something like this one:
body CHECK /volba v.s kontaktovat se vzbudila z geografick. povahy/i
This phrase reads "volba vás kontaktovat se vzbudila z geografické povahy" in
the original message, it is a poor - most likely automated - translation. (The
whole message is an obvious spam.) Considering the phrase is readable without
the UTF characters (vas, geograficke), we wanted to catch both variants with
the rule.
From what we have been able to find out, SA 3.4.1 compares the strings
byte-wise, regardless of multibyte characters, causing the rule to not match.
The rule can be sort of fixed by changing the dot character into double dot
(both characters are 2-byte) but then it fails to detect the purely ASCII
variant of the phrase.
Is there any configuration option (or is it possible to add one) that would
revert this to previous behaviour, where single dot matched both ASCII and
multibyte characters?
Thanks in advance.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7490] Match for any UTF-8 character stopped working in 3.4.1
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7490
jaroslav@thinline.cz changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jaroslav@thinline.cz
--- Comment #2 from jaroslav@thinline.cz ---
I re-tested to be sure - input message has this content type and body:
Content-Type: text/plain; charset=UTF-8
volba vás kontaktovat se vzbudila z geografické povahy
In hex (hexdump -C):
00000300 65 63 74 3a 20 74 65 73 74 0a 43 6f 6e 74 65 6e |ect: test.Conten|
00000310 74 2d 54 79 70 65 3a 20 74 65 78 74 2f 70 6c 61 |t-Type: text/pla|
00000320 69 6e 3b 20 63 68 61 72 73 65 74 3d 55 54 46 2d |in; charset=UTF-|
00000330 38 0a 0a 76 6f 6c 62 61 20 76 c3 a1 73 20 6b 6f |8..volba v..s ko|
00000340 6e 74 61 6b 74 6f 76 61 74 20 73 65 20 76 7a 62 |ntaktovat se vzb|
00000350 75 64 69 6c 61 20 7a 20 67 65 6f 67 72 61 66 69 |udila z geografi|
00000360 63 6b c3 a9 20 70 6f 76 61 68 79 0a |ck.. povahy.|
2 bytes per non-ascii character. SA 3.4.0 matches, 3.4.1 does not.
(normalize_charset is set to 1, but as I understand it, it should have no
effect in this case)
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7490] Match for any UTF-8 character stopped working in 3.4.1
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7490
--- Comment #3 from RW <rw...@googlemail.com> ---
What I was thinking was that normalize_charset might have been broken in 3.4.0
but working in 3.4.1, then any hits on non-UTF-8 mail would go away on
upgrading. The rules would have appeared to stop working when they were never
working properly in the first place.
If you've seen the rule match a UTF-8 é, then I can't explain that.
IIWY I'd just change the "." to ".{1,2}"
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7490] Match for any UTF-8 character stopped working in 3.4.1
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7490
Kevin A. McGrail <km...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |INVALID
CC| |kmcgrail@apache.org
Status|NEW |RESOLVED
--- Comment #5 from Kevin A. McGrail <km...@apache.org> ---
Workaround identified if it was a bug.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7490] Match for any UTF-8 character stopped working in 3.4.1
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7490
RW <rw...@googlemail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |rwmaillists@googlemail.com
--- Comment #1 from RW <rw...@googlemail.com> ---
I don't recall "." ever matching a multibyte character. There are long-standing
stock body rules that rely on "use byte" and that means that two byte
characters, like é in UTF-8, are seen as two separate characters.
Are you sure that you didn't previously see this work with é encoded in an
8-bit character set? One thing that can affect this is normalize_charset 1,
which converts all text to UTF-8.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7490] Match for any UTF-8 character stopped working in 3.4.1
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7490
--- Comment #4 from jaroslav@thinline.cz ---
Well, considering there are no other replies (and that this feature might've
been a bug in the first place), we'll do that.
Thanks
--
You are receiving this mail because:
You are the assignee for the bug.