You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2005/07/19 11:09:11 UTC
[Bug 4491] New: 25_replace.cf in 3.1.0 is very poor quality
http://bugzilla.spamassassin.org/show_bug.cgi?id=4491
Summary: 25_replace.cf in 3.1.0 is very poor quality
Product: Spamassassin
Version: 3.1.0
Platform: Other
OS/Version: other
Status: NEW
Severity: critical
Priority: P5
Component: Rules
AssignedTo: dev@spamassassin.apache.org
ReportedBy: mbr@freebsd.org
In my tests with spanish, french, german mails I see a massive FPS
rate with 25_replace.cf.
There are no boundary checks done. Spanisch words like tambi�n, credito
etc. are matching.
For french mails this problem exists too.
The boundaries sould be be checked with \b at the beginning,
and [^\d\p{IsAlpha}] at the end. This is needed because obfuscated
words can (and will) contain non alpha nummeric chars and boundary
checks fail if LANG=C (used for speed reasons) is used.
I've also about 5-10 exceptions (FPS words matching).
The replace tags script is not very comfortable, we do that
here differently:
body __OBFUL_42 /\b(?!vir-?gins?)%%z:VIRGIN%%s?[^\d\p{IsAlpha}]/i
body __OBFUL_57 /\b(?!cr[e�]dit)%%y:CREDIT%%[^\d\p{IsAlpha}]/i
body __OBFUL_12 /\b(?!am-?bien)%%y:AMBIEN%%[^\d\p{IsAlpha}]/i
%%z: and %%y: are different replace-sets with other tags. Of course
our version works also with singe words like the
Mail::SpamAssassin::Plugin::ReplaceTags plugin.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4491
felicity@apache.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.1.1 |3.1.2
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4491
felicity@apache.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.1.2 |3.2.0
------- Additional Comments From felicity@apache.org 2006-04-04 03:51 -------
since there's been nothing new with this in many months, punting to 3.2.0
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4491
------- Additional Comments From mbr@freebsd.org 2005-07-22 17:41 -------
Hi,
> Without looking at the rules in question, it seems to me that all except the
> first and the last are things I would expect to find in a spam. Tambien
> definitely shouldn't be hitting rules. I can't remember the translation for
> mlif to know if it is a reasonable hit or not.
Obfuscating rules should only match obfuscated words. And we defninitly need
to set proper borders.
Martin
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4491
------- Additional Comments From lwilton@earthlink.net 2005-07-21 14:35 -------
Subject: Re: 25_replace.cf in 3.1.0 is very poor quality
> Words I see matching with those rules are.
>
> tambi�n
> cr�dito
> cr�dit
> p�nis
> �rection
> mlif
Without looking at the rules in question, it seems to me that all except the
first and the last are things I would expect to find in a spam. Tambien
definitely shouldn't be hitting rules. I can't remember the translation for
mlif to know if it is a reasonable hit or not.
Are all of these really that common in non-spam portugese mail that they
should be excepted? Or is the problem that the score is too high? How many
points are these adding to a typical message? Are they really creating FPs,
or are they just adding a couple of points of score to some percentage of
the messages?
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4491
Bob@Menschel.net changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords| |triage
------- Additional Comments From Bob@Menschel.net 2005-07-19 18:56 -------
Martin, What order of magnitude are you calling "massive"? Admittedly, I don't
have as many European ham as you have, and my S/O so far is 98.44% on all the
FUZZY_* rules (did an egrep FUZZY_ on ham.log, spam.log, pipe each to wc-l). I'm
not doubting your report -- just want to get on record how bad "massive" is.
Thanks.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4491
duncf@debian.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.1.0 |3.1.1
------- Additional Comments From duncf@debian.org 2005-08-27 17:51 -------
Is this still an issue with the new scores? (I'm guessing yes, to a certain
extent.) I'm not sure if we can really make significant changes like this
post-perceptron.
I'm not sure this should hold up 3.1.0, punting to 3.1.1.
Dan are you looking at this?
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4491
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.2.0 |Future
------- Additional Comments From jm@jmason.org 2006-12-05 05:45 -------
no movement, punting from 3.2.0 as well
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4491
duncf@debian.org changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |dev@spamassassin.apache.org
AssignedTo|dev@spamassassin.apache.org |quinlan@pathname.com
Severity|critical |major
Keywords|triage |
Target Milestone|Undefined |3.1.0
------- Additional Comments From duncf@debian.org 2005-07-23 17:48 -------
Reassigning to Dan.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
You are on the CC list for the bug, or are watching someone who is.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4491
------- Additional Comments From mbr@freebsd.org 2005-07-21 03:56 -------
It is not that bad, because each HIT gives only 1 point. But about 5% of my
ham matches againt those rules.
tambi�n is the worst of those words. Almost all portugese mails are matching
here.
Words I see matching with those rules are.
tambi�n
cr�dito
cr�dit
p�nis
�rection
mlif
Martin
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4491] 25_replace.cf in 3.1.0 is very poor quality
Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4491
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|Future |3.3.0
------- Additional Comments From jm@jmason.org 2006-12-05 06:10 -------
really move this time
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.