You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2005/11/09 21:35:07 UTC

[Bug 4678] New: Two $'s in the MSG id causes 9.4 point in scoreset 2

http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678

           Summary: Two $'s in the MSG id causes 9.4 point in scoreset 2
           Product: Spamassassin
           Version: unspecified
          Platform: Other
               URL: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=337617
        OS/Version: other
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Rules
        AssignedTo: dev@spamassassin.apache.org
        ReportedBy: spamassassin@packages.debian.org


Have a look at the attached mail. It is extremely hammy, except it
contains $ in its Message-ID. This triggers all three of

 3.5 RATWARE_OUTLOOK_NONAME Bulk email fingerprint (Outlook no name)
                            found
 2.7 RATWARE_MS_HASH        Bulk email fingerprint (msgid ms hash) found
 3.2 MSGID_DOLLARS          Message-Id has pattern used in spam

No single evidence should score 9.4 points.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


felicity@apache.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.8                       |Future




------- Additional Comments From felicity@apache.org  2006-11-22 21:54 -------
no real movement on this ticket, so moving to future.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From spamassassin@dostech.ca  2006-07-15 23:52 -------
http://ruleqa.spamassassin.org/?daterev=20060715-r422180-n

These seem usable:

MSECS   SPAM%   HAM%    S/O%    RANK    SCORE   NAME   

0.00000	0.1666 	0.2499	0.400 	0.45 	0.00 	T_MSGID_DOLLARS
0.00000	0.1666 	0.2499 	0.400 	0.45 	0.00 	MSGID_DOLLARS 		

0.00000 2.4038  0.0000  1.000 	0.83 	0.00 	T_RATWARE_OUTLOOK_NONAME 		
0.00000 2.4038  0.0000 	1.000 	0.83 	0.00 	RATWARE_OUTLOOK_NONAME 	

0.00000 2.4307  0.0000  1.000 	0.83 	0.00 	T_RATWARE_MS_HASH 		
0.00000 2.4307 	0.0000  1.000 	0.83 	0.00 	RATWARE_MS_HASH



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


jbrinkmann@vegasystems.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jbrinkmann@vegasystems.de






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From falk@debian.org  2005-12-06 17:42 -------
Created an attachment (id=3295)
 --> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3295&action=view)
Further test cases

Here's two more mails falsely tagged as spam because of this problem.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From nj@leverton.org  2005-12-14 21:01 -------
Created an attachment (id=3302)
 --> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3302&action=view)
Different example - different mail path, different MTAs, still triggers.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From dallase@nmgi.com  2005-11-09 21:59 -------
i believe this is an outlook express formatted Message-ID, and that message is 
sent to a newgroup, which may strip the X-Mailer/X-Mimeole headers and keep the 
original Message-ID??  

RATWARE_OUTLOOK_NONAME expects a X-Mailer: when the msgid is in OLE format.   
RATWARE_MS_HASH expects to see the X-Mimeole: header, both of which are missing 
from your sample.

Do you know what MUA was used to send the original message???  Below this is an 
similar email conversation i had with someone about this...  needless to say, 
his sample looked very much like yours....


#################################

Hi Frank,

Well, I guess I find that hard to believe, as Outlook Express uses at least 
these following characterizing headers..

Message-ID: <00...@nmgi.com>
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.2180
X-Mimeole: Produced By Microsoft MimeOLE V6.00.2900.2180

I cant imagine there is something between them and you that is stripping these 
headers... But I don�t know any other explaination for the lack of X-Mailer: 
header on OLE email.

Sorry,
Dallas


> 
> Hi Dallas,
> The person I just sent you the examples from used Outlook 
> Express on his lap top PC to send the email.
> 

>> Frank,
>>
>> There is no X-Mailer: header... What is the mail client they are using to 
send this?
>>
>> Thanks,
>> Dallas


>>> 
>>>  Here is the forwarded email as an example of RATWARE 
>>> getting many of 
>>> my students and parents blocked.
>>> 




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From dallase@nmgi.com  2005-11-09 22:25 -------
could adding a couple mail list rules into a meta prevent this?   probably 
trivial for a spammer to spoof maillist headers though...


# catches a few spams missed by MSGID_OUTLOOK_INVALID
header __HAS_OUTLOOK_IN_MAILER  X-Mailer =~ /\bMSCRM\b|Microsoft 
(?:CDO|Outlook|Office Outlook)\b/
header __LIST_UNSUB             List-Unsubscribe =~ /\S/
header __LIST_GENERIC           Mailing-List =~ /\S/
#  __MAILMAN_21 is from 20_ratware.cf
meta __LIST_MAIL                  (__LIST_UNSUB || __LIST_GENERIC ||  
__MAILMAN_21)
meta MSGID_DOLLARS              (__OUTLOOK_DOLLARS_MSGID && && !__LIST_MAIL && !
__HAS_OUTLOOK_IN_MAILER && !__UNUSABLE_MSGID)
describe MSGID_DOLLARS          Message-Id has pattern used in spam





------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


spamassassin@dostech.ca changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.6                       |3.1.7






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From bugzilla.spamassassin.org@dale.us  2006-05-19 15:53 -------
I'm having the problem with WM5 + exchange services. 

Adding the rules from comment #11 fixes my case, but MSGID_DOLLARS still hits at
2.2.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


felicity@apache.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.1                       |3.1.2




------- Additional Comments From felicity@apache.org  2006-03-08 18:56 -------
I don't think there's a way to solve this immediately without a new score generation run.  I think we'll be 
likely to do a run after 3.1.1 is released as part of the rule updates stuff, so I'm moving the ticket to 3.1.2 
for the moment.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


felicity@apache.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.4                       |3.1.5






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


felicity@apache.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.3                       |3.1.4






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From jm@jmason.org  2006-10-16 10:11 -------
daryl: +1 on replacing those rules in the 3.2.0 ruleset, given those freqs



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From spamassassin@packages.debian.org  2005-11-09 21:35 -------
Created an attachment (id=3245)
 --> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3245&action=view)
Message exhibiting undersirable behaviour




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


Bob@Menschel.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|Undefined                   |3.1.1
            Version|unspecified                 |3.1.0




------- Additional Comments From Bob@Menschel.net  2005-11-10 07:17 -------
I'm not sure that this can be considered a "single" spam sign, but, 

MSGID_DOLLARS means we have a 12$8$8@ message id, which is usable (not
unusable), and the message is apparently not from an outlook client.
My corpus has 23 ham in October 2005.

RATWARE_MS_HASH means we have a 4+$4+$4+@ message id, and no MS MimeOLE header.
17 ham in October 2005

RATWARE_OUTLOOK_NONAME means " and there is no X-Mailer header
15 ham in October 2005

score MSGID_DOLLARS 3.120 2.163 3.171 1.716
score RATWARE_MS_HASH 2.425 1.379 2.691 1.910
score RATWARE_OUTLOOK_NONAME 3.120 1.861 3.471 2.777

Looks to me like those scores are too high ... perhaps there's a new email
client or server generating 12$8$8@ message ids that wasn't around when we did
the 3.1 scoring run? 



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


eriker-sa@f-secure.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |era+sa@iki.fi




------- Additional Comments From eriker-sa@f-secure.com  2006-05-05 07:58 -------
I regularly see complaints about these forgery rules biting operations who write
their own little mail-sending scripts and pasting in whatever headers they see
in their own inbox just for good measure. This is not completely unlike what the
spamware authors are doing, but these rules are really prone to false positives
for messages from homegrown CGIs etc. ("Homegrown" includes several OEM
solutions in use by large ISPs, probably made by some "consultant" over spring
break.)

I guess a workaround of sorts would be to amass samples of false positives from
these rules and submit them to the test corpus. Unfortunately, I am not sure
whether it would be okay for me to submit the samples I have, as I don't have
any permission from the people I received them from.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From sidney@sidney.com  2005-11-09 22:37 -------
The example email in the attachment has headers indicatinng that it comes
through ezmlm mail list software. ezmlm allows the administrator to specify a
list of headers to strip out and a list of headers to add to outgoing mail.

I did not find any specific reference to X-MimeOLE headers being on the default
list of those to strip out, but neither did a Google search come up with any
examples of mail with bot a sent by ezmlm header and a X-MimeOLE header.

I did find a case of someone complaining how ezmlm removes headers that are
useful for tracking down spammers. Even if it is configurable, it would be hard
to get all administrators of ezmlm mailing lists to set up non-default
configurations to make SpamAssassin's job easier.

It might be a case of recognizing that mail sent through ezmlm has spam-fighting
information stripped out of it and if the administrator does not do something
about it then their subscribers have to whitelist the mailing list and trust the
list administrator to keep spam off of it.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


nj@leverton.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #3302|message/rfc822              |text/plain
          mime type|                            |






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From apache@satyen.com  2006-03-27 17:09 -------
Problem is associated with Windows Mobile 5 + Microsoft Exchange.  Discussion:
http://www.emailaddresses.com/forum/showthread.php?s=&threadid=42287



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From jm@jmason.org  2006-07-12 09:42 -------
Era -- you could always run a mass-check,
http://wiki.apache.org/spamassassin/NightlyMassCheck .  that way the data stays
there, but the rules are run on it and the results submitted...



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


spamassassin@dostech.ca changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.7                       |3.1.8






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


felicity@apache.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.2                       |3.1.3




------- Additional Comments From jm@jmason.org  2006-04-03 12:03 -------
'Problem is associated with Windows Mobile 5 + Microsoft Exchange.  Discussion:
http://www.emailaddresses.com/forum/showthread.php?s=&threadid=42287'

the "fix" suggested there btw doesn't work in all cases:

# Windows Mobile 5 sent through exchange misfires - 2006-03-27
header __ME_VIAEXCHANGE Received =~ /with Microsoft Exchange Server/
score __ME_VIAEXCHANGE 0.05
meta RATWARE_OUTLOOK_NONAME       __MSGID_DOLLARS_OK && !__HAS_X_MAILER &&
!__ME_VIAEXCHANGE
meta RATWARE_MS_HASH              __MSGID_DOLLARS_OK && !__MIMEOLE_MS &&
!__ME_VIAEXCHANGE

this only fixes 2 of the reported 4 FP cases.  still, it may be worth a try at
some stage.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From jm@jmason.org  2006-11-23 02:18 -------
/me pokes daryl to check in those fixed rules and close the bug ;)



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678





------- Additional Comments From nj@leverton.org  2005-12-14 21:00 -------
Created an attachment (id=3301)
 --> (http://issues.apache.org/SpamAssassin/attachment.cgi?id=3301&action=view)
Example mail demonstrating the spurious hits




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


spamassassin@dostech.ca changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|Future                      |3.2.0




------- Additional Comments From spamassassin@dostech.ca  2006-11-23 20:52 -------
I suck.  MSGID_DOLLARS has already been dropped in r467038, the other two remain.

[dos@FC5-VPC trunk]$ svn ci -m "bug 4678: avoid some RATWARE_MS_HASH and
RATWARE_OUTLOOK_NONAME FPs"
Sending        rules/20_ratware.cf
Transmitting file data .
Committed revision 478768.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug 4678] Two $'s in the MSG id causes 9.4 point in scoreset 2

Posted by bu...@bugzilla.spamassassin.org.
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4678


felicity@apache.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.1.5                       |3.1.6






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.