You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Fred W. Bacon" <ba...@aerodyne.com> on 2004/08/11 20:40:25 UTC
false positive with FORGED_OUTLOOK_TAGS and MIME_BASE64_TEXT
We got a troubling false positive today. A message from a potential
business partner in Korea was marked as spam because the message matched
the rules FORGED_MUA_OUTLOOK, FORGED_OUTLOOK_TAGS and MIME_BASE64_TEXT.
We're using spamassassin 2.63 called from a mimedefang milter.
The original message was encoded as base64, which isn't uncommon in
Asian locales. What troubles me is that the decoded message shouldn't
have matched the FORGED_OUTLOOK_TAGS meta rule. When I looked at the
definition of the meta rule in 20_ratware.cf, there didn't seem to be
any reason that FORGED_OUTLOOK_TAGS should have matched. All of the
required tags (meta,head,html, and body) are present in the decoded
message. It is as though the rule is being checked against the base64
encoded text rather than the decoded message. Is this true? Is there a
simple way to fix this?
I haven't checked the FORGED_MUA_OUTLOOK, but I suspect it is suffering
from the same base64 encoding issue.
I've attached a sanitized version of the message below.
--
Fred W. Bacon <ba...@aerodyne.com>
Aerodyne Research, Inc.
>>From techkor@hitel.net Tue Aug 10 02:58:52 2004
Return-Path: <te...@hitel.net>
Received: from mailman.aerodyne.com ([unix socket]) by
mailman.aerodyne.com
(Cyrus v2.1.16-Invoca-RPM-2.1.16-2) with LMTP; Tue, 10 Aug 2004
02:58:52
-0400
X-Sieve: CMU Sieve 2.2
Received: from dauntless.cnchost.com (mailman.aerodyne.com [127.0.0.1])
by
mailman.aerodyne.com (8.12.10/8.12.10) with ESMTP id i7A6wo0X002703 for
<GN...@aerodyne.com>; Tue, 10 Aug 2004 02:58:51 -0400
Received: from sni17.hitel.net ([211.41.85.197]) by
dauntless.cnchost.com
(ConcentricHost SMTP MX 1.45) id CAA11806 for <gn...@aerodyne.com>; Tue,
10
Aug 2004 02:58:46 -0400 (EDT)
Errors-To: <te...@hitel.net>
Received: from 211.41.85.198 (211.41.85.198) at KTMAIL with ESMTP
Hanmir
by sni17;Tue, 10 Aug 2004 15:58:34 +0900
X-MsgID: 1092121114795206.9.sni17
Message-ID: <10...@sni17>
X-RECEIVED-IP: 211.217.207.62
Y-Message-ID: <01...@a>
From: "hitel.net" <te...@hitel.net>
To: <gn...@aerodyne.com>
Subject: *****SPAM***** My visit to aerodyne on August 16, 2004
Date: Tue, 10 Aug 2004 15:57:04 +0900
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----------=_1092121132-4296-35"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2800.1106
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106
X-Spam-Score: =====
X-Spam-Tests: 5.1
BAYES_44,FORGED_MUA_OUTLOOK,FORGED_OUTLOOK_TAGS,MIME_BASE64_TEXT
X-Scanned-By: MIMEDefang 2.39
X-Evolution-Source: imap://bacon;auth=DIGEST-MD5@imap.aerodyne.com/
This is a multi-part message in MIME format...
------------=_1092121132-4296-35
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_0161_01C47EF2.AE2EBAE0"
Content-Transfer-Encoding: binary
This is a multi-part message in MIME format...
------=_NextPart_000_0161_01C47EF2.AE2EBAE0
Content-Type: text/plain; charset="ks_c_5601-1987"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
<snipped actual message body>
------=_NextPart_000_0161_01C47EF2.AE2EBAE0
Content-Type: text/html; charset="ks_c_5601-1987"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html;
charset=ks_c_5601-1987">
<META content="MSHTML 6.00.2800.1106" name=GENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=#ffffff>
removed actual message body
</BODY></HTML>
------=_NextPart_000_0161_01C47EF2.AE2EBAE0--
------------=_1092121132-4296-35
Content-Type: text/plain; name="SpamAssassinReport.txt"
Content-Disposition: inline; filename="SpamAssassinReport.txt"
Mime-Version: 1.0
X-Mailer: MIME-tools 5.411 (Entity 5.404)
Content-Transfer-Encoding: 8bit
Spam detection software, running on the system "mailman.aerodyne.com",
has
identified this incoming email as possible spam. The original message
has been attached to this so you can view it (if it isn't spam) or block
similar future email. If you have any questions, see
bacon@aerodyne.com for details.
Content preview: removed preview
Content analysis details: (5.1 points, 5.0 required)
pts rule name description
---- ----------------------
--------------------------------------------------
-0.0 BAYES_44 BODY: Bayesian spam probability is 44 to 50%
[score: 0.4996]
1.0 MIME_BASE64_TEXT RAW: Message text disguised using base64
encoding
1.0 FORGED_OUTLOOK_TAGS Outlook can't send HTML in this format
3.1 FORGED_MUA_OUTLOOK Forged mail pretending to be from MS Outlook
------------=_1092121132-4296-35--
Re: false positive with FORGED_OUTLOOK_TAGS and MIME_BASE64_TEXT
Posted by David B Funk <db...@engineering.uiowa.edu>.
On Wed, 11 Aug 2004, Fred W. Bacon wrote:
> We got a troubling false positive today. A message from a potential
> business partner in Korea was marked as spam because the message matched
> the rules FORGED_MUA_OUTLOOK, FORGED_OUTLOOK_TAGS and MIME_BASE64_TEXT.
>
> We're using spamassassin 2.63 called from a mimedefang milter.
>
> The original message was encoded as base64, which isn't uncommon in
> Asian locales. What troubles me is that the decoded message shouldn't
>
FWIW, I had to reduce the score here for MIME_BASE64_TEXT for exactly
the same reason. (Lots of correspondence with insitutions world-wide).
I also had to back off the score on: SUBJ_ILLEGAL_CHARS &
FROM_ILLEGAL_CHARS because of many Asian mailers putting raw GB type
chars in 'From:' and 'Subject:' (even tho that violates RFC-2822).
One of them even being the Chinese Yahoo site. ;(
No clue about the FORGED_OUTLOOK_TAGS issue.
all the 'body' type rules are supposed to be applied to the message
after decoding, not the encoded message.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: false positive with FORGED_OUTLOOK_TAGS and MIME_BASE64_TEXT
Posted by Robert Menschel <Ro...@Menschel.net>.
Hello Fred,
Wednesday, August 11, 2004, 11:40:25 AM, you wrote:
FWB> We got a troubling false positive today. A message from a potential
FWB> business partner in Korea was marked as spam because the message
FWB> matched the rules FORGED_MUA_OUTLOOK, FORGED_OUTLOOK_TAGS and
FWB> MIME_BASE64_TEXT.
IMO, SA 2.5x and 2.6x have significant problems dealing with some
variations of base-64 encoding. I suspect you've run into one of those
variations.
Best bet is to lower the score for the MIME_BASE64_TEXT rule until you
can migrate to 3.0
FWB> ... What troubles me is that the decoded message shouldn't
FWB> have matched the FORGED_OUTLOOK_TAGS meta rule. When I looked at the
FWB> definition of the meta rule in 20_ratware.cf, there didn't seem to be
FWB> any reason that FORGED_OUTLOOK_TAGS should have matched. All of the
FWB> required tags (meta,head,html, and body) are present in the decoded
FWB> message. It is as though the rule is being checked against the base64
FWB> encoded text rather than the decoded message. Is this true? Is there a
FWB> simple way to fix this?
I expect the fix is to migrate to version 3.0. The devs have completely
rewritten the handling of encoded emails, and it should work much, much
better.
I've got 3.0 running privately on my PC here -- if you want to send me
the original message complete, with the original encoding, I can test it
for you.
Bob Menschel