You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Anders Norrbring <an...@norrbring.biz> on 2004/03/23 23:33:36 UTC
Outlook 2003 idiot mail client... :(
> Ämne: Re: .biz domain tagged?
>
> Anders Norrbring wrote on Tue, 23 Mar 2004 11:52:28 +0100:
>
>
> > Received: from edit (edit.the-server.net [192.168.111.30])
> > by mail.the-server.net (Postfix) with ESMTP id 47FAE1099
> > for <an...@the-server.net>; Tue, 23 Mar 2004 11:35:02 +0100 (CET)
>
> so your own machine is edit.the-server.net
>
> > Message-Id: <20...@mail.the-server.net>
>
> that's different = not added by Outlook
>
> > Received: from unknown (HELO edit) (norrbring@81.216.17.135 with login)
> > by smtp100.mail.sc5.yahoo.com with SMTP; 23 Mar 2004 10:35:32 -0000
> >
>
> Now, this is completely different and SA possibly can't determine the host
> it came from, so it uses the next received line or just skips the test,
> don't know.
>
> Conclusion: look how to enable MIDs in Outlook.
Well, I Google'd for a while... And on the page
http://extelligence.ringlet.net/roam/archives/000018.html I found;
"I could scarcely believe that, so seconds after the meeting there I was,
googling for Outlook 2003 Message-Id, and sure enough, there it was: on the
first page of results, a news bulletin which, among others, states that
Outlook 2003 only includes Message-ID's when sending e-mail through an
Exchange server. What really got my goat, though, was the reason for
omitting the Message-Id header: you see, users complained that their machine
names were visible on the Internet!"
So, it's a "feature" in my Outlook 2003 to *not* include the Message-Id
header.. Jesus!
So, how can we go about to make the MSGID_FROM_MTA_SHORT detection skip
clients which adds the mailer tag:
"X-Mailer: Microsoft Office Outlook, Build 11.0.5510", or at least Build 11?
Outlook 2003 do add that, so can the detection engine be told to check that
and not tag up the e-mails as spam because of the erroneous "Message-Id"
tag?
Anders Norrbring
Norrbring Consulting
(with a faulty mail client)
Re: Outlook 2003 idiot mail client... :(
Posted by Kai Schaetzl <ma...@conactive.com>.
Kenneth Porter wrote on Wed, 24 Mar 2004 04:55:04 -0800:
> Since the Message-ID is supposed to be added by the server if the MUA omits
> it, what servers are these clients using that fail to add the ID?
>
The problem is not the omission of the MID on the server. It's the omission of
the MID on the client.
Kai
--
Kai Sch�tzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org
Re: Outlook 2003 idiot mail client... :(
Posted by Kai Schaetzl <ma...@conactive.com>.
Anders Norrbring wrote on Wed, 24 Mar 2004 14:11:27 +0100:
> Well, I *got* the header added by Postfix alright, but SA slapped me with a
> tag 3 just in case? Now I've signed up with habeas, to prevent the SA
> tagging of my e-mails.
>
I'd suggest using a lokal SMTP proxy and let it add that MID. Sorry, I can't
suggest one in specific.
Kai
--
Kai Sch�tzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org
Re: Outlook 2003 idiot mail client... :(
Posted by Bob George <ma...@ttlexceeded.com>.
Anders Norrbring wrote:
> [...]
> > Well, I *got* the header added by Postfix alright, but SA slapped me with a
> tag 3 just in case? Now I've signed up with habeas, to prevent the SA
> tagging of my e-mails.
Anders, just keep in mind that you have NO CONTROL over what a
recipient's system tags the message as. Habeas will help if the receiver
uses those rules as originally implemented, and hopefully people haven't
gone willy-nilly changing them to spam scores. Having the header added
by postfix will help as well.
Fixing it with postfix will probably help the most. Send me a mail
off-list and I'll let you know how it scores.
- Bob
SV: Outlook 2003 idiot mail client... :(
Posted by Anders Norrbring <an...@norrbring.biz>.
> * Kenneth Porter <sh...@sewingwitch.com>:
>
> > Since the Message-ID is supposed to be added by the server if the MUA
> omits
> > it, what servers are these clients using that fail to add the ID?
>
> qmail?
Well, I *got* the header added by Postfix alright, but SA slapped me with a
tag 3 just in case? Now I've signed up with habeas, to prevent the SA
tagging of my e-mails.
Anders Norrbring
Re: Outlook 2003 idiot mail client... :(
Posted by Ralf Hildebrandt <Ra...@charite.de>.
* Kenneth Porter <sh...@sewingwitch.com>:
> Since the Message-ID is supposed to be added by the server if the MUA omits
> it, what servers are these clients using that fail to add the ID?
qmail?
--
Ralf Hildebrandt (Im Auftrag des Referat V a) Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. +49 (0)30-450 570-916
IT-Zentrum Standort Campus Mitte AIM. ralfpostfix
Re: Outlook 2003 idiot mail client... :(
Posted by Kenneth Porter <sh...@sewingwitch.com>.
--On Tuesday, March 23, 2004 11:33 PM +0100 Anders Norrbring
<an...@norrbring.biz> wrote:
> Well, I Google'd for a while... And on the page
> http://extelligence.ringlet.net/roam/archives/000018.html I found;
Found in microsoft.public.outlook:
<http://groups.google.com/groups?threadm=%23GDzS5VDEHA.3404%40TK2MSFTNGP10.
phx.gbl>
<http://groups.google.com/groups?threadm=18af01c3f66b%2413fb0f60%243501280a
%40phx.gbl>
<http://groups.google.com/groups?threadm=5cbb01c3e5bc%24f9876c00%24a001280a
%40phx.gbl>
<http://groups.google.com/groups?threadm=OZhbLZO0DHA.3496%40TK2MSFTNGP11.ph
x.gbl>
Other sightings:
<http://www.slipstick.com/emo/2003/up031211.htm>
Since the Message-ID is supposed to be added by the server if the MUA omits
it, what servers are these clients using that fail to add the ID?
Re: Outlook 2003 idiot mail client... :(
Posted by jdow <jd...@earthlink.net>.
From: "Anders Norrbring" <an...@norrbring.biz>
> > Ämne: Re: .biz domain tagged?
> >
> > Anders Norrbring wrote on Tue, 23 Mar 2004 11:52:28 +0100:
> >
> >
> > > Received: from edit (edit.the-server.net [192.168.111.30])
> > > by mail.the-server.net (Postfix) with ESMTP id 47FAE1099
> > > for <an...@the-server.net>; Tue, 23 Mar 2004 11:35:02 +0100 (CET)
> >
> > so your own machine is edit.the-server.net
> >
> > > Message-Id: <20...@mail.the-server.net>
> >
> > that's different = not added by Outlook
> >
> > > Received: from unknown (HELO edit) (norrbring@81.216.17.135 with
login)
> > > by smtp100.mail.sc5.yahoo.com with SMTP; 23 Mar 2004 10:35:32 -0000
> > >
> >
> > Now, this is completely different and SA possibly can't determine the
host
> > it came from, so it uses the next received line or just skips the test,
> > don't know.
> >
> > Conclusion: look how to enable MIDs in Outlook.
>
>
>
>
> Well, I Google'd for a while... And on the page
> http://extelligence.ringlet.net/roam/archives/000018.html I found;
>
> "I could scarcely believe that, so seconds after the meeting there I was,
> googling for Outlook 2003 Message-Id, and sure enough, there it was: on
the
> first page of results, a news bulletin which, among others, states that
> Outlook 2003 only includes Message-ID's when sending e-mail through an
> Exchange server. What really got my goat, though, was the reason for
> omitting the Message-Id header: you see, users complained that their
machine
> names were visible on the Internet!"
>
> So, it's a "feature" in my Outlook 2003 to *not* include the Message-Id
> header.. Jesus!
>
> So, how can we go about to make the MSGID_FROM_MTA_SHORT detection skip
> clients which adds the mailer tag:
> "X-Mailer: Microsoft Office Outlook, Build 11.0.5510", or at least Build
11?
> Outlook 2003 do add that, so can the detection engine be told to check
that
> and not tag up the e-mails as spam because of the erroneous "Message-Id"
> tag?
>
> Anders Norrbring
> Norrbring Consulting
> (with a faulty mail client)
Complain loudly to Microsoft. *I* am not going to change MY rules to cater
to Microsoft catering to spammers.
{^_^}
Re: Outlook 2003 idiot mail client... :(
Posted by Kai Schaetzl <ma...@conactive.com>.
Bob George wrote on Wed, 24 Mar 2004 10:59:59 -0500:
> Still, it's trivial for a spammer to get around this. It would be nice
> if this were one of their policy settings rather than a blanket default.
>
Indeed. Or, as you explained in another message, use another method to
generate a unique string. We don't want the hostname, we just want the MID
;-)
Kai
--
Kai Sch�tzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org
Re: Outlook 2003 idiot mail client... :(
Posted by Bob George <ma...@ttlexceeded.com>.
Kai Schaetzl wrote:
> Bob George wrote on Tue, 23 Mar 2004 17:55:39 -0500:
>>But only if NOT using Exchange, right? :)
>
> Well, but in that case Exchange will add it's own hostname, won't it? So,
> the problem remains the same. Or doesn't Exchange add a Received line when
> it gets the mail from the client?
After re-reading http://www.slipstick.com/emo/2003/up031211.htm where it
states:
> [...] Mark says that, as he understands it, Microsoft's position that
> they expect all mail servers to whitelist outgoing mail from Outlook
> 2003 users and add a Message-ID header to fill in the one that
> Outlook omits.
I think I might see some logic at play, although I'm not sure I agree. I
suppose Microsoft's logic is that by removing insertion of that header
from the client to the server, the decision of what to add can be
controlled centrally. In the anti-spam wars, perhaps this makes a
twisted kind of sense, in that "normal" users SHOULD be going through a
relay server (ISP or corporate).
Still, it's trivial for a spammer to get around this. It would be nice
if this were one of their policy settings rather than a blanket default.
- Bob
Re: Outlook 2003 idiot mail client... :(
Posted by Loren Wilton <lw...@earthlink.net>.
> Bob George wrote on Tue, 23 Mar 2004 17:55:39 -0500:
>
> > But only if NOT using Exchange, right? :)
> >
>
> Well, but in that case Exchange will add it's own hostname, won't it? So,
> the problem remains the same. Or doesn't Exchange add a Received line when
> it gets the mail from the client?
>
> Kai
When you use an Exchange server with Outlook the server appears to be the
source of the mail as far as the internet is concerned. Here are some
header lines from a message I sent from work and cc'ed to my earthlink
account. My actual sending machine shows up noplace in this set of headers:
Loren
Received: from usea-naimss1.unisys.com ([192.61.61.103])
by killdeer (EarthLink SMTP Server) with ESMTP id 1b6jgZ5J53NZFlr0
for <lw...@earthlink.net>; Wed, 24 Mar 2004 17:03:05 -0800 (PST)
Received: from usea-nagw2.na.uis.unisys.com ([129.224.72.18]unverified) by
usea-naimss1 with InterScan Messaging Security Suite; Wed, 24 Mar 2004
19:06:56 -0600
Received: from usea-nagw2.na.uis.unisys.com ([129.224.72.53]) by
usea-nagw2.na.uis.unisys.com with Microsoft SMTPSVC(5.0.2195.6713);
Wed, 24 Mar 2004 19:02:55 -0600
Received: from usmv-exch1.na.uis.unisys.com ([192.59.173.13]) by
usea-nagw2.na.uis.unisys.com with Microsoft SMTPSVC(5.0.2195.6713);
Wed, 24 Mar 2004 19:02:55 -0600
Received: from USMV-EXCH3.na.uis.unisys.com ([192.59.173.11]) by
usmv-exch1.na.uis.unisys.com with Microsoft SMTPSVC(5.0.2195.6713);
Wed, 24 Mar 2004 17:02:53 -0800
X-MimeOLE: Produced By Microsoft Exchange V6.0.6487.1
content-class: urn:content-classes:message
Subject: RE: [ntdev] Re: Is it advisable to test on HyperThreading machines
?
Date: Wed, 24 Mar 2004 17:02:53 -0800
Message-ID:
<F8...@USMV-EXCH3.na.uis.unisys.com>
Re: Outlook 2003 idiot mail client... :(
Posted by Kai Schaetzl <ma...@conactive.com>.
Bob George wrote on Tue, 23 Mar 2004 17:55:39 -0500:
> But only if NOT using Exchange, right? :)
>
Well, but in that case Exchange will add it's own hostname, won't it? So,
the problem remains the same. Or doesn't Exchange add a Received line when
it gets the mail from the client?
Kai
--
Kai Sch�tzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org
Re: Outlook 2003 idiot mail client... :(
Posted by Bob George <ma...@ttlexceeded.com>.
LuKreme wrote:
> On Mar 23, 2004, at 3:55 PM, Bob George wrote:
>
>> Well, it's OUTLOOK that's broken, but... a meta rule could easily be
>> created to check for both. Something like (untested):
>>
>> header CLIENT_OUTLOOK2003 X-Mailer =~ /Microsoft Office Outlook, Build
>> 11/
>> meta OUTLOOK2003ADJUST (MSGID_FROM_MTA_SHORT && CLIENT_OUTLOOK2003)
>> score OUTLOOK2003ADUST -3
>
>
> Obvious typo. You meant 3.0 of course, not -3.
>
> Right?
>
> :->
Heh, well no but then again, I thought Anders meant HE didn't want to
score MS Outlook messages quite so spammy. Others may decide to go the
other way altogether. :)
- Bbo
Re: Outlook 2003 idiot mail client... :(
Posted by LuKreme <kr...@kreme.com>.
On Mar 23, 2004, at 3:55 PM, Bob George wrote:
> Well, it's OUTLOOK that's broken, but... a meta rule could easily be
> created to check for both. Something like (untested):
>
> header CLIENT_OUTLOOK2003 X-Mailer =~ /Microsoft Office Outlook, Build
> 11/
> meta OUTLOOK2003ADJUST (MSGID_FROM_MTA_SHORT && CLIENT_OUTLOOK2003)
> score OUTLOOK2003ADUST -3
Obvious typo. You meant 3.0 of course, not -3.
Right?
:->
--
Use your key, unlock the door, see what fate might have in store. Come
explore your dreams and Creations, Enter the world of imagination.
Re: Outlook 2003 idiot mail client... :(
Posted by Kai Schaetzl <ma...@conactive.com>.
Anders Norrbring wrote on Wed, 24 Mar 2004 00:22:00 +0100:
> I just got this hint... Can it be elaborated on?
>
you can just score that rule to 0 and you are done. But it only affects
your mail.
Kai
--
Kai Sch�tzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org
SV: Outlook 2003 idiot mail client... :(
Posted by Anders Norrbring <an...@norrbring.biz>.
> > [...]
> > So, it's a "feature" in my Outlook 2003 to *not* include the Message-Id
> > header.. Jesus!
>
> But only if NOT using Exchange, right? :)
>
> > So, how can we go about to make the MSGID_FROM_MTA_SHORT detection skip
> > clients which adds the mailer tag:
> > "X-Mailer: Microsoft Office Outlook, Build 11.0.5510", or at least Build
> 11?
> > Outlook 2003 do add that, so can the detection engine be told to check
> that
> > and not tag up the e-mails as spam because of the erroneous "Message-Id"
> > tag?
>
> Well, it's OUTLOOK that's broken, but... a meta rule could easily be
> created to check for both. Something like (untested):
>
> header CLIENT_OUTLOOK2003 X-Mailer =~ /Microsoft Office Outlook, Build 11/
> meta OUTLOOK2003ADJUST (MSGID_FROM_MTA_SHORT && CLIENT_OUTLOOK2003)
> score OUTLOOK2003ADUST -3
>
> So there's a fix for outlook, and a nice way for spammers to circumvent
> the check. :) Maybe tighten the rules down more to be prudent.
>
> It might be easier to just tweak the score for MSGID_FROM_MTA_SHORT
> down, since blindly trusting the X-Mailer header isn't a great strategy
> unless you've got some other factor you can add to the meta to prevent
> simple forging.
I just got this hint... Can it be elaborated on?
rename the existing rule __MSG_ID_FROM_MTA_SHORT, then add these:
header __NO_MSGID_OUTLOOK =~ /Microsoft Office Outlook, Build
11.\d{1,2}.\d{4}/
meta MSG_ID_FROM_MTA_SHORT (__MSG_ID_FROM_MTA_SHORT && !
__NO_MSGID_OUTLOOK)
Re: Outlook 2003 idiot mail client... :(
Posted by jdow <jd...@earthlink.net>.
----- Original Message -----
From: "Bob Apthorpe" <ap...@cynistar.net>
To: <sp...@incubator.apache.org>
Sent: Wednesday, 2004 March, 24 08:11
Subject: Re: Outlook 2003 idiot mail client... :(
> On Wed, 24 Mar 2004 16:32:27 +0100 "Kai Schaetzl"
<ma...@conactive.com> wrote:
>
> > Anders Norrbring wrote on Wed, 24 Mar 2004 07:06:39 +0100:
> >
> > > Rest assured that I will, I actually paid some good money for my
Office pack
> > >
> >
> > Anders, you should complain loudly to Microsoft about this, you are a
> > customer. If only a few customers complain about the MID, but nobody
complains
> > about the missing MID then they won't re-add it.
>
> Seriously. It appears that Microsoft intentionally broke their mail
> client on the request of their (idiot) customers. This indicates that
> Microsoft listens to their customers, at least to their idiot customers.
> So you have to get Microsoft's attention somehow; you may have to
> pretend to be an idiot to get them to listen to you.
...
> Or maybe this is done purposely to degrade SpamAssassin accuracy to
> promote Microsoft's proprietary email Caller-ID scheme...
>
> /me adjusts tinfoil hat...
Bob, there is a simple enough trick to cover this, in theory. Teach
email clients on corporate networks to catch the fact that there is
no Message-ID field and that the email headers indicate it is from
Outlook 2003. In that case insert a Message-ID that looks like the
old Outlook Message-ID field. Of course, I bet the spammers do this
before the revised servers get out into the field for the rest of us.
So perhaps that Outlook rule needs to have an Outlook 2003 rule of
opposite polarity. Or else with all the discussion here, where there
are spammer scum listening, perhaps the Outlook rule has become
obsolete. I know if I were a spammer with software that mimicked or
tried to mimic Outlook I'd certainly be putting in the properly
formatted header now.
{^_^}
Re: Outlook 2003 idiot mail client... :(
Posted by Justin <li...@numbnuts.net>.
On Thu, 25 Mar 2004, Bob Apthorpe wrote:
> The RFC is crystal clear that it's primarily the MUA's job to add a
> Message-Id.
I don't mean to defend Microsoft in any way shape or form but the RFC is
"crystal clear" that a msg-id is not required. Period. The wording
leaves no room for interpretation. It's recommended but not required.
If the RFC authors wanted it to be required they would have said so.
Just because almost all MUA developers programmed their MUAs to generate a
msg-id doesn't mean that the sole standard, the RFC that defines our
beloved mail format, should be reinterpreted by each admin to fit their
personal wishes and desires. Actions like this do nothing but segregate
mail systems. Many of the popular web browsers support(ed) the
non-standard Netscape and IE HTML tags. That doesn't make those tags
official though.
Personally I think the msg-id should be assigned by the MTA if for no
other reason than for logging and possible authentication purposes. My
wishes are part of the RFC though so I'm not going to create a rule that
enforces those wishes.
My $.02
Justin
Re: Outlook 2003 idiot mail client... :(
Posted by Bob Apthorpe <ap...@cynistar.net>.
Hi,
On Thu, 25 Mar 2004 12:45:35 -0000 "John Hall" <jo...@cambridgetechgroup.com> wrote:
> "Bob Apthorpe" <ap...@cynistar.net> wrote in message
> news:20040324101131.11831030.apthorpe+sa@cynistar.net...
>
> > The Outlook missing-Message-Id problem really needs to be fixed because
> > in this case the RFCs are very clear that 'SHOULD' means 'SHALL.'
>
> It means "that there may exist valid reasons in particular circumstances to
> ignore a particular item, but the full implications must be understood and
> carefully weighed before choosing a different course".
Yes, and (to me at least) "preventing trivial information leakage by
dropping the Message-Id header entirely rather than by simple,
innovative coding in order to satisfy a minority of our customers, used
as a default setting with no apparent way to revert to previous,
expected behavior" is not a valid reason, the implications of which were
certainly not carefully weighed.
This is not an anti-Microsoft screed. We have RFCs for a reason, in this
case for defense-in-depth for debugging mail problems and fundamental
interoperability. Running roughshod over the RFCs doesn't scale; what
happens when <your favorite MTA vendor> decides that "the MTA SHOULD add
a Message-Id to messages that lack them" translates to the action "I
don't feel like coding this because only b0rked MUAs emit mail without
Message-Ids?" Vendors and customers can point fingers all day long; the
upshot is that if all vendors complied with the letter and spirit of the
appropriate RFCs, there wouldn't be a problem.
The RFC is crystal clear that it's primarily the MUA's job to add a
Message-Id. Now if Microsoft wants to let their customers break that,
they should provide that as an option that is off by default. The vendor
should leave it to the customer to decide whether to deviate from RFCs,
but otherwise comply. As it is, there's no customer choice. Users can't
fix this problem even if they want to.
Regardless, if this problem is going to be solved, it won't be solved
here.
-- Bob
Re: Outlook 2003 idiot mail client... :(
Posted by LuKreme <kr...@kreme.com>.
On Mar 26, 2004, at 11:21 AM, Justin wrote:
> The fact that the RFCs do not require a msg-id should stop SA from
> creating a rule that enforces a *non-requirement*.
No it shouldn't. SA creates rules that indicate spammishness. The
rules are run against the corpus and the ones that get the most
spam/least ham get the highest scores.
And, for the purposes of 99% of users in 99% of cases, a SHOULD in a
RFC is just as important as a MUST. A SHOULD only exists to allow some
flexibility in extremely limited and extremely specialized cases. Not
to simply ignore because you feel like it.
--
I listen to the wind, to the wind of my soul
Re: Outlook 2003 idiot mail client... :(
Posted by Justin <li...@numbnuts.net>.
On Thu, 25 Mar 2004, John Hall wrote:
> "Bob Apthorpe" <ap...@cynistar.net> wrote in message
> news:20040324101131.11831030.apthorpe+sa@cynistar.net...
>
> > The Outlook missing-Message-Id problem really needs to be fixed because
> > in this case the RFCs are very clear that 'SHOULD' means 'SHALL.'
>
> It means "that there may exist valid reasons in particular circumstances to
> ignore a particular item, but the full implications must be understood and
> carefully weighed before choosing a different course".
The fact that the RFCs do not require a msg-id should stop SA from
creating a rule that enforces a *non-requirement*. If the RFC was worded
differently then this would be a different matter entirely. It doesn't
however and we shouldn't create our own standards to hold others
accountable to.
Justin
Re: Outlook 2003 idiot mail client... :(
Posted by John Hall <jo...@cambridgetechgroup.com>.
"Bob Apthorpe" <ap...@cynistar.net> wrote in message
news:20040324101131.11831030.apthorpe+sa@cynistar.net...
> The Outlook missing-Message-Id problem really needs to be fixed because
> in this case the RFCs are very clear that 'SHOULD' means 'SHALL.'
It means "that there may exist valid reasons in particular circumstances to
ignore a particular item, but the full implications must be understood and
carefully weighed before choosing a different course".
John
Re: Outlook 2003 idiot mail client... :(
Posted by Justin <li...@numbnuts.net>.
On Thu, 25 Mar 2004, Bob George wrote:
> I suspect somewhere there's a spammer lurking on-list, developing a 100%
> RFC-compliant tool.
You mean a starving programmer hired by a spammer to develop a 100%
RFC-compliant tool, correct? Spammers aren't smart enough to do it on
their own. :)
Justin
Re: Outlook 2003 idiot mail client... :(
Posted by Bob George <ma...@ttlexceeded.com>.
Jay Levitt wrote:
> [...]
> Not to mention the fact that SA filters for spam, not RFC compliance. Even
> assuming arguendo that Outlook *isn't* complying with the RFC, if a large
> volume of ham is generated by clients that break the RFC, then that
> particular breakage is obviously not good evidence of spamming.
Thanks for saying that better than I did. There are many things that
annoy me in email (top-posting with full quotes [of PGP signatures!],
over-long signatures) but they're not spam indicators one way or the other.
I suspect somewhere there's a spammer lurking on-list, developing a 100%
RFC-compliant tool.
- Bob
Re: Outlook 2003 idiot mail client... :(
Posted by Jay Levitt <ja...@shopwatch.org>.
Bob Apthorpe wrote:
> The Outlook missing-Message-Id problem really needs to be fixed because
> in this case the RFCs are very clear that 'SHOULD' means 'SHALL.'
How do you figure? The meaning of "SHOULD" in an RFC has been pretty clear
for a few decades now... if the RFC is broken, then the RFC is broken, but
it sounds like Outlook complies.
Not to mention the fact that SA filters for spam, not RFC compliance. Even
assuming arguendo that Outlook *isn't* complying with the RFC, if a large
volume of ham is generated by clients that break the RFC, then that
particular breakage is obviously not good evidence of spamming.
Jay Levitt
Re: Outlook 2003 idiot mail client... :(
Posted by Bob Apthorpe <ap...@cynistar.net>.
On Wed, 24 Mar 2004 16:32:27 +0100 "Kai Schaetzl" <ma...@conactive.com> wrote:
> Anders Norrbring wrote on Wed, 24 Mar 2004 07:06:39 +0100:
>
> > Rest assured that I will, I actually paid some good money for my Office pack
> >
>
> Anders, you should complain loudly to Microsoft about this, you are a
> customer. If only a few customers complain about the MID, but nobody complains
> about the missing MID then they won't re-add it.
Seriously. It appears that Microsoft intentionally broke their mail
client on the request of their (idiot) customers. This indicates that
Microsoft listens to their customers, at least to their idiot customers.
So you have to get Microsoft's attention somehow; you may have to
pretend to be an idiot to get them to listen to you.
The Outlook missing-Message-Id problem really needs to be fixed because
in this case the RFCs are very clear that 'SHOULD' means 'SHALL.'
Aside: It's maybe a little unfair to claim that the Microdoft customers
that objected to RFC-compliant client-supplied Message-Id are idiots,
since including host name in the Message-Id is a form of information
leakage, divulging the names of machines on a (private?) network. This
is a somewhat legitimate concern. Tenuous but legitimate.
For the few ultra-paranoid/myopic customers that even care about
this, there are a few simple ways around this problem that don't involve
dropping the Message-Id's. The Message-Id just needs to be unique;
there's no requirement for a hostname to be divulged. Substituting
MD5(hostname) or SHA1(hostname) or CRC32(hostname) or processor-ID or
any of a wide array of easily-generatable static machine-specific unique
identifiers for hostname solves the problem.
Second, if you're going through all this trouble, you need to omit,
strip off, or otherwise mangle the Received header showing the transfer
from the client MUA to the local MTA before it exits the local network
otherwise you leak the information you just went through all that
trouble to scrub from the Message-Id.
But if you're going to break RFCs by having your MTA mangle the Received
headers of outbound messages, isn't it a simple matter to mangle the
Message-Id at the same time, rather than breaking every new copy of
Outlook? Don't forget that someone in the organization is probably still
using an older version of Outlook or another RFC-compliant MUA that
properly adds a Message-Id, so you still have to mangle the Message-Id
at the MTA before it exits the system (Rule #1: never trust the client.)
Microsoft somehow bought the counterargument and better, didn't add a
hidden checkbox or Registry entry to let you work around it. Honestly, I
don't know how to harness the level of collective idiocy needed to
convince Microsoft to undo this brokenness.
Or maybe this is done purposely to degrade SpamAssassin accuracy to
promote Microsoft's proprietary email Caller-ID scheme...
/me adjusts tinfoil hat...
-- Bob
Re: Outlook 2003 idiot mail client... :(
Posted by Kai Schaetzl <ma...@conactive.com>.
Anders Norrbring wrote on Wed, 24 Mar 2004 07:06:39 +0100:
> Rest assured that I will, I actually paid some good money for my Office pack
>
Anders, you should complain loudly to Microsoft about this, you are a
customer. If only a few customers complain about the MID, but nobody complains
about the missing MID then they won't re-add it.
Kai
--
Kai Sch�tzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org
SV: Outlook 2003 idiot mail client... :(
Posted by Anders Norrbring <an...@norrbring.biz>.
[8<]
> So, given that you'll start with a significant strike against you (3+
> points, IIRC), make sure you're not otherwise spammy. That .biz domain
> will count another few points against you. You could sign up for habeas,
> and this is exactly the sort of situation it's meant for, I believe.
> Says "Although I send from a spammy domain, this is NOT spam and I'm
> willing to risk financial penalties if I voilate this trust." That's why
> I still continue to give a negative score to habeas-tagged messages.
Rest assured that I will, I actually paid some good money for my Office pack
and not that I'm a Windows lover, but software availability makes me stay
for now. I guess I could revert to Outlook XP (2002), but it doesn't feel
'right' to do it. Much of my work demand the integration with other Office
apps, but I could do a search for other mail clients that can do the same,
if there are any.
> Quoting the article you cited:
>
> --- cut here --- cut here ---
> > Also, RFC 2821 <http://www.faqs.org/rfc/rfc2821.txt> allows the first
> > or last SMTP server in the chain to add a Message-Id if none is
> > present, but there are still many RFC-compliant SMTP servers out
> > there which do not do so, and the client has absolutely no guarantees
> > that a message will get a Message-Id header - and IMHO, every
> > message should have one.
> --- cut here --- cut here ---
>
> Well, there's another fix. Get your server admin to fix it for you?
I'm my own server admin, but frankly I'm not (yet) aware of how I could fix
it?
Anders Norrbring
Re: Outlook 2003 idiot mail client... :(
Posted by Bob George <ma...@ttlexceeded.com>.
Anders Norrbring wrote:
> [...] Here it is again, posted before... All the headers from a mail
> sent out via my Outlook.. If there's something in the headers that
> can be treated as "unique" or "safe", please let me know.
Well, realize that you can only affect what's received on YOUR system,
not others. The rule I provided would avoid YOUR system tagging Outlook
2003 messages as spam, but obviously won't help for stuff you send
elsewhere.
So, given that you'll start with a significant strike against you (3+
points, IIRC), make sure you're not otherwise spammy. That .biz domain
will count another few points against you. You could sign up for habeas,
and this is exactly the sort of situation it's meant for, I believe.
Says "Although I send from a spammy domain, this is NOT spam and I'm
willing to risk financial penalties if I voilate this trust." That's why
I still continue to give a negative score to habeas-tagged messages.
> I'm afraid Bill G. doesn't invite med to his breakfasts, so my
> complaints about the lacking header element probably won't be
> noticed.. :)
Do messages from other clients besides Outlook 2003 get labelled
correctly? That may be the easiest fix.
Quoting the article you cited:
--- cut here --- cut here ---
> Also, RFC 2821 <http://www.faqs.org/rfc/rfc2821.txt> allows the first
> or last SMTP server in the chain to add a Message-Id if none is
> present, but there are still many RFC-compliant SMTP servers out
> there which do not do so, and the client has absolutely no guarantees
> that a message will get a Message-Id header - and IMHO, every
> message should have one.
--- cut here --- cut here ---
Well, there's another fix. Get your server admin to fix it for you?
--- cut here --- cut here ---
> Oh well... score another one for muddy thinking. The end result?
> Outlook 2003 currently generates scores of messages that may very
> well be considered <http://www.slipstick.com/emo/2003/up031211.htm>
> spam
>
<http://www.mentalized.net/journal/2003/01/03/experiencing_outlook_xp/index.asp>
> by many popular spam filters out there - and for a very, very good
> reason, too.
--- cut here --- cut here ---
And there's why the world's not necessarily going to drop that scoring
for you!
I found THIS amusing: http://www.slipstick.com/emo/2003/up031211.htm
> [...] Mark says that, as he understands it, Microsoft's position that
> they expect all mail servers to whitelist outgoing mail from Outlook
> 2003 users and add a Message-ID header to fill in the one that
> Outlook omits.
- Bob
SV: Outlook 2003 idiot mail client... :(
Posted by Anders Norrbring <an...@norrbring.biz>.
> Anders Norrbring wrote:
>
> > [...]
> > So, it's a "feature" in my Outlook 2003 to *not* include the Message-Id
> > header.. Jesus!
>
> But only if NOT using Exchange, right? :)
>
> > So, how can we go about to make the MSGID_FROM_MTA_SHORT detection skip
> > clients which adds the mailer tag:
> > "X-Mailer: Microsoft Office Outlook, Build 11.0.5510", or at least Build
> 11?
> > Outlook 2003 do add that, so can the detection engine be told to check
> that
> > and not tag up the e-mails as spam because of the erroneous "Message-Id"
> > tag?
>
> Well, it's OUTLOOK that's broken, but... a meta rule could easily be
> created to check for both. Something like (untested):
>
> header CLIENT_OUTLOOK2003 X-Mailer =~ /Microsoft Office Outlook, Build 11/
> meta OUTLOOK2003ADJUST (MSGID_FROM_MTA_SHORT && CLIENT_OUTLOOK2003)
> score OUTLOOK2003ADUST -3
>
> So there's a fix for outlook, and a nice way for spammers to circumvent
> the check. :) Maybe tighten the rules down more to be prudent.
>
> It might be easier to just tweak the score for MSGID_FROM_MTA_SHORT
> down, since blindly trusting the X-Mailer header isn't a great strategy
> unless you've got some other factor you can add to the meta to prevent
> simple forging.
>
> - Bob
Here it is again, posted before... All the headers from a mail sent out via
my Outlook.. If there's something in the headers that can be treated as
"unique" or "safe", please let me know.
I'm afraid Bill G. doesn't invite med to his breakfasts, so my complaints
about the lacking header element probably won't be noticed.. :)
------------
Return-Path: <an...@norrbring.biz>
Received: from mail.the-server.net ([unix socket])
by iris (Cyrus v2.1.15) with LMTP; Tue, 23 Mar 2004 11:35:17 +0100
X-Sieve: CMU Sieve 2.2
Received: from localhost (localhost [127.0.0.1])
by mail.the-server.net (Postfix) with ESMTP id 8F02E10B7
for <an...@the-server.net>; Tue, 23 Mar 2004 11:35:17 +0100 (CET)
Received: from mail.the-server.net ([127.0.0.1]) by localhost (iris
[127.0.0.1]) (amavisd-new, port 10024) with LMTP id 00478-04-9 for
<an...@the-server.net>; Tue, 23 Mar 2004 11:35:07 +0100 (CET)
Received: from edit (edit.the-server.net [192.168.111.30])
by mail.the-server.net (Postfix) with ESMTP id 47FAE1099
for <an...@the-server.net>; Tue, 23 Mar 2004 11:35:02 +0100 (CET)
From: "Anders Norrbring" <an...@norrbring.biz>
To: <an...@the-server.net>
Subject:
Date: Tue, 23 Mar 2004 11:35:07 +0100
Organization: Norrbring Consulting
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
X-Mailer: Microsoft Office Outlook, Build 11.0.5510
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
thread-index: AcQQwoKtCVChFjvBQJmJ8aPi1IcuzQ==
Message-Id: <20...@mail.the-server.net>
X-Virus-Scanned: by Amavisd-New, Kaspersky Lab's AV & NOD32 AV at
the-server.net
X-Spam-Status: No, hits=4.1 tagged_above=3.0 required=7.0 tests=BIZ_TLD,
MSGID_FROM_MTA_SHORT
X-Spam-Level: ****
Later,
Anders.
Re: Outlook 2003 idiot mail client... :(
Posted by Bob George <ma...@ttlexceeded.com>.
Anders Norrbring wrote:
> [...]
> So, it's a "feature" in my Outlook 2003 to *not* include the Message-Id
> header.. Jesus!
But only if NOT using Exchange, right? :)
> So, how can we go about to make the MSGID_FROM_MTA_SHORT detection skip
> clients which adds the mailer tag:
> "X-Mailer: Microsoft Office Outlook, Build 11.0.5510", or at least Build 11?
> Outlook 2003 do add that, so can the detection engine be told to check that
> and not tag up the e-mails as spam because of the erroneous "Message-Id"
> tag?
Well, it's OUTLOOK that's broken, but... a meta rule could easily be
created to check for both. Something like (untested):
header CLIENT_OUTLOOK2003 X-Mailer =~ /Microsoft Office Outlook, Build 11/
meta OUTLOOK2003ADJUST (MSGID_FROM_MTA_SHORT && CLIENT_OUTLOOK2003)
score OUTLOOK2003ADUST -3
So there's a fix for outlook, and a nice way for spammers to circumvent
the check. :) Maybe tighten the rules down more to be prudent.
It might be easier to just tweak the score for MSGID_FROM_MTA_SHORT
down, since blindly trusting the X-Mailer header isn't a great strategy
unless you've got some other factor you can add to the meta to prevent
simple forging.
- Bob
- Bob
Re: Outlook 2003 idiot mail client... :(
Posted by Bob George <ma...@ttlexceeded.com>.
LuKreme wrote:
> [...]
> Personally, anyone stupid enough to run LookOut as their mail client
> deserves whatever Microshaft gives them.
Well hey now, I'm no fan of Microsoft, but I'm not ready to comdemn
anybody because of the OS or mail client they use. There's a pretty
significant portion of the population who have NO CHOICE in the matter.
I certainly hope "anti-spam" isn't turning into "anti-microsoft." That's
about as effective as a lot of other misdirected efforts going on in the
world today.
- Bob
LuKreme 2004 - idiot mail reply...
Posted by Anders Norrbring <an...@norrbring.biz>.
> On Mar 23, 2004, at 4:27 PM, Justin wrote:
> > 3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
> > may exist valid reasons in particular circumstances to ignore a
> > particular item, but the full implications must be understood and
> > carefully weighed before choosing a different course.
>
> And one of the implications to consider is being tagged as spammer scum.
>
> You makes your bed, you lies in it.
>
> Personally, anyone stupid enough to run LookOut as their mail client
> deserves whatever Microshaft gives them.
So, now we're reverting to throwing shit in the face of others? I suggest
you take out you personal aggressions somewhere else, perhaps a gym?
Anders.
Re: Outlook 2003 idiot mail client... :(
Posted by LuKreme <kr...@kreme.com>.
On Mar 23, 2004, at 4:27 PM, Justin wrote:
> 3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
> may exist valid reasons in particular circumstances to ignore a
> particular item, but the full implications must be understood and
> carefully weighed before choosing a different course.
And one of the implications to consider is being tagged as spammer scum.
You makes your bed, you lies in it.
Personally, anyone stupid enough to run LookOut as their mail client
deserves whatever Microshaft gives them.
--
Generalizations are always inaccurate. -Mugsy
Re: Outlook 2003 idiot mail client... :(
Posted by Justin <li...@numbnuts.net>.
On Tue, 23 Mar 2004, Anders Norrbring wrote:
> So, it's a "feature" in my Outlook 2003 to *not* include the Message-Id
> header.. Jesus!
Well, let me through another little twist in this. Per section 3.6.4 of
RFC 2822, a Message-ID field is not required.
"Though optional, every message SHOULD have a "Message-ID:" field."
http://www.rfc-editor.org/rfc/rfc2822.txt
Section 1.2.1 refers to the definition of "SHOULD" found in #3 RFC 2119
as:
3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.
Ya'll can debate the purpose, praticality, wording, and meaning among that
yourselves. I just wanted to point that out. Personally I've configured
Sendmail to reject messages with syntactically incorrect msg-id fields.
If a Message-ID is present, it's format is strictly defined. When I
implemented the msg-id check I used to catch a lot of mass mailers that
were poorly written. That was a few years ago. Now it catches well under
10k messages per week.
My $.02 to the thread,
Justin