FP's on BAD_ENC_HEADER in bounces from Microsoft SMTPSVC

[Nick Leverton <nj...@leverton.org>]

Microsoft SMTPSVC seems to trigger BAD_ENC_HEADER when sending bounces if 
it's been given a non-English bounce template (or whatever M$ use for 
configuring that). Even bounces to correctly encoded mail.  I've got quite 
a number of examples, and all of them have a foreign language Subject 
line, encoded in =?unicode-1-1-utf-7?, but wrapped onto more than one 
line.

Three samples attached, from different SMTPSMV servers - jal.co.uk (Japan 
Airlines), ohl.de and ifg.com, all of which are legit correspondents.  I 
removed the original pre-bounce message except for its final Received 
header - this one which bounced the mail - and I overwrote the usernames 
for their privacy, but the mail domains, the incriminating Received line, 
and the rest of the headers are original.

header BAD_ENC_HEADER  ALL =~ /=\?[^?\s]+\?[^?\s]\?\s*[^?]+\s(?!\?=)/

I think the problem is that the Subject header although encoded does have 
spaces in, which is invalid for RFC2047 (and headers can only be split on 
whitespace, so the folded headers are doubly invalid).  

Is anyone else having trouble from this ?  With a net/bayes score of 3.100, 
it doesn't need many other rules to reach spam levels.  One of those 
samples hit HTML_50_60, HTML_FONT_BIG, HTML_MESSAGE, HTML_TAG_EXIST_TBODY, 
HTML_WEB_BUGS and  NO_REAL_NAME for a total score of 5.196 (ouch).  I've 
zeroed the BAD_ENC_HEADER score for myself, but wonder if it's affecting 
others too ?

Nick

Re: FP's on BAD_ENC_HEADER in bounces from Microsoft SMTPSVC

[alan premselaar <al...@12inch.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Leverton wrote:
[snip]

> We don't have an M$ mail server (and I for one don't want one).  We're a 
> Unix shop, as qmail and qpsmtpd in our own headers shows :)  
> 
> I'm quite prepared to believe this is a MS bug, it certainly looks like it.  
> But it seems to be a long term one - seen in emails from SMTPSVC versions 
> 5.0.2195.6713 and 6.0.3790.1830.  Remote MS servers, configured for 
> foreign languages, sending genuine non-spam bounces to non-spam mails 
> cause SA to FP on this rule.
> 
> Nick

Nick,

 As much as I'd like to say "yeah, it's yet another bad MS program" ...
i'm not entirely convinced of that.  We used to run Exchange 2000 with
Japanese DSN messages and I'm certain that we didn't have this problem.
 As such, I suspect that the organizations that are using these
particular Exchange servers have probably just mis-configured them.

Of course I find it curious that they would use utf-7 encoding instead
of utf-8 (which seems more widely accepted).

Alan


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEkjeVE2gsBSKjZHQRApMVAKCd4nBjHBPAPSDdy+ZYnbovP3YqTACgkEu/
vvA7PRzYcUULfx+kTp/aEoM=
=fv/m
-----END PGP SIGNATURE-----

Re: FP's on BAD_ENC_HEADER in bounces from Microsoft SMTPSVC

[Nick Leverton <nj...@leverton.org>]

On Thursday 15 June 2006 03:43, Alan Premselaar wrote:
> Aside from the QP scatter, this subject doesn't look like it's properly
> encoded.  if memory serves, if the encoded subject needs to be broken
> across multiple lines, each line needs to have its own encoding
> start/end tags.
>
> so it should look something like:
>
> Subject: =?unicode-1-1-utf-7?Q?<encoded_part>?=
> 	=?unicode-1-1-utf-7?Q?<more_encoded_part>?=
>
> (someone correct me if i'm wrong)

In RFC 2047 section 2, it's clear you're right and M$ are wrong:

   	"... unencoded white space
   characters (such as SPACE and HTAB) are FORBIDDEN within an
   'encoded-word'.  For example, the character sequence

      =?iso-8859-1?q?this is some text?=

   would be parsed as four 'atom's, rather than as a single 'atom' ..."

RFC 822 (which 2047 was based on) says that a CRLF followed by space or tab 
is syntactically equivalent to just a space or tab.  RFC 2822 has similar 
language.  Hence encoded-words cannot be split across lines.

> Of course it's hard to tell because of the QuotedPrintable encoding
> artifacts, but it looks like your MS mail server is in some way
> misconfigured.

Yes sorry about the extra QP coding on the attachments, I normally use mutt 
so didn't realise kmail was going to mangle them.  I can resend them if 
you want, but they match what I sent except for the topmost Received line.

We don't have an M$ mail server (and I for one don't want one).  We're a 
Unix shop, as qmail and qpsmtpd in our own headers shows :)  

I'm quite prepared to believe this is a MS bug, it certainly looks like it.  
But it seems to be a long term one - seen in emails from SMTPSVC versions 
5.0.2195.6713 and 6.0.3790.1830.  Remote MS servers, configured for 
foreign languages, sending genuine non-spam bounces to non-spam mails 
cause SA to FP on this rule.

Nick

Re: FP's on BAD_ENC_HEADER in bounces from Microsoft SMTPSVC

[Alan Premselaar <al...@12inch.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Leverton wrote:
[snip]


> Subject: =3D?unicode-1-1-utf-7?Q?+kU1P4XK2YUuQGnfl- =20
> 	(+MKgw6TD8-)?=3D
> 

Aside from the QP scatter, this subject doesn't look like it's properly
encoded.  if memory serves, if the encoded subject needs to be broken
across multiple lines, each line needs to have its own encoding
start/end tags.

so it should look something like:

Subject: =?unicode-1-1-utf-7?Q?<encoded_part>?=
	=?unicode-1-1-utf-7?Q?<more_encoded_part>?=

(someone correct me if i'm wrong)

Of course it's hard to tell because of the QuotedPrintable encoding
artifacts, but it looks like your MS mail server is in some way
misconfigured.

Either that or something else is wrapping the headers and breaking the
encoding.

HTH

alan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEkMk5E2gsBSKjZHQRAtCkAKDaCCjpeUTVIzC/vYppbh8Bn0j66gCffW1v
27zlnRX/AbNzWsw7HgTj14I=
=IaOn
-----END PGP SIGNATURE-----