You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by chuckee <mi...@gmail.com> on 2018/07/25 04:10:01 UTC
Problem with new rules
I'm from a reasonably large ESP and we handle all types of emails being sent
via our servers. We've noticed a change with SpamAssassin in the last few
days/weeks which is causing problems.
The following 2 rules are causing these problems:
3.5 HDR_ORDER_FTSDMCXX_DIRECT Header order similar to spam
and
2.0 MIMEOLE_DIRECT_TO_MX MIMEOLE + direct-to-MX
The high scoring first rule, especially, is causing problems, and is
triggered if a sender is sending emails from Windows Live Mail (it seems to
be just this email client), and their email only has 1 'Received from'
header.
As an ESP we can confirm that it is extremely common for ESP's to strip out
'Received from' headers - if we didn't, then many recipient mail servers
reject emails because they look at the (often bad) reputation of the IP
address of the sender and judge an email on that. For example, if a sender
is sending an email from a hotel, their email would be judged based on the
reputation of the hotel's IP address (which is a ridiculous scenario). We
can confirm that Barracuda is one such email filter that does this.
As such, both of the above rules (in our opinion) should not be in place.
ESP's must strip out received headers to prevent legitimate emails from
getting rejected. If we leave them in to cater for those SpamAssassin rules
then many emails will get rejected based on reputation checks of the
sender's own IP address (which is against proper email protocol, but has
been happening ever since email was invented and will continue to happen).
What can be done to influence whoever made these recent rule changes to
revert things back to how they were?
Thanks
--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html
Re: Problem with new rules
Posted by RW <rw...@googlemail.com>.
On Tue, 24 Jul 2018 21:10:01 -0700 (MST)
chuckee wrote:
> As an ESP we can confirm that it is extremely common for ESP's to
> strip out 'Received from' headers - if we didn't, then many recipient
> mail servers reject emails because they look at the (often bad)
> reputation of the IP address of the sender and judge an email on
> that.
I don't find it to be all that common. Where I have seen it, the ESP
has cited privacy as the reason.
> For example, if a sender is sending an email from a hotel,
> their email would be judged based on the reputation of the hotel's IP
> address (which is a ridiculous scenario). We can confirm that
> Barracuda is one such email filter that does this.
Barracuda is based on SpamAssassin, and I would guess it's still
handling this the same way as SA, and the way it did when a disgruntled
employee posted their rules some years ago.
Deep tests look for the open proxies/relays, static IP addresses of
hacked servers and IP blocks allocated to spammers. If a spammer
controlled address submits to a service provider there's a good chance
its spam. By the look of it the only really high scoring deep rule is
the Spamhaus snowshoe list, which is pretty reliable.
Re: Problem with new rules
Posted by John Hardin <jh...@impsec.org>.
On Tue, 24 Jul 2018, chuckee wrote:
> I'm from a reasonably large ESP and we handle all types of emails being sent
> via our servers. We've noticed a change with SpamAssassin in the last few
> days/weeks which is causing problems.
> 3.5 HDR_ORDER_FTSDMCXX_DIRECT Header order similar to spam
> The high scoring first rule, especially, is causing problems, and is
> triggered if a sender is sending emails from Windows Live Mail (it seems to
> be just this email client), and their email only has 1 'Received from'
> header.
I'd asked for the headers from such a FP message so that I could
investigate WLM-specific mitigations. I've not heard anything since. Does
that mean the lower score limit is a sufficient mitigation?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Users mistake widespread adoption of Microsoft Office for
the development of a document format standard.
-----------------------------------------------------------------------
6 days until the 283rd anniversary of John Peter Zenger's acquittal
Re: Problem with new rules
Posted by John Hardin <jh...@impsec.org>.
On Wed, 25 Jul 2018, David Jones wrote:
> On 07/24/2018 11:10 PM, chuckee wrote:
>> I'm from a reasonably large ESP and we handle all types of emails being
>> sent
>> via our servers. We've noticed a change with SpamAssassin in the last few
>> days/weeks which is causing problems.
>> The following 2 rules are causing these problems:
>> 3.5 HDR_ORDER_FTSDMCXX_DIRECT Header order similar to spam
>> and
>> 2.0 MIMEOLE_DIRECT_TO_MX MIMEOLE + direct-to-MX
>>
>
> Feel free to set these scores to zero or 0.01 for a quick fix to your
> solution while this gets sorted out.
>
>> The high scoring first rule, especially, is causing problems, and is
>> triggered if a sender is sending emails from Windows Live Mail (it seems to
>> be just this email client), and their email only has 1 'Received from'
>> header.
So there are no headers for the internal-to-Windows-Live handoffs? Windows
Live Mail is acting as a MUA w/r/t your MTA?
Or are you stripping Received headers *before* your SA scan?
>> As an ESP we can confirm that it is extremely common for ESP's to strip out
>> 'Received from' headers - if we didn't, then many recipient mail servers
>> reject emails because they look at the (often bad) reputation of the IP
>> address of the sender and judge an email on that. For example, if a sender
>> is sending an email from a hotel, their email would be judged based on the
>> reputation of the hotel's IP address (which is a ridiculous scenario). We
>> can confirm that Barracuda is one such email filter that does this.
>
> You shouldn't have to strip out Received headers to prevent legit emails from
> getting rejected. Either this is from using too aggressive RBLs or you need
> to add to your trusted_networks to look past some Received headers. Another
> option I use is to make meta rules for certain OK Received headers to
> subtract a few points.
>
>> As such, both of the above rules (in our opinion) should not be in place.
>> ESP's must strip out received headers to prevent legitimate emails from
>> getting rejected. If we leave them in to cater for those SpamAssassin rules
>> then many emails will get rejected based on reputation checks of the
>> sender's own IP address (which is against proper email protocol, but has
>> been happening ever since email was invented and will continue to happen).
>
>> What can be done to influence whoever made these recent rule changes to
>> revert things back to how they were?
>
> I guess someone needs to look back through the SVN commits to see when this
> was introduced.
Both were me, based on the S/O in the masscheck corpus.
MIMEOLE_DIRECT_TO_MX has been around for a couple of years now,
HDR_ORDER_FTSDMCXX_DIRECT is recent.
SPAM% HAM% S/O RANK SCORE NAME
12.5652 0.0008 1.000 1.00 0.00 HDR_ORDER_FTSDMCXX_DIRECT
14.3808 0.0229 0.998 0.94 1.00 MIMEOLE_DIRECT_TO_MX
I'm a bit surprised that HDR_ORDER_FTSDMCXX_DIRECT is problematic
because it's partially based on suspicious header ordering. I can see
MIMEOLE_DIRECT_TO_MX being problematic because that's essentially "MSFT
MUA sending direct to my MTA".
I'll set a lower score limit on HDR_ORDER_FTSDMCXX_DIRECT.
I'd be willing to investigate an exclusion for Windows Live Mail, based on
the assumption that MSFT Got It Wrong Again.
If you can provide the headers from such a FP message (zipped, in private
mail) I will see what I can do to tune it.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The promise of nuclear power: electricity too cheap to meter
The reality of nuclear power: FUD too cheap to meter
-----------------------------------------------------------------------
10 days until the 283rd anniversary of John Peter Zenger's acquittal
Re: Problem with new rules
Posted by David Jones <dj...@ena.com>.
On 07/24/2018 11:10 PM, chuckee wrote:
> I'm from a reasonably large ESP and we handle all types of emails being sent
> via our servers. We've noticed a change with SpamAssassin in the last few
> days/weeks which is causing problems.
> The following 2 rules are causing these problems:
> 3.5 HDR_ORDER_FTSDMCXX_DIRECT Header order similar to spam
> and
> 2.0 MIMEOLE_DIRECT_TO_MX MIMEOLE + direct-to-MX
>
Feel free to set these scores to zero or 0.01 for a quick fix to your
solution while this gets sorted out.
> The high scoring first rule, especially, is causing problems, and is
> triggered if a sender is sending emails from Windows Live Mail (it seems to
> be just this email client), and their email only has 1 'Received from'
> header.
> As an ESP we can confirm that it is extremely common for ESP's to strip out
> 'Received from' headers - if we didn't, then many recipient mail servers
> reject emails because they look at the (often bad) reputation of the IP
> address of the sender and judge an email on that. For example, if a sender
> is sending an email from a hotel, their email would be judged based on the
> reputation of the hotel's IP address (which is a ridiculous scenario). We
> can confirm that Barracuda is one such email filter that does this.
>
You shouldn't have to strip out Received headers to prevent legit emails
from getting rejected. Either this is from using too aggressive RBLs or
you need to add to your trusted_networks to look past some Received
headers. Another option I use is to make meta rules for certain OK
Received headers to subtract a few points.
> As such, both of the above rules (in our opinion) should not be in place.
> ESP's must strip out received headers to prevent legitimate emails from
> getting rejected. If we leave them in to cater for those SpamAssassin rules
> then many emails will get rejected based on reputation checks of the
> sender's own IP address (which is against proper email protocol, but has
> been happening ever since email was invented and will continue to happen).
>
> What can be done to influence whoever made these recent rule changes to
> revert things back to how they were?
>
I guess someone needs to look back through the SVN commits to see when
this was introduced.
> Thanks
>
I filter for about 60,000 mailboxes and I don't see any hits in my mail
logs for either of those rules in the past 3 days on my production mail
flow.
--
David Jones