You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Marcus Schopen <li...@localguru.de> on 2016/12/08 15:54:26 UTC
how to parse back through forwarding headers to find the true
source IP
Hi,
some of my users forward external mails to my host. In some cases those
forwarding hosts don't filter spam. How do I parse back through
forwarding headers to find the true source IP and run dnsrbl checks on
that IP. I don't want to reject those mails in case of spam, so that the
forwarding host will become a backscatter, but just marking them. I
tried to set the forwarding host IPs to trusted_networks, which helps
with wrong dnswl.org checks, but RBL checks are disabled then. Any ideas
how to handle that?
Ciao
Marcus
Re: how to parse back through forwarding headers to find the true
source IP
Posted by Marcus Schopen <li...@localguru.de>.
Hi,
Am Donnerstag, den 08.12.2016, 16:25 +0000 schrieb RW:
> On Thu, 08 Dec 2016 16:54:26 +0100
> Marcus Schopen wrote:
>
> > Hi,
> >
> > some of my users forward external mails to my host. In some cases
> > those forwarding hosts don't filter spam. How do I parse back through
> > forwarding headers to find the true source IP and run dnsrbl checks on
> > that IP. I don't want to reject those mails in case of spam, so that
> > the forwarding host will become a backscatter, but just marking them.
> > I tried to set the forwarding host IPs to trusted_networks, which
> > helps with wrong dnswl.org checks, but RBL checks are disabled then.
> > Any ideas how to handle that?
>
> You need to put them in internal networks for spamassassin to do
> last-external checks.
Thanks to all for helping!
The forwarded mails go this way:
Scammer -> 62.146.106.13[2-3] -> 62.146.106.2[1-6] -> MY_IP
My current setup looks like this now:
-----------
trusted_networks MY_IP
# udag.de forwarding: forwarding servers connecting MY_IP
trusted_networks 62.146.106.21
trusted_networks 62.146.106.22
trusted_networks 62.146.106.23
trusted_networks 62.146.106.24
trusted_networks 62.146.106.25
trusted_networks 62.146.106.26
# udag.de forwarding: original receiving servers
trusted_networks 62.146.106.132
trusted_networks 62.146.106.133
-----------
I didn't set any internal_networks, because "If trusted_networks is set
and internal_networks is not, the value of trusted_networks will be used
for this parameter."[1], so in my understanding my internal_networks are
equal to trusted_networks.
After adding above rules to my SA config I checked some incoming mails
and filtering seems to be correct. The forwarding servers
62.146.106.2[1-6] are not listed e.g. on SPAMCOP, INPS.de etc. so it
must be the scammer's IP and DNSWL check doesn't come up with a
RCVD_IN_DNSWL_LOW hit, which one gets if 62.146.106.2[1-6] hosts are
checked directly:
--------
Dec 9 18:55:09 server mimedefang.pl[19467]: uB9Ht5SU012194:
MDLOG,uB9Ht5SU012194,spam,22.67 BAYES_50 DIGEST_MULTIPLE DKIM_SIGNED
FROM_EXCESS_BASE64 HTML_IMAGE_ONLY_24 HTML_MESSAGE MIME_HTML_ONLY
PYZOR_CHECK RAZOR2_CF_RANGE_51_100 RAZOR2_CF_RANGE_E8_51_100
RAZOR2_CHECK RCVD_IN_BL_SPAMCOP_NET RCVD_IN_BRBL_LASTEXT
RCVD_IN_DNSBL_INPS_DE RCVD_IN_SBL RCVD_IN_SBL_CSS RP_MATCHES_RCVD
T_DKIM_INVALID URIBL_ABUSE_SURBL URIBL_BLACK URIBL_DBL_SPAM URIBL_SBL
URIBL_SBL_A,62.146.106.23,<srs0
+jcbd=xy=r-resources.com=noreply-n-ztzccx-mspy.kqtc@udag.de>,<ma...@mydomain.de>,Subject ...
--------
I checked another user, who's forwarding mails from ISP Strato to my
host and there is a strange received header set by forwarder side. In
this case mails go this way:
Scammer -> 81.169.145.98 -> 81.169.146.14[4-9] -> MY_IP
But receiving IP smtp.rzone.de[81.169.145.98] never comes up in the
Received header. The header looks like this:
-----
Received: from srv544.mailer-service.de ([62.138.228.44])
by smtp.rzone.de (RZmta 39.10 OK)
with ESMTP id A02f69sB9H4Aw9o
for <ma...@mydomain.de>;
Fri, 9 Dec 2016 18:04:10 +0100 (CET)
-----
How can SA parse for back to the original receiving host IP
smtp.rzone.de[81.169.145.98], if they just come up with "smtp.rzone.de
(RZmta 39.10 OK)". Would that nevertheless work or is it in this case
only possible to put the outgoing servers 81.169.146.14[4-9] to my
trusted_networks and does that make sense at all then?
Ciao
Marcus
[1]
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html
--
,---- [ Marcus Schopen ]
| (0>
| //\
| V_/_ D-33602 Bielefeld
|
`----
Re: how to parse back through forwarding headers to find the true
source IP
Posted by RW <rw...@googlemail.com>.
On Thu, 08 Dec 2016 16:54:26 +0100
Marcus Schopen wrote:
> Hi,
>
> some of my users forward external mails to my host. In some cases
> those forwarding hosts don't filter spam. How do I parse back through
> forwarding headers to find the true source IP and run dnsrbl checks on
> that IP. I don't want to reject those mails in case of spam, so that
> the forwarding host will become a backscatter, but just marking them.
> I tried to set the forwarding host IPs to trusted_networks, which
> helps with wrong dnswl.org checks, but RBL checks are disabled then.
> Any ideas how to handle that?
You need to put them in internal networks for spamassassin to do
last-external checks.
Re: how to parse back through forwarding headers to find the true
source IP
Posted by RW <rw...@googlemail.com>.
On Thu, 08 Dec 2016 12:18:52 -0500
Kris Deugau wrote:
> Automated forwarding should just be another SMTP hop, and adding the
> third-party host to trusted_networks should work just fine[1].
This works if you haven't defined internal_networks at all, in
which case internal and trusted are assumed to be the same. In the
general case putting them into trusted_networks only fixes the
whitelist rules.
If both internal_networks and trusted_networks are defined then
forwarding servers should ideally be both trusted and internal.
Re: how to parse back through forwarding headers to find the true
source IP
Posted by RW <rw...@googlemail.com>.
On Thu, 08 Dec 2016 17:50:33 -0500
Kris Deugau wrote:
> Yes, both. Strictly speaking these should only go in
> trusted_networks, since these are not *your* hosts, but the way the
> lookups are defined you'll probably need to put them in
> internal_networks.
>
> I took a look at the stock rules and it seems the "black" DNSBL rules
> almost all use -lastexternal, where the "white" ones use
> -firsttrusted. I'm not sure why things are divided that way; I would
> have expected it to be more along the lines of per-DNSBL grouping to
> one or the other.
The idea that internal_networks represents your own servers is a
misleading one. The point of it is to find the relevant MX handover,
which in this case is the one into the forwarders network. It's not
just about DNSBLs, there are other rules that look at the rDNS etc on
this handover. Lastexternal tests are mostly unsafe to run on anything
but an MX handover.
The reason why the trusted network may extent beyond the internal
network is that sometime other peoples' networks may contain submission
servers which SA can't identify unless they record authentication.
Putting such a network into internal_networks may cause FPs because an
unidentified submission client can look like a spammer delivering
directly to MX.
Putting addresses only into trusted effectively turns-off a lot of
useful tests, and it doesn't cause any DNSBL lookups that wouldn't run
anyway.
The reason that whitelists run on firsttrusted is that it's the only IP
address that's both worth testing and unforgeable.
Re: how to parse back through forwarding headers to find the true
source IP
Posted by Kris Deugau <kd...@vianet.ca>.
(Please keep mail on the list)
Marcus Schopen wrote:
> Hi Kris,
>
> thanks for your time!
>
> Am Donnerstag, den 08.12.2016, 12:18 -0500 schrieb Kris Deugau:
>>> On 12/8/2016 10:54 AM, Marcus Schopen wrote:
>>>> Hi,
>>>>
>>>> some of my users forward external mails to my host. In some cases those
>>>> forwarding hosts don't filter spam. How do I parse back through
>>>> forwarding headers to find the true source IP and run dnsrbl checks on
>>>> that IP. I don't want to reject those mails in case of spam, so that the
>>>> forwarding host will become a backscatter, but just marking them. I
>>>> tried to set the forwarding host IPs to trusted_networks, which helps
>>>> with wrong dnswl.org checks, but RBL checks are disabled then. Any ideas
>>>> how to handle that?
>>
>> Not sure what you mean by "RBL checks are disabled then"; do you mean
>> that none of them fire when expected?
>
> On this page I found
>
> https://spamassassin.apache.org/full/3.1.x/doc/Mail_SpamAssassin_Conf.html
>
> trusted_networks: "DNS blacklist checks will never query for hosts on
> these networks."
That means that IPs *in* trusted_networks won't be checked. IPs that
*hand off to* IPs in trusted_networks will be:
Scammer -> Third party forwarder -> Your MX
In the minimal setup, trusted_networks only contains your MX, and the
only IP SA will consider for IP DNSBL lookups is the third party forwarder.
Adding the IP (or IPs) of that forwarder to trusted_networks means SA
will do IP DNSBL lookups on the scammer's IP, assuming it can be parsed
from the forwarder's Received header(s).
Also note, that link is a little stale since IIRC SpamAssassin's trust
path system got some minor revisions and tweaks in 3.2.
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html
covers the current release.
> The mails are forwarded automatically and from the logs of the last
> months I see the same 6 IPs which they are sent from. These IPs are
> separate from emails which were sent via their smtp relay, so they split
> up forwarding IPs from "normal" smtp relay, which is good.
>
> May I ask your for your configuration? Do you put the forwarding IPs to
> "internal_networks" then?
No; I have internal hosts listed in internal_networks and/or
msa_networks as appropriate, a modest set of other local IPs in
trusted_networks, and ~100+ IPs from customer forwarding.
+--trusted_networks---------------------------+
| +--internal_networks--+ +--msa_networks--+ |
| | 192.168.2.0/24 | | 192.168.3.0/29 | |
| +---------------------+ +----------------+ |
| us: 192.168.0.0/22 |
| Them: |
| customer A: |
| 10.0.3.3 |
| 10.0.3.4 |
| 192.168.255.252 |
| customer B: |
| 172.17.34.56 |
+---------------------------------------------+
I *think* there are cases where it's also correct to end up with
msa_networks and internal_networks overlapping, and I've found a few
mail flow paths where adding the remote system to internal_networks
seemed to be the only way to get the DNS lookups pointed to the correct
IP, but those have been rare.
>
> Here is an example for the headers:
>
> ---------
> Return-Path: <SRS0
> +f3sO=XX=e.smdm-mail.de=replies-EMID0AA064V3J7P010JM80B7JJB011IQOI7@udag.de>
> Received: from athosian.udag.de (athosian.udag.de [62.146.106.25])
> by mxrelay.medienlabor.de (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP
> id uB8Dg6xg008952 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384
> bits=256 verify=NOT) for <ma...@mydomain.de>; Thu, 8 Dec 2016 14:42:09
> +0100
>
> Received: from exspamix-02.udag.de (exspamix-02.udag.de
> [62.146.106.133]) by athosian.udag.de (Postfix) with ESMTP id
> 772FD142C2F for <ma...@externaldomain.de>; Thu, 8 Dec 2016 14:42:06
> +0100 (CET)
>
> Received: from mx68e167.fagms.de ([62.144.68.167]) by
> exspamix-02.udag.de with esmtps
> (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.86)
> (envelope-from
> <re...@e.smdm-mail.de>)
> id 1cEyxZ-0008bX-A5 for marcus@externaldomain.de; Thu, 08 Dec 2016
> 14:42:06 +0100
> ---------
>
> At the external, forwarding provider udag.de the mail was received from
> mx68e167.fagms.de (the host I'd like to check by dnsrbl) by
> exspamix-02.udag.de. My receiving MX is mxrelay.medienlabor.de, which
> received the mail from athosian.udag.de then.
>
> Whichs hosts do I have to put to "internal_networks" then, only the
> forwarding host athosian.udag.de, which connects the my server
> (mxrelay.medienlabor.de) or the receiving host exspamix-02.udag.de too?
Yes, both. Strictly speaking these should only go in trusted_networks,
since these are not *your* hosts, but the way the lookups are defined
you'll probably need to put them in internal_networks.
I took a look at the stock rules and it seems the "black" DNSBL rules
almost all use -lastexternal, where the "white" ones use -firsttrusted.
I'm not sure why things are divided that way; I would have expected it
to be more along the lines of per-DNSBL grouping to one or the other.
I can't recall now what subtle difference I found that made putting
third-party relays in internal_networks unusable, but since I built the
configuration it's been working well here to just use trusted_networks.
(Probably some local rule group that relied on the difference between
"trusted" hosts and "internal" hosts.)
-kgd
Re: how to parse back through forwarding headers to find the true
source IP
Posted by Kris Deugau <kd...@vianet.ca>.
> On 12/8/2016 10:54 AM, Marcus Schopen wrote:
>> Hi,
>>
>> some of my users forward external mails to my host. In some cases those
>> forwarding hosts don't filter spam. How do I parse back through
>> forwarding headers to find the true source IP and run dnsrbl checks on
>> that IP. I don't want to reject those mails in case of spam, so that the
>> forwarding host will become a backscatter, but just marking them. I
>> tried to set the forwarding host IPs to trusted_networks, which helps
>> with wrong dnswl.org checks, but RBL checks are disabled then. Any ideas
>> how to handle that?
Not sure what you mean by "RBL checks are disabled then"; do you mean
that none of them fire when expected?
*Some* DNSBL rules will fire differently in this case depending on
whether they were defined with -lastexternal or -firsttrusted, but most
should work fine and do what you want.
I do this here for customers who have domain mail hosted with a third
party, with mail forwarded to their ISP account with us.
Kevin A. McGrail wrote:
> I would answer that you can't. Unless they forward you the source or
> the original email as an attachment, many times that information is lost.
That depends on whether this is automated forwarding at the mail system
level, or by-hand forwarding in any mail client.
Automated forwarding should just be another SMTP hop, and adding the
third-party host to trusted_networks should work just fine[1].
Mail that was forwarded by hand, yeah, you're usually stuck unless you
can teach your users how to properly forward as attachment. Inline
forwards rarely include *any* origin IP info, or any headers at all
other than From, To, CC, Subject, and Date.
-kgd
[1] Unless the third party mail host is one who routes forwarded mail
for a single domain through outbound IPs across half a dozen /8 ranges
(yes, really, I have observed this personally). In which case you can
either give up on getting correct origin IP information, or play
trusted_networks whack-an-IP until the results are "good enough".
Re: how to parse back through forwarding headers to find the true
source IP
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 12/8/2016 10:54 AM, Marcus Schopen wrote:
> Hi,
>
> some of my users forward external mails to my host. In some cases those
> forwarding hosts don't filter spam. How do I parse back through
> forwarding headers to find the true source IP and run dnsrbl checks on
> that IP. I don't want to reject those mails in case of spam, so that the
> forwarding host will become a backscatter, but just marking them. I
> tried to set the forwarding host IPs to trusted_networks, which helps
> with wrong dnswl.org checks, but RBL checks are disabled then. Any ideas
> how to handle that?
>
> Ciao
> Marcus
>
>
I would answer that you can't. Unless they forward you the source or
the original email as an attachment, many times that information is lost.
Regards,
KAM