You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Matthias Leisi <ma...@leisi.net> on 2017/10/07 21:41:30 UTC
Whitelisting DKIM-signed domains
Last week at the 41st M3AAWG meeting in Toronto there was considerable interest in domain-based whitelisting information when I presented the dnswl.org project. Obviously, this needs to be authenticated, and that’s what we have DKIM for.
We created an experimental list dwl.dnswl.org (subject to change without prior notice yaddayadda, with minimal infrastructure etc - don’t use it in production yet!), which works like a regular domain- or hostname-based blacklist would. More details are here https://www.dnswl.org/?p=311 <https://www.dnswl.org/?p=311>, but in a nutshell that’s how it could be implemented in SpamAssassin (put it in your local.cf or in some similarly convenient place):
ifplugin Mail::SpamAssassin::Plugin::AskDNS
askdns DNSWL_DWL_HI _DKIMDOMAIN_.dwl.dnswl.org A /^127\.\d+\.\d+\.3/
tflags DNSWL_DWL_HI nice net
describe DNSWL_DWL_HI dwl.dnswl.org high trust
score DNSWL_DWL_HI -5
askdns DNSWL_DWL_MED _DKIMDOMAIN_.dwl.dnswl.org A /^127\.\d+\.\d+\.2/
tflags DNSWL_DWL_MED nice net
describe DNSWL_DWL_MED dwl.dnswl.org medium trust
score DNSWL_DWL_MED -2
askdns DNSWL_DWL_LOW _DKIMDOMAIN_.dwl.dnswl.org A /^127\.\d+\.\d+\.1/
tflags DNSWL_DWL_LOW nice net
describe DNSWL_DWL_LOW dwl.dnswl.org low trust
score DNSWL_DWL_LOW -1
askdns DNSWL_DWL_NONE _DKIMDOMAIN_.dwl.dnswl.org A /^127\.\d+\.\d+\.0/
tflags DNSWL_DWL_NONE nice net
describe DNSWL_DWL_NONE dwl.dnswl.org listed, but no particular trust information available
score DNSWL_DWL_NONE -0.1
endif # Mail::SpamAssassin::Plugin::AskDNS
Note that this only works on DKIM-signed domains (DKIM_VALID).
Any inputs or thoughts are highly appreciated.
— Matthias, for the dnswl.org project
Re: Whitelisting DKIM-signed domains
Posted by Ralph Seichter <m1...@monksofcool.net>.
On 08.10.17 11:55, Matthias Leisi wrote:
> If the DKIM signature does not validate, the rules do not fire.
My bad, I had missed the sentence "Askdns rules awaiting for a tag
which never receives its value never result in a DNS query" in
http://search.cpan.org/dist/Mail-SpamAssassin/lib/Mail/SpamAssassin/Plugin/AskDNS.pm
-Ralph
Re: Whitelisting DKIM-signed domains
Posted by Matthias Leisi <ma...@leisi.net>.
> I have a primary and several secondary domains tied to a DNSWL ID. All
Currently, all domains in a given DNSWL Id share the same trust score. This may change over time, but we want to get some experience first. As a starting point, the trust of the domains is derived from the trust in the IPs - this is likely to change as we go along.
> of these domains can be used to send emails to public mailing lists.
> Some mailing lists break DKIM signatures by modifying subject line or
> body of emails. How does this affect the SpamAssassin score when people
> use the rules you suggested in your blog? In particular, is there a
> "score punishment" if some badly configured mailing lists break valid
> DKIM signatures with their junk additions?
If the DKIM signature does not validate, the rules do not fire. Therefore there also is no „punishment“ for invalid signatures.
Mailing lists breaking DKIM signature are an issue, but one that is outside of the scope of the whitelisting rules. There are ESPs who add a second DKIM signature to outgoing messages - maybe that would be the way to go for mailing lists as well, but I guess this is not an easy or straightforward solution for all cases.
— Matthias
Re: Whitelisting DKIM-signed domains
Posted by Ralph Seichter <m1...@monksofcool.net>.
On 07.10.17 23:41, Matthias Leisi wrote:
> More details are here https://www.dnswl.org/?p=311
Since the blog did not explain it, I'm asking here:
I have a primary and several secondary domains tied to a DNSWL ID. All
of these domains can be used to send emails to public mailing lists.
Some mailing lists break DKIM signatures by modifying subject line or
body of emails. How does this affect the SpamAssassin score when people
use the rules you suggested in your blog? In particular, is there a
"score punishment" if some badly configured mailing lists break valid
DKIM signatures with their junk additions?
-Ralph
Re: Whitelisting DKIM-signed domains
Posted by RW <rw...@googlemail.com>.
On Sat, 7 Oct 2017 15:12:42 -0700 (PDT)
John Hardin wrote:
> On Sat, 7 Oct 2017, Matthias Leisi wrote:
>
> > Note that this only works on DKIM-signed domains (DKIM_VALID).
>
> ...then shouldn't those negatively-scored rules be meta'd with &&
> DKIM_VALID?
It's doing lookups on domains extracted from valid signatures.
Re: Whitelisting DKIM-signed domains
Posted by John Hardin <jh...@impsec.org>.
On Sat, 7 Oct 2017, Matthias Leisi wrote:
> Note that this only works on DKIM-signed domains (DKIM_VALID).
...then shouldn't those negatively-scored rules be meta'd with && DKIM_VALID?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The most glaring example of the cognitive dissonance on the left
is the concept that human beings are inherently good, yet at the
same time cannot be trusted with any kind of weapon, unless the
magic fairy dust of government authority gets sprinkled upon them.
-- Moshe Ben-David
-----------------------------------------------------------------------
191 days since the first commercial re-flight of an orbital booster (SpaceX)
Re: Whitelisting DKIM-signed domains
Posted by Matthias Leisi <ma...@leisi.net>.
> I assume that eventually this DNS query would respond with high trust:
>
> # dig alertsp.chase.com.dwl.dnswl.org
I wondered why this query suddenly appeared from dozens and dozens of sources in the log :)
That is a good example, in that it shows one point to discuss: subdomains. At least in the dnswl.org data we currently have only very few subdomains (because historically we did not care about this). However in practice I believe this is pretty widely used (especially if third parties send email on behalf of the domain owner), so we need to pick up fast on this.
The parent domain is listed in our database (chase.com.dwl.dnswl.org 127.0.2.2). I’m not sure whether the reputation of a parent zone should be „inherited“ by a child zone.
Additionally it is a good example in terms of the score, which we currently calculate based on the score of the IPs associated with this entry. A lot of the JPMChase IPs are on trust-level hi, a few on medium, which is enough to result in an average medium score. There is room for improvement there :)
— Matthias
Re: Whitelisting DKIM-signed domains
Posted by RW <rw...@googlemail.com>.
On Wed, 11 Oct 2017 07:13:29 -0400
Rupert Gallagher wrote:
> The problem I see here is the number of people who really want to
> push blacklists and whitelists, as if they were a magic thing to add
> to their served to catch spam and blame for the failures. Why would
> you trust list B and W knowing that they can be corrupted?
It's a matter of evidence, if you see a list hitting 1 in a few spams
and no ham it seems relevant to take account of that.
Even if a spammer gained control of dnswl the worst they could do is
take a few points off their spam. Worst case scenario is that my
detection rate in SA would drop to 99.5%.
> Are you aware that the
> communications between your server and the remote service can be
> altered to fool you into accepting a cryptolocker?
It's an A record lookup. Is your point is that no-one should every do
DNS look-ups?
> There are privacy
> and secutity considerations that are completely ignored here.
Not really, there's no way of knowing whether the look-up was
generated from spam or ham.
> If you
> are serious about e-mail, stop looking for magic. It is a waste of
> human resources. I would rather see an open debate and collaboration
> on closing the loopholes of the RFC standard while making sure the
> servers implementations are sound and complete.
That's substantially harder, less effective and not prevented by people
doing practical spam filtering.
> I speak out of experience, as I catch 98% of spam without any magic.
Comparing figures like that is practically meaningless, but FWIW I'm
currently catching 99.7% of the spam reaching SA and I consider that
poor.
Re: Whitelisting DKIM-signed domains
Posted by David Jones <dj...@ena.com>.
On 10/11/2017 06:13 AM, Rupert Gallagher wrote:
> The problem I see here is the number of people who really want to push
> blacklists and whitelists, as if they were a magic thing to add to their
> served to catch spam and blame for the failures. Why would you trust
> list B and W knowing that they can be corrupted? Why letting them know
> about your contacts? Are you aware that the communications between your
> server and the remote service can be altered to fool you into accepting
> a cryptolocker? There are privacy and secutity considerations that are
> completely ignored here. If you are serious about e-mail, stop looking
> for magic. It is a waste of human resources. I would rather see an open
> debate and collaboration on closing the loopholes of the RFC standard
> while making sure the servers implementations are sound and complete. I
> speak out of experience, as I catch 98% of spam without any magic.
>
>
Care to elaborate how you "catch 98% of spam without any magic?" Does
it involve SpamAssassin?
--
David Jones
Re: Whitelisting DKIM-signed domains
Posted by Matthias Leisi <ma...@leisi.net>.
I’ll just pick out one particular argument, as RW touched upon the others:
| Why would you trust list B and W knowing that they can be corrupted?
That was one specific concern in the design of dnswl.org <http://dnswl.org/>, which we documented eg here: https://www.dnswl.org/?page_id=23 <https://www.dnswl.org/?page_id=23> („How is this different from other whitelisting services?“)
Like many other lists, the cost of running dnswl.org <http://dnswl.org/> is paid by receivers - those doing more than 100’000 queries per day on the IP-based list in our case are asked to get a subscription and to rsync the data locally (we may extend that to the domain-based list, but that is still in experimental stage anyway). Thus the commercial incentives of the organisation (to the degree that they would actually matter) are very much aligned with the receivers, basically ruling out any benefits of corruption.
Yes, about once a year there is someone claiming „i just paid a subscription, now list me!“. In these cases, we send them a „thanks, but no thanks“ note, give them a refund on the subscription, and remove their account.
— Matthias
Re: Whitelisting DKIM-signed domains
Posted by Rupert Gallagher <ru...@protonmail.com>.
The problem I see here is the number of people who really want to push blacklists and whitelists, as if they were a magic thing to add to their served to catch spam and blame for the failures. Why would you trust list B and W knowing that they can be corrupted? Why letting them know about your contacts? Are you aware that the communications between your server and the remote service can be altered to fool you into accepting a cryptolocker? There are privacy and secutity considerations that are completely ignored here. If you are serious about e-mail, stop looking for magic. It is a waste of human resources. I would rather see an open debate and collaboration on closing the loopholes of the RFC standard while making sure the servers implementations are sound and complete. I speak out of experience, as I catch 98% of spam without any magic.
Sent from ProtonMail Mobile
On Sun, Oct 8, 2017 at 4:18 PM, David Jones <dj...@ena.com> wrote:
> On 10/08/2017 08:42 AM, Rupert Gallagher wrote: > You are blinded by your purpose. > > > > On Sun, Oct 8, 2017 at 9:45 AM, Matthias Leisi > wrote: >> > Am 08.10.2017 um 00:55 schrieb Rupert Gallagher : > > Whitelisting >> DKIM-signed domains is a bad idea for at least two reasons: >> mass-mailing services, and spammers who send from real addresses of >> people whose passwords were easy to guess. This is not whitelisting >> any and all DKIM-signed domain (that would obviously be foolish). This >> is about whitelisting DKIM-signed domains with a positive reputation. >> And „whitelisting" here means, that some points are deducted from the >> SpamAssassin result. — Matthias No one is forcing anyone to add this to their SA setup. I personally think it's an excellent idea and have been promoting a similar concept with a long list of whitelist_auth entries of reputable domains based on the envelope-from which is mostly aligned with SPF_PASS. The good far outweighs the bad if you have a well-tuned MTA doing most of the rejecting of bad senders before the message makes it to SA. Adding some of this logic to SA can only improve things. Let's say your own bank (i.e. chase.com) sends you an email about a loan or a credit card which is legitimate. They DKIM signed it as alertsp.chase.com and the envelope-from was no-reply@alertsp.chase.com. Now a spammer sends the exact same email body but used their own DKIM and envelope-from domain. Both emails it hit SPF_PASS, DKIM_VALID, and DKIM_VALID_AU in SA. How are you going to allow in the chase.com email and block the other one? You have to use something based on authorization (SPF) and/or authentication (DKIM) to trust the alertsp.chase.com domain. whitelist_auth no-reply@alertsp.chase.com I assume that eventually this DNS query would respond with high trust: # dig alertsp.chase.com.dwl.dnswl.org It's already listed on a few other Internet whitelists. Then you can train the spammer's email as spam in your Bayesian DB or add custom content rules to score this email high and the real chase.com email will score low. -- David Jones @leisi.net> @leisi.net>
Re: Whitelisting DKIM-signed domains
Posted by David Jones <dj...@ena.com>.
On 10/08/2017 08:42 AM, Rupert Gallagher wrote:
> You are blinded by your purpose.
>
>
>
> On Sun, Oct 8, 2017 at 9:45 AM, Matthias Leisi <matthias@leisi.net
> <ma...@leisi.net>> wrote:
>> > Am 08.10.2017 um 00:55 schrieb Rupert Gallagher : > > Whitelisting
>> DKIM-signed domains is a bad idea for at least two reasons:
>> mass-mailing services, and spammers who send from real addresses of
>> people whose passwords were easy to guess. This is not whitelisting
>> any and all DKIM-signed domain (that would obviously be foolish). This
>> is about whitelisting DKIM-signed domains with a positive reputation.
>> And „whitelisting" here means, that some points are deducted from the
>> SpamAssassin result. — Matthias
No one is forcing anyone to add this to their SA setup. I personally
think it's an excellent idea and have been promoting a similar concept
with a long list of whitelist_auth entries of reputable domains based on
the envelope-from which is mostly aligned with SPF_PASS. The good far
outweighs the bad if you have a well-tuned MTA doing most of the
rejecting of bad senders before the message makes it to SA. Adding some
of this logic to SA can only improve things.
Let's say your own bank (i.e. chase.com) sends you an email about a loan
or a credit card which is legitimate. They DKIM signed it as
alertsp.chase.com and the envelope-from was no-reply@alertsp.chase.com.
Now a spammer sends the exact same email body but used their own DKIM
and envelope-from domain. Both emails it hit SPF_PASS, DKIM_VALID, and
DKIM_VALID_AU in SA.
How are you going to allow in the chase.com email and block the other
one? You have to use something based on authorization (SPF) and/or
authentication (DKIM) to trust the alertsp.chase.com domain.
whitelist_auth no-reply@alertsp.chase.com
I assume that eventually this DNS query would respond with high trust:
# dig alertsp.chase.com.dwl.dnswl.org
It's already listed on a few other Internet whitelists.
Then you can train the spammer's email as spam in your Bayesian DB or
add custom content rules to score this email high and the real chase.com
email will score low.
--
David Jones
Re: Whitelisting DKIM-signed domains
Posted by Rupert Gallagher <ru...@protonmail.com>.
You are blinded by your purpose.
On Sun, Oct 8, 2017 at 9:45 AM, Matthias Leisi <ma...@leisi.net> wrote:
>> Am 08.10.2017 um 00:55 schrieb Rupert Gallagher : > > Whitelisting DKIM-signed domains is a bad idea for at least two reasons: mass-mailing services, and spammers who send from real addresses of people whose passwords were easy to guess. This is not whitelisting any and all DKIM-signed domain (that would obviously be foolish). This is about whitelisting DKIM-signed domains with a positive reputation. And „whitelisting" here means, that some points are deducted from the SpamAssassin result. — Matthias @protonmail.com>
Re: Whitelisting DKIM-signed domains
Posted by Matthias Leisi <ma...@leisi.net>.
> Am 08.10.2017 um 00:55 schrieb Rupert Gallagher <ru...@protonmail.com>:
>
> Whitelisting DKIM-signed domains is a bad idea for at least two reasons: mass-mailing services, and spammers who send from real addresses of people whose passwords were easy to guess.
This is not whitelisting any and all DKIM-signed domain (that would obviously be foolish). This is about whitelisting DKIM-signed domains with a positive reputation. And „whitelisting“ here means, that some points are deducted from the SpamAssassin result.
— Matthias
Re: Whitelisting DKIM-signed domains
Posted by Matthias Leisi <ma...@leisi.net>.
> Am 08.10.2017 um 01:01 schrieb Benny Pedersen <me...@junc.eu>:
>
> so report spam to dnswl ?
That’s always very welcome :) This was recently updated and included in the self service. If logged in on https://www.dnswl.org/selfservice/ you’ll see a section labelled „Spam Reporting“.
Simple emails to admins/at/dnswl.org are also welcome, but the form helps us to get things automatically managed and spam reports becoming more effective.
— Matthias
Re: Whitelisting DKIM-signed domains
Posted by Benny Pedersen <me...@junc.eu>.
Rupert Gallagher skrev den 2017-10-08 00:55:
> Whitelisting DKIM-signed domains is a bad idea for at least two
> reasons: mass-mailing services, and spammers who send from real
> addresses of people whose passwords were easy to guess.
so report spam to dnswl ?
Re: Whitelisting DKIM-signed domains
Posted by Georg Faerber <ge...@riseup.net>.
On 17-10-07 18:55:35, Rupert Gallagher wrote:
> Whitelisting DKIM-signed domains is a bad idea for at least two
> reasons: mass-mailing services, and spammers who send from real
> addresses of people whose passwords were easy to guess.
I second this.
Cheers,
Georg
Re: Whitelisting DKIM-signed domains
Posted by Rupert Gallagher <ru...@protonmail.com>.
Whitelisting DKIM-signed domains is a bad idea for at least two reasons: mass-mailing services, and spammers who send from real addresses of people whose passwords were easy to guess.
Sent from ProtonMail Mobile
On Sat, Oct 7, 2017 at 11:41 PM, Matthias Leisi <ma...@leisi.net> wrote:
> Last week at the 41st M3AAWG meeting in Toronto there was considerable interest in domain-based whitelisting information when I presented the dnswl.org project. Obviously, this needs to be authenticated, and that’s what we have DKIM for.
>
> We created an experimental list dwl.dnswl.org (subject to change without prior notice yaddayadda, with minimal infrastructure etc - don’t use it in production yet!), which works like a regular domain- or hostname-based blacklist would. More details are here https://www.dnswl.org/?p=311, but in a nutshell that’s how it could be implemented in SpamAssassin (put it in your local.cf or in some similarly convenient place):
>
> ifplugin Mail::SpamAssassin::Plugin::AskDNS
>
> askdns DNSWL_DWL_HI _DKIMDOMAIN_.dwl.dnswl.org A /^127.d+.d+.3/
> tflags DNSWL_DWL_HI nice net
> describe DNSWL_DWL_HI dwl.dnswl.org high trust
> score DNSWL_DWL_HI -5
>
> askdns DNSWL_DWL_MED _DKIMDOMAIN_.dwl.dnswl.org A /^127.d+.d+.2/
> tflags DNSWL_DWL_MED nice net
> describe DNSWL_DWL_MED dwl.dnswl.org medium trust
> score DNSWL_DWL_MED -2
>
> askdns DNSWL_DWL_LOW _DKIMDOMAIN_.dwl.dnswl.org A /^127.d+.d+.1/
> tflags DNSWL_DWL_LOW nice net
> describe DNSWL_DWL_LOW dwl.dnswl.org low trust
> score DNSWL_DWL_LOW -1
>
> askdns DNSWL_DWL_NONE _DKIMDOMAIN_.dwl.dnswl.org A /^127.d+.d+.0/
> tflags DNSWL_DWL_NONE nice net
> describe DNSWL_DWL_NONE dwl.dnswl.org listed, but no particular trust information available
> score DNSWL_DWL_NONE -0.1
>
> endif # Mail::SpamAssassin::Plugin::AskDNS
>
> Note that this only works on DKIM-signed domains (DKIM_VALID).
>
> Any inputs or thoughts are highly appreciated.
>
> — Matthias, for the dnswl.org project