You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Brian Conry <bc...@bestpractical.com> on 2023/01/06 22:23:50 UTC
Looking for advice about limiting DNS queries
Hi,
First things first:
* SpamAssassin version: 3.4.2
* Debian 10
* SA is created and invoked as a Perl object by a MIMEDefang filter
What I'm looking for is a way to tell SA to only run DNS checks on names
that it finds in the headers of the message, i.e. to not scan the body
of the message for names.
The motivation for this is that some of the mail addresses we operate
are for security response teams that regularly receive mail that
contains reports about things like signs of malware.
For example a report from a security appliance that it saw a system
doing DNS queries for a known bitcoin mining malware domain.
The problem is that SA is picking that name from the body of the mail
message and running the full set of DNS checks on it. This includes the
various DNSBL lookups, which are fine, as well as things like DKIM that
require records from within the domain.
The result of this is that every time one of our mail servers handles a
message with one of these reports it makes DNS queries that will trigger
monitoring on our network for devices that might be infected with
bitcoin mining malware. Fortunately the servers in question don't also
handle the warnings that we receive about this possible malware so we
don't have a feedback loop.
I've looked through the debug-level logging of the rule processing and
am fairly confident in my assessment of the problem - I can see
information about which rules are being invoked and triggering DNS
queries and all of that seems fine, but what I didn't notice was
anything covering how SA created the list of domains to check from the
mail message.
I don't think that there's any configuration or options to do what I'm
asking, but I wanted to ask some experts before making any changes to
our configs.
Thank you,
Brian Conry
Re: Looking for advice about limiting DNS queries
Posted by "Kevin A. McGrail" <km...@apache.org>.
I am 99% sure you will be unable to implement that in SA natively and
easily without something such as a milter. Using mimedefang, we have
significant code to allow people to submit samples to create the KAM
ruleset and maintain the RBL. In short, I think we have solved the exact
problem you're talking about and happy to give you pointers.
However, you could add the recipient address to a welcome list entry. That
would effectively make scanning disabled.
I'm assuming that won't work.
KAM
On Fri, Jan 6, 2023, 17:24 Brian Conry <bc...@bestpractical.com> wrote:
> Hi,
>
> First things first:
> * SpamAssassin version: 3.4.2
> * Debian 10
> * SA is created and invoked as a Perl object by a MIMEDefang filter
>
> What I'm looking for is a way to tell SA to only run DNS checks on names
> that it finds in the headers of the message, i.e. to not scan the body
> of the message for names.
>
> The motivation for this is that some of the mail addresses we operate
> are for security response teams that regularly receive mail that
> contains reports about things like signs of malware.
>
> For example a report from a security appliance that it saw a system
> doing DNS queries for a known bitcoin mining malware domain.
>
> The problem is that SA is picking that name from the body of the mail
> message and running the full set of DNS checks on it. This includes the
> various DNSBL lookups, which are fine, as well as things like DKIM that
> require records from within the domain.
>
> The result of this is that every time one of our mail servers handles a
> message with one of these reports it makes DNS queries that will trigger
> monitoring on our network for devices that might be infected with
> bitcoin mining malware. Fortunately the servers in question don't also
> handle the warnings that we receive about this possible malware so we
> don't have a feedback loop.
>
> I've looked through the debug-level logging of the rule processing and
> am fairly confident in my assessment of the problem - I can see
> information about which rules are being invoked and triggering DNS
> queries and all of that seems fine, but what I didn't notice was
> anything covering how SA created the list of domains to check from the
> mail message.
>
> I don't think that there's any configuration or options to do what I'm
> asking, but I wanted to ask some experts before making any changes to
> our configs.
>
> Thank you,
> Brian Conry
>
Re: Looking for advice about limiting DNS queries
Posted by jeremy ardley <je...@ardley.org>.
On 8/1/23 13:57, Brian Conry wrote:
> Hello again,
>
> I'm replying to my own message because I don't want to single out
> anyone who has already replied. There was value in each of your
> responses.
>
> This is going to be a long email, and for that you have my apologies,
> but I can't think of any way to make it shorter without it sounding
> like "dude, just trust me, I know what I'm doing", and I don't want to
> be that guy.
>
> Thank you for the time and thought that went into the replies. I
> sincerely appreciate your concern for the effectiveness of our spam
> filtering.
>
> One thing is clear to me from those response is that I did a poor job
> explaining the situation, and for that you also have my apologies.
>
> This is my attempt to correct that error.
>
> First, any configuration change I would make would be scoped to _only_
> the one (1) recipient address in question - nothing will be changed
> about what SA does for any other address that we handle.
>
> Second, this recipient address is for a queue in an RTIR [1]
> installation for one of our hosted customers. The purpose of this
> queue is to receive reports of suspicious activity from elsewhere, and
> the queue is worked by trained security professionals. This address
> is already configured to receive all messages no matter what the spam
> score is. It is their job to look at them and assess whether or not
> it represents an incident, and if so what response is needed.
>
> Third, to expand on something I alluded to briefly, the emails in
> question are generated by a security appliance on our customer's
> network, in accordance with their security policy and posture. The
> warnings we're getting when our mail server performs these DNS queries
> are coming from _our_ network infrastructure, which is AWS.
>
> As I understand things, I have several options.
>
> Option 1) Do nothing with any configuration.
> We will continue to get notification from AWS of this suspicious
> activity, often several times in one day, that we then have to go and
> correlate with mail logs to confirm that the suspicious DNS queries
> were, in fact, related to the spam filtering of an email, and that the
> email in question was for this queue that is specifically for
> receiving that sort of content.
> The danger with this is that we will become lax in our checking of the
> mail logs and that this will essentially devolve into a variant on
> option 2, but with more work.
>
> Option 2) Tell our network monitoring (AWS) to ignore these findings
> for our mail servers.
> This seems fairly reasonable, as we know that our mail servers will
> make these queries semi-regularly as they are running the spam
> filtering on the messages for this recipient.
> The downside is that it will also turn off all notification to us if
> similar content were to be received at another address, potentially
> one that isn't handled by trained security professionals, or even if
> (heaven forbid) our mail servers were to be compromised by bitcoin
> mining malware. That last one shouldn't be possible due to other
> controls, but there's no denying that there is some added risk in
> auto-ignoring these warnings.
>
> Option 3) Skip all spam checking for this recipient.
> It is, after all, associated with an incident response queue, expected
> to receive email messages with body content that contains names of, or
> even links directly to, known malware domains, and is staffed by
> security professionals.
> And yet, this all-or-nothing approach feels like it's sacrificing some
> possible good, which leads to my questions regarding a hypothetical
> Option 4.
>
> Option 4) Targeted disabling of specific checks for this _one_
> recipient that preserves as much value as possible for the remaining
> checks.
>
> I think there are several variants here, and this is where I know that
> I don't have the expertise necessary to make the optimal decision.
>
> From what I can tell from the reports, the only queries that are
> triggering the security alert for our mail servers are the ones made
> for records in the (known malware) domain or one of its subdomains.
>
> In the debug logs that I inspected there were three queries:
> 1) malwaredomain.com/A
> 2) subdomain.malwaredomain.com/A
> 3) malwaredomain.com/NS
> As best as I can tell, the results of these queries were all used for
> additional DNSRBL queries, but if AWS is noticing that part of the
> context they aren't letting on.
>
> Variant A) disable all DNSRBL checks. :(
>
> Variant B) disable only those RBLs that ask for the information that
> triggers these queries. This is an improvement, but it also skips
> those same checks on everything in the message headers.
>
> Variant C) disable some/most/all checks for names found only in the
> message body. This would provide full checking of all names found in
> the headers and skip only that content in which we expect to find
> trouble.
>
> I believe I can do variants A) and B), so worst case would be choosing
> B, but I'm willing to put in some additional work to implement variant
> C) if that is possible.
>
> If you've made it this far, I congratulate you on your endurance and
> thank you for your time.
>
> Thanks,
> Brian
>
> [1] https://github.com/bestpractical/rtir#readme
It appears on brief inspection that the core of your problem is AWS
complaining about suspicious activity from your DNS queries relating to
spam.
If this is so, you have the option to run your own DNS, and/or delegate
your DNS forwarding to a service that is more helpful to your operations
than AWS.
In relation to some anti-spam sites used to check sender IPs, the advice
is to not use any well-known DNS forwarder such as google 8.8.8.8.
Running your own DNS server seems to be more acceptable to the anti-spam
sites.
.
--
Jeremy
Re: Looking for advice about limiting DNS queries
Posted by Jared Hall <ja...@jaredsec.com>.
On 1/8/2023 12:57 AM, Brian Conry wrote:
> ...
> Third, to expand on something I alluded to briefly, the emails in
> question are generated by a security appliance on our customer's
> network, in accordance with their security policy and posture. The
> warnings we're getting when our mail server performs these DNS queries
> are coming from _our_ network infrastructure, which is AWS.
>
> As I understand things, I have several options.
> ...
No, there are more options than that; 210 by my count including hybrid
solutions. Others here probably have more.
For you, I recommend the use of Shortcircuiting in association with
whitelist_from_rcvd.
Make sure the Shortcircuit module is loaded. Check SA's v320.pre file.
Then, In your local.cf add this:
score USER_IN_WHITELIST -100
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
shortcircuit USER_IN_WHITELIST on
endif # Mail::SpamAssassin::Plugin::Shortcircuit
whitelist_from_rcvd <SE...@DOMAIN> <ENVELOPE_DOMAIN_FROM>
With SA v3.2, I personally don't trust the DKIM and SPF modules but if
the offending security server emails pass DKIM/SPF checks, then you can
play around with using the whitelist_auth function:
whitelist_auth <SE...@DOMAIN>
Now, your mail server is happy. Emails to the security people from any
other source will be subjected to the full checks of SA, so they will be
happy. The rest of your users will be happy.
Keep it simple. Happiness abounds :)
Re: Looking for advice about limiting DNS queries
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2023-01-06 at 17:23:50 UTC-0500 (Fri, 6 Jan 2023 16:23:50 -0600)
Brian Conry <bc...@bestpractical.com>
is rumored to have said:
> Hi,
>
> First things first:
> * SpamAssassin version: 3.4.2
> * Debian 10
I don't know what the Debian versioning status is, but that is a very
old and very likely broken SA if it has not had 3.4.6 patches
backported.
> * SA is created and invoked as a Perl object by a MIMEDefang filter
On 2023-01-08 at 00:57:47 UTC-0500 (Sat, 7 Jan 2023 23:57:47 -0600)
Brian Conry <bc...@bestpractical.com>
is rumored to have said:
> First, any configuration change I would make would be scoped to _only_
> the one (1) recipient address in question - nothing will be changed
> about what SA does for any other address that we handle.
Easy: in your mimedefang-filter file, just skip the entire SA check if a
message is for that one recipient. Note that you cannot exclude users
from milter handling inside Postfix itself. (Your Option 3)
Slightly harder: in your mimedefang-filter file, load alternative rules
for that one recipient which excludes (zero scores) URIDNSBL rules.
(variation on Option 4)
More work: in your mimedefang-filter file, selectively exempt or
special-rule messages for that one recipient based on something more
selective than just the recipient, e.g. sender, size, subject, etc.
(variation on Option 4)
Hardest: slap some sense into whoever has decreed particular DNS queries
to be suspect. Policing mail system DNS with the same rules one uses to
protect random Windows PCs driven directly by humans from their users is
unworkable by design.
It may well be that simply excluding RTIR inbound traffic entirely from
SA checks would be the wisest choice. If you are using the Bayes rules
with auto-learning (which you may be by default) there's a risk of
mis-training on that anomalous mail flow and applying the "learning" to
everyone else's mail.
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
Re: Looking for advice about limiting DNS queries
Posted by Brian Conry <bc...@bestpractical.com>.
Hello again,
I'm replying to my own message because I don't want to single out anyone
who has already replied. There was value in each of your responses.
This is going to be a long email, and for that you have my apologies,
but I can't think of any way to make it shorter without it sounding like
"dude, just trust me, I know what I'm doing", and I don't want to be
that guy.
Thank you for the time and thought that went into the replies. I
sincerely appreciate your concern for the effectiveness of our spam
filtering.
One thing is clear to me from those response is that I did a poor job
explaining the situation, and for that you also have my apologies.
This is my attempt to correct that error.
First, any configuration change I would make would be scoped to _only_
the one (1) recipient address in question - nothing will be changed
about what SA does for any other address that we handle.
Second, this recipient address is for a queue in an RTIR [1]
installation for one of our hosted customers. The purpose of this queue
is to receive reports of suspicious activity from elsewhere, and the
queue is worked by trained security professionals. This address is
already configured to receive all messages no matter what the spam score
is. It is their job to look at them and assess whether or not it
represents an incident, and if so what response is needed.
Third, to expand on something I alluded to briefly, the emails in
question are generated by a security appliance on our customer's
network, in accordance with their security policy and posture. The
warnings we're getting when our mail server performs these DNS queries
are coming from _our_ network infrastructure, which is AWS.
As I understand things, I have several options.
Option 1) Do nothing with any configuration.
We will continue to get notification from AWS of this suspicious
activity, often several times in one day, that we then have to go and
correlate with mail logs to confirm that the suspicious DNS queries
were, in fact, related to the spam filtering of an email, and that the
email in question was for this queue that is specifically for receiving
that sort of content.
The danger with this is that we will become lax in our checking of the
mail logs and that this will essentially devolve into a variant on
option 2, but with more work.
Option 2) Tell our network monitoring (AWS) to ignore these findings for
our mail servers.
This seems fairly reasonable, as we know that our mail servers will make
these queries semi-regularly as they are running the spam filtering on
the messages for this recipient.
The downside is that it will also turn off all notification to us if
similar content were to be received at another address, potentially one
that isn't handled by trained security professionals, or even if (heaven
forbid) our mail servers were to be compromised by bitcoin mining
malware. That last one shouldn't be possible due to other controls, but
there's no denying that there is some added risk in auto-ignoring these
warnings.
Option 3) Skip all spam checking for this recipient.
It is, after all, associated with an incident response queue, expected
to receive email messages with body content that contains names of, or
even links directly to, known malware domains, and is staffed by
security professionals.
And yet, this all-or-nothing approach feels like it's sacrificing some
possible good, which leads to my questions regarding a hypothetical
Option 4.
Option 4) Targeted disabling of specific checks for this _one_ recipient
that preserves as much value as possible for the remaining checks.
I think there are several variants here, and this is where I know that I
don't have the expertise necessary to make the optimal decision.
From what I can tell from the reports, the only queries that are
triggering the security alert for our mail servers are the ones made for
records in the (known malware) domain or one of its subdomains.
In the debug logs that I inspected there were three queries:
1) malwaredomain.com/A
2) subdomain.malwaredomain.com/A
3) malwaredomain.com/NS
As best as I can tell, the results of these queries were all used for
additional DNSRBL queries, but if AWS is noticing that part of the
context they aren't letting on.
Variant A) disable all DNSRBL checks. :(
Variant B) disable only those RBLs that ask for the information that
triggers these queries. This is an improvement, but it also skips those
same checks on everything in the message headers.
Variant C) disable some/most/all checks for names found only in the
message body. This would provide full checking of all names found in
the headers and skip only that content in which we expect to find trouble.
I believe I can do variants A) and B), so worst case would be choosing
B, but I'm willing to put in some additional work to implement variant
C) if that is possible.
If you've made it this far, I congratulate you on your endurance and
thank you for your time.
Thanks,
Brian
[1] https://github.com/bestpractical/rtir#readme
Re: Looking for advice about limiting DNS queries
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 06.01.23 16:23, Brian Conry wrote:
>What I'm looking for is a way to tell SA to only run DNS checks on
>names that it finds in the headers of the message, i.e. to not scan
>the body of the message for names.
the URIDNSBL does this and it produces very good results.
By disabling this you are lowering SA effectivity.
>The motivation for this is that some of the mail addresses we operate
>are for security response teams that regularly receive mail that
>contains reports about things like signs of malware.
>For example a report from a security appliance that it saw a system
>doing DNS queries for a known bitcoin mining malware domain.
I remember solving problem like this - malware scanner was reporting our
filter looking for log4j hostname.
It happened because the machine was target of nessus scan...
you should relax such "security" checks, because, yes, SA can check for
domain mentioned in e-mail passing it.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Chernobyl was an Windows 95 beta test site.