You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2017/10/13 21:55:10 UTC
[Bug 7477] New: Direct DNS Querying Per DNSBL Zone
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
Bug ID: 7477
Summary: Direct DNS Querying Per DNSBL Zone
Product: Spamassassin
Version: unspecified
Hardware: All
OS: All
Status: NEW
Severity: enhancement
Priority: P2
Component: Libraries
Assignee: dev@spamassassin.apache.org
Reporter: guenther@rudersport.de
Target Milestone: Undefined
Feature request for a configuration option to set nameservers (similar to the
existing dns_servers setting) on a per-DNSBL zone basis.
This feature would be useful for local and custom third-party DNSBLs, but also
in certain environments where admins don't have full control of the nameservers
used (see following in-depth discussion). Besides directly specifying a
nameserver, this feature also can query for the DNSBLs authoritative
nameservers on its own.
This would of course be a non-default option, and should come with a clear
warning not to use unless the admin knows why and what he is doing. For that
reason, the setting's documentation is not located in the main Conf docs.
I'd like to thank Invaluement, Rob McEwen, who permitted me to work on and
spend time on this by funding my work on this project as part of the PCCC team
and to publish it under the Apache License, Version 2.0.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
--- Comment #4 from Karsten Bräckelmann <gu...@rudersport.de> ---
(In reply to Benny Pedersen from comment #3)
> could we change to sqlite ?
>
> so pr dnsbl changes to pr sqlite db cache ?, why keep it dns when all nearly
> dont like to have dns data sync ?
This feature is about specific configuration for a(ny given) DNSBL, while still
using the general configuration for all other DNSBLs.
No part of this about DNS is new, so using SQLite instead of DNS simply is not
an option.
> is dns really that unstable that it really needs fixing in spamassassin ?
This is not about fixing unstable DNS.
> possible begin to use enlists with ip listning ?
>
> why do i say it again, i find enlist very use full to have local url listed,
That has nothing to do with this feature request.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
Henrik Krohns <ap...@hege.li> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |KlausRusch@atmedia.net
--- Comment #5 from Henrik Krohns <ap...@hege.li> ---
*** Bug 3500 has been marked as a duplicate of this bug. ***
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
--- Comment #1 from Karsten Bräckelmann <gu...@rudersport.de> ---
Created attachment 5464
--> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5464&action=edit
per-zone resolver
Current state. Not fully operational, thus not yet committing to SVN but
placing here.
- Direct querying implemented. Future improvement should include periodically
refreshing of authoritative nameservers and possibly rotation of server used.
- Caching of the results is work-in-progress, commented out since it is not in
a useable state.
Comments and feedback highly welcome.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
--- Comment #2 from Karsten Bräckelmann <gu...@rudersport.de> ---
In-depth discussion by Rob McEven. Included in the patch, pasted here for
convenience.
Direct DNS Query Per DNSBL
This is an optional (non-default!) feature in SpamAssassin. It is assigned to a
DNSBL on a DNSBL-by-DNSBL basis. Therefore, if this feature is assigned to one
particular DNSBL, all of the other DNSBLs continue to operate normally, WITHOUT
using or being impacted by this feature.
This feature provides two main functions:
(1) This enables the queries for a particular DNSBL to be sent DIRECTLY to a
particular specified IP (or hostname), which can be either an authoritative DNS
server, or a caching DNS server (whatever IP or hostname is specified in the
setting for that particular DNSBL that is making use of this feature)
-OR-
(2) SpamAssassin can look up the NS records for a particular DNSBL (using the
default DNS server for that lookup), resolve those host names to IP addresses,
cache that DNS information as instructed by the authoritative TTL values, then
start sending the actual DNSBL lookups DIRECTLY to one of those IPs, thus
bypassing the default caching DNS server and going DIRECTLY to the
authoritative DNS servers for that particular DNSBL. The NS records forthe
DNSBL and the actual "a" record answers to the queries - are BOTH cached
internally by SA, according to the TTL values provided by the DNSBLs
authoritative DNS servers answers. When those expire, that information is then
re-fetched, and then an IP to query against is selected again. Typically, the
A-record of each DNSBL lookup is going to have a very short TTL value. But
these are still cached since, in real-world spam filtering environments, MANY
such lookups on the same item can often occur very rapidly within a small
number of seconds. Therefore, these are also cached per the DNSBL's TTL value
for these A-record lookups.
WHY IS THIS NEEDED? WHEN WOULD THIS BE USED?
Sometimes, SpamAssassin is installed in environments where the administrator
does not necessarily have 100% control over the DNS server, or needs to use a
DNS server that is NOT the default DNS server for that server, for a particular
DNSBL. It is not uncommon for custom global DNS settings for that server to be
overwritten by a hosting or service provider. Many hosting companies often try
to default DNS settings to using Google's DNS servers instead of a
locally-hosted caching DNS server. (This is VERY common!) And some SpamAssassin
installations are in environments where the SA adminstrator simply has
insufficient privileges to even be allowed to install their own locally-hosted
caching DNS server. Other similar problems are common.
At the same time, some DNSBLs either require that they are accessed from the
users own IP space and/or they dont allow queries to come from large
providers DNS servers. For example, many DNSBLs block queries that come from
both Googles and OpenDNSs DNS servers. Yet, again, it is extremely common for
hosting companies to default to Google and OpenDNS DNS servers. At the same
time, many subscription-based DNSBLs require that the queries come DIRECTLY
from the userss IP space, and NOT be routed through Googles or their ISPs
dns servers (and this would include both subscription-only DNSBLs, such as
invaluement.com -AND- direct query subscriptions for more public DNSBLs like
SpamHaus, which often require a subscription for various usage levels and
scenarios). This feature therefore enables a way for a SpamAssassin
administrator to ensure that the DNSBL lookups are going to come from the SA
installations own IP space, and are NOT subject to undesirable global DNS
changes that would disrupt access to that DNSBL. This also gives the DNSBL
provider granularity of control by allowing the DNSBL provider to ONLY use this
feature for particular DNSBLs, and give them the ability to set specific custom
settings per DNSBL. In other words, each DNSBL this is applied to can have its
own custom setting! And, again, those DNSBLs for which this is not implemented
will continue to operate in the SAME manner as before this feature was
introduced into SpamAssassin.
WHEN NOT TO USE THIS (AND LIMITATIONS):
For those SpamAssassin installations where the administrator has a great deal
of control over the server and can install their own locally-hosted caching DNS
server for use by SA, such an administrator should continue to do that and
should NOT use this feature. Also, this solution may not be appropriate for
extremely high-volume processing. However, if someone is in a high-volume
environment, they normally would (and should!) have these other resources and
permissions and level of control to be able to manage their own caching DNS
server, without fear of having their settings overwritten, or the other
problems described earlier! The main target of this feature is those
SpamAssassin installations where the administrator doesnt have as much
control, and such SA installations are typically smaller (such as typically
providing spam filtering for fewer than 1,000 mailboxes). This new feature
probably shouldnt be considered as superior to using your own locally-hosted
caching DNS server, when the SA admin does have that option (with a reliable
implementation that isn't subject to unwanted alterations).
TECHNOLOGY NOTES:
One challenge for this product is that fact that SA child processes are
stateless. Therefore, this is implemented by using the Cache::FastMmap
shared-memory cache for storing the DNSBL answers. Each SA child process
spawned by SPAMD then reads and writes values from/to Cache::FastMmap. This is
a Unix-specific program. Therefore, those who maintain Windows version of
SpamAssassin might not have this feature, unless they are able to program an
equivalent functionality. That might possibly be achieved through the use of a
Windows port of Redis? (Since Redis is similar to Cache::FastMmap, and has been
ported to Windows.) But this would need to be researched further, along with
the licensing requirements of Redis or any other 3rd party programs used to
implement this functionality on a Windows port of SpamAssassin.
--
You are receiving this mail because:
You are the assignee for the bug.
Re: [Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2022-04-11 at 11:23:40 UTC-0400 (Mon, 11 Apr 2022 08:23:40 -0700)
Michael Peddemors <mi...@linuxmagic.com>
is rumored to have said:
> On 2022-04-11 06:18, bugzilla-daemon@spamassassin.apache.org wrote:
>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
>>
>> Henrik Krohns <ap...@hege.li> changed:
>>
>> What |Removed |Added
>> ----------------------------------------------------------------------------
>> Target Milestone|4.0.0 |Future
>>
>> --- Comment #6 from Henrik Krohns <ap...@hege.li> ---
>> Would need some cleaning up, but no time to look for 4.0.0. Is there
>> _really_
>> demand for this feature, when resolvers are configurable pretty much
>> everywhere.. postponing..
>>
>
> There is demand out there I am sure, with recent changes to some RBL
> operators policies, and things like DoH out in the wild.
>
> There can be cases where certain RBL's are blocked from queries at the
> upstream provider.
>
> Just an opinion.. Sometimes SA administrators may not have operational
> control over resolver decisions.
I think it would be irresponsible to put this in SA and that it would
cause more trouble than it can be worth.
Mail systems should use local caching resolvers under common control
with the mail system. They SHOULD NOT use any resolver with 'safety'
features designed to protect end users by breaking resolution. They
SHOULD NOT use unaccountable free resolvers. FOR ANYTHING.
This sort of thing belongs in a dedicated resolver with forwarding
options like dnsmasq or unbound, NOT in SA. I don't think we should be
providing support for systems that are fundamentally misdesigned in
order to make the use of a 3rd-party resolver a top priority.
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
Re: [Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by Henrik K <he...@hege.li>.
On Mon, Apr 11, 2022 at 08:23:40AM -0700, Michael Peddemors wrote:
>
> Just an opinion.. Sometimes SA administrators may not have operational
> control over resolver decisions.
Sorry I just don't buy it. In what kind of environment an administrator can
customize SA, but doesn't have root permissions to install or configure a
resolver? No one has commented in the bug in years that they would need
this feature (.. which is inferior compared to a resolver, even admitted in
the patch description).
Of course anyone is free to polish the patch into current trunk, and it can
be then considered.
Re: [Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by Michael Peddemors <mi...@linuxmagic.com>.
On 2022-04-11 06:18, bugzilla-daemon@spamassassin.apache.org wrote:
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
>
> Henrik Krohns <ap...@hege.li> changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> Target Milestone|4.0.0 |Future
>
> --- Comment #6 from Henrik Krohns <ap...@hege.li> ---
> Would need some cleaning up, but no time to look for 4.0.0. Is there _really_
> demand for this feature, when resolvers are configurable pretty much
> everywhere.. postponing..
>
There is demand out there I am sure, with recent changes to some RBL
operators policies, and things like DoH out in the wild.
There can be cases where certain RBL's are blocked from queries at the
upstream provider.
Just an opinion.. Sometimes SA administrators may not have operational
control over resolver decisions.
--
"Catch the Magic of Linux..."
------------------------------------------------------------------------
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
------------------------------------------------------------------------
604-682-0300 Beautiful British Columbia, Canada
This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
Henrik Krohns <ap...@hege.li> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|4.0.0 |Future
--- Comment #6 from Henrik Krohns <ap...@hege.li> ---
Would need some cleaning up, but no time to look for 4.0.0. Is there _really_
demand for this feature, when resolvers are configurable pretty much
everywhere.. postponing..
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
Henrik Krohns <he...@hege.li> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.4.3 |4.0.0
CC| |hege@hege.li
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
Kevin A. McGrail <km...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |kmcgrail@apache.org
Target Milestone|Undefined |3.4.3
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7477] Direct DNS Querying Per DNSBL Zone
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7477
Benny Pedersen <me...@junc.eu> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |me@junc.eu
--- Comment #3 from Benny Pedersen <me...@junc.eu> ---
could we change to sqlite ?
so pr dnsbl changes to pr sqlite db cache ?, why keep it dns when all nearly
dont like to have dns data sync ?
is dns really that unstable that it really needs fixing in spamassassin ?
possible begin to use enlists with ip listning ?
why do i say it again, i find enlist very use full to have local url listed,
and it does not need dns to do this, if the enlist data is in sqlite it's still
lowmen even it there is millions random domains to be blacklisted
is sqlite to slow for this ?
imho could we distribute data to be compiled to sqlite, to keep lowmem still,
so update channels can provide data to be added to sqlite, and then have rule
set loaded from sqlite to memmory if its low amount of data to be loaded
just my own feedback, good or bad, dont know
--
You are receiving this mail because:
You are the assignee for the bug.