You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Michael B Allen <io...@gmail.com> on 2015/06/09 05:03:21 UTC
Must-Have Plugins?
So I have had SA running for about 2 days on a very small site with a
handful of users. I've been running the default config just to see how
well it would do by itself. Unfortunately quite a lot of spam is
getting through. So far 40 of 142 spams have passed.
So my question is, what is the best way to improve things? Is there
any particular must-have plugins? What is the one thing I can do to a
default install that is going to give me the biggest return on
invested effort?
Mike
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 09.06.2015 um 05:03 schrieb Michael B Allen:
> So I have had SA running for about 2 days on a very small site with a
> handful of users. I've been running the default config just to see how
> well it would do by itself. Unfortunately quite a lot of spam is
> getting through. So far 40 of 142 spams have passed.
>
> So my question is, what is the best way to improve things? Is there
> any particular must-have plugins? What is the one thing I can do to a
> default install that is going to give me the biggest return on
> invested effort?
train your bayes, preferred a global one to benfit all users from the
same training
BAYES_00 9125 73.91 %
BAYES_05 306 2.47 %
BAYES_20 328 2.65 %
BAYES_40 285 2.30 %
BAYES_50 1149 9.30 %
BAYES_60 108 0.87 %
BAYES_80 98 0.79 %
BAYES_95 92 0.74 %
BAYES_99 854 6.91 %
BAYES_999 781 6.32 %
DELIVERED 14631 92.37 %
DNSWL 13696 86.47 %
SPF 6633 41.88 %
SPF/DKIM WL 3392 21.41 %
SHORTCIRCUIT 3488 22.02 %
BLOCKED 1491 9.41 %
[sa-milt@mail-gw:~]$ spamfilter-blocked.sh | grep "Jun 8" | wc -l
287
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 09.06.2015 um 20:29 schrieb John Hardin:
> On Tue, 9 Jun 2015, David Jones wrote:
>
>> Some of the best and easiest things you can enable to block spam are
>> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
>
>> - Enable greylisting. This is just about the only way you can block
>> zero-hour spam from compromised accounts that come from legit mail
>> servers before they get listed in RBLs.
>
> Just bear in mind some commercial organizations may be very hostile to
> anything that delays delivery of mail, regardless of how much it would
> reduce spam.
>
> Two things that I have found very useful at the MTA level are:
>
> (1) Delay sending your SMTP banner a second or two and reject any sender
> that starts sending information before that. This is a built-in option
> in Sendmail, google "greet_pause"
for recent postfix with postcreen it is "postscreen_greet_wait"
postscreen_greet_action = enforce
postscreen_greet_wait = ${stress?2}${stress:11}s
the 11 seconds are not randomly, many spambots have a internal timeout
of 10 seconds and at begin of 2015/01 change it from 10 to 11 was a
impressive dropdown of visible rejects in mailgraph
additoonally if you use postfix consider
"postscreen_whitelist_interfaces = !ip-of-backup-mx, static:all" and add
for every domain a backup-mx to that interface, the stats below are
unique IP's over the last month and "Honeypot-Only" tried the backup MX,
got a temporary reject and never tried on the primary IP
Default-MX: 62419
Honeypot-MX: 25943
Honeypot-Only: 18382
Re: Must-Have Plugins?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 10.06.15 04:34, Amir Caspi wrote:
>To: Matus UHLAR - fantomas <uh...@fantomas.sk>
>Cc: users@spamassassin.apache.org
pleaase, avoid personal mail. The list is for public discussion.
>Subject: Re: Must-Have Plugins?
>
>On Jun 10, 2015, at 12:32 AM, Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
>
>> FEATURE(`block_bad_helo')
>> define(`confALLOW_BOGUS_HELO', `False')
>
>Argh, unfortunately, that feature is only on sendmail 8.14 and higher,
> which means RHEL/CentOS 6 or higher. For those of us running RHEL/CentOS
> 5, that's only available via a custom install. =(
It was available before and I have used it as HACK()
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"To Boot or not to Boot, that's the question." [WD1270 Caviar]
Re: Must-Have Plugins?
Posted by Amir Caspi <ce...@3phase.com>.
On Jun 19, 2015, at 6:02 PM, Philip Prindeville <ph...@redfish-solutions.com> wrote:
> Given how many vulnerabilities CentOS 5 has, why would you want to keep running that?
Because, while "I wish I could upgrade ... various circumstances prevent that right now."
It is fully patched, FWIW.
--- Amir
thumbed via iPhone
Re: Must-Have Plugins?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On 06/10/2015 04:34 AM, Amir Caspi wrote:
> On Jun 10, 2015, at 12:32 AM, Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
>
>> FEATURE(`block_bad_helo')
>> define(`confALLOW_BOGUS_HELO', `False')
> Argh, unfortunately, that feature is only on sendmail 8.14 and higher, which means RHEL/CentOS 6 or higher. For those of us running RHEL/CentOS 5, that's only available via a custom install. =(
>
> Does anyone know of a reputable RPM distro for sendmail 8.14+ for CentOS 5? I can't find anything decent via Google, everything is for CentOS 6 or higher.
>
> (My server setup requires RPMs, so I can't build from a source tarball. I could potentially use the source RPM from CentOS 6 to get a custom RPM for 5, although even that is problematic.)
>
> Bleh. I wish I could upgrade this server to a newer OS, but various circumstances prevent that right now.
>
> --- Amir
>
Given how many vulnerabilities CentOS 5 has, why would you want to keep
running that?
Re: Must-Have Plugins?
Posted by Amir Caspi <ce...@3phase.com>.
On Jun 10, 2015, at 12:32 AM, Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
> FEATURE(`block_bad_helo')
> define(`confALLOW_BOGUS_HELO', `False')
Argh, unfortunately, that feature is only on sendmail 8.14 and higher, which means RHEL/CentOS 6 or higher. For those of us running RHEL/CentOS 5, that's only available via a custom install. =(
Does anyone know of a reputable RPM distro for sendmail 8.14+ for CentOS 5? I can't find anything decent via Google, everything is for CentOS 6 or higher.
(My server setup requires RPMs, so I can't build from a source tarball. I could potentially use the source RPM from CentOS 6 to get a custom RPM for 5, although even that is problematic.)
Bleh. I wish I could upgrade this server to a newer OS, but various circumstances prevent that right now.
--- Amir
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 10.06.2015 um 13:17 schrieb Kevin A. McGrail:
> On 6/10/2015 2:32 AM, Matus UHLAR - fantomas wrote:
>>
>>> I'm not sure whether or not I have enabled requiring valid rDNS... given
>>> how many legitimate mailservers out there don't have proper rDNS,
>>
>> how many? I'm happy to block them for years...
> From what I've see, the effectivness and false positives depends mostly
> on the countries sending email to your users.
>
> Off the cuff, I'd say it has it's roots in the mid 90's when AOL was a
> 1000lb gorilla and pushed the rptr for mail servers. I think much of
> Europe and NA complied.
>
> However, I see a TON of legit mail, for example, coming out of the
> Pacific Rim countries with no rptr's
well, combine strict PTR rules with DNSWL and SPF, the one which have
*something* sane are not hurted and the rest need to learn some basics
Re: Must-Have Plugins?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 6/10/2015 2:32 AM, Matus UHLAR - fantomas wrote:
>
>> I'm not sure whether or not I have enabled requiring valid rDNS... given
>> how many legitimate mailservers out there don't have proper rDNS,
>
> how many? I'm happy to block them for years...
From what I've see, the effectivness and false positives depends mostly
on the countries sending email to your users.
Off the cuff, I'd say it has it's roots in the mid 90's when AOL was a
1000lb gorilla and pushed the rptr for mail servers. I think much of
Europe and NA complied.
However, I see a TON of legit mail, for example, coming out of the
Pacific Rim countries with no rptr's.
regards,
KAM
Re: Must-Have Plugins?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On Jun 9, 2015, at 12:29 PM, John Hardin <jh...@impsec.org> wrote:
>> (2) Check the HELO the other guy sends and reject if it's not a FQDN
>> (i.e. it's not got any periods at all). This probably shouldn't be done
>> on mail originating locally, but for mail coming in from the Internet the
>> other MTA should always be sending a FQDN in the HELO. A non-FQDN HELO
>> is a pretty good sign of a spambot sending from a compromised workstation
>> or PC directly to your MTA.
On 09.06.15 12:48, Amir Caspi wrote:
>Do you have a sendmail line (or lines) that we can pretty much copy/paste
> to implement this, for those of us who are not total sendmail experts and
> too lazy (or busy, or incompetent) to Google something like this? =)
FEATURE(`block_bad_helo')
define(`confALLOW_BOGUS_HELO', `False')
>I'm not sure whether or not I have enabled requiring valid rDNS... given
> how many legitimate mailservers out there don't have proper rDNS,
how many? I'm happy to block them for years...
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
We are but packets in the Internet of life (userfriendly.org)
Re: Must-Have Plugins?
Posted by John Hardin <jh...@impsec.org>.
On Tue, 9 Jun 2015, Amir Caspi wrote:
> On Jun 9, 2015, at 12:29 PM, John Hardin <jh...@impsec.org> wrote:
>
>> (2) Check the HELO the other guy sends and reject if it's not a FQDN
>> (i.e. it's not got any periods at all). This probably shouldn't be done
>> on mail originating locally, but for mail coming in from the Internet
>> the other MTA should always be sending a FQDN in the HELO. A non-FQDN
>> HELO is a pretty good sign of a spambot sending from a compromised
>> workstation or PC directly to your MTA.
>
> Do you have a sendmail line (or lines) that we can pretty much
> copy/paste to implement this, for those of us who are not total sendmail
> experts and too lazy (or busy, or incompetent) to Google something like
> this? =)
I don't do that in sendmail, it's one of the checks in my milter-regex
config. My example milter-regex config is here:
http://www.impsec.org/~jhardin/antispam/milter-regex.conf
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The yardstick you should use when considering whether to support a
given piece of legislation is "what if my worst enemy is chosen to
administer this law?"
-----------------------------------------------------------------------
Re: Must-Have Plugins?
Posted by Amir Caspi <ce...@3phase.com>.
On Jun 9, 2015, at 12:29 PM, John Hardin <jh...@impsec.org> wrote:
> (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. it's not got any periods at all). This probably shouldn't be done on mail originating locally, but for mail coming in from the Internet the other MTA should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty good sign of a spambot sending from a compromised workstation or PC directly to your MTA.
Do you have a sendmail line (or lines) that we can pretty much copy/paste to implement this, for those of us who are not total sendmail experts and too lazy (or busy, or incompetent) to Google something like this? =)
I'm not sure whether or not I have enabled requiring valid rDNS... given how many legitimate mailservers out there don't have proper rDNS, I'm hesitant to turn that into a poison pill at the sendmail level. (Even my own $DAYJOB company failed to have rDNS on their mailservers until I showed them bounces from some strict recipient MTAs...)
Cheers.
-- Amir
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 24.06.2015 um 02:00 schrieb Philip Prindeville:
>
>
> On 06/19/2015 01:07 PM, Dianne Skoll wrote:
>> On Fri, 19 Jun 2015 12:51:28 -0600
>> Philip Prindeville <ph...@redfish-solutions.com> wrote:
>>
>> [stuff]
>>
>>> With this, we avoid ever accepting about 98% of the SPAM that we’d
>>> otherwise receive.
>> Really? 98%? I find that surprising. We get quite a lot of spam
>> from gmail, hotmail, yahoo etc. that would pass all of your tests.
>>
>> Regards,
>>
>> Dianne.
>
> I should have mentioned we also blacklist yahoo... and are thinking
> about blocking google, too
don't forget facebbok (xfdw) and gmail, but why not just blcklist
anything except whitelisted? :-)
Re: Must-Have Plugins?
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Tue, 23 Jun 2015 18:00:27 -0600
Philip Prindeville <ph...@redfish-solutions.com> wrote:
> I should have mentioned we also blacklist yahoo... and are thinking
> about blocking google, too.
I see. If we did this, then yes, we'd probably stop a lot of spam
(though nowhere near 98%) but we'd also lose 98% of our customers,
or more like 99.999%.
Implementing a spam filter for 20 probably highly-techie recipients
imposes vastly different constraints than doing it for hundreds of
thousands of mostly non-technical people. :)
Regards,
Dianne.
Re: Must-Have Plugins?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On 06/19/2015 01:07 PM, Dianne Skoll wrote:
> On Fri, 19 Jun 2015 12:51:28 -0600
> Philip Prindeville <ph...@redfish-solutions.com> wrote:
>
> [stuff]
>
>> With this, we avoid ever accepting about 98% of the SPAM that we’d
>> otherwise receive.
> Really? 98%? I find that surprising. We get quite a lot of spam
> from gmail, hotmail, yahoo etc. that would pass all of your tests.
>
> Regards,
>
> Dianne.
I should have mentioned we also blacklist yahoo... and are thinking
about blocking google, too.
Re: Must-Have Plugins?
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Fri, 19 Jun 2015 12:51:28 -0600
Philip Prindeville <ph...@redfish-solutions.com> wrote:
[stuff]
> With this, we avoid ever accepting about 98% of the SPAM that we’d
> otherwise receive.
Really? 98%? I find that surprising. We get quite a lot of spam
from gmail, hotmail, yahoo etc. that would pass all of your tests.
Regards,
Dianne.
Re: Must-Have Plugins?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Jun 19, 2015, at 3:28 PM, David Jones <dj...@ena.com> wrote:
>> From: Philip Prindeville <ph...@redfish-solutions.com>
>> Sent: Friday, June 19, 2015 3:53 PM
>> To: David Jones
>> Cc: users@spamassassin.apache.org
>> Subject: Re: Must-Have Plugins?
>
>> On Jun 19, 2015, at 2:35 PM, David Jones <dj...@ena.com> wrote:
>
>>>
>>>> But I’m on a LOT of high volume mailing lists (like mozilla-general and netdev) that get heavily spammed.
>>>
>>> Filtering mailing lists is a slightly different ballgame than filtering regular email. Some of the items listed above
>>> don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) since they are like proxies of the
>>> original sender's mail server.
>>>
>>> Dave
>
>> Sorry, I also meant that many of those mailing lists are harvested… so my address has been bought and sold many, many times.
>
> I see. For email addresses that have gotten on those lists, what I have found to be effective is to
> focus more on the reputation of the sending mail server. Some mail servers like mailchimp,
> sendgrid, constant contact, etc. will get these addresses but you can safely unsubscribe from
> them and eventually get off of their lists over a few weeks.
> Most of the other mail servers can be blocked by the major RBLs and the other techniques you
> mentioned in your original post. There are safe ways to whitelist specific sending IPs for domains
> where you don't have to put in a risky whitelist entry at the MTA level that will open you up to
> spoofing problems. This usually requires a little scripting to pull data that has been vetted by the
> SA level then tie it back to the MTA level.
> The rest that gets to SA needs to be handled by properly trained Bayes, DBLs, custom rules
> like KAM.cf, etc.
Well that was interesting. How long at Hetzner been hosting one of the spam assassin.apache.org mirrors?
Because they have a very low reputation with us.
So your message, which includes the text “spamassassin.apache.org” got flagged and quarantined.
More than a little ironic…
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>From: Philip Prindeville <ph...@redfish-solutions.com>
>Sent: Friday, June 19, 2015 3:53 PM
>To: David Jones
>Cc: users@spamassassin.apache.org
>Subject: Re: Must-Have Plugins?
>On Jun 19, 2015, at 2:35 PM, David Jones <dj...@ena.com> wrote:
>>
>>> But I’m on a LOT of high volume mailing lists (like mozilla-general and netdev) that get heavily spammed.
>>
>> Filtering mailing lists is a slightly different ballgame than filtering regular email. Some of the items listed above
>> don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) since they are like proxies of the
>> original sender's mail server.
>>
>> Dave
>Sorry, I also meant that many of those mailing lists are harvested… so my address has been bought and sold many, many times.
I see. For email addresses that have gotten on those lists, what I have found to be effective is to
focus more on the reputation of the sending mail server. Some mail servers like mailchimp,
sendgrid, constant contact, etc. will get these addresses but you can safely unsubscribe from
them and eventually get off of their lists over a few weeks.
Most of the other mail servers can be blocked by the major RBLs and the other techniques you
mentioned in your original post. There are safe ways to whitelist specific sending IPs for domains
where you don't have to put in a risky whitelist entry at the MTA level that will open you up to
spoofing problems. This usually requires a little scripting to pull data that has been vetted by the
SA level then tie it back to the MTA level.
The rest that gets to SA needs to be handled by properly trained Bayes, DBLs, custom rules
like KAM.cf, etc.
Re: Must-Have Plugins?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Jun 19, 2015, at 2:35 PM, David Jones <dj...@ena.com> wrote:
>
>> But I’m on a LOT of high volume mailing lists (like mozilla-general and netdev) that get heavily spammed.
>
> Filtering mailing lists is a slightly different ballgame than filtering regular email. Some of the items listed above
> don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) since they are like proxies of the
> original sender's mail server.
>
> Dave
Sorry, I also meant that many of those mailing lists are harvested… so my address has been bought and sold many, many times.
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>>> From: Philip Prindeville <ph...@redfish-solutions.com>
>>
>>> On Jun 9, 2015, at 12:29 PM, John Hardin <jh...@impsec.org> wrote:
>>
>>>> On Tue, 9 Jun 2015, David Jones wrote:
>>>>
>>>>> Some of the best and easiest things you can enable to block spam are
>>>>> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
>>>>
>>>>> - Enable greylisting. This is just about the only way you can block
>>>>> zero-hour spam from compromised accounts that come from legit mail
>>>>> servers before they get listed in RBLs.
>>>>
>>>> Just bear in mind some commercial organizations may be very hostile to anything that delays delivery of mail, regardless of how much it would reduce spam.
>>>>
>>>> Two things that I have found very useful at the MTA level are:
>>>>
>>>> (1) Delay sending your SMTP banner a second or two and reject any sender that starts sending information before that. This is a built-in option in Sendmail, google "greet_pause".
>>>>
>>>> (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. it's not got any periods at all). This probably shouldn't be done on mail originating locally, but for mail coming in from the Internet the other MTA should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty good sign of a spambot sending from a compromised workstation or PC directly to your MTA.
>>>>
>>>> I have some other MTA checks in place, but these two block the most.
>>
>>
>>> We use MimeDefang and it actually calls SpamAssassin. But before we accept connections (or email), we make a bunch of tests.
>>
>>> We use Geo::IP to block a bunch of ISP’s and countries that are hostile in filter_relay().
>>
>>> These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.
>>
>>> Then we apply more tests in filter_helo(). Note, these are only for connections that arrive on port 25, not on port 587
>>> (which is for local clients to submit, and is authenticated with a username/password pair).
>>
>>> We accept connections from 127.0.0.1 with no further checks.
>>
>>> We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the required brackets. We block any bracketed >dotted quads that have an invalid quad (i.e. 300.300.300.300) or where the address they claim to be from is different >from the actual address we’re seeing (no one behind a NATting firewall should be connecting to us unless they know their >[static] external address).
>>
>>> We block anyone listed by ZenBL.
>>
>>> We block (a) anyone claiming to be us, (b) anyone claiming to be “localhost” or “localhost.localdomain”.
>>
>>> We block a list of names that are always fraudulent, like “paypal.com” (without any subdomains), or >“smtp.communitel.net” which seems to get spoofed a LOT, or “mail.com” or “mail.ru”. We block hostnames that don’t >have a domain (no dots).
>>
>>> Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including the A and PTR records not agreeing).
>>
>>> With this, we avoid ever accepting about 98% of the SPAM that we’d otherwise receive.
>>
>> Good information. Thanks for taking the time to document it for this list.
>> How many mailboxes do you filter with this configuration?
>Not that many. About 20.
That's cool. Mail filtering (probably any filtering) is not a one-size-fits-all. I am able to do many of the things
you mentioned above but some things will break too many legit senders with incorrect FCrDNS (PTR = HELO).
>But I’m on a LOT of high volume mailing lists (like mozilla-general and netdev) that get heavily spammed.
Filtering mailing lists is a slightly different ballgame than filtering regular email. Some of the items listed above
don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) since they are like proxies of the
original sender's mail server.
Dave
Re: Must-Have Plugins?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Jun 19, 2015, at 1:01 PM, David Jones <dj...@ena.com> wrote:
>> From: Philip Prindeville <ph...@redfish-solutions.com>
>
>> On Jun 9, 2015, at 12:29 PM, John Hardin <jh...@impsec.org> wrote:
>
>>> On Tue, 9 Jun 2015, David Jones wrote:
>>>
>>>> Some of the best and easiest things you can enable to block spam are
>>>> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
>>>
>>>> - Enable greylisting. This is just about the only way you can block
>>>> zero-hour spam from compromised accounts that come from legit mail
>>>> servers before they get listed in RBLs.
>>>
>>> Just bear in mind some commercial organizations may be very hostile to anything that delays delivery of mail, regardless of how much it would reduce spam.
>>>
>>> Two things that I have found very useful at the MTA level are:
>>>
>>> (1) Delay sending your SMTP banner a second or two and reject any sender that starts sending information before that. This is a built-in option in Sendmail, google "greet_pause".
>>>
>>> (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. it's not got any periods at all). This probably shouldn't be done on mail originating locally, but for mail coming in from the Internet the other MTA should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty good sign of a spambot sending from a compromised workstation or PC directly to your MTA.
>>>
>>> I have some other MTA checks in place, but these two block the most.
>
>
>> We use MimeDefang and it actually calls SpamAssassin. But before we accept connections (or email), we make a bunch of tests.
>
>> We use Geo::IP to block a bunch of ISP’s and countries that are hostile in filter_relay().
>
>> These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.
>
>> Then we apply more tests in filter_helo(). Note, these are only for connections that arrive on port 25, not on port 587
>> (which is for local clients to submit, and is authenticated with a username/password pair).
>
>> We accept connections from 127.0.0.1 with no further checks.
>
>> We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the required brackets. We block any bracketed >dotted quads that have an invalid quad (i.e. 300.300.300.300) or where the address they claim to be from is different >from the actual address we’re seeing (no one behind a NATting firewall should be connecting to us unless they know their >[static] external address).
>
>> We block anyone listed by ZenBL.
>
>> We block (a) anyone claiming to be us, (b) anyone claiming to be “localhost” or “localhost.localdomain”.
>
>> We block a list of names that are always fraudulent, like “paypal.com” (without any subdomains), or >“smtp.communitel.net” which seems to get spoofed a LOT, or “mail.com” or “mail.ru”. We block hostnames that don’t >have a domain (no dots).
>
>> Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including the A and PTR records not agreeing).
>
>> With this, we avoid ever accepting about 98% of the SPAM that we’d otherwise receive.
>
> Good information. Thanks for taking the time to document it for this list.
> How many mailboxes do you filter with this configuration?
Not that many. About 20.
But I’m on a LOT of high volume mailing lists (like mozilla-general and netdev) that get heavily spammed.
-Philip
>
>> -Philip
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>From: Philip Prindeville <ph...@redfish-solutions.com>
>On Jun 9, 2015, at 12:29 PM, John Hardin <jh...@impsec.org> wrote:
>> On Tue, 9 Jun 2015, David Jones wrote:
>>
>>> Some of the best and easiest things you can enable to block spam are
>>> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
>>
>>> - Enable greylisting. This is just about the only way you can block
>>> zero-hour spam from compromised accounts that come from legit mail
>>> servers before they get listed in RBLs.
>>
>> Just bear in mind some commercial organizations may be very hostile to anything that delays delivery of mail, regardless of how much it would reduce spam.
>>
>> Two things that I have found very useful at the MTA level are:
>>
>> (1) Delay sending your SMTP banner a second or two and reject any sender that starts sending information before that. This is a built-in option in Sendmail, google "greet_pause".
>>
>> (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. it's not got any periods at all). This probably shouldn't be done on mail originating locally, but for mail coming in from the Internet the other MTA should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty good sign of a spambot sending from a compromised workstation or PC directly to your MTA.
>>
>> I have some other MTA checks in place, but these two block the most.
>We use MimeDefang and it actually calls SpamAssassin. But before we accept connections (or email), we make a bunch of tests.
>We use Geo::IP to block a bunch of ISP’s and countries that are hostile in filter_relay().
>These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.
>Then we apply more tests in filter_helo(). Note, these are only for connections that arrive on port 25, not on port 587
>(which is for local clients to submit, and is authenticated with a username/password pair).
>We accept connections from 127.0.0.1 with no further checks.
>We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the required brackets. We block any bracketed >dotted quads that have an invalid quad (i.e. 300.300.300.300) or where the address they claim to be from is different >from the actual address we’re seeing (no one behind a NATting firewall should be connecting to us unless they know their >[static] external address).
>We block anyone listed by ZenBL.
>We block (a) anyone claiming to be us, (b) anyone claiming to be “localhost” or “localhost.localdomain”.
>We block a list of names that are always fraudulent, like “paypal.com” (without any subdomains), or >“smtp.communitel.net” which seems to get spoofed a LOT, or “mail.com” or “mail.ru”. We block hostnames that don’t >have a domain (no dots).
>Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including the A and PTR records not agreeing).
>With this, we avoid ever accepting about 98% of the SPAM that we’d otherwise receive.
Good information. Thanks for taking the time to document it for this list.
How many mailboxes do you filter with this configuration?
>-Philip
Re: Must-Have Plugins?
Posted by Philip Prindeville <ph...@redfish-solutions.com>.
On Jun 9, 2015, at 12:29 PM, John Hardin <jh...@impsec.org> wrote:
> On Tue, 9 Jun 2015, David Jones wrote:
>
>> Some of the best and easiest things you can enable to block spam are
>> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
>
>> - Enable greylisting. This is just about the only way you can block
>> zero-hour spam from compromised accounts that come from legit mail
>> servers before they get listed in RBLs.
>
> Just bear in mind some commercial organizations may be very hostile to anything that delays delivery of mail, regardless of how much it would reduce spam.
>
> Two things that I have found very useful at the MTA level are:
>
> (1) Delay sending your SMTP banner a second or two and reject any sender that starts sending information before that. This is a built-in option in Sendmail, google "greet_pause".
>
> (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. it's not got any periods at all). This probably shouldn't be done on mail originating locally, but for mail coming in from the Internet the other MTA should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty good sign of a spambot sending from a compromised workstation or PC directly to your MTA.
>
> I have some other MTA checks in place, but these two block the most.
We use MimeDefang and it actually calls SpamAssassin. But before we accept connections (or email), we make a bunch of tests.
We use Geo::IP to block a bunch of ISP’s and countries that are hostile in filter_relay().
These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.
Then we apply more tests in filter_helo(). Note, these are only for connections that arrive on port 25, not on port 587 (which is for local clients to submit, and is authenticated with a username/password pair).
We accept connections from 127.0.0.1 with no further checks.
We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the required brackets. We block any bracketed dotted quads that have an invalid quad (i.e. 300.300.300.300) or where the address they claim to be from is different from the actual address we’re seeing (no one behind a NATting firewall should be connecting to us unless they know their [static] external address).
We block anyone listed by ZenBL.
We block (a) anyone claiming to be us, (b) anyone claiming to be “localhost” or “localhost.localdomain”.
We block a list of names that are always fraudulent, like “paypal.com” (without any subdomains), or “smtp.communitel.net” which seems to get spoofed a LOT, or “mail.com” or “mail.ru”. We block hostnames that don’t have a domain (no dots).
Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including the A and PTR records not agreeing).
With this, we avoid ever accepting about 98% of the SPAM that we’d otherwise receive.
-Philip
Re: Must-Have Plugins?
Posted by John Hardin <jh...@impsec.org>.
On Wed, 10 Jun 2015, Bill Cole wrote:
>> > (2) Check the HELO the other guy sends and reject if it's not a FQDN
>> > (i.e. it's not got any periods at all).
>>
>> or if it's your FQDN, or your IP - they should use their FQDN, not yours.
>
> And if you don't/can't use a greeting pause, these are useful in catching
> many of the bots that fast-talk.
Absolutely. I see a lot of instances where the first couple of tries from
a given IP are blocked by greet-pause, and then after a bit there are
several more from the same IP blocked by invalid HELO.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
...much of our country's counterterrorism security spending is not
designed to protect us from the terrorists, but instead to protect
our public officials from criticism when another attack occurs.
-- Bruce Schneier
-----------------------------------------------------------------------
Re: Must-Have Plugins?
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 9 Jun 2015, at 14:39, Matus UHLAR - fantomas wrote:
> On 09.06.15 11:29, John Hardin wrote:
>> Two things that I have found very useful at the MTA level are:
>>
>> (1) Delay sending your SMTP banner a second or two and reject any
>> sender that starts sending information before that. This is a
>> built-in option in Sendmail, google "greet_pause".
>
> even 15...
Based on the implementation in Sendmail on a variety of systems with up
to a few million sessions/day, Postfix's 'postscreen' implementation on
smaller (mostly *much* smaller) sites and CommunigatePro's
implementation on a couple of middling sites, the value of adding
greet_pause time starts dropping fast around 3 seconds and there's no
point going past 5. You can actually start seeing occasional collateral
damage around 10 seconds with some high-volume legitimate senders timing
out their 'first try' runs aggressively. 15 is another breakpoint,
apparently because there is widespread lore in high-speed sending that
says it is not worth waiting any longer for a greeting banner. For
high-volume sites it is also important to understand that SMTP session
lives from SYN-ACK to FIN without a greeting pause are mostly under 10
seconds; adding 3 seconds to that means you must support MANY more
concurrent SMTP connections.
FWIW, the best implementation of fast-talk detection is Postfix's
postscreen. It is optimized to put this and other low-cost exclusion
tricks outside of a relatively heavy smtpd and was designed with an eye
on the many years of demonstration by Sendmail and others of how to
minimize the impact of adding delays. By not delaying clients that have
recently passed a delay, it assures that most of your high-volume
legitimate senders don't contribute to performance issues.
>
>> (2) Check the HELO the other guy sends and reject if it's not a FQDN
>> (i.e. it's not got any periods at all).
>
> or if it's your FQDN, or your IP - they should use their FQDN, not
> yours.
And if you don't/can't use a greeting pause, these are useful in
catching many of the bots that fast-talk. Also useful are checks for
improper use of IP HELOs (e.g. unbracketed IPs, not valid "IP literals")
and checking for *.local and *.localdomain HELO args. A handful of
innocent morons run MTAs that use such names, but they tend not to
survive long doing so.
Re: Must-Have Plugins?
Posted by John Hardin <jh...@impsec.org>.
On Tue, 9 Jun 2015, Matus UHLAR - fantomas wrote:
> On 09.06.15 11:29, John Hardin wrote:
>> Two things that I have found very useful at the MTA level are:
>>
>> (1) Delay sending your SMTP banner a second or two and reject any sender
>> that starts sending information before that. This is a built-in option in
>> Sendmail, google "greet_pause".
>
> even 15...
>
>> (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e.
>> it's not got any periods at all).
>
> or if it's your FQDN, or your IP - they should use their FQDN, not yours.
Agreed. That's one of the other checks I have, but it rarely hits. Then
again, my volume is tiny... :)
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The yardstick you should use when considering whether to support a
given piece of legislation is "what if my worst enemy is chosen to
administer this law?"
-----------------------------------------------------------------------
Re: Must-Have Plugins?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 09.06.15 11:29, John Hardin wrote:
>Two things that I have found very useful at the MTA level are:
>
>(1) Delay sending your SMTP banner a second or two and reject any
>sender that starts sending information before that. This is a
>built-in option in Sendmail, google "greet_pause".
even 15...
>(2) Check the HELO the other guy sends and reject if it's not a FQDN
>(i.e. it's not got any periods at all).
or if it's your FQDN, or your IP - they should use their FQDN, not yours.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
One OS to rule them all, One OS to find them,
One OS to bring them all and into darkness bind them
Re: Must-Have Plugins?
Posted by John Hardin <jh...@impsec.org>.
On Tue, 9 Jun 2015, David Jones wrote:
> Some of the best and easiest things you can enable to block spam are
> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
> - Enable greylisting. This is just about the only way you can block
> zero-hour spam from compromised accounts that come from legit mail
> servers before they get listed in RBLs.
Just bear in mind some commercial organizations may be very hostile to
anything that delays delivery of mail, regardless of how much it would
reduce spam.
Two things that I have found very useful at the MTA level are:
(1) Delay sending your SMTP banner a second or two and reject any sender
that starts sending information before that. This is a built-in option in
Sendmail, google "greet_pause".
(2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e.
it's not got any periods at all). This probably shouldn't be done on mail
originating locally, but for mail coming in from the Internet the other
MTA should always be sending a FQDN in the HELO. A non-FQDN HELO is a
pretty good sign of a spambot sending from a compromised workstation or PC
directly to your MTA.
I have some other MTA checks in place, but these two block the most.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Maxim V: Close air support and friendly fire should be easier to
tell apart.
-----------------------------------------------------------------------
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 10.06.2015 um 15:49 schrieb Michael B Allen:
> By "librarified" I mean the DNS "server" is just a code context that
> can be constructed with it's own config precisely and only as needed
> by the software that will be querying it (possibly temporarily if it's
> just client-only activity like a barrage of DNS queries fired in
> reaction to an email that fails other spam tests). It should not be
> necessary to change the resolver configuration or behavior of the
> entire system and everything running on it if only one component in
> the system needs this special feature (in this case a query limit and
> private cache). That is just bad programming philosophy and it the
> source of a lot of bad behavior in software (and DNS is a very good
> example of this actually)
1: SA has a option to define the nameserver without touch resolv.conf
2: why would somebody not re-use already cached data for any software
Caching nameserver vs. resolver library (was Re: Must-Have
Plugins?)
Posted by Dianne Skoll <df...@roaringpenguin.com>.
[I have lost the attribution, but someone wrote:]
> >That's not what I'm saying. It should not be necessary to run a
> >full-blown DNS server for SA to do it's queries. It should be
> >possible to call a library and create a DNS context that has all of
> >it's own parameters and then use that in an isolated way.
That's silly. The whole point of a caching DNS server is that *all*
applications on the system benefit from the cache. If you cache within
each process, then you waste memory and make redundant DNS lookups.
> >Then other services on the system are completely unaffected. Don't
> >tell me someone has never tweaked some parameter in your supposedly
> >caching-only nameserver and inadvertantly broken something or
> >wished they could tweak something and can't because of the
> >dependencies.
Umm... that has never happened to me and I've been administering UNIX
systems since 1990. The whole point of caching nameservers is that
they're dead easy to set up and once they're running, you just leave
them alone.
Regards,
Dianne.
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>>>> given that install unbound as local resolver takes 2 minutes it's even not
>>>> worth to argue on that topic and a spamfilter without RBL's and URIBL's is
>>>> just nonsense
>>
>>>I have installed a caching DNS server before (albeit probably about 15
>>>years ago). But it just shouldn't be necessary.
>>
>> It can be necessary if you have enough mail volume.
>That's not what I'm saying. It should not be necessary to run a
>full-blown DNS server for SA to do it's queries. It should be possible
>to call a library and create a DNS context that has all of it's own
>parameters and then use that in an isolated way. Then other services
>on the system are completely unaffected. Don't tell me someone has
>never tweaked some parameter in your supposedly caching-only
>nameserver and inadvertantly broken something or wished they could
>tweak something and can't because of the dependencies. And it's very
>possible that the queries might be for different names using custom
>query parameters in an async way and so on in which case the system
>resolver API might not be ideal.
You missed my point which I clarified yesterday in a previous post.
>I'm not pooh-poohing your advice. I'm just saying the DNS bits should
>be librarified so that these things don't even need extra thought.
>This stuff might be what you do all the time but I don't. I do this
>once every few years. This is the sort of thing that makes people
>switch to "cloud services".
If you don't do this kind of work often, I completely understand it's
hard to keep up with everything. I am sure I can't keep up with
some of the stuff that you do everyday. One option is to use
something like http://efa-project.org/ so it can be handled for
you automatically by smart people like Shawn that do this everyday.
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 11.06.2015 um 19:28 schrieb Michael B Allen:
> On Thu, Jun 11, 2015 at 10:03 AM, RW <rw...@googlemail.com> wrote:
>> You don't need a full-blown DNS server, you just need a resolver which
>> is typically ~ 100kB plus whatever space you want for caching.
>> Mine is currently using 9MB of resident memory compared with 103MB
>> for a single spamd child process. This handles DNS more efficiently
>> than SpamAssassin could.
>>
>> The closest SA itself could get is to have a dedicated process -
>
> No. You don't need a process at all. It would just be some functions
> that are called (preferably async but that's easy to make optional).
> Depending on the size of the cache it could be in memory or a disk
> file. Then it would be properly "librarified" and isolated
sorry, but that is pure nonsense
there is a *damned good* reason why different things are running in
different processes and it would be pretty stupid having the dns cache
in memory of a SA process
* the MTA before SA is making DNS requests anyways
* a spf-policyd called by the MTA is making DNS requests anyways
* SA will re-use that already present caches
with your proposal you need to manage a *shared cache* between the spamd
childs (ignoring here that there are dozens of ways to integrate SA into
a mailsystem) or waste memory by having each process his own cache view
not able to resuse it
and to be honest: in all the time you wasted by writing a lot of
nonsense you could have installed a proper resolver and just accept that
thounsands of mail-operators out there doing the same for decades now in
high-load setups which scale have a good reason to do so instead wasting
time by re-invent the wheel
Re: Must-Have Plugins?
Posted by Michael B Allen <io...@gmail.com>.
On Thu, Jun 11, 2015 at 10:03 AM, RW <rw...@googlemail.com> wrote:
> On Wed, 10 Jun 2015 18:45:10 -0400
> Michael B Allen wrote:
>
>> On Wed, Jun 10, 2015 at 9:56 AM, David Jones <dj...@ena.com> wrote:
>> >>> given that install unbound as local resolver takes 2 minutes it's
>> >>> even not worth to argue on that topic and a spamfilter without
>> >>> RBL's and URIBL's is just nonsense
>> >
>> >>I have installed a caching DNS server before (albeit probably about
>> >>15 years ago). But it just shouldn't be necessary.
>> >
>> > It can be necessary if you have enough mail volume.
>>
>> That's not what I'm saying. It should not be necessary to run a
>> full-blown DNS server for SA to do it's queries.
>
> You don't need a full-blown DNS server, you just need a resolver which
> is typically ~ 100kB plus whatever space you want for caching.
> Mine is currently using 9MB of resident memory compared with 103MB
> for a single spamd child process. This handles DNS more efficiently
> than SpamAssassin could.
>
> The closest SA itself could get is to have a dedicated process -
No. You don't need a process at all. It would just be some functions
that are called (preferably async but that's easy to make optional).
Depending on the size of the cache it could be in memory or a disk
file. Then it would be properly "librarified" and isolated.
But this thread is getting a little wonky so I apologize in advance
for not respond further.
Mike
Re: Must-Have Plugins?
Posted by RW <rw...@googlemail.com>.
On Wed, 10 Jun 2015 18:45:10 -0400
Michael B Allen wrote:
> On Wed, Jun 10, 2015 at 9:56 AM, David Jones <dj...@ena.com> wrote:
> >>> given that install unbound as local resolver takes 2 minutes it's
> >>> even not worth to argue on that topic and a spamfilter without
> >>> RBL's and URIBL's is just nonsense
> >
> >>I have installed a caching DNS server before (albeit probably about
> >>15 years ago). But it just shouldn't be necessary.
> >
> > It can be necessary if you have enough mail volume.
>
> That's not what I'm saying. It should not be necessary to run a
> full-blown DNS server for SA to do it's queries.
You don't need a full-blown DNS server, you just need a resolver which
is typically ~ 100kB plus whatever space you want for caching.
Mine is currently using 9MB of resident memory compared with 103MB
for a single spamd child process. This handles DNS more efficiently
than SpamAssassin could.
The closest SA itself could get is to have a dedicated process -
essentially reinventing the wheel with a heavyweight perl resolver. But
that isn't a general solution because spamd isn't the only way of
using the SA libraries. The generic solution would be to use a shared
database with all the overheads and complications that that implies.
Re: Must-Have Plugins?
Posted by Michael B Allen <io...@gmail.com>.
On Wed, Jun 10, 2015 at 9:56 AM, David Jones <dj...@ena.com> wrote:
>>> given that install unbound as local resolver takes 2 minutes it's even not
>>> worth to argue on that topic and a spamfilter without RBL's and URIBL's is
>>> just nonsense
>
>>I have installed a caching DNS server before (albeit probably about 15
>>years ago). But it just shouldn't be necessary.
>
> It can be necessary if you have enough mail volume.
That's not what I'm saying. It should not be necessary to run a
full-blown DNS server for SA to do it's queries. It should be possible
to call a library and create a DNS context that has all of it's own
parameters and then use that in an isolated way. Then other services
on the system are completely unaffected. Don't tell me someone has
never tweaked some parameter in your supposedly caching-only
nameserver and inadvertantly broken something or wished they could
tweak something and can't because of the dependencies. And it's very
possible that the queries might be for different names using custom
query parameters in an async way and so on in which case the system
resolver API might not be ideal.
I'm not pooh-poohing your advice. I'm just saying the DNS bits should
be librarified so that these things don't even need extra thought.
This stuff might be what you do all the time but I don't. I do this
once every few years. This is the sort of thing that makes people
switch to "cloud services".
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by Noel Butler <no...@ausics.net>.
On 11/06/2015 00:18, Dianne Skoll wrote:
> On Wed, 10 Jun 2015 13:56:49 +0000
> David Jones <dj...@ena.com> wrote:
>
> [One should run a caching DNS server on a mail server.]
>
>> We are giving you solid advice based on real experiences where we
>> ran into problems and worked around them. Just try to enable RBLs
>> and see how it works for you.
>
> I'm not disputing that running a caching DNS server is a good idea, but
> you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
> Spamhaus, for example, has a TTL of 1 minute on its A records. (Check
> out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.)
>
> Quite a number of years ago, I ran an analysis of the mail logs on a
> very busy server and found an abysmally low cache hit rate (about 30%)
> and that was in the day when Spamhaus had a 15-minute TTL.
30% is an excellent hit rate, however -
The longer the TTL the higher the cache hit
The longer the TTL the higher the collateral damage
It's why most run 1-10 min TTL's, might not seem much, but take for
example in the mid 90's when AOL was useless at dealing with spam
issues, a listing of 10 mins could deny thousands of messages back then,
and that helped "prompt" them into getting their act together,
especially when a number of DNSBL's were doing it, so they kicked off
their user (who often retuned 30 mins later courtesy of AOL's world wide
flood of freebie CD's), and blocks where removed quick enough to
minimise more innocents getting caught up.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 10 Jun 2015, at 10:26, Kevin A. McGrail wrote:
> On 6/10/2015 10:18 AM, Dianne Skoll wrote:
>> I'm not disputing that running a caching DNS server is a good idea,
>> but
>> you may be quite surprised at the low cache hit rate for IP-based
>> DNSBLs.
> IMO, the primary goal of a caching-only nameserver is in fact, not the
> caching, but rather the unique source IP so as to avoid running into
> DNS limits placed on RBL queries from some BL providers that you can
> run afoul of when sharing a DNS server.
>
> Caching is really just icing on the cake coupled with the simplest way
> to get a local DNS server up and running, no?
Not at all scales and styles of mail system. The MTA does lookups at
connect and at each command that mostly block progress, and them if the
message makes it to SA, virtually all of those lookups and often closely
related ones will be done again, often in another process running as a
different user which might (OS-dependent) mean that a record in the
meager cache kept by the OS won't be used for the second lookup.
I no longer have access to the data I gathered on this when I was
handling a big-ish system with multiple then-hefty MX gateways doing
spam filtering, but my memory is still sound enough that I can say the
difference between (1) talking to The Official Enterprise DNS Server on
the other side of a router that handled all recursive resolution and (2)
using a machine-local caching forwarder on each MX forwarding to a
shared caching recursive resolver on a common LAN was most of the median
SMTP session life. My recollection is that (1) meant most sessions took
~7 seconds or longer, (2) dropped it to near 3 seconds. A number of
things have changed in the past decade that might substantially change
that effect even in a similar site, but I think most of the effect
(proliferation of DNS-based tactics like SPF & DKIM and many more usable
DNSBLs and particularly URIBLs) can only make a cache more helpful, even
if the help is marginal. On the other hand, even legitimate operations
seem to think every DNS record should expire before today's close of
business, and that makes caching less possible.
Also, a smaller site gets less benefit at all from a DNS cache. If
you've got a few dozen users getting the same mail simultaneously in
parallel, you win. If you don't HAVE a few dozen users and most of your
users get no spam and little mail, you have a cache that's pretty
sparse. You still avoid the problems of looking like part of an abusive
behemoth when you forward to Google or of getting self-serving lies from
the local ISP browser-aid resolver, so it remains worthwhile.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 6/10/2015 10:18 AM, Dianne Skoll wrote:
> I'm not disputing that running a caching DNS server is a good idea, but
> you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
IMO, the primary goal of a caching-only nameserver is in fact, not the
caching, but rather the unique source IP so as to avoid running into DNS
limits placed on RBL queries from some BL providers that you can run
afoul of when sharing a DNS server.
Caching is really just icing on the cake coupled with the simplest way
to get a local DNS server up and running, no?
Regards,
KAM
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by David B Funk <db...@engineering.uiowa.edu>.
On Wed, 10 Jun 2015, David Jones wrote:
>> [One should run a caching DNS server on a mail server.]
>
> My point was that running a local caching server is the only way one
> can know exactly how the lookups are happening. If you point to a
> DNS server that you don't manage, it could be forwarding to an ISP's
> DNS caches which will aggregate your queries in with others and could
> cause unexpected results for those RBLs that limit queries.
One other technical benefit to running a local caching server is that if
SA is configured to talk to it va the localhost (loopback) interface there
are MTU advantages.
Most loopback interfaces have a MTU of 16K (or bigger) and will handle large
UDP packets without fragementation. In general DNS transactions are fastest
via UDP if you don't have fragementation issues.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Wed, 10 Jun 2015 14:56:40 +0000
David Jones <dj...@ena.com> wrote:
> My point was that running a local caching server is the only way one
> can know exactly how the lookups are happening.
Ah, true. I missed that point I guess.
Regards,
Dianne.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by David Jones <dj...@ena.com>.
>[One should run a caching DNS server on a mail server.]
>> We are giving you solid advice based on real experiences where we
>> ran into problems and worked around them. Just try to enable RBLs
>> and see how it works for you.
>I'm not disputing that running a caching DNS server is a good idea, but
>you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
>Spamhaus, for example, has a TTL of 1 minute on its A records. (Check
>out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.)
>Quite a number of years ago, I ran an analysis of the mail logs on a
>very busy server and found an abysmally low cache hit rate (about 30%)
>and that was in the day when Spamhaus had a 15-minute TTL.
My point was that running a local caching server is the only way one
can know exactly how the lookups are happening. If you point to a
DNS server that you don't manage, it could be forwarding to an ISP's
DNS caches which will aggregate your queries in with others and could
cause unexpected results for those RBLs that limit queries.
I have 8 mail filters that run a local caching DNS server which forward
to a pair of DNS caches running rbldnsd for a local copy of a number
of RBL zones including my own private RBL. This configuration has to
provide some caching benefits when I get blasted by mass marketing
campaigns. Postfix should keep my local cache populated so when SA
asks for the same DNS information it would be a few milliseconds
response.
I should take some time to do some real analysis as you have done.
Thanks for the info and link.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by Reindl Harald <h....@thelounge.net>.
Am 11.06.2015 um 03:33 schrieb Dianne Skoll:
> On Thu, 11 Jun 2015 01:00:45 +0200
> Reindl Harald <h....@thelounge.net> wrote:
>
>> cache-min-ttl: 600
>
> Even a 10-minute cache time buys you very little. My original analysis
> assumed a 15-minute TTL
calling 32% cache hits on a single day "very little" is questionable
server stats for thread 0: 436986 queries, 140106 answers from cache,
296880 recursions, 4202 prefetch
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Thu, 11 Jun 2015 01:00:45 +0200
Reindl Harald <h....@thelounge.net> wrote:
> cache-min-ttl: 600
Even a 10-minute cache time buys you very little. My original analysis
assumed a 15-minute TTL.
Regards,
Dianne.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by Reindl Harald <h....@thelounge.net>.
Am 10.06.2015 um 16:18 schrieb Dianne Skoll:
> On Wed, 10 Jun 2015 13:56:49 +0000
> David Jones <dj...@ena.com> wrote:
>
> [One should run a caching DNS server on a mail server.]
>
>> We are giving you solid advice based on real experiences where we
>> ran into problems and worked around them. Just try to enable RBLs
>> and see how it works for you.
>
> I'm not disputing that running a caching DNS server is a good idea, but
> you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
> Spamhaus, for example, has a TTL of 1 minute on its A records. (Check
> out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.)
yes, to exceed the volume quicker and only if your resolver has a bad
configuration and that's even one reason more to use a local cache
msg-cache-size: 96m
neg-cache-size: 96m
rrset-cache-size: 192m
cache-min-ttl: 600
cache-max-ttl: 10800
DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Wed, 10 Jun 2015 13:56:49 +0000
David Jones <dj...@ena.com> wrote:
[One should run a caching DNS server on a mail server.]
> We are giving you solid advice based on real experiences where we
> ran into problems and worked around them. Just try to enable RBLs
> and see how it works for you.
I'm not disputing that running a caching DNS server is a good idea, but
you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
Spamhaus, for example, has a TTL of 1 minute on its A records. (Check
out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.)
Quite a number of years ago, I ran an analysis of the mail logs on a
very busy server and found an abysmally low cache hit rate (about 30%)
and that was in the day when Spamhaus had a 15-minute TTL.
Anyway, run through the exercise yourself; it's eye-opening.
My original post was here (back when I used to be David, so don't
let the signature confuse you...)
http://spamassassin.1065346.n5.nabble.com/Fwd-Asrg-draft-levine-iprangepub-01-tp28778p28802.html
Regards,
Dianne.
Re: Must-Have Plugins?
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 10 Jun 2015, at 10:55, Alex Regan wrote:
> Hi,
>
>>> Not everyone is running a dedicated mail server. My server is an
>>> everything-server running on a hosted VPS that only has a few
>>> "users"
>>> that get significant amounts of email. I'm not sure I want another
>>> daemon that can break or take up clock cycles and memory on a system
>>> processing 10 spams / hour (of which the DNSBL service might catch
>>> 2?). At least not yet, but I suppose I could change my mind. At the
>>> moment not that many spams are getting through.
>>
>>> Mike
>>
>> You asked for help, we provided it. It's fine if you ignore the
>> advice,
>> but it's good advice if you want to take your mail filtering to the
>> next
>> level in the future.
>
> And it's not just for mail filtering. Unless you have the smallest of
> systems, it's good general practice to implement a caching name server
> for authentication, the email system itself, and general queries from
> other services as well.
Right. A modern MTA using very simple internal spam exclusion tactics
(validity of sender domains, existence of valid PTR for the client IP,
etc) does multiple DNS queries per message. A local cache entry is
always an order of magnitude faster to get than one on the other side of
some router, so unless you're using an OS with a system name cache (e.g
Solaris' nscd, MacOS' Directory Services, etc.) that can handle many
records, your MTA mostly waits for DNS. Adding in DNSBLs used by the MTA
and variations like URIBLs used in SA, and you can end up doing the same
queries because of the same message far enough apart between the MTA and
the filtering that both go off to the net for resolution because the
system cache is anemic or absent.
But there's an even better reason: trustworthiness. ISPs and others
providing 'free access' resolvers have a history of shaping their
replies in their own interest. Sometimes this is also well-intentioned
(e.g. redirecting "bad" domain names) but while such tactics can be
protective for web browsing users, it can cripple mail servers trying to
block spam.
Re: Must-Have Plugins?
Posted by Alex Regan <my...@gmail.com>.
Hi,
>> Not everyone is running a dedicated mail server. My server is an
>> everything-server running on a hosted VPS that only has a few "users"
>> that get significant amounts of email. I'm not sure I want another
>> daemon that can break or take up clock cycles and memory on a system
>> processing 10 spams / hour (of which the DNSBL service might catch
>> 2?). At least not yet, but I suppose I could change my mind. At the
>> moment not that many spams are getting through.
>
>> Mike
>
> You asked for help, we provided it. It's fine if you ignore the advice,
> but it's good advice if you want to take your mail filtering to the next
> level in the future.
And it's not just for mail filtering. Unless you have the smallest of
systems, it's good general practice to implement a caching name server
for authentication, the email system itself, and general queries from
other services as well.
Regards,
Alex
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>> given that install unbound as local resolver takes 2 minutes it's even not
>> worth to argue on that topic and a spamfilter without RBL's and URIBL's is
>> just nonsense
>I have installed a caching DNS server before (albeit probably about 15
>years ago). But it just shouldn't be necessary.
It can be necessary if you have enough mail volume.
>By "librarified" I mean the DNS "server" is just a code context that
>can be constructed with it's own config precisely and only as needed
>by the software that will be querying it (possibly temporarily if it's
>just client-only activity like a barrage of DNS queries fired in
>reaction to an email that fails other spam tests). It should not be
>necessary to change the resolver configuration or behavior of the
>entire system and everything running on it if only one component in
>the system needs this special feature (in this case a query limit and
>private cache). That is just bad programming philosophy and it the
>source of a lot of bad behavior in software (and DNS is a very good
>example of this actually).
You obviously don't understand how DNS works in relation to RBLs.
We are giving you solid advice based on real experiences where we
ran into problems and worked around them. Just try to enable RBLs
and see how it works for you.
>Not everyone is running a dedicated mail server. My server is an
>everything-server running on a hosted VPS that only has a few "users"
>that get significant amounts of email. I'm not sure I want another
>daemon that can break or take up clock cycles and memory on a system
>processing 10 spams / hour (of which the DNSBL service might catch
>2?). At least not yet, but I suppose I could change my mind. At the
>moment not that many spams are getting through.
>Mike
You asked for help, we provided it. It's fine if you ignore the advice,
but it's good advice if you want to take your mail filtering to the next
level in the future.
Re: Must-Have Plugins?
Posted by Michael B Allen <io...@gmail.com>.
On Wed, Jun 10, 2015 at 7:25 AM, Reindl Harald <h....@thelounge.net> wrote:
>
>
> Am 10.06.2015 um 13:21 schrieb Kevin A. McGrail:
>>
>> On 6/10/2015 12:45 AM, Michael B Allen wrote:
>>>
>>> But I just can't
>>> bring myself to install a caching DNS server and run everything
>>> through localhost. This is why software should be librarified.
>>
>> I strongly advise you to install a caching DNS server and using a few RBLs
>
>
> +1
>
> i can't understand "I just can't bring myself to install a caching DNS
> server and run everything through localhost" because that is the way to go
> to avoid exceed RBL limits and bad resolvers
>
>
> "This is why software should be librarified" is nonsense in that context -
> the library also needs to ask a dns server at the end of the day and the
> server needs to be 100% trustable when it comes to email
>
> given that install unbound as local resolver takes 2 minutes it's even not
> worth to argue on that topic and a spamfilter without RBL's and URIBL's is
> just nonsense
I have installed a caching DNS server before (albeit probably about 15
years ago). But it just shouldn't be necessary.
By "librarified" I mean the DNS "server" is just a code context that
can be constructed with it's own config precisely and only as needed
by the software that will be querying it (possibly temporarily if it's
just client-only activity like a barrage of DNS queries fired in
reaction to an email that fails other spam tests). It should not be
necessary to change the resolver configuration or behavior of the
entire system and everything running on it if only one component in
the system needs this special feature (in this case a query limit and
private cache). That is just bad programming philosophy and it the
source of a lot of bad behavior in software (and DNS is a very good
example of this actually).
Not everyone is running a dedicated mail server. My server is an
everything-server running on a hosted VPS that only has a few "users"
that get significant amounts of email. I'm not sure I want another
daemon that can break or take up clock cycles and memory on a system
processing 10 spams / hour (of which the DNSBL service might catch
2?). At least not yet, but I suppose I could change my mind. At the
moment not that many spams are getting through.
Mike
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 10.06.2015 um 13:21 schrieb Kevin A. McGrail:
> On 6/10/2015 12:45 AM, Michael B Allen wrote:
>> But I just can't
>> bring myself to install a caching DNS server and run everything
>> through localhost. This is why software should be librarified.
> I strongly advise you to install a caching DNS server and using a few RBLs
+1
i can't understand "I just can't bring myself to install a caching DNS
server and run everything through localhost" because that is the way to
go to avoid exceed RBL limits and bad resolvers
"This is why software should be librarified" is nonsense in that context
- the library also needs to ask a dns server at the end of the day and
the server needs to be 100% trustable when it comes to email
given that install unbound as local resolver takes 2 minutes it's even
not worth to argue on that topic and a spamfilter without RBL's and
URIBL's is just nonsense
Re: Must-Have Plugins?
Posted by John Hardin <jh...@impsec.org>.
On Wed, 10 Jun 2015, Kevin A. McGrail wrote:
> On 6/10/2015 12:45 AM, Michael B Allen wrote:
>> But I just can't
>> bring myself to install a caching DNS server and run everything
>> through localhost. This is why software should be librarified.
>
> I strongly advise you to install a caching DNS server and using a few RBLs.
Just a minor nit: It's not the "caching" part that's important here, it's
the "not forwarding" part. If you set up a caching DNS server that just
forwards to your ISP's DNS servers, you haven't addressed the "BL blocked
due to query volume" problem.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
...much of our country's counterterrorism security spending is not
designed to protect us from the terrorists, but instead to protect
our public officials from criticism when another attack occurs.
-- Bruce Schneier
-----------------------------------------------------------------------
Re: Must-Have Plugins?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 6/10/2015 12:45 AM, Michael B Allen wrote:
> But I just can't
> bring myself to install a caching DNS server and run everything
> through localhost. This is why software should be librarified.
I strongly advise you to install a caching DNS server and using a few RBLs.
regards,
KAM
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>> Some of the best and easiest things you can enable to block spam are
>> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
>> - Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the
>> majority of junk before it reaches SA. Just make sure you are below their
>> free threshold limit. One important way to do this is to make sure your
>> SA server isn't pointed to an Internet caching DNS server that would join
>> your queries with others. Install a local caching DNS server that does not
>> forward to another caching DNS server and change /etc/resolv.conf to use
>> 127.0.0.1.
>Well that sounds like a must-have feature to me. But I just can't
>bring myself to install a caching DNS server and run everything
>through localhost. This is why software should be librarified.
What OS are you running? It's normally a very simple install to get a caching
DNS server running locally since the default configurations usually come ready
to do exactly what you need in this case. Google "caching dns server howto"
plus your OS and you will see it's pretty easy.
You can try using RBLs with your existing DNS server configuration. If it's a
dedicated DNS server for your network, then you have a good chance of
staying below the free thresholds for a low volume server. If it's a major
ISP's DNS server then the odds will be against you.
http://www.spamhaus.org/faq/section/DNSBL%20Usage#366
Try "dig 138.178.203.192.zen.spamhaus.org" or nslookup to see if you get
back a response of 127.0.0.4. If so, you should be good.
>> - Enable DNS checks:
>> Make sure the sending mail server's SMTP HELO is a valid domain.
>> Make sure the sender address (MAIL FROM) is a valid domain.
>> Make sure the sending mail server has a PTR record. Some can go farther with
>> this one and require the PTR match the SMTP HELO for FCrDNS but there are
>> many legit mail servers out there that don't have this setup properly so I can
>> only check to make sure a PTR record exists. Later in SA I add points for rule
>> RDNS_NONE that penalizes for incorrect FCrDNS.
>Is this done with postfix rules or SA rules? Where can I learn more
>about this? Doesn't SA already do this stuff?
This should be done in your MTA if possible before it's handed off to SA. Some of
this information is exposed to SA in headers but some isn't. High volume servers
need this logic at the MTA to keep processing times low. Only a small percentage
of mail makes it to SA in my environment and most of that is going to be clean.
In a low volume environment, you can send most of the mail through SA and still
keep the processing times down low. My MailScanner batches average about 5 to
10 messages and complete in 4 to 5 seconds normally with ClamAV and Eset
Nod32 AV scanners.
Re: Must-Have Plugins?
Posted by Michael B Allen <io...@gmail.com>.
On Tue, Jun 9, 2015 at 8:36 AM, David Jones <dj...@ena.com> wrote:
>>>On 08.06.15 23:03, Michael B Allen wrote:
>>>So I have had SA running for about 2 days on a very small site with a
>>>handful of users. I've been running the default config just to see how
>>>well it would do by itself. Unfortunately quite a lot of spam is
>>>getting through. So far 40 of 142 spams have passed.
>>>
>>>So my question is, what is the best way to improve things? Is there
>>>any particular must-have plugins? What is the one thing I can do to a
>>>default install that is going to give me the biggest return on
>>>invested effort?
>
>>network checks like razor/pyzor/dcc (they all require third-party programs)
>>TextCat (if you and your users are able to set up ok_languages)
>
> +1 on the razor/pyzor/dcc but they can be challenging to get working
> TextCat is good and easy to enable.
>
> Some of the best and easiest things you can enable to block spam are
> outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
> - Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the
> majority of junk before it reaches SA. Just make sure you are below their
> free threshold limit. One important way to do this is to make sure your
> SA server isn't pointed to an Internet caching DNS server that would join
> your queries with others. Install a local caching DNS server that does not
> forward to another caching DNS server and change /etc/resolv.conf to use
> 127.0.0.1.
Well that sounds like a must-have feature to me. But I just can't
bring myself to install a caching DNS server and run everything
through localhost. This is why software should be librarified.
> - Enable DNS checks:
> Make sure the sending mail server's SMTP HELO is a valid domain.
> Make sure the sender address (MAIL FROM) is a valid domain.
> Make sure the sending mail server has a PTR record. Some can go farther with
> this one and require the PTR match the SMTP HELO for FCrDNS but there are
> many legit mail servers out there that don't have this setup properly so I can
> only check to make sure a PTR record exists. Later in SA I add points for rule
> RDNS_NONE that penalizes for incorrect FCrDNS.
Is this done with postfix rules or SA rules? Where can I learn more
about this? Doesn't SA already do this stuff?
Sounds like I'm just going to stick with bayes. But suprisingly my
spam intake has slowed. I don't even have the 200 spams yet.
Thanks,
Mike
Re: Must-Have Plugins?
Posted by Reindl Harald <h....@thelounge.net>.
Am 09.06.2015 um 17:23 schrieb Alex Regan:
>> My top hit counts from last week from dnsblcount.pl script (using
>> postscreen so the numbers are most likely skewed based on ordering and
>> thresholds being met with multiple RBL hits):
>
> Where did you find dnsblcount.pl? Or is this is your own? That sounds
> like a great compliment to pflogsumm and mailgraph. I'm also using
> postscreen. How difficult do you think it would be to update it to
> account for those rejects too?
it counts those rejects but postscreen only logs the RBL with the
higehst score or in case more with identical hit the one with the
fastest response
https://www.google.at/search?q=dnsblcount.pl
http://www.joreybump.com/dnsblcount/
spamhaus.org 12455
inps.de 4527
barracudacentral.org 3366
sorbs.net 2878
junkemailfilter.com 1073
thelounge.net 709
spamcop.net 129
mailspike.net 113
swinog.ch 8
manitu.net 2
psbl.org 2
spameatingmonkey.net 1
=================================
Total DNSBL rejections: 25263
Re: Must-Have Plugins?
Posted by Alex Regan <my...@gmail.com>.
Hi,
> My top hit counts from last week from dnsblcount.pl script (using
> postscreen so the numbers are most likely skewed based on ordering and
> thresholds being met with multiple RBL hits):
Where did you find dnsblcount.pl? Or is this is your own? That sounds
like a great compliment to pflogsumm and mailgraph. I'm also using
postscreen. How difficult do you think it would be to update it to
account for those rejects too?
> - Enable greylisting. This is just about the only way you can block zero-hour spam
> from compromised accounts that come from legit mail servers before they get
> listed in RBLs. I use SQLgrey with Postfix and was able to ease it in slowly with
> it's feature called discrimination mode.
I'm also using sqlgrey with DB_CLUSTER to support communications between
bayes on multiple systems. Do you know anything about using policyd for
this? I've seen a few references to it recently, and heard it may be a
better replacement for use with postfix?
> Inside SA add the KAM.cf rules (Google for it) and update them a couple of times
> each day. These rules are a must!
http://www.peregrinehw.com/downloads/SpamAssassin/contrib/KAM.cf
> I also have added CRM114 and BOGOFILTER plugins which are similar to BAYES
> but don't require the manual training. These are fairly difficult to install but
> provide a good complement to BAYES scoring and actually help automatically
> train my BAYES database.
CRM114 looks interesting, but is it still being developed? Can you
briefly describe what's involved in setting it up?
BOGOFILTER also looks interesting, but are you concerned it's
duplicating some of the rules or efforts of SA itself?
I'm really interested in TxRep:
http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_TxRep.html
> Setup ClamAV and add the UNOFFICIAL SIGS.
Yes, everyone should be using at least the sanesecurity sigs. A script
to download and manage the sigs is here:
http://sourceforge.net/projects/unofficial-sigs/
Regards,
Alex
Re: Must-Have Plugins?
Posted by Amir Caspi <ce...@3phase.com>.
On Jun 9, 2015, at 12:51 PM, RW <rw...@googlemail.com> wrote:
> Bogofilter is pretty easy to use without a plugin. Typically it's just
> a matter of piping your mail through bogofilter -e -p
> In general the most efficient way to score-in an external filter is to
> run it separately and have SA score the result - by scoring
> headers or otherwise.
My SA setup is that mail is piped to spamc for individual users, as follows:
[host] $ cat /etc/procmailrc
HOSTNAME=`hostname`
:0 fw
| /usr/bin/spamc -u "${LOGNAME}@${HOSTNAME}" -s 1500000
[host] $ cat ~/.procmailrc
SPAM_FOLDER="$HOME/mail/Spam"
:0 :$HOME/spamassassin.lock
* X-Spam-Status: Yes
$SPAM_FOLDER
I'm unfortunately not a procmail expert... in fact, I barely understand how it works at all. Given the above, how would I incorporate bogofilter? Would it be as "simple" as adding a line like:
| /usr/bin/bogofilter -e -p
into /etc/procmailrc right above the pipe into spamc? And then, presumably, I'd add custom rules (into SA's local.cf) that would score the Bogo results?
Thanks.
--- Amir
Re: Must-Have Plugins?
Posted by RW <rw...@googlemail.com>.
On Tue, 9 Jun 2015 12:36:58 +0000
David Jones wrote:
> I also have added CRM114 and BOGOFILTER plugins which are similar to
> BAYES but don't require the manual training.
They need manual training to the same extent that Bayes needs it.
> These are fairly difficult to install
Bogofilter is pretty easy to use without a plugin. Typically it's just
a matter of piping your mail through bogofilter -e -p
In general the most efficient way to score-in an external filter is to
run it separately and have SA score the result - by scoring
headers or otherwise. The reason for using the Bogofilter plugin
is that it plumbs Bogofilter into SA autotraining. Personally I'd only
rely on that as a last resort.
> but provide a good complement to BAYES scoring
One thing that can make bogofilter more complementary is to configure
it for multi-word tokenization. It makes it good with spam that
has spammy language, but few spammy words, which Bayes can struggle
with. This is also the kind of spam on which most custom body rules are
targeted.
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>> - Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the
>> majority of junk before it reaches SA. Just make sure you are below their
>> free threshold limit. One important way to do this is
>"One important way to do this" in terms of the Spamhaus threshold limit
>is to not be such a tightwad and poney up for the Spamhaus commercial
>service. ;-)
>They do a cheaper version than the RSync feed, you can just query their
>servers directly.
>Spamhaus do a fantastic job. They deserve charitable donations from
>generous mail sysadmins !!!
We filter a lot of mailboxes and pay spamhaus several thousands of dollars each year.
The invaluement.com RBL is very effective and only costs a few hundred dollars a year.
>> - Enable greylisting.
>Ewww...
>I hate people who operate graylisting. Its a lazy "tarnish everyone
>with the same brush" approach to anti-spam.
>In this day and age, you don't need it. Decent network checks, properly
>configured Spamassassin and you should be able to achieve a very
>respectable spam catching rate.
I respectfully disagree. I too hated greylisting for years until recently when I found a
way to slowly ease it in for my users who didn't even detect it. There is no other way
to block brand new spam campaigns from compromised accounts. These mail servers
normally have a good reputation so they are not on any RBLs yet. Greylisting puts a
"speed bump" in place so the RBLs have time to catch up.
Spammers pay "sweat shops" to devise new spam that will get through the major filters
like SA and commercial products as zero-hour spam. Bayes is often ineffective against
these new campaigns. When a new spam campaign like this hits the Internet, the world
takes a little while to detect and react to it so you have to put some kind of buffer like
greylisting in place.
I whitelist a lot of major ISPs and known good senders to bypass greylisting so this only
impacts mostly small mail servers that don't have any compromised account detection.
My company's support team used to get several calls a week calls about blacklisting
senders that turned out to be compromised accounts but now we might get one or two
every 3 months now. By the time the blacklist entry was added, RBLs had already been
blocking them so I just remove the new entry to keep my lists clean.
If you have a large mail filtering environment with a lot of very old email accounts that have
become bought and sold on spammer lists, this is a must. I guess if you are only filtering
for a few hundred accounts, you can do a lot of things differently and be fine.
Re: Must-Have Plugins?
Posted by Ben <be...@list-subs.com>.
> - Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the
> majority of junk before it reaches SA. Just make sure you are below their
> free threshold limit. One important way to do this is
"One important way to do this" in terms of the Spamhaus threshold limit
is to not be such a tightwad and poney up for the Spamhaus commercial
service. ;-)
They do a cheaper version than the RSync feed, you can just query their
servers directly.
Spamhaus do a fantastic job. They deserve charitable donations from
generous mail sysadmins !!!
> - Enable greylisting.
Ewww...
I hate people who operate graylisting. Its a lazy "tarnish everyone
with the same brush" approach to anti-spam.
In this day and age, you don't need it. Decent network checks, properly
configured Spamassassin and you should be able to achieve a very
respectable spam catching rate.
Re: Must-Have Plugins?
Posted by David Jones <dj...@ena.com>.
>>On 08.06.15 23:03, Michael B Allen wrote:
>>So I have had SA running for about 2 days on a very small site with a
>>handful of users. I've been running the default config just to see how
>>well it would do by itself. Unfortunately quite a lot of spam is
>>getting through. So far 40 of 142 spams have passed.
>>
>>So my question is, what is the best way to improve things? Is there
>>any particular must-have plugins? What is the one thing I can do to a
>>default install that is going to give me the biggest return on
>>invested effort?
>network checks like razor/pyzor/dcc (they all require third-party programs)
>TextCat (if you and your users are able to set up ok_languages)
+1 on the razor/pyzor/dcc but they can be challenging to get working
TextCat is good and easy to enable.
Some of the best and easiest things you can enable to block spam are
outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
- Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the
majority of junk before it reaches SA. Just make sure you are below their
free threshold limit. One important way to do this is to make sure your
SA server isn't pointed to an Internet caching DNS server that would join
your queries with others. Install a local caching DNS server that does not
forward to another caching DNS server and change /etc/resolv.conf to use
127.0.0.1.
Look in this list archives for other RBLs and DBLs that have been recently
recommended. My most effective RBL is sip.invaluement.com based on my
logs. This is a subscription-based RBL that is reasonably priced and worth
every penny.
My top hit counts from last week from dnsblcount.pl script (using
postscreen so the numbers are most likely skewed based on ordering and
thresholds being met with multiple RBL hits):
sip.invaluement.com 1323365
b.barracudacentral.org 464976
zen.spamhaus.org 243244
sip24.invaluement.com 238086
dbl.spamhaus.org 72507
- Enable DNS checks:
Make sure the sending mail server's SMTP HELO is a valid domain.
Make sure the sender address (MAIL FROM) is a valid domain.
Make sure the sending mail server has a PTR record. Some can go farther with
this one and require the PTR match the SMTP HELO for FCrDNS but there are
many legit mail servers out there that don't have this setup properly so I can
only check to make sure a PTR record exists. Later in SA I add points for rule
RDNS_NONE that penalizes for incorrect FCrDNS.
- Enable greylisting. This is just about the only way you can block zero-hour spam
from compromised accounts that come from legit mail servers before they get
listed in RBLs. I use SQLgrey with Postfix and was able to ease it in slowly with
it's feature called discrimination mode.
- Block the newer TLDs until you need to allow them. Spammers are registering
the new TLDs like ".link" and ".click" then sending spam from it. There was a
recent thread in this mailing list talking about how to get a list of them.
- Block TLDs that often contain nothing but spam. Not everyone can do this but
in my environment, I am able to block .br, .hu, .ro, .ru, .tw, .vn, etc. I have a
small whitelist built over time that I allow trusted IPs to bypass this check but
most of the time this is email from infected PCs that are members in a botnet
sending from all over the world with these country codes.
Inside SA add the KAM.cf rules (Google for it) and update them a couple of times
each day. These rules are a must!
I also have added CRM114 and BOGOFILTER plugins which are similar to BAYES
but don't require the manual training. These are fairly difficult to install but
provide a good complement to BAYES scoring and actually help automatically
train my BAYES database.
Setup ClamAV and add the UNOFFICIAL SIGS.
Re: Must-Have Plugins?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 08.06.15 23:03, Michael B Allen wrote:
>So I have had SA running for about 2 days on a very small site with a
>handful of users. I've been running the default config just to see how
>well it would do by itself. Unfortunately quite a lot of spam is
>getting through. So far 40 of 142 spams have passed.
>
>So my question is, what is the best way to improve things? Is there
>any particular must-have plugins? What is the one thing I can do to a
>default install that is going to give me the biggest return on
>invested effort?
network checks like razor/pyzor/dcc (they all require third-party programs)
TextCat (if you and your users are able to set up ok_languages)
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease