Re: Adding RBLs to default SA rules

[Dave Jones <da...@apache.org>]

A rule with a score of zero is disabled and not even attempted.  Back 
when some RBLs went from free to paid, SA admins all around the world 
had to set some rules to a score of zero.  Our mail volume is so high 
that I have to disable the SURBL rules like this or I get a nasty email 
from them with a quote to continue using them.  BTW, I did a 1 month 
trial with their rbldnsd feed and I didn't find any value in it.  In 
fact, it had too many false positives.  When I told them that they just 
sent me a link showing how good they were.

A local caching DNS server in front of the rbldnsd will help since your 
MTA and SA will often be requesting similar DNS records.

FYI, the Invaluement RBL combined with Spamhaus will do wonders for your 
mail filtering.  The IVM RBL is very cheap and worth every penny.  We 
pay a few hundred to Rob for IVM and a few thousand for Spamhaus.  Then 
with a few extra SA rules for IVM, that handles the majority of our spam 
with no false positives.

Dave

On 06/15/2017 05:05 AM, Bryan Vest wrote:
> On RBL's if my thinking is correct even if the score is zero it will still
> slow down the processing as it would still go look at the RBL. That is why
> the system I manage temporarily mirror's RBL's on a local machine using
> rbldnsd. Though we expire all records if we have seen no activity from that
> ip in 4 hours, if they hit again from an external RBL they are put back in
> the local mirror.
> 
> With response times of anywhere from 10ms to 60ms even longer if the RBL is
> having a bad day. When you are processing 5,000,000 inbound per day that
> adds up pretty quick to wasted time waiting.
> 
> How RBL's should work vs. how some of them do work is always a topic of
> debate. The only one I trust without question is spamhaus which I think is
> free to a point then becomes a paid service as your mail load grows.
> 
> I love the idea but would be cautious about which RBL's to use and document
> that part so there is no question of how it works and what it does.
> 
> --Bryan
> 
> 
> 
> On Wed, Jun 14, 2017 at 6:21 PM, Dave Jones <da...@apache.org> wrote:
> 
>> On 06/14/2017 08:53 AM, Kevin A. McGrail wrote:
>>
>>> Some comments in-line below:
>>>
>>> On 6/14/2017 9:47 AM, Dave Jones wrote:
>>>
>>>> There are soooooo many bugs already open.  I looked around BZ some last
>>>> week and it seems like there is no real recent movement on anything.  I
>>>> would like to create a BZ to get a DMARC plugin started but on the users
>>>> mailing list a few weeks back it didn't seem to get any traction or enough
>>>> interest.  It seems like so many people are using SA in many different ways
>>>> that the momentum to move things forward is fragmented. The great thing
>>>> about SA is that it's very flexible but that is also a negative thing too.
>>>>
>>> I would appreciate it if you would still open a bug.  I am trying to make
>>> that a central place for ideas.
>>>
>>>
>> I commented on your existing bug just now.
>>
>> Also, you might try asking someone to write something or post a draft
>>> patch, etc.  A little bit of movement can become a snowball.
>>>
>>>> Maybe there are some out there that took some of my techniques and are
>>>> trying them out but it seems that everyone is kinda set in their ways. Mail
>>>> filtering is changing with SPF, DKIM, DMARC, ARC, etc.  RBLs are still very
>>>> important but SA currently doesn't use enough of them by default.  I
>>>> understand they have to meet some requirements before they can be included
>>>> in SA by default which is why I am pushing this in BZ.
>>>>
>>> Remember that at least my POV is that SA is a framework so the goal for
>>> me is to support RBLs in general.  But I'm very happy to support you in
>>> this endeavor.
>>>
>>
>> Many don't understand how RBLs work so it would be helpful if we could
>> include some more common RBLs in the default rules and maybe set the
>> default score to zero and document somewhere how to enable them by setting
>> up a non-zero score in their local.cf.
>>
>> Dave
>>
>>
> 

Re: Adding RBLs to default SA rules

[Dave Jones <da...@apache.org>]

Have you checked out the Invaluement RBL?  Rob will give you a free trial.


On 06/15/2017 08:53 AM, Bryan Vest wrote:
> I am corrected then, if a score of 0 will not even hit the RBL then that
> wipes out most of what I said.
> 
> The way our internal rbldnsd works it never has more than about 120k
> entries in it, we dont rsync from anyone it is maintained by perl scripts
> that have been doing the job pretty well for around 7 years now.
> 
> Our spamhaus feed is included in our contract with the mail system vendor.
> 
> --Bryan
> 
> On Thu, Jun 15, 2017 at 9:19 AM, Dave Jones <da...@apache.org> wrote:
> 
>> A rule with a score of zero is disabled and not even attempted.  Back when
>> some RBLs went from free to paid, SA admins all around the world had to set
>> some rules to a score of zero.  Our mail volume is so high that I have to
>> disable the SURBL rules like this or I get a nasty email from them with a
>> quote to continue using them.  BTW, I did a 1 month trial with their
>> rbldnsd feed and I didn't find any value in it.  In fact, it had too many
>> false positives.  When I told them that they just sent me a link showing
>> how good they were.
>>
>> A local caching DNS server in front of the rbldnsd will help since your
>> MTA and SA will often be requesting similar DNS records.
>>
>> FYI, the Invaluement RBL combined with Spamhaus will do wonders for your
>> mail filtering.  The IVM RBL is very cheap and worth every penny.  We pay a
>> few hundred to Rob for IVM and a few thousand for Spamhaus.  Then with a
>> few extra SA rules for IVM, that handles the majority of our spam with no
>> false positives.
>>
>> Dave
>>
>> On 06/15/2017 05:05 AM, Bryan Vest wrote:
>>
>>> On RBL's if my thinking is correct even if the score is zero it will still
>>> slow down the processing as it would still go look at the RBL. That is why
>>> the system I manage temporarily mirror's RBL's on a local machine using
>>> rbldnsd. Though we expire all records if we have seen no activity from
>>> that
>>> ip in 4 hours, if they hit again from an external RBL they are put back in
>>> the local mirror.
>>>
>>> With response times of anywhere from 10ms to 60ms even longer if the RBL
>>> is
>>> having a bad day. When you are processing 5,000,000 inbound per day that
>>> adds up pretty quick to wasted time waiting.
>>>
>>> How RBL's should work vs. how some of them do work is always a topic of
>>> debate. The only one I trust without question is spamhaus which I think is
>>> free to a point then becomes a paid service as your mail load grows.
>>>
>>> I love the idea but would be cautious about which RBL's to use and
>>> document
>>> that part so there is no question of how it works and what it does.
>>>
>>> --Bryan
>>>
>>>
>>>
>>> On Wed, Jun 14, 2017 at 6:21 PM, Dave Jones <da...@apache.org> wrote:
>>>
>>> On 06/14/2017 08:53 AM, Kevin A. McGrail wrote:
>>>>
>>>> Some comments in-line below:
>>>>>
>>>>> On 6/14/2017 9:47 AM, Dave Jones wrote:
>>>>>
>>>>> There are soooooo many bugs already open.  I looked around BZ some last
>>>>>> week and it seems like there is no real recent movement on anything.  I
>>>>>> would like to create a BZ to get a DMARC plugin started but on the
>>>>>> users
>>>>>> mailing list a few weeks back it didn't seem to get any traction or
>>>>>> enough
>>>>>> interest.  It seems like so many people are using SA in many different
>>>>>> ways
>>>>>> that the momentum to move things forward is fragmented. The great thing
>>>>>> about SA is that it's very flexible but that is also a negative thing
>>>>>> too.
>>>>>>
>>>>>> I would appreciate it if you would still open a bug.  I am trying to
>>>>> make
>>>>> that a central place for ideas.
>>>>>
>>>>>
>>>>> I commented on your existing bug just now.
>>>>
>>>> Also, you might try asking someone to write something or post a draft
>>>>
>>>>> patch, etc.  A little bit of movement can become a snowball.
>>>>>
>>>>> Maybe there are some out there that took some of my techniques and are
>>>>>> trying them out but it seems that everyone is kinda set in their ways.
>>>>>> Mail
>>>>>> filtering is changing with SPF, DKIM, DMARC, ARC, etc.  RBLs are still
>>>>>> very
>>>>>> important but SA currently doesn't use enough of them by default.  I
>>>>>> understand they have to meet some requirements before they can be
>>>>>> included
>>>>>> in SA by default which is why I am pushing this in BZ.
>>>>>>
>>>>>> Remember that at least my POV is that SA is a framework so the goal for
>>>>> me is to support RBLs in general.  But I'm very happy to support you in
>>>>> this endeavor.
>>>>>
>>>>>
>>>> Many don't understand how RBLs work so it would be helpful if we could
>>>> include some more common RBLs in the default rules and maybe set the
>>>> default score to zero and document somewhere how to enable them by
>>>> setting
>>>> up a non-zero score in their local.cf.
>>>>
>>>> Dave
>>>>
>>>>
>>>>
>>>
>>
> 

Re: Adding RBLs to default SA rules

[Bryan Vest <mu...@gmail.com>]

I am corrected then, if a score of 0 will not even hit the RBL then that
wipes out most of what I said.

The way our internal rbldnsd works it never has more than about 120k
entries in it, we dont rsync from anyone it is maintained by perl scripts
that have been doing the job pretty well for around 7 years now.

Our spamhaus feed is included in our contract with the mail system vendor.

--Bryan

On Thu, Jun 15, 2017 at 9:19 AM, Dave Jones <da...@apache.org> wrote:

> A rule with a score of zero is disabled and not even attempted.  Back when
> some RBLs went from free to paid, SA admins all around the world had to set
> some rules to a score of zero.  Our mail volume is so high that I have to
> disable the SURBL rules like this or I get a nasty email from them with a
> quote to continue using them.  BTW, I did a 1 month trial with their
> rbldnsd feed and I didn't find any value in it.  In fact, it had too many
> false positives.  When I told them that they just sent me a link showing
> how good they were.
>
> A local caching DNS server in front of the rbldnsd will help since your
> MTA and SA will often be requesting similar DNS records.
>
> FYI, the Invaluement RBL combined with Spamhaus will do wonders for your
> mail filtering.  The IVM RBL is very cheap and worth every penny.  We pay a
> few hundred to Rob for IVM and a few thousand for Spamhaus.  Then with a
> few extra SA rules for IVM, that handles the majority of our spam with no
> false positives.
>
> Dave
>
> On 06/15/2017 05:05 AM, Bryan Vest wrote:
>
>> On RBL's if my thinking is correct even if the score is zero it will still
>> slow down the processing as it would still go look at the RBL. That is why
>> the system I manage temporarily mirror's RBL's on a local machine using
>> rbldnsd. Though we expire all records if we have seen no activity from
>> that
>> ip in 4 hours, if they hit again from an external RBL they are put back in
>> the local mirror.
>>
>> With response times of anywhere from 10ms to 60ms even longer if the RBL
>> is
>> having a bad day. When you are processing 5,000,000 inbound per day that
>> adds up pretty quick to wasted time waiting.
>>
>> How RBL's should work vs. how some of them do work is always a topic of
>> debate. The only one I trust without question is spamhaus which I think is
>> free to a point then becomes a paid service as your mail load grows.
>>
>> I love the idea but would be cautious about which RBL's to use and
>> document
>> that part so there is no question of how it works and what it does.
>>
>> --Bryan
>>
>>
>>
>> On Wed, Jun 14, 2017 at 6:21 PM, Dave Jones <da...@apache.org> wrote:
>>
>> On 06/14/2017 08:53 AM, Kevin A. McGrail wrote:
>>>
>>> Some comments in-line below:
>>>>
>>>> On 6/14/2017 9:47 AM, Dave Jones wrote:
>>>>
>>>> There are soooooo many bugs already open.  I looked around BZ some last
>>>>> week and it seems like there is no real recent movement on anything.  I
>>>>> would like to create a BZ to get a DMARC plugin started but on the
>>>>> users
>>>>> mailing list a few weeks back it didn't seem to get any traction or
>>>>> enough
>>>>> interest.  It seems like so many people are using SA in many different
>>>>> ways
>>>>> that the momentum to move things forward is fragmented. The great thing
>>>>> about SA is that it's very flexible but that is also a negative thing
>>>>> too.
>>>>>
>>>>> I would appreciate it if you would still open a bug.  I am trying to
>>>> make
>>>> that a central place for ideas.
>>>>
>>>>
>>>> I commented on your existing bug just now.
>>>
>>> Also, you might try asking someone to write something or post a draft
>>>
>>>> patch, etc.  A little bit of movement can become a snowball.
>>>>
>>>> Maybe there are some out there that took some of my techniques and are
>>>>> trying them out but it seems that everyone is kinda set in their ways.
>>>>> Mail
>>>>> filtering is changing with SPF, DKIM, DMARC, ARC, etc.  RBLs are still
>>>>> very
>>>>> important but SA currently doesn't use enough of them by default.  I
>>>>> understand they have to meet some requirements before they can be
>>>>> included
>>>>> in SA by default which is why I am pushing this in BZ.
>>>>>
>>>>> Remember that at least my POV is that SA is a framework so the goal for
>>>> me is to support RBLs in general.  But I'm very happy to support you in
>>>> this endeavor.
>>>>
>>>>
>>> Many don't understand how RBLs work so it would be helpful if we could
>>> include some more common RBLs in the default rules and maybe set the
>>> default score to zero and document somewhere how to enable them by
>>> setting
>>> up a non-zero score in their local.cf.
>>>
>>> Dave
>>>
>>>
>>>
>>
>