You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Shivram Krishnan <ro...@gmail.com> on 2017/08/14 22:00:36 UTC
Operators Blacklist Survey
Hi,
I am a graduate student at the University of Southern California and am
currently researching on the impact of false positives in blacklists. I am
aware that spamassassin uses blacklists in its rule based system to stop
spam messages. But since it is a rule based system, even if there are false
positives in blacklists, there may be other rules which can influence
spamassassin to mark it correctly. There are several other blacklists which
are used to stop different attacks (eg phishing, DDoS, malware hosting
etc). I was wondering if operators in general use external
blacklists(uribl, spamhaus, spamcop etc) in the form of rule based system
(like spamassassin) or use it outrightly to block all IPs listed in them.
It will be great if you can take this four question survey, which can help
me understand the usage of blacklists by operators. The survey consists of
these questions -
1) The size of the network(s) you manage(in terms of customers)
2) List of external blacklists used.
3) How these blacklists are used? whether in a rule based system or
outrightly blocked or both
4) If external blacklists are used in a non-rule based system, how do you
overcome false positives?
The link to the survey is below -
https://docs.google.com/forms/d/e/1FAIpQLSe-hgYD-ifkFMyPHrqYL0b7jAkbWjOKiAQjh-oI4mYeiVQg2g/viewform
Shivram
Re: Operators Blacklist Survey
Posted by Rupert Gallagher <ru...@protonmail.com>.
Re. 1: undisclosed
Re. 2: null
Re. 3: not applicable
Re. 4: not applicable
We do not use external blacklists to filter spam.
We use own method, with a ham/spam ratio of 98/02. Spam is not delivered: we reject it upfront. Our business clients are very happy.
No, we are not protonmail. Protonmail delivers spam that we would have rejected upfront.
Sent from ProtonMail Mobile
On Tue, Aug 15, 2017 at 12:00 AM, Shivram Krishnan <ro...@gmail.com> wrote:
> Hi,
>
> I am a graduate student at the University of Southern California and am currently researching on the impact of false positives in blacklists. I am aware that spamassassin uses blacklists in its rule based system to stop spam messages. But since it is a rule based system, even if there are false positives in blacklists, there may be other rules which can influence spamassassin to mark it correctly. There are several other blacklists which are used to stop different attacks (eg phishing, DDoS, malware hosting etc). I was wondering if operators in general use external blacklists(uribl, spamhaus, spamcop etc) in the form of rule based system (like spamassassin) or use it outrightly to block all IPs listed in them.
>
> It will be great if you can take this four question survey, which can help me understand the usage of blacklists by operators. The survey consists of these questions -
> 1) The size of the network(s) you manage(in terms of customers)
> 2) List of external blacklists used.
> 3) How these blacklists are used? whether in a rule based system or outrightly blocked or both
> 4) If external blacklists are used in a non-rule based system, how do you overcome false positives?
>
> The link to the survey is below -
>
> https://docs.google.com/forms/d/e/1FAIpQLSe-hgYD-ifkFMyPHrqYL0b7jAkbWjOKiAQjh-oI4mYeiVQg2g/viewform
>
> Shivram
Re: Operators Blacklist Survey
Posted by David Jones <dj...@ena.com>.
On 08/15/2017 12:46 PM, Shivram Krishnan wrote:
> Thanks for the response Dianne.
>
> Rule-based systems like spamassassin make room for false positives from
> any one of the rules. For instance , a blacklist can have a false
> positive, but there may be other rules which may not agree with the
> blacklist. An ensemble of such rules allows make spamassassin to be more
> accurate.
>
> In case of non-rule based systems like firewall, an inaccurate blacklist
> can prove costly when the firewall drops legitimate traffic based on
> inaccurate blacklists. I was reading about graylists on cisco firewalls
> <https://www.cisco.com/c/en/us/td/docs/security/security_management/cisco_security_manager/security_manager/4-1/user/guide/CSMUserGuide_wrapper/fwbotnet.pdf>, where
> the network operators could use the graylists to generate alerts to the
> operator to act upon. A network operator can treat a third-party
> blacklist as a graylist and generate alerts. Is this common?
>
>
Another issue you are going to find is that SpamAssassin can be
installed in many different ways and in many different "positions" of
the mail flow. Some will have SA secondary to blacklists done by the
MTA (Postfix, Sendmail, Exim, etc.). When done at the MTA level, each
MTA can use blacklists differently. For example, I use postscreen and
weighted RBLs to combine the results of about 25 blacklists and
whitelists to get an aggregate score. Some people might only use 2 to 5
blacklists in their MTA that outright block if there is a single hit.
This is the traditional method that I used years ago but is way to risky
compared to postscreen weighting of blacklists.
There are not many default blacklists and whitelists in SA. A mail
administrator has to manually add many extras to get it up to being
useful IMHO. This requires careful analysis of your mail flow as each
SA instance has varying requirements and unique characteristics. There
is a basic commonality to blocking spam but it's not as common as you
might think until you read this list for several years and see all of
the differences.
My style of spam blocking is heavy on the reputation side which includes
blacklists and whitelists. I define safe senders based on certain
whitelists and valid opt-out processing of approved senders based on
SPF, DKIM and DMARC. I suspect many on this list would not agree with
this tactic but that's OK. Each mail flow is different. The bottom
line is if you don't get any complaints from your customers, you are
doing something right.
If you tune everything correctly (which takes a lot of time and effort),
then you basically have to bypass blacklists for the major providers
like Office 365, Gmail, Yahoo, AOL, etc. and rely primarily on content
filtering in SA to block the spam. A few blacklists like Spamhaus and
Invaluement have figured this out and don't list these large mail
services providers but there are still many that don't which causes
problems.
Filtering outbound mail has different challenges than inbound mail. You
have to have some form of compromised account detection based on unusual
activity which has nothing to do with blacklists or whitelists. Plus
you need to carefully filter outbound mail using properly configured
last-external rules for blacklists so your own customer IPs are excluded
from blacklists but further hops back are evaluated against blacklists.
>
> On Tue, Aug 15, 2017 at 12:24 PM, Dianne Skoll <dfs@roaringpenguin.com
> <ma...@roaringpenguin.com>> wrote:
>
> On Tue, 15 Aug 2017 12:02:23 -0500
> Shivram Krishnan <rorrykeys@gmail.com <ma...@gmail.com>>
> wrote:
>
> > Thanks for the response Bill. I have got a couple of responses from
> > this group, which agree with what you are saying - they have their
> > own custom techniques to prevent spam and reduce false positives. If
> > thats the case, who uses third-party generated blacklists?
>
> I think you'll find a lot of people use them. My instincts tell me the
> userbase falls into three sets of administrators:
>
> 1) Admins of large organizations that can afford reputable lists
> like Spamhaus,
> etc. and use them.
>
> 2) Admins of tiny mail servers who are highly aggressive and use
> blacklists like kids popping candy and who don't care overly-much
> about false positives.
>
> 3) Admins of small to medium organizations who use commercial
> anti-spam filters or commercial email hosts that make use of
> blacklists by default, and who probably don't really understand the
> ramifications of using blacklists.
>
> My $0.02: Blacklists can be useful, but I would never reject based
> solely
> on an IP being blacklisted. Also, I don't use third-party
> blacklists, though
> I do use a set of DNSBLs that my company controls.
>
> Regards,
>
> Dianne.
>
>
--
David Jones
Re: Operators Blacklist Survey
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Tue, 15 Aug 2017 12:46:59 -0500
Shivram Krishnan <ro...@gmail.com> wrote:
> Rule-based systems like spamassassin make room for false positives
> from any one of the rules. For instance , a blacklist can have a
> false positive, but there may be other rules which may not agree with
> the blacklist. An ensemble of such rules allows make spamassassin to
> be more accurate.
I didn't know any of that! Thanks! Wow! Consider my mind blown.
> In case of non-rule based systems like firewall, an inaccurate
> blacklist can prove costly when the firewall drops legitimate traffic
> based on inaccurate blacklists. I was reading about graylists on
> cisco firewalls...
That's not a standard use of the term "graylist"
> where the network operators could use the graylists to generate
> alerts to the operator to act upon. A network operator can treat a
> third-party blacklist as a graylist and generate alerts. Is this
> common?
Not if sysadmins want a life. If I did something like that on our
systems, I'd be getting multiple alerts per second.
Regards,
Dianne.
Re: Operators Blacklist Survey
Posted by Shivram Krishnan <ro...@gmail.com>.
Thanks for the response Dianne.
Rule-based systems like spamassassin make room for false positives from any
one of the rules. For instance , a blacklist can have a false positive, but
there may be other rules which may not agree with the blacklist. An
ensemble of such rules allows make spamassassin to be more accurate.
In case of non-rule based systems like firewall, an inaccurate blacklist
can prove costly when the firewall drops legitimate traffic based on
inaccurate blacklists. I was reading about graylists on cisco firewalls
<https://www.cisco.com/c/en/us/td/docs/security/security_management/cisco_security_manager/security_manager/4-1/user/guide/CSMUserGuide_wrapper/fwbotnet.pdf>,
where
the network operators could use the graylists to generate alerts to the
operator to act upon. A network operator can treat a third-party blacklist
as a graylist and generate alerts. Is this common?
On Tue, Aug 15, 2017 at 12:24 PM, Dianne Skoll <df...@roaringpenguin.com>
wrote:
> On Tue, 15 Aug 2017 12:02:23 -0500
> Shivram Krishnan <ro...@gmail.com> wrote:
>
> > Thanks for the response Bill. I have got a couple of responses from
> > this group, which agree with what you are saying - they have their
> > own custom techniques to prevent spam and reduce false positives. If
> > thats the case, who uses third-party generated blacklists?
>
> I think you'll find a lot of people use them. My instincts tell me the
> userbase falls into three sets of administrators:
>
> 1) Admins of large organizations that can afford reputable lists like
> Spamhaus,
> etc. and use them.
>
> 2) Admins of tiny mail servers who are highly aggressive and use
> blacklists like kids popping candy and who don't care overly-much
> about false positives.
>
> 3) Admins of small to medium organizations who use commercial
> anti-spam filters or commercial email hosts that make use of
> blacklists by default, and who probably don't really understand the
> ramifications of using blacklists.
>
> My $0.02: Blacklists can be useful, but I would never reject based solely
> on an IP being blacklisted. Also, I don't use third-party blacklists,
> though
> I do use a set of DNSBLs that my company controls.
>
> Regards,
>
> Dianne.
>
Re: Operators Blacklist Survey
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Tue, 15 Aug 2017 12:02:23 -0500
Shivram Krishnan <ro...@gmail.com> wrote:
> Thanks for the response Bill. I have got a couple of responses from
> this group, which agree with what you are saying - they have their
> own custom techniques to prevent spam and reduce false positives. If
> thats the case, who uses third-party generated blacklists?
I think you'll find a lot of people use them. My instincts tell me the
userbase falls into three sets of administrators:
1) Admins of large organizations that can afford reputable lists like Spamhaus,
etc. and use them.
2) Admins of tiny mail servers who are highly aggressive and use
blacklists like kids popping candy and who don't care overly-much
about false positives.
3) Admins of small to medium organizations who use commercial
anti-spam filters or commercial email hosts that make use of
blacklists by default, and who probably don't really understand the
ramifications of using blacklists.
My $0.02: Blacklists can be useful, but I would never reject based solely
on an IP being blacklisted. Also, I don't use third-party blacklists, though
I do use a set of DNSBLs that my company controls.
Regards,
Dianne.
Re: Operators Blacklist Survey
Posted by Rupert Gallagher <ru...@protonmail.com>.
Companies with an MBA director who happened to read the "blacklist" buzzword somewhere and think they will look good by using it themselves.
Sent from ProtonMail Mobile
On Tue, Aug 15, 2017 at 7:02 PM, Shivram Krishnan <ro...@gmail.com> wrote:
> Thanks for the response Bill. I have got a couple of responses from this group, which agree with what you are saying - they have their own custom techniques to prevent spam and reduce false positives. If thats the case, who uses third-party generated blacklists?
>
> On Mon, Aug 14, 2017 at 11:01 PM, Bill Cole <sa...@billmail.scconsult.com> wrote:
>
>> On 14 Aug 2017, at 18:00, Shivram Krishnan wrote:
>>
>>> Hi,
>>>
>>> I am a graduate student at the University of Southern California and am
>>> currently researching on the impact of false positives in blacklists.
>>
>> Apparently they don't bother with a mandatory Research Methodology course for grad students any more. That's disappointing.
>>
>>> I am
>>> aware that spamassassin uses blacklists in its rule based system to stop
>>> spam messages. But since it is a rule based system, even if there are false
>>> positives in blacklists, there may be other rules which can influence
>>> spamassassin to mark it correctly. There are several other blacklists which
>>> are used to stop different attacks (eg phishing, DDoS, malware hosting
>>> etc). I was wondering if operators in general use external
>>> blacklists(uribl, spamhaus, spamcop etc) in the form of rule based system
>>> (like spamassassin) or use it outrightly to block all IPs listed in them.
>>
>> Asking that question HERE assures that you will get a badly skewed sample.
>>
>> The majority of SA users do not read this list. The majority of email admins do not use SA. Many who do use DNSBLs don't understand that they do so, because the mail filtering is in a box they were told they never need to touch or is done externally by a filtering provider who won't tell customers what they use. A very large fraction of legitimate mail, possibly a majority, flows between and within a few large providers who do not use SA, may or may not cooperate with and/or use publicly available DNSBLs, and will never admit to using anything other than their own tools for spam filtering.
>>
>>> It will be great if you can take this four question survey, which can help
>>> me understand the usage of blacklists by operators.
>>
>> Unfortunately my current answers would be very unusual, because I recently lost the job where I actively managed mail systems for pay, and the micro-systems I manage for myself and friends who ask for help are tiny and ridiculously unrepresentative.
>>
>> But no matter, I'll act like I still have that job or the one before it or any of the others I've had managing mail systems in the age of DNSBLs.
>>
>>> The survey consists of
>>> these questions -
>>> 1) The size of the network(s) you manage(in terms of customers)
>>
>> That is confidential and proprietary business information which I am not authorized to share.
>>
>>> 2) List of external blacklists used.
>>
>> That is confidential and proprietary business information which I am not authorized to share.
>>
>>> 3) How these blacklists are used? whether in a rule based system or
>>> outrightly blocked or both
>>
>> That is confidential and proprietary business information which I am not authorized to share.
>>
>>> 4) If external blacklists are used in a non-rule based system, how do you
>>> overcome false positives?
>>
>> That is confidential and proprietary business information which I am not authorized to share.
>>
>> I expect that a large percentage of professional email admins would answer identically. I would not recommend trusting any who answered substantively.
>>
>> I would also recommend against sharing this message with your faculty advisor. Some questions cannot be answered accurately or meaningfully by taking surveys of those willing to answer. Spam control is an operational security facility. People doing it who understand their jobs will not discuss the details.
Re: Operators Blacklist Survey
Posted by Shivram Krishnan <ro...@gmail.com>.
Thanks for the response Bill. I have got a couple of responses from this
group, which agree with what you are saying - they have their own custom
techniques to prevent spam and reduce false positives. If thats the case,
who uses third-party generated blacklists?
On Mon, Aug 14, 2017 at 11:01 PM, Bill Cole <
sausers-20150205@billmail.scconsult.com> wrote:
> On 14 Aug 2017, at 18:00, Shivram Krishnan wrote:
>
> Hi,
>>
>>
>> I am a graduate student at the University of Southern California and am
>> currently researching on the impact of false positives in blacklists.
>>
>
> Apparently they don't bother with a mandatory Research Methodology course
> for grad students any more. That's disappointing.
>
> I am
>> aware that spamassassin uses blacklists in its rule based system to stop
>> spam messages. But since it is a rule based system, even if there are
>> false
>> positives in blacklists, there may be other rules which can influence
>> spamassassin to mark it correctly. There are several other blacklists
>> which
>> are used to stop different attacks (eg phishing, DDoS, malware hosting
>> etc). I was wondering if operators in general use external
>> blacklists(uribl, spamhaus, spamcop etc) in the form of rule based system
>> (like spamassassin) or use it outrightly to block all IPs listed in them.
>>
>
> Asking that question HERE assures that you will get a badly skewed sample.
>
> The majority of SA users do not read this list. The majority of email
> admins do not use SA. Many who do use DNSBLs don't understand that they do
> so, because the mail filtering is in a box they were told they never need
> to touch or is done externally by a filtering provider who won't tell
> customers what they use. A very large fraction of legitimate mail, possibly
> a majority, flows between and within a few large providers who do not use
> SA, may or may not cooperate with and/or use publicly available DNSBLs, and
> will never admit to using anything other than their own tools for spam
> filtering.
>
> It will be great if you can take this four question survey, which can help
>> me understand the usage of blacklists by operators.
>>
>
> Unfortunately my current answers would be very unusual, because I recently
> lost the job where I actively managed mail systems for pay, and the
> micro-systems I manage for myself and friends who ask for help are tiny and
> ridiculously unrepresentative.
>
> But no matter, I'll act like I still have that job or the one before it or
> any of the others I've had managing mail systems in the age of DNSBLs.
>
> The survey consists of
>> these questions -
>> 1) The size of the network(s) you manage(in terms of customers)
>>
>
> That is confidential and proprietary business information which I am not
> authorized to share.
>
> 2) List of external blacklists used.
>>
>
> That is confidential and proprietary business information which I am not
> authorized to share.
>
> 3) How these blacklists are used? whether in a rule based system or
>> outrightly blocked or both
>>
>
> That is confidential and proprietary business information which I am not
> authorized to share.
>
> 4) If external blacklists are used in a non-rule based system, how do you
>> overcome false positives?
>>
>
> That is confidential and proprietary business information which I am not
> authorized to share.
>
> I expect that a large percentage of professional email admins would answer
> identically. I would not recommend trusting any who answered substantively.
>
> I would also recommend against sharing this message with your faculty
> advisor. Some questions cannot be answered accurately or meaningfully by
> taking surveys of those willing to answer. Spam control is an operational
> security facility. People doing it who understand their jobs will not
> discuss the details.
>
>
>
Re: Operators Blacklist Survey
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 14 Aug 2017, at 18:00, Shivram Krishnan wrote:
> Hi,
>
>
> I am a graduate student at the University of Southern California and
> am
> currently researching on the impact of false positives in blacklists.
Apparently they don't bother with a mandatory Research Methodology
course for grad students any more. That's disappointing.
> I am
> aware that spamassassin uses blacklists in its rule based system to
> stop
> spam messages. But since it is a rule based system, even if there are
> false
> positives in blacklists, there may be other rules which can influence
> spamassassin to mark it correctly. There are several other blacklists
> which
> are used to stop different attacks (eg phishing, DDoS, malware hosting
> etc). I was wondering if operators in general use external
> blacklists(uribl, spamhaus, spamcop etc) in the form of rule based
> system
> (like spamassassin) or use it outrightly to block all IPs listed in
> them.
Asking that question HERE assures that you will get a badly skewed
sample.
The majority of SA users do not read this list. The majority of email
admins do not use SA. Many who do use DNSBLs don't understand that they
do so, because the mail filtering is in a box they were told they never
need to touch or is done externally by a filtering provider who won't
tell customers what they use. A very large fraction of legitimate mail,
possibly a majority, flows between and within a few large providers who
do not use SA, may or may not cooperate with and/or use publicly
available DNSBLs, and will never admit to using anything other than
their own tools for spam filtering.
> It will be great if you can take this four question survey, which can
> help
> me understand the usage of blacklists by operators.
Unfortunately my current answers would be very unusual, because I
recently lost the job where I actively managed mail systems for pay, and
the micro-systems I manage for myself and friends who ask for help are
tiny and ridiculously unrepresentative.
But no matter, I'll act like I still have that job or the one before it
or any of the others I've had managing mail systems in the age of
DNSBLs.
> The survey consists of
> these questions -
> 1) The size of the network(s) you manage(in terms of customers)
That is confidential and proprietary business information which I am not
authorized to share.
> 2) List of external blacklists used.
That is confidential and proprietary business information which I am not
authorized to share.
> 3) How these blacklists are used? whether in a rule based system or
> outrightly blocked or both
That is confidential and proprietary business information which I am not
authorized to share.
> 4) If external blacklists are used in a non-rule based system, how do
> you
> overcome false positives?
That is confidential and proprietary business information which I am not
authorized to share.
I expect that a large percentage of professional email admins would
answer identically. I would not recommend trusting any who answered
substantively.
I would also recommend against sharing this message with your faculty
advisor. Some questions cannot be answered accurately or meaningfully by
taking surveys of those willing to answer. Spam control is an
operational security facility. People doing it who understand their jobs
will not discuss the details.