You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Lars Jørgensen <la...@kb.dk> on 2011/07/04 16:28:42 UTC
Lowering spam threshold
Hi,
We still get quite a bit of spam through and instead of fiddling with scores, I was thinking about lowering the threshold. Currently tag is at 6.2 and kill at 6.9. Would it be unwise to lower these? What thresholds are other people on this list using?
--
Lars
Re: Lowering spam threshold [avoid discarding at high cost]
Posted by John Hardin <jh...@impsec.org>.
On Fri, 8 Jul 2011, Andrzej Adam Filip wrote:
> John Hardin <jh...@impsec.org> wrote:
>> On Fri, 8 Jul 2011, Lars Jørgensen wrote:
>>
>>>>> $sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level
>>>>> $sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail)
>>>
>>>> That seems a little aggressive to me. Personally I'd prefer a larger
>>>> margin of error for FPs, and would set the discard level to 9 or 10
>>>> (unless the "evasive actions" include "quarantine for review").
>>>
>>> "evasive actions" do indeed include quarantine. No-quarantine-cutoff is set at 20, which may be a bit high, but we got room for it.
>>
>> So, tag at 5.2, quarantine at 6.2, discard at 20? That sounds
>> reasonable to me, assuming the quarantine is readily accessible for
>> review.
>
> If you want to treat email as *RELIABLE* delivery service then
> avoid discarding at high cost - reject in SMTP session to make
> *sending host* responsible for sending bounce message.
> [ It can be done using milters with both sendmail and postfix ]
Granted, and agreed. I was using "discard" generically here.
> I do remember situation in which receiving MTA simply discarded
> important message from one of my users and it took a few days for
> sender *and recipient* to find out that message has been silently
> discarded:
> *sender assumed that recipient reads it in silence,
> * recipient assumed in silence that those [...] longer have not sent it yet
>
> I can treat it as funny *today* but it was not funny.
Nope. Especially when they're CEOs.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The difference between ignorance and stupidity is that the stupid
desire to remain ignorant. -- Jim Bacon
-----------------------------------------------------------------------
12 days until the 42nd anniversary of Apollo 11 landing on the Moon
Re: Lowering spam threshold [avoid discarding at high cost]
Posted by Andrzej Adam Filip <an...@gmail.com>.
John Hardin <jh...@impsec.org> wrote:
> On Fri, 8 Jul 2011, Lars Jørgensen wrote:
>
>>>> $sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level
>>>> $sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail)
>>
>>> That seems a little aggressive to me. Personally I'd prefer a larger
>>> margin of error for FPs, and would set the discard level to 9 or 10
>>> (unless the "evasive actions" include "quarantine for review").
>>
>> "evasive actions" do indeed include quarantine. No-quarantine-cutoff is set at 20, which may be a bit high, but we got room for it.
>
> So, tag at 5.2, quarantine at 6.2, discard at 20? That sounds
> reasonable to me, assuming the quarantine is readily accessible for
> review.
If you want to treat email as *RELIABLE* delivery service then
avoid discarding at high cost - reject in SMTP session to make
*sending host* responsible for sending bounce message.
[ It can be done using milters with both sendmail and postfix ]
I do remember situation in which receiving MTA simply discarded
important message from one of my users and it took a few days for
sender *and recipient* to find out that message has been silently
discarded:
*sender assumed that recipient reads it in silence,
* recipient assumed in silence that those [...] longer have not sent it yet
I can treat it as funny *today* but it was not funny.
--
[pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu
The power to destroy a planet is insignificant when compared to the
power of the Force.
-- Darth Vader
RE: Lowering spam threshold
Posted by John Hardin <jh...@impsec.org>.
On Fri, 8 Jul 2011, Lars Jørgensen wrote:
>>> $sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level
>>> $sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail)
>
>> That seems a little aggressive to me. Personally I'd prefer a larger
>> margin of error for FPs, and would set the discard level to 9 or 10
>> (unless the "evasive actions" include "quarantine for review").
>
> "evasive actions" do indeed include quarantine. No-quarantine-cutoff is set at 20, which may be a bit high, but we got room for it.
So, tag at 5.2, quarantine at 6.2, discard at 20? That sounds reasonable
to me, assuming the quarantine is readily accessible for review.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The United States has become a place where entertainers and
professional athletes are mistaken for people of importance.
-- Maureen Johnson Smith Long
-----------------------------------------------------------------------
12 days until the 42nd anniversary of Apollo 11 landing on the Moon
RE: Lowering spam threshold
Posted by Lars Jørgensen <la...@kb.dk>.
> > $sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level
> > $sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail)
> That seems a little aggressive to me. Personally I'd prefer a larger
> margin of error for FPs, and would set the discard level to 9 or 10
> (unless the "evasive actions" include "quarantine for review").
"evasive actions" do indeed include quarantine. No-quarantine-cutoff is set at 20, which may be a bit high, but we got room for it.
Lars
RE: Lowering spam threshold
Posted by John Hardin <jh...@impsec.org>.
On Wed, 6 Jul 2011, Lars Jørgensen wrote:
> $sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level
> $sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail)
That seems a little aggressive to me. Personally I'd prefer a larger
margin of error for FPs, and would set the discard level to 9 or 10
(unless the "evasive actions" include "quarantine for review").
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
[People] are socialists because they are blinded by envy and
ignorance. -- economist Ludwig von Mises (1881-1973)
-----------------------------------------------------------------------
Tomorrow: Robert Heinlein's 104th birthday
Re: Lowering spam threshold
Posted by Ned Slider <ne...@unixmail.co.uk>.
On 06/07/11 09:17, Lars Jørgensen wrote:
>> I think many people run with tag at 5.0 and discard at 10.0
>
> I should have mentioned that we are running amavisd-new. I thought that was the de facto way of integrating spamassassin into a mail gateway, but reading this list reveals that most people probably doesn't do that. Makes me wonder if I am doing the wrong thing?
>
> Amavisd-new has further settings as to thresholds, and these are the ones I put in as of today (after reading other peoples tips here, thank you everybody):
>
> $sa_tag_level_deflt = -10; # add spam info headers if at, or above that level
> $sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level
> $sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail)
> $sa_dsn_cutoff_level = 7.4; # spam level beyond which a DSN is not sent
>
> Does above scores make sense?
>
>
Yes, makes perfect sense to other amavisd-new users. I currently tag at
5.0 (the default SA score) and quarantine at 6.0. I also set the DSN
cut-off level to be the same as quarantine as I don't want to send DSNs.
If you are finding spam is getting through untagged with the default SA
score of 5.0 then I would look to write some additional rules to target
those spam that are getting through rather than lowering the score below
the SA default of 5.0. This list can help you with that if you provide
examples.
Additionally, I have very carefully hand trained bayes with only
confirmed spam/ham and tweaked the scores to be more representative of
the faith I have in my bayes data. I find many cases where bayes alone
will identify spam and have scored bayes_99 accordingly.
The main "problem" I see with SA is that I reject all the easy spam
(>90%) at the smtp level so SA only really gets to see the more
difficult and less obvious stuff. If SA saw all spam then the detection
rates out of the box would be extremely high, but with only the more
difficult samples to chew on detection rates inevitably drop and are
artificially lowered. As a result it can appear that a lot of spam is
getting through when in reality the overall percentage is still really
small. That last 1% is just hard to catch without increasing the risk of
false positives.
Re: Lowering spam threshold
Posted by Michael Scheidell <mi...@secnap.com>.
On 7/6/11 4:17 AM, Lars Jørgensen wrote:
>> I think many people run with tag at 5.0 and discard at 10.0
> I should have mentioned that we are running amavisd-new. I thought that was the de facto way of integrating spamassassin into a mail gateway, but reading this list reveals that most people probably doesn't do that. Makes me wonder if I am doing the wrong thing?
>
> Amavisd-new has further settings as to thresholds, and these are the ones I put in as of today (after reading other peoples tips here, thank you everybody):
>
join the amavisd-new list. you will get direct answers to your
questions from a very active, knoledgable group.
(and, NO, don't lower your spam threshold.. SA rules are scored to
assume a default of 5.0 to mark spam. )
if you are getting too much spam, then NORMAL SA assistance in SA group
is your best bet. if amavisd issues including what those additional
settings do, then the amavis group
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
RE: Lowering spam threshold
Posted by Lars Jørgensen <la...@kb.dk>.
> I think many people run with tag at 5.0 and discard at 10.0
I should have mentioned that we are running amavisd-new. I thought that was the de facto way of integrating spamassassin into a mail gateway, but reading this list reveals that most people probably doesn't do that. Makes me wonder if I am doing the wrong thing?
Amavisd-new has further settings as to thresholds, and these are the ones I put in as of today (after reading other peoples tips here, thank you everybody):
$sa_tag_level_deflt = -10; # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail)
$sa_dsn_cutoff_level = 7.4; # spam level beyond which a DSN is not sent
Does above scores make sense?
--
Lars
Re: Lowering spam threshold
Posted by Anthony Cartmell <li...@fonant.com>.
> The default spam threshold, and the one that all of the generated scores
> are targeted at, is 5.0 - you already seem to be running at an elevated
> score, so I wouldn't see any issues with dropping your tag score back to
> the default of 5.0
>
> I think many people run with tag at 5.0 and discard at 10.0
I tag at 4.0 and quarantine for 30 days at 8.0 and above, using
MailScanner. Works well with my rules and typical mail, but every
installation will be slightly different.
Anthony
--
www.fonant.com - Quality web sites
Re: Lowering spam threshold
Posted by John Hardin <jh...@impsec.org>.
On Mon, 4 Jul 2011, Lars Jørgensen wrote:
> We still get quite a bit of spam through and instead of fiddling with
> scores, I was thinking about lowering the threshold. Currently tag is at
> 6.2 and kill at 6.9. Would it be unwise to lower these? What thresholds
> are other people on this list using?
The default spam threshold, and the one that all of the generated scores
are targeted at, is 5.0 - you already seem to be running at an elevated
score, so I wouldn't see any issues with dropping your tag score back to
the default of 5.0
I think many people run with tag at 5.0 and discard at 10.0
I'd suggest that a 0.7-point spread between tag and discard is a little
too aggressive.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Warning Labels we'd like to see #1: "If you are a stupid idiot while
using this product you may hurt yourself. And it won't be our fault."
-----------------------------------------------------------------------
Today: the 235th anniversary of the Declaration of Independence
Re: Lowering spam threshold
Posted by a....@ukgrid.net.
Currently I have it at 4.8
Quoting Lars Jørgensen <la...@kb.dk>:
> Hi,
>
> We still get quite a bit of spam through and instead of fiddling
> with scores, I was thinking about lowering the threshold. Currently
> tag is at 6.2 and kill at 6.9. Would it be unwise to lower these?
> What thresholds are other people on this list using?
>
>
> --
> Lars
>