You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2010/03/26 16:42:29 UTC
[Bug 6390] New: KHOP_SC_TOP200 scores don't take into account scbl
score.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
Summary: KHOP_SC_TOP200 scores don't take into account scbl
score.
Product: Spamassassin
Version: 3.3.1
Platform: Other
OS/Version: All
Status: NEW
Severity: minor
Priority: P5
Component: Rules
AssignedTo: dev@spamassassin.apache.org
ReportedBy: scheidell@secnap.net
72_scores.cf:score KHOP_SC_CIDR16 2.200 1.291 2.200
1.291
72_scores.cf:score KHOP_SC_CIDR24 2.649 1.198 2.649
1.198
72_scores.cf:score KHOP_SC_TOP200 4.000 3.944 4.000
3.944
72_scores.cf:score KHOP_SC_TOP_CIDR16 2.797 0.728 2.797
0.728
72_scores.cf:score KHOP_SC_TOP_CIDR24 0.001 0.000 0.001
0.000
72_scores.cf:score KHOP_SC_TOP_CIDR8 2.200 0.001 2.200
0.001
it looks like all the other SC rules (scores) take into account 'net', so that
if you have the ip already in spamcop (you have to be in spam cop to be in sc
anyway), the other rules seem to take that into account.
however, if (again, if you are in top200, you ARE in spamcop), so with rbls's
enabled, and net enabled, anything that is in sc_200 will score 5.3 points:
4 points for being in sc_top200, 1.3 points for being in spamcop itself.
this will put it over the top for (99%?) of all SA installations.
suggest manual tweaks on score for 'net' scores, lower by the spamcop bl score.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6390] KHOP_SC_TOP200 scores don't take into account scbl score.
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
Adam Katz <an...@khopis.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |antispam@khopis.com
--- Comment #4 from Adam Katz <an...@khopis.com> 2010-03-29 19:46:29 UTC ---
(In reply to comment #2)
> This is pretty much a duplicate of bug 6114. Still open, mind you, so it
> almost feels like these rules shouldn't have been published yet.
I didn't know those were out of my sandbox. ALL khop-sc rules are marked
nopublish.
(In reply to comment #3)
> Oh, and the last update of 20_khop_sc_bug_6114.cf with respect to this issue
> appears to be more than 2 months old! Adam?
Well crap. Until last night, I was stalled on figuring out how to automate
checkins from my channel to svn. I gave up on security and stored my password
in cleartext on my system, relying upon the system's physical security instead.
khop-sc-neighbors is now synced daily nightly with svn (the channel itself
updates every four hours).
(In reply to comment #0)
> 72_scores.cf:score KHOP_SC_CIDR16 2.200 1.291 2.200 1.291
> 72_scores.cf:score KHOP_SC_CIDR24 2.649 1.198 2.649 1.198
> 72_scores.cf:score KHOP_SC_TOP200 4.000 3.944 4.000 3.944
> 72_scores.cf:score KHOP_SC_TOP_CIDR16 2.797 0.728 2.797 0.728
> 72_scores.cf:score KHOP_SC_TOP_CIDR24 0.001 0.000 0.001 0.000
> 72_scores.cf:score KHOP_SC_TOP_CIDR8 2.200 0.001 2.200 0.001
First, this was not published in 3.3.0 or 3.3.1 and has not migrated anywhere
on the 3.3 branch, so it will miss 3.3.2 as well. This affects only the trunk,
which is where the above quote originates.
Second, those scores are out of control. The GA is supposed to notice the
overlap with RCVD_IN_BL_SPAMCOP_NET, which KHOP_SC_TOP200 overlaps completely.
The fact that it compensated for anything BUT the top200 is laughable, and the
score on top-cidr24 is baffling (if scored that low, why was it promoted in the
first place?).
Scoring notes to come tomorrow, I've got to run for now.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6390] KHOP_SC_TOP200 scores don't take into account scbl score.
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
Adam Katz <an...@khopis.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
--- Comment #7 from Adam Katz <an...@khopis.com> 2011-03-04 15:29:42 EST ---
(In reply to comment #6)
> Hey, I just thought of a way to handle this. Would it be possible to disable
> these scores entirely if network rules are enabled, and only enable if network
> is disabled? That would eliminate the overlap and make these rules useful.
KHOP_SC_TOP200 and KHOP_SPAMHAUS_DROP* use this to zero their score on
net+DNSEval runs. I believe the overlap from the other rules is beneficial.
As noted in comment #5, this overlap is intentional and is scored very low on
net runs. When run without net support, it is scored higher. When using the
net rules but DNSEval is disabled, it is also scored higher.
I have adjusted the channel's scores given the current performance from
http://ruleqa.spamassassin.org/20110226-r1074804-n?srcpath=khop_sc_. I also
lessened the weight of some of the rules. Spamhaus DROP actually earned more
weight due to its surprisingly low average spam scores (it had been zeroed).
score KHOP_SC_CIDR8 0.3 0.1 0.3 0.1
score KHOP_SC_TOP_CIDR8 0.2 0 0.2 0
score KHOP_SC_CIDR16 1.0 0.2 1.0 0.2
score KHOP_SC_TOP_CIDR16 2.0 0.3 2.0 0.4
score KHOP_SC_CIDR24 0.1 0 0.1 0
score KHOP_SC_TOP_CIDR24 2.7 0.5 2.7 0.5
score KHOP_SC_TOP200 4 0 4 0 # unnecessary if DNSBLs work
score KHOP_SPAMHAUS_DROP 1 0.2 1 0.2
score KHOP_SPAMHAUS_DROP_LE 2 0 2 0 # adds to KHOP_SPAMHAUS_DROP
score KHOP_PSBL_CIDR24 2 0.6 2 0.6
if (! plugin(Mail::SpamAssassin::Plugin::DNSEval) )
score KHOP_SC_CIDR8 (0) (0.2) (0) (0.2) # BRBL(98%)
score KHOP_SC_TOP_CIDR8 (0) (0.9) (0) (0.9) # BRBL(98%)
score KHOP_SC_CIDR16 (0) (1.5) (0) (1.5) # BRBL(99%), PBL(98%)
score KHOP_SC_TOP_CIDR16 (0) (1.7) (0) (1.7) # BRBL(99%), PBL(94%)
score KHOP_SC_CIDR24 (0) (0.9) (0) (0.9) # SC(99) BRBL(96) MSPIKE(96)
score KHOP_SC_TOP_CIDR24 (0) (2.5) (0) (2.5) # MSPIKE(99) SC(98) BRBL(97)
...
score KHOP_SC_TOP200 (0) (4.4) (0) (4.4) # SC(99) PSBL(99) ...
score KHOP_SPAMHAUS_DROP (0) (3) (0) (3) # SBL(93)
score KHOP_PSBL_CIDR24 (0) (1.5) (0) (1.5) # BRBL(98) XBL(95)
endif
I believe my response to this bug early last year was sufficient to close it.
These rules exist in SVN with regular updates purely for vetting purposes since
SpamAssassin currently lacks a mechanism to automatically publish timely
updates. Anybody looking to use these rules should subscribe to my sa-update
channel, which will pull daily+ updates to these CIDR regexps.
Any remaining issue with these rules accidentally getting published is a bug
related to the "nopublish" tflag rather than with this rule set.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6390] KHOP_SC_TOP200 scores don't take into account scbl score.
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
AXB <al...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |alex.uribl@gmail.com
--- Comment #1 from AXB <al...@gmail.com> 2010-03-26 15:48:48 UTC ---
(In reply to comment #0)
> 72_scores.cf:score KHOP_SC_CIDR16 2.200 1.291 2.200
> 1.291
> 72_scores.cf:score KHOP_SC_CIDR24 2.649 1.198 2.649
> 1.198
> 72_scores.cf:score KHOP_SC_TOP200 4.000 3.944 4.000
> 3.944
> 72_scores.cf:score KHOP_SC_TOP_CIDR16 2.797 0.728 2.797
> 0.728
> 72_scores.cf:score KHOP_SC_TOP_CIDR24 0.001 0.000 0.001
> 0.000
> 72_scores.cf:score KHOP_SC_TOP_CIDR8 2.200 0.001 2.200
> 0.001
>
> it looks like all the other SC rules (scores) take into account 'net', so that
> if you have the ip already in spamcop (you have to be in spam cop to be in sc
> anyway), the other rules seem to take that into account.
>
> however, if (again, if you are in top200, you ARE in spamcop), so with rbls's
> enabled, and net enabled, anything that is in sc_200 will score 5.3 points:
> 4 points for being in sc_top200, 1.3 points for being in spamcop itself.
>
> this will put it over the top for (99%?) of all SA installations.
>
> suggest manual tweaks on score for 'net' scores, lower by the spamcop bl score.
IMO, depending on the kind of traffic you have, its safest to disable these
rules completely. (worked for me)
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6390] KHOP_SC_TOP200 scores don't take into account scbl score.
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
--- Comment #5 from Adam Katz <an...@khopis.com> 2010-03-30 22:44:27 UTC ---
Here is how the rules are currently scored on my khop-sc-neighbors channel.
This ensures that the rules do next to nothing when run with net checks, but
come back up (and then some) for net checks that lack DNSEval support (which is
to say, the DNSBLs).
score KHOP_SC_CIDR8 0.1 0.01 0.1 0.01
score KHOP_SC_TOP_CIDR8 0.9 0.1 0.9 0.1
score KHOP_SC_CIDR16 1.6 0.5 1.6 0.5
score KHOP_SC_TOP_CIDR16 2.0 0.5 2.0 0.5
score KHOP_SC_CIDR24 2.5 0.6 2.5 0.6
score KHOP_SC_TOP_CIDR24 2.7 0.6 2.7 0.6
score KHOP_SC_TOP200 4 0 4 0 # unnecessary if DNSBLs work
# Bump these up to compensate for expected but absent overlap (94+% noted)
if (! plugin(Mail::SpamAssassin::Plugin::DNSEval) )
score KHOP_SC_CIDR8 (0) (0.1) (0) (0.2) # BRBL(98%)
score KHOP_SC_TOP_CIDR8 (0) (0.9) (0) (0.9) # BRBL(98%)
score KHOP_SC_CIDR16 (0) (1.5) (0) (1.5) # BRBL(99%), PBL(98%)
score KHOP_SC_TOP_CIDR16 (0) (1.7) (0) (1.7) # BRBL(99%), PBL(94%)
score KHOP_SC_CIDR24 (0) (1.9) (0) (1.9) # SC(99) BRBL(96) ANBREP(96)
score KHOP_SC_TOP_CIDR24 (0) (2.1) (0) (2.1) # MSPIKE(99) SC(98) ...
# BRBL(97) PSBL(97) HOSTKARMA(97)
score KHOP_SC_TOP200 (0) (4.4) (0) (4.4) # SC(99) PSBL(99) ...
# HOSTKARMA(99) SEMBLACK(99) BRBL(98) ANBREP(94)
endif
It's nice to see what the GA has to say about these rules, but I would never
take the resulting scores if DNSBLs are accessible.
At the moment, BRBL=1.644/1.449, PSBL=2.7, SC=SPAMCOP=1.246/1.347, and
PBL=3.558/3.335. This overlap puts almost all (>96%) of the KHOP_SC_TOP200
hits at 5.590/5.349 *before* the 3.944 is added by the GA's score for this
rule, and 9.534/9.440 total. Though it does have a 1.000 S/O and these /are/
the worst offenders, I think this rule is best left absent from DNSEval-enabled
net checks.
A CIDR8 is often out of the relevant network administrator's control. For this
reason, we CANNOT score CIDR8 rules high. I've got the lesser at near-nothing
and the top at 0.9. CIDR8s aside, my scoring is wholly more conservative than
the GA.
(In reply to comment #2)
> This is pretty much a duplicate of bug 6114. Still open, mind you, so it
> almost feels like these rules shouldn't have been published yet.
I just closed that bug since it was mostly about getting the rules into a
sandbox for testing and I now have access to do that and a script that syncs
nightly.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6390] KHOP_SC_TOP200 scores don't take into account scbl score.
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
--- Comment #6 from Warren Togami <wt...@gmail.com> 2010-12-26 02:18:02 UTC ---
Hey, I just thought of a way to handle this. Would it be possible to disable
these scores entirely if network rules are enabled, and only enable if network
is disabled? That would eliminate the overlap and make these rules useful.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6390] KHOP_SC_TOP200 scores don't take into account scbl score.
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
--- Comment #2 from Karsten Bräckelmann <gu...@rudersport.de> 2010-03-26 15:59:54 UTC ---
This is pretty much a duplicate of bug 6114. Still open, mind you, so it almost
feels like these rules shouldn't have been published yet.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6390] KHOP_SC_TOP200 scores don't take into account scbl score.
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6390
--- Comment #3 from Karsten Bräckelmann <gu...@rudersport.de> 2010-03-26 16:03:27 UTC ---
Oh, and the last update of 20_khop_sc_bug_6114.cf with respect to this issue
appears to be more than 2 months old! Adam?
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.