Re[2]: it's getting worse again

[Robert Menschel <Ro...@Menschel.net>]

Hello Florin,

Wednesday, April 6, 2005, 5:40:10 PM, you wrote:

FA> So what is the reason why BAYES_99 is scored so low?

The algorithm/process that determines scores came out with a low score
like that.  It seemed a good bet for this new version.  Many of us
have decided that it wasn't, and we've increased the score for it in
our systems.

One of the strengths of SA is its flexibility -- if you want to be
more aggressive, raise the score(s) or lower the threshold. If you
want to be more conservative, lower the score(s) or raise the
threshold. SA's scores are aimed at a fairly conservative target,
since false positives are horrendously worse than false negatives.

>> I'm a little puzzled what you're asking for, then;  addon rulesets are
>> available from SARE, and somewhere there's a tool to automatically check
>> for updates on those rules.

FA> My impression, after a quick perusal, was that any mentions about SARE
FA> and the like are pretty well "hidden" on the SA main website.
FA> Yes, there is a mention, but there's a big fat "Use at your own risk"
FA> warning at the top of the page. What would a new user think?

A new user should think, and think twice, before using SARE rules.
A new user that doesn't read this list probably should think three or
four times before using SARE rules, and should do so slowly and
carefully, if at all, and only after reading the documentation within
those rules files.

A user who has read this list and sees how many people use SARE rules,
should also be capable of looking at the documentation within those
rules and deciding which ones might be worth trying. (And should
probably do so slowly and carefully anyway.)

>> If you're really not interested in tweaking your SA setup

FA> I've a fairly demanding job, i've a few pretty convoluted personal
FA> projects i'm involved in, i've a family and other details that typically
FA> show up if one is not an archetypal pale-faced geek-in-the-basement.
FA> I do try to take care of my personal webserver (to which i'm the sole
FA> admin), mailserver (SA, Postfix, Cyrus, Squirrelmail), VoIP PBX, etc.,
FA> despite the schedule overload.

FA> And these days i was looking at SA and i'm, like, "it's not gonna
FA> happen, i don't have time for this." I chose to play the dumb user on
FA> purpose, just because i can't fix everything myself.

I hold down a 50-60 hour work week, family, volunteer time for NPOs,
plus personal interests, and still find time to fight spam via SARE,
because it's that important to me. Personal preference.

If you don't want to spend the time required to tweak SA to a high
enough performance (you probably don't need the 99.9% accuracy I
want), the you can buy someone else's package and let them worry about
the tweaking.

There's a balance point -- some time invested vs some gain received.
The question becomes whether the gain received does balance the time
invested, and only you can answer that question.

Bob Menschel

Re: Re[2]: it's getting worse again

[Florin Andrei <fl...@andrei.myip.org>]

On Thu, 2005-04-07 at 11:24 -0400, Kevin Sullivan wrote:

> If you think about this, it isn't surprising.  At the time the mass-checks 
> ran for 3.0.2, the 3.0.2 rules caught almost all of the spam of that time. 
> (If they didn't, then people wrote rules until they did.)  So dynamic 
> systems like Bayes and SURBL didn't add much. and thus scored low.  This 
> will be the case during every release.
> But now, between releases, spammers write spam which evades the "standard" 
> rules.  Sure, there will be new "standard" rules for the next SA release, 
> but until then dynamic systems like Bayes and SURBL are all that are 
> catching some spam.

This looks like the smoking gun.

Wouldn't make sense, then, to pre-emptively bump up the scores for the
dynamic systems right from the beginning? They will be slightly over-
rated at first, become perfectly rated after a while, then start getting
under-rated as more time passes.
That will certainly be better than the current situation, when the
dynamic rules are perfectly rated only at first, but immediately they
start to "decay" (well, not really, but i'm trying to find a concise
metaphor).

Or how about this:
In the big SA config file, add a parameter that controls the overall
weight of the static versus the dynamic things. Release the next SA
version with that parameter set so that it gives more importance to the
static systems. Tell users (in a prominent, obvious, even intrusive
fashion!) to adjust that parameter every now and then, to give more
importance to the dynamic systems as time passes.
Or even, heck, make SA track time and automatically increase the
importance of the dynamic systems as time passes. That will make it, of
course, one of those scary self-adaptive systems that pull the carpet
from under sysadmin's feet :-) but if it stops the spam, then who cares.

I believe you're right, that's what's causing me problems - the spammers
started to learn the static rules and are evading them. Well, if that's
true, then SA must provide a mechanism to control that. An overall
static-vs-dynamic "balance button" might be a good idea. Or not. <shrug>

I will try to bump up the Bayes rules and see where that goes.

Thanks everyone.

-- 
Florin Andrei

http://florin.myip.org/