New rule type suggestion

[Alexander Litvinov <la...@ac-sw.com>]

Is it possible to write automatic program/script that will use mass check 
results and write meta rules that score is some other rules triggered, like 
multiple MUA, multiple RBLs and other written by hands.

It is possible that spam triggers few low-score rules but combination of riles 
is very spam-specific. Is it possible to write script for determinating such 
groups of rules ? If yes, it seems to me good idea to write meta rules for 
this.

Re: New rule type suggestion

[Mike Laiosa <mi...@void.recurse.net>]

I've been thinking about this for quite a while.  The hypothesis is 
that the SA rules are not linearly independent.

SA essentially implements a simple neural network.  From that point of 
view, adding one (or more) hidden layers would probably improve its 
effectiveness.  The problem with simply adding hidden layers is that it 
makes it unclear how individual rules affect the final score, and the 
ability to tweak rule weights is important to a lot of us.  I figure 
something like this would work best if it showed up in SA like the 
bayes stuff does: an alternate classifier whose score is used as an SA 
rule.  Of course since the input to the multi-layer perceptron would be 
which SA rules matched, there may be strange side effects when mixing 
that result in with the rules in SA's own linear combination, but 
experimentation is needed to find out for sure.

I was hoping to do some experimentation with that idea last summer, but 
I got distracted with one thing, and then another, until my brain had a 
stack overflow.  Maybe I'll get around to it next summer, if someone 
doesn't beat me to it.

Alexander's idea of automatically finding the linear dependancies and 
representing them with meta-rules may prove to be a less invasive 
solution to the problem.  Neural network techniques (specifically 
methods of training multi-layer perceptrons) may be a way to generate 
the groups of rules Alexander is suggesting.

Mike


On Wednesday, January 7, 2004, at 09:05  PM, Alexander Litvinov wrote:

> Is it possible to write automatic program/script that will use mass 
> check
> results and write meta rules that score is some other rules triggered, 
> like
> multiple MUA, multiple RBLs and other written by hands.
>
> It is possible that spam triggers few low-score rules but combination 
> of riles
> is very spam-specific. Is it possible to write script for 
> determinating such
> groups of rules ? If yes, it seems to me good idea to write meta rules 
> for
> this.
>