low scores?

[Rich <rg...@ellerbach.com>]

I have recently upgrades from 2.x to 3.0.1 and have been watching the
scores for stuff that is real spam. I had a bunch of up-weighted scores in
2.x but I didn't move those over to the new version while I evaluated what
the new version was doing. What I don't understand are what seem to be
extremely low scores for various tests, for instance this is the report:

Content analysis details:   (1.9 points, 5.0 required)                    
                                                                          
                                         pts rule name             
description                                                        ----
---------------------- ---------------------------------------
0.0 HTML_40_50             BODY: Message is 40% to 50% HTML               
                    0.0 HTML_MESSAGE           BODY: HTML included in
message                                      1.9 BAYES_99              
BODY: Bayesian spam probability is 99 to 100%
                            [score: 1.0000]

on a message that had a content preview of:

Content preview:  <a href="http://imsodamtired.com/?wid=100049"> Why b u  
                      y from World Wide Meds?<br><br> # No Prescription
Required<br> #                               Discrete & Confidential
Packag i n g<br> # World Wide Shipping<br> #                          
Quality Generic Medi.c.ations<br> # 1 0 0 % M0ney Back Guarant e e<br>    
                    </a> <br><br><br><br><br><br> <a

etc. (i.e. no-doubt-about-it spam) yet there are zero scores for the two
HTML tests and only! 1.9 for the BAYES_99 test. I don't run any network
tests because I'm behind a corporate firewall and they are unreliable in
this environment.

My question is why are these score so low? If 5 is a typical spam/ham
these messages should be scoring close to that based on the bayes_99
alone.

If the engine is expecting to be able to use network tests for these then
shouldn't the default scores be higher if those tests are turned off?

Rich

Re: low scores?

[Jim Maul <jm...@elih.org>]

Rich wrote:
> I have recently upgrades from 2.x to 3.0.1 and have been watching the
> scores for stuff that is real spam. I had a bunch of up-weighted scores in
> 2.x but I didn't move those over to the new version while I evaluated what
> the new version was doing. What I don't understand are what seem to be
> extremely low scores for various tests, for instance this is the report:
> 
> Content analysis details:   (1.9 points, 5.0 required)                    
>                                                                           
>                                          pts rule name             
> description                                                        ----
> ---------------------- ---------------------------------------
> 0.0 HTML_40_50             BODY: Message is 40% to 50% HTML               
>                     0.0 HTML_MESSAGE           BODY: HTML included in
> message                                      1.9 BAYES_99              
> BODY: Bayesian spam probability is 99 to 100%
>                             [score: 1.0000]
> 
> on a message that had a content preview of:
> 
> Content preview:  <a href="http://imsodamtired.com/?wid=100049"> Why b u  
>                       y from World Wide Meds?<br><br> # No Prescription
> Required<br> #                               Discrete & Confidential
> Packag i n g<br> # World Wide Shipping<br> #                          
> Quality Generic Medi.c.ations<br> # 1 0 0 % M0ney Back Guarant e e<br>    
>                     </a> <br><br><br><br><br><br> <a
> 
> etc. (i.e. no-doubt-about-it spam) yet there are zero scores for the two
> HTML tests and only! 1.9 for the BAYES_99 test. I don't run any network
> tests because I'm behind a corporate firewall and they are unreliable in
> this environment.
> 
> My question is why are these score so low? If 5 is a typical spam/ham
> these messages should be scoring close to that based on the bayes_99
> alone.
> 
> If the engine is expecting to be able to use network tests for these then
> shouldn't the default scores be higher if those tests are turned off?
> 
> Rich
> 
> 

The SA scores are generated based on the scores of other rules and takes 
into account overlap of certain rules.  From what i understand, BAYES_99 
is scored what it is because a lot of messages that triggered this rule 
also triggered other rules and as such the score for it was lowered.  If 
you dont run this other rules however (i would imagine network tests 
would be some of them) then i would suggest you bump up the scores for 
the tests you are running to compensate for the lack of other tests 
being run.  This is exactly what i did.  My BAYES_99 has been running at 
4.5 with no problems for a while now.  The ability to change the scores 
of tests is there for exactly this reason - because everyones system is 
different.  Dont be afraid to override the defaults, but be sure to 
watch closely after you do to check for false positives.

-Jim