bayes danger with high spam/ham ratio?

[Marcel van der Goot <vd...@earthlink.net>]

Hi,

I get at least a 100 times more spam than ham. If I train the Bayes
filter with that, is there any danger that it will just learn that
all email is spam? After all, that would give it a better than 99%
hit ratio ...

-Marcel

Re: bayes danger with high spam/ham ratio?

[Matt Kettler <mk...@evi-inc.com>]

At 05:56 PM 4/6/2004, Marcel van der Goot wrote:
>I get at least a 100 times more spam than ham. If I train the Bayes
>filter with that, is there any danger that it will just learn that
>all email is spam? After all, that would give it a better than 99%
>hit ratio ...

In practical situations, no.

The bayes database could be viewed as having tokens in one of three classes.
         1) Those which exist only in spam
         2) Those which exist only in ham
         3) Those which exist in both.

The "high spam ratio" effect will increase the scores of tokens in 
categories 1 and 3, but won't have any real effect on category 2.

Also, if you look at the way token probabilities are assigned, SA typically 
favors neutral scores for items in category 3, even if they are more common 
in spam than nonspam. Run sa-learn --dump's output through sort and you'll 
get a feeling for this effect.

(And don't be too alarmed if you see several "non-spam" words in the spam 
pile.. It's not as bad as you might think.. keep in mind that most emails 
have at least 20 words, so one bad-token won't hurt)

Bayes poison can make more tokens fall into #3, which worsens the situation 
slightly, but there are still likely to be a lot of tokens which exist only 
in #2. After all, a spammer isn't going to be able to optimize his bayes 
poison to uniformly cover all terms you commonly use in your everyday nonspam.

So a high spam to nonspam ratio might edge up your non-spam bayes scores 
slightly, but it's not going to have a huge impact.

(for reference, my current training ratio is about 22:1 spam:ham)