You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spamassassin.apache.org by Apache Wiki <wi...@apache.org> on 2006/12/30 15:01:42 UTC

[Spamassassin Wiki] Update of "BayesStopList" by JustinMason

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Spamassassin Wiki" for change notification.

The following page has been changed by JustinMason:
http://wiki.apache.org/spamassassin/BayesStopList

The comment on the change is:
add note about history

------------------------------------------------------------------------------
  
  }}}
  
+ == History ==
+ 
+ (this part by jm, who selected the original set)
+ 
+ The original selection of words in the stop-list was based on words that scored around 0.4 to 0.6 in Bayes score after learning a "typical" spam/nonspam corpus, with a very large number of hits -- in other words, they were both (a) very common and (b) likely to always be ignored by the Bayes code anyway, since they were always going to fall within the $MIN_PROB_STRENGTH range.  They weren't chosen just as "common" words.
+ 
+ However, it's arguable that the stop-list makes an assumption that everyone speaks English -- in some non-English-language countries, a nonspam corpus may contain no English terms while the spam corpus is mostly-English, in which case those stop-words would actually make good spam signs.
+ 
+