You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spamassassin.apache.org by Apache Wiki <wi...@apache.org> on 2006/12/30 15:01:42 UTC
[Spamassassin Wiki] Update of "BayesStopList" by JustinMason
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Spamassassin Wiki" for change notification.
The following page has been changed by JustinMason:
http://wiki.apache.org/spamassassin/BayesStopList
The comment on the change is:
add note about history
------------------------------------------------------------------------------
}}}
+ == History ==
+
+ (this part by jm, who selected the original set)
+
+ The original selection of words in the stop-list was based on words that scored around 0.4 to 0.6 in Bayes score after learning a "typical" spam/nonspam corpus, with a very large number of hits -- in other words, they were both (a) very common and (b) likely to always be ignored by the Bayes code anyway, since they were always going to fall within the $MIN_PROB_STRENGTH range. They weren't chosen just as "common" words.
+
+ However, it's arguable that the stop-list makes an assumption that everyone speaks English -- in some non-English-language countries, a nonspam corpus may contain no English terms while the spam corpus is mostly-English, in which case those stop-words would actually make good spam signs.
+
+