You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Dan Mahoney, System Admin" <da...@prime.gushi.org> on 2005/09/08 11:04:04 UTC
global bayes database?
Hey all,
As my bayes database and my training of spamassassin is much better than
that of most of my users...
Is there any way to augment user bayes DB with a call to mine? (and/or
mailnull's, which is the default catch-all box for a few domains with only
one valid address and a ton of dictionary spam (which thus autofeeds to
spamassassin -r)
-Dan
--
"This Is Not Goodbye!"
-DM, August 11th 2001, 10 PMish Chicago Time
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: [sa-list] Re: global bayes database?
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Fri, 9 Sep 2005, Michael Parker wrote:
> Oh, you want an entirely different Bayes storage module, one that
> doesn't exist. You're more than welcome to create your own, perldoc
> Mail::SpamAssassin::BayesStore to get a sense of the API that you must
> implement. I'll leave the issues surrounding combining of bayes
> databases as an exercise to the reader (suggest you search the archives
> for previous msgs on this topic).
Interesting. The API looks fairly straightforward.
Support for full-blown-multi-bayes isn't something I could see myself
implementing right now, based purely on time constraints, but I don't
think it's particularly hard.
Still, the tweak to add a call that does nspam_nham_get, and if it's less
than the required number for effective bayes, uses the system bayes DBs
for scanning (but not learning, if autolearn is deemed appropriate) should
be easy enough.
I've searched the users list -- my issues with token collision are nil --
I'm sure everything I've got is in the "new" format since any attempts I
made to try and get my original users stuff in crashed my systems.
I'm going to note this more for anyone else who searches this list than
for myself -- scoring on multiple bayes counts could have disasterous
circumstances. Since it *has* to be read-only...
(since everyone gets the same spam -- see
http://article.gmane.org/gmane.mail.spam.spamassassin.general/60376 )
..any admin must realize that for all they know, their user could work
for Pfizer or SmithKline -- and you could be tagging all their legit
workmail as bad. Normal bayes prevents this (or forces the user to accept
that since they don't consider the names of drugs a bad thing, they have
to deal with the spam).
To pull an old phrase...one mans junk is another's treasure.
Still, this could be as simple as calling the bayes algorithm twice, once
as $user, once as $system -- and maintaining a different (probably
slightly lower) set of scores for $system.
Given, maybe the multi-bayes option should even be off by default for
users with a good corpus (define good...200 messages? a thousand?).
But I know that since I get more email, use pine and a shell, and
religiously shuffle all my spam to spamassassin -r, that I'm more likely
to have a complete corpus than those users which use outlook and have to
rely on the automatic learning features.
Okay, I've babbled enough.
-Dan
--
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: global bayes database?
Posted by Michael Parker <pa...@pobox.com>.
Dan Mahoney, System Admin wrote:
>
> Right, but this isn't exactly what I was looking for. Basically, I'm
> looking for a system whereby if a users bayes corpus isn't primed
> properly, it can refer to others, as sort of an if --> then system,
> rather than manually overriding it.
>
> As SQL becomes the recommended standard, I'm hoping this feature
> becomes more popular, as I would *love* to see SA do scoring and
> training based on BOTH system-bayes files as well as user-bayes.
>
> Maybe I should just ask for a pony :)
>
Oh, you want an entirely different Bayes storage module, one that
doesn't exist. You're more than welcome to create your own, perldoc
Mail::SpamAssassin::BayesStore to get a sense of the API that you must
implement. I'll leave the issues surrounding combining of bayes
databases as an exercise to the reader (suggest you search the archives
for previous msgs on this topic).
Michael
Re: global bayes database?
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Thu, 8 Sep 2005, Michael Parker wrote:
> Dan Mahoney, System Admin wrote:
>
>> On Thu, 8 Sep 2005, Matt Kettler wrote:
>>
>>> At 05:04 AM 9/8/2005, Dan Mahoney, System Admin wrote:
>>>
>>>> As my bayes database and my training of spamassassin is much better
>>>> than that of most of my users...
>>>>
>>>> Is there any way to augment user bayes DB with a call to mine?
>>>
>>>
>>> one way is in your local.cf just set everyone to use the same bayes
>>> database and make it world accessible:
>>
>>
>> I'm using SQL. I'm sorry, I should have mentioned that. The SQL docs
>> don't say anything about it.
>>
>>
> You mean this portion of the SQL docs?
>
> In addition to the global configuration directives there is a user
> preference:
>
> bayes_sql_override_username someusername
Right, but this isn't exactly what I was looking for. Basically, I'm
looking for a system whereby if a users bayes corpus isn't primed
properly, it can refer to others, as sort of an if --> then system, rather
than manually overriding it.
As SQL becomes the recommended standard, I'm hoping this feature becomes
more popular, as I would *love* to see SA do scoring and training based on
BOTH system-bayes files as well as user-bayes.
Maybe I should just ask for a pony :)
-Dan
> This directive, if used, will override the username used for storing
> data in the database. This could be used to group users together to
> share bayesian filter data. You can also use this config option to
> trick sa-learn to learn data as a specific user.
>
>
> Michael
>
>
--
"Of course she's gonna be upset! You're dealing with a woman here Dan,
what the hell's wrong with you?"
-S. Kennedy, 11/11/01
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: [sa-list] Re: global bayes database?
Posted by Michael Parker <pa...@pobox.com>.
Dan Mahoney, System Admin wrote:
> On Thu, 8 Sep 2005, Matt Kettler wrote:
>
>> At 05:04 AM 9/8/2005, Dan Mahoney, System Admin wrote:
>>
>>> As my bayes database and my training of spamassassin is much better
>>> than that of most of my users...
>>>
>>> Is there any way to augment user bayes DB with a call to mine?
>>
>>
>> one way is in your local.cf just set everyone to use the same bayes
>> database and make it world accessible:
>
>
> I'm using SQL. I'm sorry, I should have mentioned that. The SQL docs
> don't say anything about it.
>
>
You mean this portion of the SQL docs?
In addition to the global configuration directives there is a user
preference:
bayes_sql_override_username someusername
This directive, if used, will override the username used for storing
data in the database. This could be used to group users together to
share bayesian filter data. You can also use this config option to
trick sa-learn to learn data as a specific user.
Michael
Re: [sa-list] Re: global bayes database?
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Thu, 8 Sep 2005, Matt Kettler wrote:
> At 05:04 AM 9/8/2005, Dan Mahoney, System Admin wrote:
>> As my bayes database and my training of spamassassin is much better than
>> that of most of my users...
>>
>> Is there any way to augment user bayes DB with a call to mine?
>
> one way is in your local.cf just set everyone to use the same bayes database
> and make it world accessible:
I'm using SQL. I'm sorry, I should have mentioned that. The SQL docs
don't say anything about it.
-Dan
--
"Why are you wearing TWO grounding straps?"
-John Evans, Ezzi Computers August 23, 2001
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: global bayes database?
Posted by Matt Kettler <mk...@comcast.net>.
At 05:04 AM 9/8/2005, Dan Mahoney, System Admin wrote:
>As my bayes database and my training of spamassassin is much better than
>that of most of my users...
>
>Is there any way to augment user bayes DB with a call to mine?
one way is in your local.cf just set everyone to use the same bayes
database and make it world accessible:
bayes_path {somepath}/bayes
bayes_file_mode 0777
Notes:
bayes_path needs to end in /bayes, as it's really a path plus half a
filename. SA will append "_toks" and "_seen" to this to create bayes_toks
and bayes_seen.
bayes_mode needs to be 0777 not 0666. It's sometimes used in directory
creation and works more like a umask than a mode.