You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Josh Trutwin <jo...@trutwins.homeip.net> on 2004/09/08 22:55:43 UTC

3.0.0-rc3 - Migrate Bayes/AWL to SQL?

I've been reading through the 3.0 docs figured I'd upgrade a test box from 2.64 and play with it.  I'm looking to use the SQL AWL/Bayes feature since I'm already using the userpref feature.  

My best guess is that after upgrading the configuration file to use the SQL settings, sa-learn --sync would migrate the current AWL/Bayes files in DB_File format to SQL.  Is that correct?

Josh

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Marco van den Bovenkamp <ma...@linuxgoeroe.dhs.org>.
Josh Trutwin wrote:

> But the man page says:
> 
> <snip>
> --restore=filename
> Performs a restore of the Bayes database defined by filename.
> 
> WARNING: This is a destructive operation, previous Bayes data will be wiped out.
> </snip>
> 
> That doesn't sound promising!

Remember that your *SQL* Bayes db is empty at that point...

-- 

		Regards,

			Marco.


Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Wed, 8 Sep 2004 17:00:15 -0400
Theo Van Dinter <fe...@kluge.net> wrote:

> I believe the SQL docs have more info

Doesn't appear to - there is no mention of either the words "restore" or "dump" in the SQL docs or the UPGRADE file.  Might be a nice addition before 3.0 goes final.

> but IIRC you need to --dump
> the DBM file and --restore into SQL.   I have no experience with the
> Bayes-SQL code though.

Ok, that's making a little more sense.  From the 3.0 sa-learn man page it looks like the options would be:

v2.64: sa-learn --backup somefile.dump
v3.00: sa-learn --restore somefile.dump

But the man page says:

<snip>
--restore=filename
Performs a restore of the Bayes database defined by filename.

WARNING: This is a destructive operation, previous Bayes data will be wiped out.
</snip>

That doesn't sound promising!

Josh

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Theo Van Dinter <fe...@kluge.net>.
On Wed, Sep 08, 2004 at 03:55:43PM -0500, Josh Trutwin wrote:
> My best guess is that after upgrading the configuration file to use the SQL settings, sa-learn --sync would migrate the current AWL/Bayes files in DB_File format to SQL.  Is that correct?

--sync syncronizes the journal and the dbm file.

I believe the SQL docs have more info, but IIRC you need to --dump the DBM
file and --restore into SQL.   I have no experience with the Bayes-SQL code
though.

-- 
Randomly Generated Tagline:
Oh, cruel fate.  Why do you mock me?
 
 		-- Homer Simpson
 		   Bart the Daredevil

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Michael Parker <pa...@pobox.com>.
On Thu, Sep 09, 2004 at 09:01:15AM -0500, Josh Trutwin wrote:
> 
> My previous install used a global whitelist / bayes database for all
> accounts.  That wasn't the best idea but it worked for a while.  If
> you re-train with sa-learn does the AWL also get re-built?
> 

Bayes and AWL have nothing to do with each other.

Michael

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Wed, 8 Sep 2004 21:19:52 -0500
Michael Parker <pa...@pobox.com> wrote:

You with the SPF folks on spf.pobox.com?

> It imports as whatever user you run sa-learn as.

Yeah, I'm seeing this behavior.  I hacked my qmail-scanner to pass in
the email address as the username to spamc for my vmailmgr accounts,
maybe I can do something similar with sa-learn?

> sa-learn doesn't have that functionality, yet.  You can use
> bayes_sql_override_username to trick sa-learn into learning as
> whoever you want to learn as.

I had thought about doing something like that, copying local.cf for
each account, update it and run sa-learn with the -C parameter to set
the config file.

> @GLOBAL doesn't really make sense for AWL, but an
> awl_sql_override_username (similar to the one for bayessql) might
> work.  Feel free to open up a bug at
> http://bugzilla.spamassassin.org/

Actually a -u flag to sa-learn would be a better "bug" to open for my
needs.  :)

My previous install used a global whitelist / bayes database for all
accounts.  That wasn't the best idea but it worked for a while.  If
you re-train with sa-learn does the AWL also get re-built?

Thanks for your help,

Josh

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Michael Parker <pa...@pobox.com>.
On Wed, Sep 08, 2004 at 05:47:03PM -0500, Josh Trutwin wrote:
> On Wed, 8 Sep 2004 16:04:00 -0500
> Michael Parker <pa...@pobox.com> wrote:
> 
> > To migrate AWL data you can use the convert_awl_dbm_to_sql script in
> > the tools directory.
> 
> Looks like I have it working though I think I'm going to give up on
> importing my bayes db into MySQL and re-train it because it imports
> the entire database under one user - namely root.
> 

It imports as whatever user you run sa-learn as.

> For sa-learn ham/spam there doesn't seem to be a way to actually set
> the user so I can do something like this with vmailmgr virtualhosts:
> 
> sa-learn --spam /home/josh/Maildir/Spam --user=josh@domain1.com
> sa-learn --spam /home/anita/Maildir/Spam --user=anita@domain1.com
> 

sa-learn doesn't have that functionality, yet.  You can use
bayes_sql_override_username to trick sa-learn into learning as whoever
you want to learn as.

> For AWL, is there a @GLOBAL username like with SQL userprefs where you
> can say this whitelist entry is global?  

@GLOBAL doesn't really make sense for AWL, but an
awl_sql_override_username (similar to the one for bayessql) might
work.  Feel free to open up a bug at http://bugzilla.spamassassin.org/

Michael

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Wed, 8 Sep 2004 16:04:00 -0500
Michael Parker <pa...@pobox.com> wrote:

> To migrate AWL data you can use the convert_awl_dbm_to_sql script in
> the tools directory.

Looks like I have it working though I think I'm going to give up on
importing my bayes db into MySQL and re-train it because it imports
the entire database under one user - namely root.

For sa-learn ham/spam there doesn't seem to be a way to actually set
the user so I can do something like this with vmailmgr virtualhosts:

sa-learn --spam /home/josh/Maildir/Spam --user=josh@domain1.com
sa-learn --spam /home/anita/Maildir/Spam --user=anita@domain1.com

I don't want Josh's Bayes SQL data to conflict with Anita's if I can
avoid it.  It looks like sa-learn uses whatever username the person
running sa-learn is logged in as, this won't work for virtual domains
as multiple domains are "owned" by one user.  In a vmailmgr setup the
user might be "domain1" and I'd want bayes/awl records to show up with
usernames: josh@domain1.com, anita@domain1.com, etc.

For AWL, is there a @GLOBAL username like with SQL userprefs where you
can say this whitelist entry is global?  

This is looking pretty nifty, I'm hoping that I can work around these
starting issues.

Thanks!

Josh

Re: [OT] 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Wed, 8 Sep 2004 16:16:06 -0500
Josh Trutwin <jo...@trutwins.homeip.net> wrote:

> > Meta: In the future, please limit your line length, it makes it
> > easier
> >       to reply.
> 
> Really?  I have it set to wrap at 70 chars.  Should it be less? 
> (Sylpheed Claws client).  Let me try this: "Wrap before sending"
> config.

Seems to have helped...

Josh

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Wed, 8 Sep 2004 16:04:00 -0500
Michael Parker <pa...@pobox.com> wrote:

> Meta: In the future, please limit your line length, it makes it
> easier
>       to reply.

Really?  I have it set to wrap at 70 chars.  Should it be less? 
(Sylpheed Claws client).  Let me try this: "Wrap before sending"
config.

> No, that doesn't work.  Hmmm...maybe a good candidate for the
> README.bayes file, at the very least the wiki.

Agreed.

> If you want to switch from DBM based bayes to SQL based bayes, you
> should first upgrade the DBM db, via sa-learn --sync.  Then you can
> sa-learn --backup > backup.txt, switch over your config in local.cf
> (according to the text in README.bayes), and then run
> sa-learn --restore backup.txt

Despite the: "WARNING: This is a destructive operation, previous Bayes
data will be wiped out." message in the sa-learn man page?  :)

> To migrate AWL data you can use the convert_awl_dbm_to_sql script in
> the tools directory.

Thanks for the tip.

Josh

Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Lucas Albers <ad...@cs.montana.edu>.
It would be nice if we had full complete directions on upgrading to sql
based for SA 3.0.
Step by step.

-- 
Luke Computer Science System Administrator
Security Administrator,College of Engineering
Montana State University-Bozeman,Montana



Re: 3.0.0-rc3 - Migrate Bayes/AWL to SQL?

Posted by Michael Parker <pa...@pobox.com>.
On Wed, Sep 08, 2004 at 03:55:43PM -0500, Josh Trutwin wrote:
> 
> My best guess is that after upgrading the configuration file to use
> the SQL settings, sa-learn --sync would migrate the current AWL/Bayes
> files in DB_File format to SQL.  Is that correct?
> 

Meta: In the future, please limit your line length, it makes it easier
      to reply.


No, that doesn't work.  Hmmm...maybe a good candidate for the
README.bayes file, at the very least the wiki.

If you want to switch from DBM based bayes to SQL based bayes, you
should first upgrade the DBM db, via sa-learn --sync.  Then you can
sa-learn --backup > backup.txt, switch over your config in local.cf
(according to the text in README.bayes), and then run
sa-learn --restore backup.txt

To migrate AWL data you can use the convert_awl_dbm_to_sql script in
the tools directory.

Michael