You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Justin Mason <jm...@jmason.org> on 2006/08/15 16:08:58 UTC
Re: [Maia-users] SA BAYES TIMING INFO
could it be that local_tests_only is *not* set to 1? in other words,
that network results are being used in bayes training? That
slows things down quite a lot.
--j.
David Morton writes:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> (This message is now CC'd to both maia-users and spamassassin mailing lists )
> ( Continuing the thread in SpamAssassin ML RE: slow sql bayes store)
>
> Alexandre Ghisoli wrote:
>
> > DB Server
> > Actually, we got perfs problem with this one, probably related to
> > Software RAID - new LSI Raid cards ordered
> > PostgreSQL 8.1.4
> > AMD Opteron 3GHz
> > 1GB RAM
> > 2x IDE HDD, software raid
>
> > 0.000 0 124718 0 non-token data: ntokens
>
> > 2006-08-15 09:57:55 Maia: [process-quarantine-sub] TIMING [total 24368
> > ms] - msg-prep: 2 (0%), train-bayes: 23700 (97%), delete-mail: 666 (3%),
> > rundown: 0 (0%)
>
>
> Ok. This looks like the best example yet of what I'm looking for. Good job
> presenting that data. :)
>
> Furthermore, from the parts I have quoted above, I think I can say without a
> doubt that *something* is messed up here. Even with software raid, that box
> should be able to handle learning a message faster than 24 seconds. Actually,
> unless you get a very good card, the opty might be able to handle the raid stuff
> better than many hardware raid cards.
>
> 124k rows should not be a problem for a database. I'm really thinking there's
> an algorithm problem withing the bayes learning code. It's making too many sql
> calls, or has a big 'O' problem... something.
>
> ( spamassasin folks, the original full message is archived at
> http://www.renaissoft.com/pipermail/maia-users/2006-August/007188.html )
>
> To the spamassassin mailing list: These results seem typical of the reports I
> have seen. It has spanned both mysql and postgresql, several OS's, SCSI or IDE,
> RAID or not. The only consistent thing is that it is slow.
>
> There is also ageneral consensus that it seems like it got really slow around
> the time 3.x was installed, though we haven't yet had any solid reports to go
> back and forth and test it empirically.
>
> - --
> David Morton
> Maia Mailguard - http://www.maiamailguard.com
> Morton Software Design and Consulting - http://www.dgrmm.net
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2.2 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFE4dBwUy30ODPkzl0RAmqTAKCfXa7x3A9d/n93RYswkqkRVK+eNwCdFeQS
> ZG+cxXgJ1I/jvIXEbhb8onc=
> =S7Jk
> -----END PGP SIGNATURE-----
Re: [Maia-users] SA BAYES TIMING INFO
Posted by David Morton <mo...@dgrmm.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Justin Mason wrote:
> could it be that local_tests_only is *not* set to 1? in other words,
> that network results are being used in bayes training? That
> slows things down quite a lot.
As far as I can see, there's no connection... bayes wouldn't have any use for that.
Plus, in some error logs, I'm getting a stack trace that shows it times out in
_put_tokens which is pretty much the database side of things.
- --
David Morton
Maia Mailguard - http://www.maiamailguard.com
Morton Software Design and Consulting - http://www.dgrmm.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFE4dbZUy30ODPkzl0RAjyUAKC7v6K8ql+gy5mBaK2wVbg8eIAMPACfQR/J
iwZEDceHOkqj/szSRGCJWPw=
=2GR2
-----END PGP SIGNATURE-----