You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Ralf Hildebrandt <Ra...@charite.de> on 2004/09/08 10:07:08 UTC
timing/performance issues
I'm using spamassassin from within amavisd-new. Loggin shows that
amavis's SA-check is the most time-consuming step while processing an
email (>>80% of the total time needed).
How can I run spamassassin against some mails to find out what part of
spamassassin (maybe some sort of external DNS query) takes how much
time?
I was thinking of a sort of "profile" (subrouting x takes y seconds)
to find out where my particular installation of SA is "slow".
--
Ralf Hildebrandt (i.A. des IT-Zentrum) Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. +49 (0)30-450 570-916
IT-Zentrum Standort Campus Mitte AIM. ralfpostfix
Re: timing/performance issues
Posted by Ralf Hildebrandt <Ra...@charite.de>.
* Ralf Hildebrandt <Ra...@charite.de>:
> I'm using spamassassin from within amavisd-new. Loggin shows that
> amavis's SA-check is the most time-consuming step while processing an
> email (>>80% of the total time needed).
>
> How can I run spamassassin against some mails to find out what part of
> spamassassin (maybe some sort of external DNS query) takes how much
> time?
>
> I was thinking of a sort of "profile" (subrouting x takes y seconds)
> to find out where my particular installation of SA is "slow".
By trial and error I found out that the activation of bayes makes
things unbearably slow.
use_bayes 0
and all is well,
use_bayes 1
bayes_auto_learn 1
bayes_learn_to_journal 1
bayes_path /home/sharedspam/.spamassassin/bayes
bayes_file_mode 777
bayes_ignore_header ReSent-Date
bayes_ignore_header ReSent-From
bayes_ignore_header ReSent-Message-ID
bayes_ignore_header ReSent-Subject
bayes_ignore_header ReSent-To
and things get slow. Why is that?
--
_________________________________________________
Charité - Universitätsmedizin Berlin
_________________________________________________
Ralf Hildebrandt
i.A. des IT-Zentrum | Netzwerkdienste
Stabsstelle des Klinikumsvorstandes
Campus Mitte
Schumannstr. 20/21 | D-10177 Berlin
Tel. +49 30 450 570155 | Fax +49 30 450 570916
Ralf.Hildebrandt@Charite.de
http://www.charite.de
Re: timing/performance issues
Posted by Lucas Albers <ad...@cs.montana.edu>.
and my dbase has a good number of entries:
sa-learn --dump| head
0.000 0 2 0 non-token data: bayes db version
0.000 0 333304 0 non-token data: nspam
0.000 0 237785 0 non-token data: nham
0.000 0 5807722 0 non-token data: ntokens
Ralf Hildebrandt said:
> * Lucas Albers <ad...@cs.montana.edu>:
>
>> I've had good results doing bayes learn_to_journal and then running a
>> rebuild every hour.
>
> Whoa, hourly? I can try that.
>
>> This runs quick, even with concurrent access's.
>> Bayes get's updated quickly.
>> Bayes is only locked for a few seconds every hour, less than 3 seconds.
>
> --
> _________________________________________________
>
> Charité - Universitätsmedizin Berlin
> _________________________________________________
>
> Ralf Hildebrandt
> i.A. des IT-Zentrum | Netzwerkdienste
> Stabsstelle des Klinikumsvorstandes
> Campus Mitte
> Schumannstr. 20/21 | D-10177 Berlin
> Tel. +49 30 450 570155 | Fax +49 30 450 570916
> Ralf.Hildebrandt@Charite.de
> http://www.charite.de
>
>
--
Luke Computer Science System Administrator
Security Administrator,College of Engineering
Montana State University-Bozeman,Montana
Re: timing/performance issues
Posted by Ralf Hildebrandt <Ra...@charite.de>.
* Lucas Albers <ad...@cs.montana.edu>:
> I've had good results doing bayes learn_to_journal and then running a
> rebuild every hour.
Whoa, hourly? I can try that.
> This runs quick, even with concurrent access's.
> Bayes get's updated quickly.
> Bayes is only locked for a few seconds every hour, less than 3 seconds.
--
_________________________________________________
Charité - Universitätsmedizin Berlin
_________________________________________________
Ralf Hildebrandt
i.A. des IT-Zentrum | Netzwerkdienste
Stabsstelle des Klinikumsvorstandes
Campus Mitte
Schumannstr. 20/21 | D-10177 Berlin
Tel. +49 30 450 570155 | Fax +49 30 450 570916
Ralf.Hildebrandt@Charite.de
http://www.charite.de
Re: timing/performance issues
Posted by Lucas Albers <ad...@cs.montana.edu>.
I've had good results doing bayes learn_to_journal and then running a
rebuild every hour.
This runs quick, even with concurrent access's.
Bayes get's updated quickly.
Bayes is only locked for a few seconds every hour, less than 3 seconds.
Ralf Hildebrandt said:
> * Justin Mason <jm...@jmason.org>:
>
>> perldoc Devel::DProf -- that's the perl profiler. but as you said,
>> it now appears to be bayes -- it could be that if a scan is taking
>> a *very* long time, what's actually taking place is a Bayes expiration
>> run, which happens once every N days (typically).
>
> Bayes expiration is done daily at nighttime by a cron job
>
> --
> _________________________________________________
>
> Charité - Universitätsmedizin Berlin
> _________________________________________________
>
> Ralf Hildebrandt
> i.A. des IT-Zentrum | Netzwerkdienste
> Stabsstelle des Klinikumsvorstandes
> Campus Mitte
> Schumannstr. 20/21 | D-10177 Berlin
> Tel. +49 30 450 570155 | Fax +49 30 450 570916
> Ralf.Hildebrandt@Charite.de
> http://www.charite.de
>
>
--
Luke Computer Science System Administrator
Security Administrator,College of Engineering
Montana State University-Bozeman,Montana
Re: timing/performance issues
Posted by Ralf Hildebrandt <Ra...@charite.de>.
* Justin Mason <jm...@jmason.org>:
> perldoc Devel::DProf -- that's the perl profiler. but as you said,
> it now appears to be bayes -- it could be that if a scan is taking
> a *very* long time, what's actually taking place is a Bayes expiration
> run, which happens once every N days (typically).
Bayes expiration is done daily at nighttime by a cron job
--
_________________________________________________
Charité - Universitätsmedizin Berlin
_________________________________________________
Ralf Hildebrandt
i.A. des IT-Zentrum | Netzwerkdienste
Stabsstelle des Klinikumsvorstandes
Campus Mitte
Schumannstr. 20/21 | D-10177 Berlin
Tel. +49 30 450 570155 | Fax +49 30 450 570916
Ralf.Hildebrandt@Charite.de
http://www.charite.de