You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Justin Mason <jm...@jmason.org> on 2004/10/23 21:25:25 UTC
Re: spamd still burning CPU in 3.0.1
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
BTW, SpamAssassin *is* CPU-intensive. It's designed that way ;)
- --j.
Tim B writes:
> email builder wrote:
> > I hurried out and installed 3.0.1, thinking one of those memory/language
> > improvements mentioned in the release notes were going to be my savior...
> >
> > Sadly, 3.0.1's spamd has the same CPU-intensive behavior here. I am soooo at
> > a loss; tried everything I've read... spent days reading... please, anyone
> > have anything more?
> >
> > If spamd isn't I/O bound, my memory isn't swapping, I have no other processes
> > that are out of control, I can't for the life of me figure out why this is
> > happening.
> >
> > Again, my specs:
> >
> > A sample from top:
> >
> > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> > 1401 maildrop 16 0 39744 34m 6840 R 28.3 3.4 3:04.18 spamd
> >
> > spamd children average around 30% CPU, but even 50% not too unusual.
> >
> > load average is around 15 to 18 during the middle of the day
> >
> > And this is how I start spamd:
> >
> > LANG=en_US; export LANG; TMPDIR=/tmp/spamassassin; export TMPDIR
> > spamd -d -q -x --max-children=5 -H /etc/razor -u maildrop -r
> > /var/run/spamd/spamd.pid
> >
> > (also tried with -L to no avail)
> >
> > /tmp/spamassassin is mounted with tmpfs
> >
> > prefs/bayes/awl all in SQL, but bayes/awl not being used right now
> >
> > we also run named on the same machine
> >
> > if it's important, this is 3.0.1, downloaded and compiled manually (not a
> > CPAN install)
> >
> > I have installed no custom rulesets, nothing extra beside whatever comes 100%
> > stock. This is a Fedora Core 2 machine (2.8P-IV hyperthreaded, 1GB RAM)
> >
> > spamc is called from maildrop as such:
> >
> > if ( $SIZE < 262144 )
> > {
> > exception {
> > xfilter "/usr/bin/spamc -u $LOGNAME"
> > }
> > }
> >
> > (also tried running inside of amavis to no avail)
> >
> > Any advice or even just pointers on any more reading I can do would be highly
> > appreciated!
> >
> >
> >>>What in the world is going on? Isn't it true that spamd (beside DCC)
> >>
> >>does
> >>
> >>>its thing w/out disk I/O? If so, what else could be chewing up so much
> >>
> >>CPU?
> >>
> >>I don't know - The same thing happens to me a couple of times a day, and I
> >>only get about 350 messages per day. Today it was at 12:25p:
> >>
> >>12:25:07 4496 511804 99.13 2532 9420 65088
> >>432884 86.93
> >>
> >>12:25:07 0 91 5.47 2.35 0.89 <<<<<<< LA
> >>
> >>When this happens, the HDD is constantly active. I'm using v2.64 with
> >>network checks. The load average for the 21 hrs of this day is about 0.1
> >
> >
> >
> >
> >
> > __________________________________
> > Do you Yahoo!?
> > Yahoo! Mail - Helps protect you from nasty viruses.
> > http://promotions.yahoo.com/new_mail
> >
>
> Have you tried doing a force-expire on your bayes db?
>
> I found this helped me. Disabling autoexpire, and twice a day running
> sa-learn --force-expire
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Exmh CVS
iD8DBQFBerAlMJF5cimLx9ARAkSzAJ4ziHgz6iLos/0Obf7OxcFEBxs3gwCfRCvG
3pCNDv79pho0WFpZWnvVDEA=
=6UpA
-----END PGP SIGNATURE-----
Re: spamd still burning CPU in 3.0.1
Posted by Michael Parker <pa...@pobox.com>.
On Thu, Oct 28, 2004 at 09:54:42AM -0400, Jeff Koch wrote:
>
> You are correct and I apologize to the SA team. I cannot characterize the
> problem as a bug - SA 3.0 is just much slower and resource intensive than
> SA 2.64. If I understand you correctly you are just testing Bayes. Our
> production testing involved using SA as a whole. And I again suggest that
> SA 3.0 be compared against previous versions (like 2.64) in a real world
> production test. Maybe the answer is to publish a cheat sheet of new
> features in 3.0 that need to be turned off in order to achieve the
> throughput of 2.64.
>
Actually, it tests most local rules (auto learning with just the
BAYES_* rules is kind a boring if you know what I mean).
I have done comparative tests against 2.6x and 3.0.x and at the time
found 3.0.x some percentage slower but it was a percentage close to
the number of extra rules 3.0.x was running at the time. Since then
the number of active rules in 3.0 have been reduced. Sorry, it was a
PITA to run the benchmark on 2.6x since it doesn't provide some of the
features that 3.0 provides, so I doubt I would do it again.
I think the majority of problems people are seeing can be solved with
some tuning. Indeed, 3.0 is a slightly different animal than 2.6x
was. It is better and more feature rich than any previous versions.
With features come complexity and unfortunately the odd problem here
or there. As developers we can not cover every possible setup, for
that we must rely on users. I love to help when I can, especially if
you're running SQL (prefs,bayes or awl). Feel free to hop on IRC and
ask for help or ask here, or if you find a bug file a report.
<plug>
Come to ApacheCon. I'll be there and I won't speak for anyone else,
but I bet you can find help/tips from folks there. Perhaps a BOF for
SpamAssassin is in order.
</plug>
Michael
Re: spamd still burning CPU in 3.0.1
Posted by Jeff Koch <je...@intersessions.com>.
You are correct and I apologize to the SA team. I cannot characterize the
problem as a bug - SA 3.0 is just much slower and resource intensive than
SA 2.64. If I understand you correctly you are just testing Bayes. Our
production testing involved using SA as a whole. And I again suggest that
SA 3.0 be compared against previous versions (like 2.64) in a real world
production test. Maybe the answer is to publish a cheat sheet of new
features in 3.0 that need to be turned off in order to achieve the
throughput of 2.64.
At 01:41 AM 10/28/2004, Michael Parker wrote:
>On Thu, Oct 28, 2004 at 01:09:57AM -0400, Jeff Koch wrote:
> >
> > We figure that we'd have to reduce the email load on each server by 50% in
> > order to use SA 3.0 and thereby need twice as many servers. However, we're
> > going to wait until the SA developers take the memory and load issues
> > seriously and fix the problem. Maybe if enough users complain they'll do
> > some high volume production test comparisons of 3.0 with previous versions
> > and sort out the problem.
> >
>
>I believe this is an entirely unfair characterization of the
>development team. In all cases where recent memory issues have
>cropped we've worked to resolve them. As for load and speed issues, I
>personally take these very seriously. I would guess I benchmark bayes
>on the average of twice a day. The benchmark pumps 300+ msgs per
>minute through my server, 6.5+ million SQL queries averaging around
>3200 queries per second on my MySQL server.
>
>If anyone has a reproducible memory or load issue I highly encourage
>you to file a bug so that we can start tracking it down.
>
>Michael
Best Regards,
Jeff Koch, Intersessions
Re: spamd still burning CPU in 3.0.1
Posted by Jeff Koch <je...@intersessions.com>.
Although we don't load balance it would be relatively easy to do if the
incoming mailserver was really having trouble. We would just duplicate the
machine using another mx record. With qmail, incoming SMTP concurrency
would reach our max and the first machine would stop accepting new
connections forcing connections to the secondary mx record. I guess it's
not really load balancing - more like failover.
At 04:18 AM 10/28/2004, email builder wrote:
> > We have two production mailservers running SA spamd. The first handles
> > about 5,000 incoming emails per hour, does spam filtering with SA and virus
>
>Can I ask you how you load balance between the two machines (obviously if one
>handles 5000/hr and the other 2,500, it's not straight round robin)? I am
>not sure I understand how to fork requests between more than one machine when
>calling it through spamc.... (I don't understand how to configure this) ???
>
> > filtering with qmailscanner and forwards the filtered mail to a server
> > handling the pop accounts. We're using SA 2.64 with Bayes, AWL, Razor and
> > about half of the RBL's. The machine is a 2.8Ghz P4 with 1.0GB RAM and SCSI
> >
> > hard drive. CPU usuage runs between 25-40% and system load runs 1.50 to
> > 2.20 with isolated spikes to 7.0.
> >
> > The second machine is a 2Ghz Athlon with 1.0GB RAM and an IDE drive. It
> > does spam and virus filtering with SA 2.64 and qmailscanner and also
> > handles POP3 sessions with vpopmail. We use Bayes, AWL, Razor and the same
> > RBL's. It handles approx 2,500 emails per hour (with peaks of 5K
> > emails/hour) and approx 2,000 pop3 sessions per hour (peaks of 5K
> > pops/hour). CPU usage runs about 20% with peaks to 50% and system load
> > averages 0.80 with peaks of 16.0.
> >
> > We are pretty satisfied with the above setup. We tried moving one of the
> > servers to SA 3.0 in order to use the new MySQL Bayes features but got
> > absolutely killed on CPU usage and system load - that lasted about a day
> > and we reverted to 2.64.
>
>Wow, OK, I am starting to think I am actually not insane and that my system
>really isn't b0rked and is merely at its performance limit. Thanks Rick and
>Jeff for the reality check!
>
> > We figure that we'd have to reduce the email load on each server by 50% in
> > order to use SA 3.0 and thereby need twice as many servers. However, we're
> > going to wait until the SA developers take the memory and load issues
> > seriously and fix the problem. Maybe if enough users complain they'll do
> > some high volume production test comparisons of 3.0 with previous versions
> > and sort out the problem.
>
>This is disturbing. I'm surprised the CPU thing has not been a topic of
>conversation (I see the memory one is).... does anyone know if the developers
>are looking at this at all???
>
>Thanks again!
>
>
> > At 09:33 PM 10/27/2004, email builder wrote:
> > > > email builder wrote:
> > > > >>email builder wrote:
> > > > >>How much email are you processing ?
> > > > >
> > > > >
> > > > > Well, just the other day we had an average of 48 msgs/min (max
> > 255/min)
> > > > get
> > > > > run
> > > > > through SA. Can't say today yet because can't run our stats tools
> > until
> > > > the
> > > > > busy hours are over cuz SA is hogging the CPU. ;)
> > > >
> > > > Hi,
> > > >
> > > > Your CPU is over loaded. At 48 a minute it should run just ok on a 2.8
> > > > Ghz machine, much over that it's going to start having problems. On
> > our
> > > > 2.4 Ghz (not HT) processor if I process over 35 a minute I start having
> > > > problems with load.
> > >
> > >I have two reactions to this:
> > >
> > >1) I like the glimmer of hope and the idea that throwing hardware at the
> > >problem can solve it
> > >
> > >2) Throwing hardware at problems is usually avoiding fixing the *real*
> > >problem. According to other posters on this list, my load is not
> > excessive
> > >for a modern-day 2.xGHz machine. I will have to re-read some messages,
> > but I
> > >believe responders to my posts on the "[OT] Email Servers" thread quoted
> > >similar machine specs and higher load than me and said they did not have
> > load
> > >problems. I'd love to hear that I am mistaken and that it's just a matter
> > of
> > >too little hardware, but I am skeptical...
> > >
> > > > I'd recommend upgrading to a dual server or perhaps putting in a second
> > > > server with round robin DNS (or if you can do it, a load balancer).
> > >
> > >We've been thinking about a multiple-machine email solution and have been
> > >wondering about architecture. Since SA seems to be the *only* email
> > server
> > >module that causes us grief (even amavisd-new/clamav is nicer to our
> > >machine!!), and although it seems strange not to go with a separate file
> > >server or database server machine (or to otherwise split up SMTP and IMAP,
> > >etc), I am starting to think (as you suggest) that just adding a separate
> > SA
> > >server is going to get us the biggest performance increase. What are
> > >people's opinions and experience setting up separate/multiple SA servers?
> > >Are there any good links for reading about such setups on the wiki or
> > >anywhere else?
> > >
> > > > SA is that CPU intensive, it really is. Maybe try adding RBL's in
> > front
> > > > of the MTA to reduce the number of messages you have to scan, that's
> > > > what we do.
> > >
> > >Ha! Yeah, this message rate is *WITH* something like 10 RBL's in Postfix
> > up
> > >front. W/out that, we'd *really* be drowning. :)
> > >
> > >Many thanks!
>
>
>__________________________________________________
>Do You Yahoo!?
>Tired of spam? Yahoo! Mail has the best spam protection around
>http://mail.yahoo.com
Best Regards,
Jeff Koch, Intersessions
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> On Thu, Oct 28, 2004 at 01:09:57AM -0400, Jeff Koch wrote:
> >
> > We figure that we'd have to reduce the email load on each server by 50%
> in
> > order to use SA 3.0 and thereby need twice as many servers. However,
> we're
> > going to wait until the SA developers take the memory and load issues
> > seriously and fix the problem. Maybe if enough users complain they'll do
> > some high volume production test comparisons of 3.0 with previous
> versions
> > and sort out the problem.
> >
>
> I believe this is an entirely unfair characterization of the
> development team. In all cases where recent memory issues have
> cropped we've worked to resolve them. As for load and speed issues, I
> personally take these very seriously. I would guess I benchmark bayes
> on the average of twice a day. The benchmark pumps 300+ msgs per
> minute through my server, 6.5+ million SQL queries averaging around
> 3200 queries per second on my MySQL server.
>
> If anyone has a reproducible memory or load issue I highly encourage
> you to file a bug so that we can start tracking it down.
Well, my excessive load is reproducable every week day between 8am and 4pm.
:)
I am not entirely ready to pin the blame on the SA developers but I am
running vanilla 3.0.1 and have done every tweak I have found here or on the
wiki all to no avail. I don't really know how to file a "bug" for such a
nebulous problem...
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: spamd still burning CPU in 3.0.1
Posted by Michael Parker <pa...@pobox.com>.
On Thu, Oct 28, 2004 at 01:09:57AM -0400, Jeff Koch wrote:
>
> We figure that we'd have to reduce the email load on each server by 50% in
> order to use SA 3.0 and thereby need twice as many servers. However, we're
> going to wait until the SA developers take the memory and load issues
> seriously and fix the problem. Maybe if enough users complain they'll do
> some high volume production test comparisons of 3.0 with previous versions
> and sort out the problem.
>
I believe this is an entirely unfair characterization of the
development team. In all cases where recent memory issues have
cropped we've worked to resolve them. As for load and speed issues, I
personally take these very seriously. I would guess I benchmark bayes
on the average of twice a day. The benchmark pumps 300+ msgs per
minute through my server, 6.5+ million SQL queries averaging around
3200 queries per second on my MySQL server.
If anyone has a reproducible memory or load issue I highly encourage
you to file a bug so that we can start tracking it down.
Michael
Re: spamd still burning CPU in 3.0.1
Posted by Rick Beebe <ri...@yale.edu>.
> Spamd1 - 4 handle an average of 1.5 million messages per day, 810 per
> minute.
> Each box is configured to a max child of 128, and usually hover around
> 70% cpu idle, and 500 megs of ram free.
Very impressive. I have a single spamd box, running 3.0.1, with four
3gHz Xeons and 4-gigs of memory. It's handling about 150,000 messages a
day with peaks in the 350/minute range. I also have around 70% cpu idle
and 500 megs of memory free most of the time. I've never used any swap.
> However when we tested 3.0 on one of the live spamd boxes, even after
> throttling the max-child down to say 64, the cpu's are pegged, and
> around 500 megs into swap.
>
> Another interesting thing I noticed, when SA 2.63 is set to 128 children
> it only spawns them as needed. SA 3.0 likes to spawn the full number of
> children no matter what!
How many spamd processes are alive (on average) on your 2.63 box? That's
the number that I would choose to startup under 3.0. It's a different
philosophy: 2.6 starts up children as needed, up to the max. So as mail
comes in there is a performance hit to start these processes. 3.0 starts
up however many you ask for at the beginning so that they're already
running and ready to go. Since my spamd box is dedicated, I prefer the
new prefork model.
FWIW, I start up 40 children. Each one claims a memory size of about 43
meg. I'm running the standard rules plus 16 of the SARE rule sets.
--
_______________________________________________________________________
Rick Beebe (203) 785-6416
Manager, Systems & Network Engineering FAX: (203) 785-3481
ITS-Med Production Systems Richard.Beebe@yale.edu
Yale University School of Medicine
Suite 124, 100 Church Street South http://its.med.yale.edu
New Haven, CT 06519
_______________________________________________________________________
Re: spamd still burning CPU in 3.0.1
Posted by Ryan Pavely <pa...@nac.net>.
We have a nice e-mail setup with 5 inbound mx boxes (Qmail +
QmailScanner + ClamD), 4 spamd boxes, 2 outbound smtp, 1 imap/pop
server, and a pq (problem queue) box that mx can re-route mail to if
there is a customer issue.
Every box is a Dual CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2399.33-MHz
686-class CPU) w/ 2-4 gigs of ram.
Running FreeBSD 5.2.1
Our spamd boxes are running SA 2.63. We created a spamd-beta box
running 3.0 for a few e-mail boxes and LOVE the upgrades.
Spamd1 - 4 handle an average of 1.5 million messages per day, 810 per
minute.
Each box is configured to a max child of 128, and usually hover around
70% cpu idle, and 500 megs of ram free.
However when we tested 3.0 on one of the live spamd boxes, even after
throttling the max-child down to say 64, the cpu's are pegged, and
around 500 megs into swap.
Another interesting thing I noticed, when SA 2.63 is set to 128 children
it only spawns them as needed. SA 3.0 likes to spawn the full number of
children no matter what!
Do I stay with 2.63 (which is behind the times these days and misses too
much spam)
Do I add more machines?
Do I wait for some memory/cpu improvements in some future version of SA.
Ryan Pavely
Director Research And Development
Net Access Corporation
Jeff Koch wrote:
>
> We have two production mailservers running SA spamd. The first handles
> about 5,000 incoming emails per hour, does spam filtering with SA and
> virus filtering with qmailscanner and forwards the filtered mail to a
> server handling the pop accounts. We're using SA 2.64 with Bayes, AWL,
> Razor and about half of the RBL's. The machine is a 2.8Ghz P4 with
> 1.0GB RAM and SCSI hard drive. CPU usuage runs between 25-40% and
> system load runs 1.50 to 2.20 with isolated spikes to 7.0.
>
> The second machine is a 2Ghz Athlon with 1.0GB RAM and an IDE drive.
> It does spam and virus filtering with SA 2.64 and qmailscanner and
> also handles POP3 sessions with vpopmail. We use Bayes, AWL, Razor and
> the same RBL's. It handles approx 2,500 emails per hour (with peaks of
> 5K emails/hour) and approx 2,000 pop3 sessions per hour (peaks of 5K
> pops/hour). CPU usage runs about 20% with peaks to 50% and system load
> averages 0.80 with peaks of 16.0.
>
> We are pretty satisfied with the above setup. We tried moving one of
> the servers to SA 3.0 in order to use the new MySQL Bayes features but
> got absolutely killed on CPU usage and system load - that lasted about
> a day and we reverted to 2.64.
>
> We figure that we'd have to reduce the email load on each server by
> 50% in order to use SA 3.0 and thereby need twice as many servers.
> However, we're going to wait until the SA developers take the memory
> and load issues seriously and fix the problem. Maybe if enough users
> complain they'll do some high volume production test comparisons of
> 3.0 with previous versions and sort out the problem.
>
>
>
>
>
>
> At 09:33 PM 10/27/2004, email builder wrote:
>
>> > email builder wrote:
>> > >>email builder wrote:
>> > >>How much email are you processing ?
>> > >
>> > >
>> > > Well, just the other day we had an average of 48 msgs/min (max
>> 255/min)
>> > get
>> > > run
>> > > through SA. Can't say today yet because can't run our stats
>> tools until
>> > the
>> > > busy hours are over cuz SA is hogging the CPU. ;)
>> >
>> > Hi,
>> >
>> > Your CPU is over loaded. At 48 a minute it should run just ok on a
>> 2.8
>> > Ghz machine, much over that it's going to start having problems.
>> On our
>> > 2.4 Ghz (not HT) processor if I process over 35 a minute I start
>> having
>> > problems with load.
>>
>> I have two reactions to this:
>>
>> 1) I like the glimmer of hope and the idea that throwing hardware at the
>> problem can solve it
>>
>> 2) Throwing hardware at problems is usually avoiding fixing the *real*
>> problem. According to other posters on this list, my load is not
>> excessive
>> for a modern-day 2.xGHz machine. I will have to re-read some
>> messages, but I
>> believe responders to my posts on the "[OT] Email Servers" thread quoted
>> similar machine specs and higher load than me and said they did not
>> have load
>> problems. I'd love to hear that I am mistaken and that it's just a
>> matter of
>> too little hardware, but I am skeptical...
>>
>> > I'd recommend upgrading to a dual server or perhaps putting in a
>> second
>> > server with round robin DNS (or if you can do it, a load balancer).
>>
>> We've been thinking about a multiple-machine email solution and have
>> been
>> wondering about architecture. Since SA seems to be the *only* email
>> server
>> module that causes us grief (even amavisd-new/clamav is nicer to our
>> machine!!), and although it seems strange not to go with a separate file
>> server or database server machine (or to otherwise split up SMTP and
>> IMAP,
>> etc), I am starting to think (as you suggest) that just adding a
>> separate SA
>> server is going to get us the biggest performance increase. What are
>> people's opinions and experience setting up separate/multiple SA
>> servers?
>> Are there any good links for reading about such setups on the wiki or
>> anywhere else?
>>
>> > SA is that CPU intensive, it really is. Maybe try adding RBL's in
>> front
>> > of the MTA to reduce the number of messages you have to scan, that's
>> > what we do.
>>
>> Ha! Yeah, this message rate is *WITH* something like 10 RBL's in
>> Postfix up
>> front. W/out that, we'd *really* be drowning. :)
>>
>> Many thanks!
>>
>>
>>
>> __________________________________
>> Do you Yahoo!?
>> Yahoo! Mail Address AutoComplete - You start. We finish.
>> http://promotions.yahoo.com/new_mail
>
>
> Best Regards,
>
> Jeff Koch
Re: spamd still burning CPU in 3.0.1
Posted by Jeff Koch <je...@intersessions.com>.
To clarify - the first server handles 700 domains and the second 250. The
first is only handling virus and spam filtering for incoming email while
the second is doing that plus pop3 and outgoing mail.
The first is also SCSI which seems to help alot - especially for qmail. Oh,
also in both machines we use MySQL per-user spam preferences which puts
another big load on the servers.
At 04:36 AM 10/28/2004, John Andersen wrote:
>On Thursday 28 October 2004 12:18 am, email builder wrote:
> > > We have two production mailservers running SA spamd. The first handles
> > > about 5,000 incoming emails per hour, does spam filtering with SA and
> > > virus
> >
> > Can I ask you how you load balance between the two machines (obviously if
> > one handles 5000/hr and the other 2,500, it's not straight round robin)?
>
>I saw nothing in his post to suggest he balanced load, or even that the
>two servers were serving the same domains.
>
>I just took it at face value that with 3.0.1 they couldn't keep up, but
>falling back to 2.64 he could carry the load.
>
>--
>_____________________________________
>John Andersen
Best Regards,
Jeff Koch, Intersessions
Re: spamd still burning CPU in 3.0.1
Posted by John Andersen <js...@pen.homeip.net>.
On Thursday 28 October 2004 12:18 am, email builder wrote:
> > We have two production mailservers running SA spamd. The first handles
> > about 5,000 incoming emails per hour, does spam filtering with SA and
> > virus
>
> Can I ask you how you load balance between the two machines (obviously if
> one handles 5000/hr and the other 2,500, it's not straight round robin)?
I saw nothing in his post to suggest he balanced load, or even that the
two servers were serving the same domains.
I just took it at face value that with 3.0.1 they couldn't keep up, but
falling back to 2.64 he could carry the load.
--
_____________________________________
John Andersen
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> We have two production mailservers running SA spamd. The first handles
> about 5,000 incoming emails per hour, does spam filtering with SA and virus
Can I ask you how you load balance between the two machines (obviously if one
handles 5000/hr and the other 2,500, it's not straight round robin)? I am
not sure I understand how to fork requests between more than one machine when
calling it through spamc.... (I don't understand how to configure this) ???
> filtering with qmailscanner and forwards the filtered mail to a server
> handling the pop accounts. We're using SA 2.64 with Bayes, AWL, Razor and
> about half of the RBL's. The machine is a 2.8Ghz P4 with 1.0GB RAM and SCSI
>
> hard drive. CPU usuage runs between 25-40% and system load runs 1.50 to
> 2.20 with isolated spikes to 7.0.
>
> The second machine is a 2Ghz Athlon with 1.0GB RAM and an IDE drive. It
> does spam and virus filtering with SA 2.64 and qmailscanner and also
> handles POP3 sessions with vpopmail. We use Bayes, AWL, Razor and the same
> RBL's. It handles approx 2,500 emails per hour (with peaks of 5K
> emails/hour) and approx 2,000 pop3 sessions per hour (peaks of 5K
> pops/hour). CPU usage runs about 20% with peaks to 50% and system load
> averages 0.80 with peaks of 16.0.
>
> We are pretty satisfied with the above setup. We tried moving one of the
> servers to SA 3.0 in order to use the new MySQL Bayes features but got
> absolutely killed on CPU usage and system load - that lasted about a day
> and we reverted to 2.64.
Wow, OK, I am starting to think I am actually not insane and that my system
really isn't b0rked and is merely at its performance limit. Thanks Rick and
Jeff for the reality check!
> We figure that we'd have to reduce the email load on each server by 50% in
> order to use SA 3.0 and thereby need twice as many servers. However, we're
> going to wait until the SA developers take the memory and load issues
> seriously and fix the problem. Maybe if enough users complain they'll do
> some high volume production test comparisons of 3.0 with previous versions
> and sort out the problem.
This is disturbing. I'm surprised the CPU thing has not been a topic of
conversation (I see the memory one is).... does anyone know if the developers
are looking at this at all???
Thanks again!
> At 09:33 PM 10/27/2004, email builder wrote:
> > > email builder wrote:
> > > >>email builder wrote:
> > > >>How much email are you processing ?
> > > >
> > > >
> > > > Well, just the other day we had an average of 48 msgs/min (max
> 255/min)
> > > get
> > > > run
> > > > through SA. Can't say today yet because can't run our stats tools
> until
> > > the
> > > > busy hours are over cuz SA is hogging the CPU. ;)
> > >
> > > Hi,
> > >
> > > Your CPU is over loaded. At 48 a minute it should run just ok on a 2.8
> > > Ghz machine, much over that it's going to start having problems. On
> our
> > > 2.4 Ghz (not HT) processor if I process over 35 a minute I start having
> > > problems with load.
> >
> >I have two reactions to this:
> >
> >1) I like the glimmer of hope and the idea that throwing hardware at the
> >problem can solve it
> >
> >2) Throwing hardware at problems is usually avoiding fixing the *real*
> >problem. According to other posters on this list, my load is not
> excessive
> >for a modern-day 2.xGHz machine. I will have to re-read some messages,
> but I
> >believe responders to my posts on the "[OT] Email Servers" thread quoted
> >similar machine specs and higher load than me and said they did not have
> load
> >problems. I'd love to hear that I am mistaken and that it's just a matter
> of
> >too little hardware, but I am skeptical...
> >
> > > I'd recommend upgrading to a dual server or perhaps putting in a second
> > > server with round robin DNS (or if you can do it, a load balancer).
> >
> >We've been thinking about a multiple-machine email solution and have been
> >wondering about architecture. Since SA seems to be the *only* email
> server
> >module that causes us grief (even amavisd-new/clamav is nicer to our
> >machine!!), and although it seems strange not to go with a separate file
> >server or database server machine (or to otherwise split up SMTP and IMAP,
> >etc), I am starting to think (as you suggest) that just adding a separate
> SA
> >server is going to get us the biggest performance increase. What are
> >people's opinions and experience setting up separate/multiple SA servers?
> >Are there any good links for reading about such setups on the wiki or
> >anywhere else?
> >
> > > SA is that CPU intensive, it really is. Maybe try adding RBL's in
> front
> > > of the MTA to reduce the number of messages you have to scan, that's
> > > what we do.
> >
> >Ha! Yeah, this message rate is *WITH* something like 10 RBL's in Postfix
> up
> >front. W/out that, we'd *really* be drowning. :)
> >
> >Many thanks!
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: spamd still burning CPU in 3.0.1
Posted by Jeff Koch <je...@intersessions.com>.
We have two production mailservers running SA spamd. The first handles
about 5,000 incoming emails per hour, does spam filtering with SA and virus
filtering with qmailscanner and forwards the filtered mail to a server
handling the pop accounts. We're using SA 2.64 with Bayes, AWL, Razor and
about half of the RBL's. The machine is a 2.8Ghz P4 with 1.0GB RAM and SCSI
hard drive. CPU usuage runs between 25-40% and system load runs 1.50 to
2.20 with isolated spikes to 7.0.
The second machine is a 2Ghz Athlon with 1.0GB RAM and an IDE drive. It
does spam and virus filtering with SA 2.64 and qmailscanner and also
handles POP3 sessions with vpopmail. We use Bayes, AWL, Razor and the same
RBL's. It handles approx 2,500 emails per hour (with peaks of 5K
emails/hour) and approx 2,000 pop3 sessions per hour (peaks of 5K
pops/hour). CPU usage runs about 20% with peaks to 50% and system load
averages 0.80 with peaks of 16.0.
We are pretty satisfied with the above setup. We tried moving one of the
servers to SA 3.0 in order to use the new MySQL Bayes features but got
absolutely killed on CPU usage and system load - that lasted about a day
and we reverted to 2.64.
We figure that we'd have to reduce the email load on each server by 50% in
order to use SA 3.0 and thereby need twice as many servers. However, we're
going to wait until the SA developers take the memory and load issues
seriously and fix the problem. Maybe if enough users complain they'll do
some high volume production test comparisons of 3.0 with previous versions
and sort out the problem.
At 09:33 PM 10/27/2004, email builder wrote:
> > email builder wrote:
> > >>email builder wrote:
> > >>How much email are you processing ?
> > >
> > >
> > > Well, just the other day we had an average of 48 msgs/min (max 255/min)
> > get
> > > run
> > > through SA. Can't say today yet because can't run our stats tools until
> > the
> > > busy hours are over cuz SA is hogging the CPU. ;)
> >
> > Hi,
> >
> > Your CPU is over loaded. At 48 a minute it should run just ok on a 2.8
> > Ghz machine, much over that it's going to start having problems. On our
> > 2.4 Ghz (not HT) processor if I process over 35 a minute I start having
> > problems with load.
>
>I have two reactions to this:
>
>1) I like the glimmer of hope and the idea that throwing hardware at the
>problem can solve it
>
>2) Throwing hardware at problems is usually avoiding fixing the *real*
>problem. According to other posters on this list, my load is not excessive
>for a modern-day 2.xGHz machine. I will have to re-read some messages, but I
>believe responders to my posts on the "[OT] Email Servers" thread quoted
>similar machine specs and higher load than me and said they did not have load
>problems. I'd love to hear that I am mistaken and that it's just a matter of
>too little hardware, but I am skeptical...
>
> > I'd recommend upgrading to a dual server or perhaps putting in a second
> > server with round robin DNS (or if you can do it, a load balancer).
>
>We've been thinking about a multiple-machine email solution and have been
>wondering about architecture. Since SA seems to be the *only* email server
>module that causes us grief (even amavisd-new/clamav is nicer to our
>machine!!), and although it seems strange not to go with a separate file
>server or database server machine (or to otherwise split up SMTP and IMAP,
>etc), I am starting to think (as you suggest) that just adding a separate SA
>server is going to get us the biggest performance increase. What are
>people's opinions and experience setting up separate/multiple SA servers?
>Are there any good links for reading about such setups on the wiki or
>anywhere else?
>
> > SA is that CPU intensive, it really is. Maybe try adding RBL's in front
> > of the MTA to reduce the number of messages you have to scan, that's
> > what we do.
>
>Ha! Yeah, this message rate is *WITH* something like 10 RBL's in Postfix up
>front. W/out that, we'd *really* be drowning. :)
>
>Many thanks!
>
>
>
>__________________________________
>Do you Yahoo!?
>Yahoo! Mail Address AutoComplete - You start. We finish.
>http://promotions.yahoo.com/new_mail
Best Regards,
Jeff Koch
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
All,
> email builder wrote:
> >
> > >How much email are you processing ?
> >
> > Well, just the other day we had an average of 48 msgs/min (max 255/min)
> > get run through SA. Can't say today yet because can't run our stats
> > tools until the busy hours are over cuz SA is hogging the CPU. ;)
>
> Hi,
>
> Your CPU is over loaded. At 48 a minute it should run just ok on a 2.8
> Ghz machine, much over that it's going to start having problems. On our
> 2.4 Ghz (not HT) processor if I process over 35 a minute I start having
> problems with load.
>
> I'd recommend upgrading to a dual server or perhaps putting in a second
> server with round robin DNS (or if you can do it, a load balancer).
>
> SA is that CPU intensive, it really is. Maybe try adding RBL's in front
> of the MTA to reduce the number of messages you have to scan, that's
> what we do.
>
> Regards,
>
> Rick
Just to top off this thread, I wanted to let all the wonderful people who
offered their system stats/specs know that we added a 2nd machine that is a
dedicated SA server where the only other app running is MySQL (for Bayes/AWL)
and things are humming along very nicely. One server apparently just
couldn't handle the load we had.
Thanks again all!
__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail
Re: spamd still burning CPU in 3.0.1
Posted by Mike Brodbelt <m....@acu.ac.uk>.
email builder wrote:
>>I'd recommend upgrading to a dual server or perhaps putting in a second
>>server with round robin DNS (or if you can do it, a load balancer).
>
>
> also, what do people think about a multiple cpu machine vs more than one
> machine? dumb question? (two machines always are faster than one dual-cpu machine)
I tend to prefer dual CPU machines as servers. They have the large
advantage that if some badly behaved process wedges the processor, you
just get one CPU running flat out, and the machine remains responsive.
Now you can ameliorate this sort of thing with ulimits and such like,
but nothing beats having a second CPU in the box, IMO. It also gives you
redundancy in case of failure, as most dual CPU machines will run fine
on 1 CPU for a while.
Regarding your problems, I'm sure there must be somethin wrong - SA
should not be that CPU intensive. I'm running it on a dual CPU Athlon
2000, using spamd via a sendmail milter. CPU usage for the individula
spamd child processes never seems to go much above about 8-9%, and even
that is momentary. After a 65 day uptime, some of my spamd processes
(quickly pasted from top) look more or less like this:-
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
11257 spamd 9 0 21200 17M 5144 S 0.0 1.6 0:18 spamd
4965 spamd 9 0 31632 27M 5096 S 0.0 2.6 0:12 spamd
29212 spamd 9 0 21792 17M 5060 S 0.0 1.7 0:11 spamd
16522 spamd 9 0 20816 16M 5040 S 0.0 1.6 0:07 spamd
The ps listing showing command line used and stuff:-
30559 ? S 0:02 /usr/sbin/spamd -x -u spamd -m 10 -d
--pidfile=/var/run/spamassassin/spamd.pid
10873 ? S 0:23 \_ spamd child
11257 ? S 0:18 \_ spamd child
29212 ? S 0:11 \_ spamd child
4965 ? S 0:12 \_ spamd child
16522 ? S 0:07 \_ spamd child
16724 ? S 0:06 \_ spamd child
24921 ? S 0:03 \_ spamd child
25813 ? S 0:03 \_ spamd child
29211 ? S 0:01 \_ spamd child
29899 ? S 0:00 \_ spamd child
As you can see, they're hardly a serious drain. Now my system is much
less busy - throughput generally sits around 5-10 messages/minute, and
the system serves only about 50 users. Even so, what you're seeing looks
excessive to my eyes. I'm using Debian Woody, with SpamAssassin 3.0.0
from backports.org.
HTH,
Mike.
Re: spamd still burning CPU in 3.0.1
Posted by jdow <jd...@earthlink.net>.
Ayup modulo a typu I do. Don't forget I am an old troglodyte
paleocomputer type who is quite contented with a few remote ssh
logins to 60 line command line sessions. That's MUCH lighter weight
than playing with X. I do have some sanity left, ya know. Here's a
piece of top at the moment - yeah 256m is closer to the point:
=======8<======
18:28:31 up 22 days, 9:16, 7 users, load average: 1.40, 2.73, 2.75
78 processes: 77 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 1.9% user 1.9% system 0.0% nice 0.0% iowait 96.1% idle
Mem: 255776k av, 150048k used, 105728k free, 0k shrd, 29424k
buff
74864k actv, 27032k in_d, 3176k in_c
Swap: 1161168k av, 69772k used, 1091396k free 46476k
cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
2716 jdow 16 0 1168 1168 852 R 2.3 0.4 0:00 0 top
18700 root 18 0 240 4 0 S 1.5 0.0 0:31 0 sshd
1 root 15 0 104 80 56 S 0.0 0.0 0:28 0 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 keventd
3 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kapmd
4 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0
ksoftirqd_CPU
9 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 bdflush
5 root 15 0 0 0 0 SW 0.0 0.0 1:06 0 kswapd
6 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kscand/DMA
7 root 15 0 0 0 0 SW 0.0 0.0 21:03 0
kscand/Normal
8 root 15 0 0 0 0 SW 0.0 0.0 0:00 0
kscand/HighMe
10 root 15 0 0 0 0 SW 0.0 0.0 0:07 0 kupdated
....
=======9<>8======
(oh look, the scissors are mating! I wonder if the babies are paperclips
or wire coathangers....)
Of course, keeping Linux up these days is nothing like the 2.0.36 days
when I had a machine up for about 460 days before I took it down to
put in a second NIC card when we got DSL service here.
The machine WILL run X. It is "a trifle slow." The same machine will
probably run XP, too. It'll even be faster than the slowest XP machine
I have seen. But I'd be WAY to impatient to use it.
Of course, there is a fellow who put a functional web browser into a
Commodore 64. So while spamd might be out of the question in 256k
bringing up a capable system in 256k should be easy, lots of room.
Haven't you ever run across the HTTP server in a minipic and one
serial memory chip?
{^_-} Joanne.
----- Original Message -----
From: "einheit elf" <ei...@mainphrame.com>
To: <us...@spamassassin.apache.org>
Sent: 2004 October, 28, Thursday 15:37
Subject: Re: spamd still burning CPU in 3.0.1
> Jdow, think about what you're saying - do you really expect us to believe
> that you could even boot redhat 9, let alone do anything useful, in 256 k
> of RAM? I was able to bring up a slackware machine with 4 MB RAM, and even
> run a web server (slowly), but that's about the practical limit, and you
> claim to be running SA in 1/16th that amount of RAM?
>
> ;)
>
> jdow said:
> > Redhat 9 does. It's rather slow. But it does get there. It's for two
users
> > only. But we're both in the 1000+ emails a day class users.
> >
> > {^_^}
> > ----- Original Message -----
> > From: "einheit elf" <ei...@mainphrame.com>
> >>
> >> jdow said:
> >>
> >> > I normally run SpamAssassin (2.63) on a slow machine, a
> >> > 166MHz Pentium with only 256k of ram.
> >>
> >> Pray tell, what OS enables you to run SA with only a quarter Meg of
RAM?
> >>
> >> einheit
> >>
> >
> >
>
>
> --
>
Re: spamd still burning CPU in 3.0.1
Posted by einheit elf <ei...@mainphrame.com>.
Jdow, think about what you're saying - do you really expect us to believe
that you could even boot redhat 9, let alone do anything useful, in 256 k
of RAM? I was able to bring up a slackware machine with 4 MB RAM, and even
run a web server (slowly), but that's about the practical limit, and you
claim to be running SA in 1/16th that amount of RAM?
;)
jdow said:
> Redhat 9 does. It's rather slow. But it does get there. It's for two users
> only. But we're both in the 1000+ emails a day class users.
>
> {^_^}
> ----- Original Message -----
> From: "einheit elf" <ei...@mainphrame.com>
>>
>> jdow said:
>>
>> > I normally run SpamAssassin (2.63) on a slow machine, a
>> > 166MHz Pentium with only 256k of ram.
>>
>> Pray tell, what OS enables you to run SA with only a quarter Meg of RAM?
>>
>> einheit
>>
>
>
--
Re: spamd still burning CPU in 3.0.1
Posted by jdow <jd...@earthlink.net>.
Redhat 9 does. It's rather slow. But it does get there. It's for two users
only. But we're both in the 1000+ emails a day class users.
{^_^}
----- Original Message -----
From: "einheit elf" <ei...@mainphrame.com>
>
> jdow said:
>
> > I normally run SpamAssassin (2.63) on a slow machine, a
> > 166MHz Pentium with only 256k of ram.
>
> Pray tell, what OS enables you to run SA with only a quarter Meg of RAM?
>
> einheit
>
Re: [OT] spamd still burning CPU in 3.0.1
Posted by Rick Macdougall <ri...@nougen.com>.
einheit elf wrote:
> jdow said:
>
>
>>I normally run SpamAssassin (2.63) on a slow machine, a
>>166MHz Pentium with only 256k of ram.
>
>
> Pray tell, what OS enables you to run SA with only a quarter Meg of RAM?
ReallyTinyOS ?
Re: spamd still burning CPU in 3.0.1
Posted by einheit elf <ei...@mainphrame.com>.
jdow said:
> I normally run SpamAssassin (2.63) on a slow machine, a
> 166MHz Pentium with only 256k of ram.
Pray tell, what OS enables you to run SA with only a quarter Meg of RAM?
einheit
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> Out of addled curiosity (not pointing specifically at you David) why has
> nobody mentioned the traditional "SpamAssassin is slow" mantra, "Try
> more memory?"
I think because memory does not seem to be an issue for me. I have 1GB RAM
and each spamd process sits at around 34MB. I don't have any swapping at
all. I could throw another gig in, but that doesn't really seem to be the
problem from what I can tell. Am I overlooking something?
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: spamd still burning CPU in 3.0.1
Posted by jdow <jd...@earthlink.net>.
From: "David Brodbeck" <gu...@gull.us>
> On Wed, 27 Oct 2004 21:43:16 -0700 (PDT), email builder wrote
> > Thanks. We were thinking about a NFS server, but SA concerns seemed
> > more important. If both can coexist peacefully, this may be the
> > exact same solution that we use.
>
> It seems like it'd be a good match. NFS is highly I/O intensive, but
doesn't
> use the CPU much. SA is highly CPU intensive, but does relatively little
I/O.
> Just make sure you have enough memory; you don't want SA's hunger for RAM
to
> starve the machine for disk cache space, and you sure as heck don't want
any
> swapping.
Out of addled curiosity (not pointing specifically at you David) why has
nobody mentioned the traditional "SpamAssassin is slow" mantra, "Try
more memory?" I normally run SpamAssassin (2.63) on a slow machine, a
166MHz Pentium with only 256k of ram. I also have a test install, pending
retiring the old machine, that is a 2GHz Athlon with 1 gig of ram. The
newer machine appears to run much faster than the ratio of the CPU clock
speeds would allow. Usually I get much less than a 1:1 speed improvement
when upgrading the CPU, based on past observations. (Kernel compiles are
not NEARLY 13 times as fast on the newer machine, for example.) I
attribute much of the difference to having a massive overload of memory
on the newer machine.
Now, it is a problem if a spamd balloons to 100megs or more and stays
there. It is not particularly a problem if 3.0.1 uses twice or three
times the memory of 2.63. Throw more memory at it. Memory is cheaper
than your time spent trying to work around large Bayes files and large
rule sets.
{^_-} Joanne, being controversial again. (And those going to
ApacheCon PLEASE buttonhole the geek who has this list
going through a spam filter. I've had dozens of novel
attacks come through recently. And I can't post them to
the list for the SARE people. For example a modification
of the drug rules is now needed for "m or tga ge" and
"a pp lica ti o n". Spam is food for this list, at least
as attachments.)
Re: spamd still burning CPU in 3.0.1
Posted by David Brodbeck <gu...@gull.us>.
On Wed, 27 Oct 2004 21:43:16 -0700 (PDT), email builder wrote
> Thanks. We were thinking about a NFS server, but SA concerns seemed
> more important. If both can coexist peacefully, this may be the
> exact same solution that we use.
It seems like it'd be a good match. NFS is highly I/O intensive, but doesn't
use the CPU much. SA is highly CPU intensive, but does relatively little I/O.
Just make sure you have enough memory; you don't want SA's hunger for RAM to
starve the machine for disk cache space, and you sure as heck don't want any
swapping.
Re: spamd still burning CPU in 3.0.1
Posted by Gavin Cato <ga...@corp.nexon.com.au>.
> 89 a minute! Wow! What else do you run on that machine? (Do you run your
> other email server software there or is it a dedicated SA box? Do you also
> run a virus scanner for example?)
Hiya,
It runs FreeBSD 4.8 (with SMP kernel of course) and sendmail + SA 3.0.1 -
that's it, nothing else apart from mrtg and some perl scripts I run in cron
to check the sanity of running processes and generate some stats for me. It
used to run SA 2.64, I honestly didn't notice any major increase in system
load when I upgraded to 3.0 (which was less than a fortnight ago).
We "pipe" MX records through it for customer domains that wish to be spam
filtered, it works pretty well.
We have another server which is identical in hardware but instead runs
postfix + ClamAV. This machine handles a *lot* more mail (not everyone wants
their mail piped through our SA server - but they do want it AV checked) and
I actually just had a new machine arrive yesterday that I'll be building up
to replace it, as that box is quite heavily loaded.
I'd actually like to convert our SA server into running postfix, but as it's
already running quite nicely with plain old sendmail, it's not something
that I really have the need to start mucking around with. Maybe in 6-12
months when we have more domains going through it.
The only times I've seen our SA server busy is when there was a network
problem, it didn't receive mail for a while then it suddenly got hammered
hard when the network connectivity was restored, and the queues started
connecting to the box, that's the only time I've wanted more CPU & RAM, that
took a fair while for it to work itself out.
Cheers
Gav
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
--- Gavin Cato <ga...@corp.nexon.com.au> wrote:
> > This is what I don't get. If you can handle an avg of 500/hr, which....
> oh,
> > wait... that's per hour. Ah, OK. That's 8/min. I'm doing an avg of
> 48/min
> > (255/min max). But I swear someone else had a throughput higher than
> that
> > who was not having CPU issues.
> >
> > ANYONE? What kind of throughput are people able to get from SA on a
> single
> > processor machine?
>
> Hi,
>
> I'm averaging 89 a minute on the previously mentioned dual 1.1ghz P3 - I
> know it's not single CPU but to be honest I'd have expected a single 2.8 to
> be faster.
Yikes. Maybe I got too excited (about just adding hardware) too soon. I
agree that a 2.8 (HT no less) should be at least as fast, although 45 is my
average and although it's not as easy to just get an average for peak hours,
I think my daytime average is more like 75 or possibly as many as yours.
89 a minute! Wow! What else do you run on that machine? (Do you run your
other email server software there or is it a dedicated SA box? Do you also
run a virus scanner for example?)
__________________________________
Do you Yahoo!?
Read only the mail you want - Yahoo! Mail SpamGuard.
http://promotions.yahoo.com/new_mail
Re: spamd still burning CPU in 3.0.1
Posted by Gavin Cato <ga...@corp.nexon.com.au>.
> This is what I don't get. If you can handle an avg of 500/hr, which.... oh,
> wait... that's per hour. Ah, OK. That's 8/min. I'm doing an avg of 48/min
> (255/min max). But I swear someone else had a throughput higher than that
> who was not having CPU issues.
>
> ANYONE? What kind of throughput are people able to get from SA on a single
> processor machine?
Hi,
I'm averaging 89 a minute on the previously mentioned dual 1.1ghz P3 - I
know it's not single CPU but to be honest I'd have expected a single 2.8 to
be faster.
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> >>I'd recommend upgrading to a dual server or perhaps putting in a second
> >>server with round robin DNS (or if you can do it, a load balancer).
> >
> >
> > also, what do people think about a multiple cpu machine vs more than one
> > machine? dumb question? (two machines always are faster than one
> dual-cpu machine)
>
> I believe I answered your other post with my machine specs, can't remember.
Yes you did, and your replies are highly appreciated!!
> Anyways, I highly recommend dual cpu with SA. I also highly recommend
> running spamd on another server and letting your mailserver/toaster call
> it with spamc. We run SA on our NFS server which is a Sparc Enterpise,
> 2gb ram, dual cpu, and it is handling all the spam processing for three
> toasters with no problem. It also serves our Maildirs to each toaster.
Thanks. We were thinking about a NFS server, but SA concerns seemed more
important. If both can coexist peacefully, this may be the exact same
solution that we use.
> Even with all the NFS traffic load is under 5 on a very busy afternoon.
> My max this month was 3200 messages per hour according to qmail-mrtg. My
> avg is around 500 messages per hour.
This is what I don't get. If you can handle an avg of 500/hr, which.... oh,
wait... that's per hour. Ah, OK. That's 8/min. I'm doing an avg of 48/min
(255/min max). But I swear someone else had a throughput higher than that
who was not having CPU issues.
ANYONE? What kind of throughput are people able to get from SA on a single
processor machine?
Does anyone have a gathering of that kind of info? (would be nice to see on
the wiki)
TIA!
__________________________________
Do you Yahoo!?
Read only the mail you want - Yahoo! Mail SpamGuard.
http://promotions.yahoo.com/new_mail
Re: spamd still burning CPU in 3.0.1
Posted by Dave Goodrich <ld...@tls.net>.
email builder wrote:
>>I'd recommend upgrading to a dual server or perhaps putting in a second
>>server with round robin DNS (or if you can do it, a load balancer).
>
>
> also, what do people think about a multiple cpu machine vs more than one
> machine? dumb question? (two machines always are faster than one dual-cpu machine)
I believe I answered your other post with my machine specs, can't remember.
Anyways, I highly recommend dual cpu with SA. I also highly recommend
running spamd on another server and letting your mailserver/toaster call
it with spamc. We run SA on our NFS server which is a Sparc Enterpise,
2gb ram, dual cpu, and it is handling all the spam processing for three
toasters with no problem. It also serves our Maildirs to each toaster.
Even with all the NFS traffic load is under 5 on a very busy afternoon.
My max this month was 3200 messages per hour according to qmail-mrtg. My
avg is around 500 messages per hour.
DAve
--
Systems Administrator
http://www.tls.net
Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> I'd recommend upgrading to a dual server or perhaps putting in a second
> server with round robin DNS (or if you can do it, a load balancer).
also, what do people think about a multiple cpu machine vs more than one
machine? dumb question? (two machines always are faster than one dual-cpu machine)
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> email builder wrote:
> >>email builder wrote:
> >>How much email are you processing ?
> >
> >
> > Well, just the other day we had an average of 48 msgs/min (max 255/min)
> get
> > run
> > through SA. Can't say today yet because can't run our stats tools until
> the
> > busy hours are over cuz SA is hogging the CPU. ;)
>
> Hi,
>
> Your CPU is over loaded. At 48 a minute it should run just ok on a 2.8
> Ghz machine, much over that it's going to start having problems. On our
> 2.4 Ghz (not HT) processor if I process over 35 a minute I start having
> problems with load.
I have two reactions to this:
1) I like the glimmer of hope and the idea that throwing hardware at the
problem can solve it
2) Throwing hardware at problems is usually avoiding fixing the *real*
problem. According to other posters on this list, my load is not excessive
for a modern-day 2.xGHz machine. I will have to re-read some messages, but I
believe responders to my posts on the "[OT] Email Servers" thread quoted
similar machine specs and higher load than me and said they did not have load
problems. I'd love to hear that I am mistaken and that it's just a matter of
too little hardware, but I am skeptical...
> I'd recommend upgrading to a dual server or perhaps putting in a second
> server with round robin DNS (or if you can do it, a load balancer).
We've been thinking about a multiple-machine email solution and have been
wondering about architecture. Since SA seems to be the *only* email server
module that causes us grief (even amavisd-new/clamav is nicer to our
machine!!), and although it seems strange not to go with a separate file
server or database server machine (or to otherwise split up SMTP and IMAP,
etc), I am starting to think (as you suggest) that just adding a separate SA
server is going to get us the biggest performance increase. What are
people's opinions and experience setting up separate/multiple SA servers?
Are there any good links for reading about such setups on the wiki or
anywhere else?
> SA is that CPU intensive, it really is. Maybe try adding RBL's in front
> of the MTA to reduce the number of messages you have to scan, that's
> what we do.
Ha! Yeah, this message rate is *WITH* something like 10 RBL's in Postfix up
front. W/out that, we'd *really* be drowning. :)
Many thanks!
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail
Re: spamd still burning CPU in 3.0.1
Posted by Rick Macdougall <ri...@nougen.com>.
email builder wrote:
>>email builder wrote:
>>How much email are you processing ?
>
>
> Well, just the other day we had an average of 48 msgs/min (max 255/min) get
> run
> through SA. Can't say today yet because can't run our stats tools until the
> busy hours are over cuz SA is hogging the CPU. ;)
Hi,
Your CPU is over loaded. At 48 a minute it should run just ok on a 2.8
Ghz machine, much over that it's going to start having problems. On our
2.4 Ghz (not HT) processor if I process over 35 a minute I start having
problems with load.
I'd recommend upgrading to a dual server or perhaps putting in a second
server with round robin DNS (or if you can do it, a load balancer).
SA is that CPU intensive, it really is. Maybe try adding RBL's in front
of the MTA to reduce the number of messages you have to scan, that's
what we do.
Regards,
Rick
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> email builder wrote:
> >>BTW, SpamAssassin *is* CPU-intensive. It's designed that way ;)
> >
> >
> > But not as CPU intensive as I am seeing. According to others on this
> list, I
> > should not be seeing a mere five spamd children completely dominating a
> > 2.8GHz(HT) processor.
>
> How much email are you processing ?
Well, just the other day we had an average of 48 msgs/min (max 255/min) get
run
through SA. Can't say today yet because can't run our stats tools until the
busy hours are over cuz SA is hogging the CPU. ;)
> Have you tried turning off razor ?
Yes. DCC too.
> Are you running a big local dns cache ?
Nothing unusual. From what I understand, DNS queries should not have any
relationship to CPU usage.
> Is the machine swapping at all ?
Not a bit. We definitely have some process blockage, but I presume that is
because the CPU is being overused.
Thanks!
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail
Re: spamd still burning CPU in 3.0.1
Posted by Rick Macdougall <ri...@nougen.com>.
email builder wrote:
>>BTW, SpamAssassin *is* CPU-intensive. It's designed that way ;)
>
>
> But not as CPU intensive as I am seeing. According to others on this list, I
> should not be seeing a mere five spamd children completely dominating a
> 2.8GHz(HT) processor.
How much email are you processing ?
Have you tried turning off razor ?
Are you running a big local dns cache ?
Is the machine swapping at all ?
Regards,
Rick
Re: spamd still burning CPU in 3.0.1
Posted by Gavin Cato <ga...@corp.nexon.com.au>.
Definitely not, I'm using SA 3.0.1 on a dual 1.13ghz P3 with 2gb RAM with
SCSI, processing a fair bit of mail.
I have 25 spamd children running, and the load is typically like this ;
> w
9:46AM up 9 days, 13:06, 1 user, load averages: 1.14, 1.46, 1.59
Cheers
Gav
On 28/10/04 9:13 AM, "email builder" <em...@yahoo.com> wrote:
>> BTW, SpamAssassin *is* CPU-intensive. It's designed that way ;)
>
> But not as CPU intensive as I am seeing. According to others on this list, I
> should not be seeing a mere five spamd children completely dominating a
> 2.8GHz(HT) processor.
>
Re: spamd still burning CPU in 3.0.1
Posted by email builder <em...@yahoo.com>.
> BTW, SpamAssassin *is* CPU-intensive. It's designed that way ;)
But not as CPU intensive as I am seeing. According to others on this list, I
should not be seeing a mere five spamd children completely dominating a
2.8GHz(HT) processor.
> Tim B writes:
> > email builder wrote:
> > > I hurried out and installed 3.0.1, thinking one of those
> memory/language
> > > improvements mentioned in the release notes were going to be my
> savior...
> > >
> > > Sadly, 3.0.1's spamd has the same CPU-intensive behavior here. I am
> soooo at
> > > a loss; tried everything I've read... spent days reading... please,
> anyone
> > > have anything more?
> > >
> > > If spamd isn't I/O bound, my memory isn't swapping, I have no other
> processes
> > > that are out of control, I can't for the life of me figure out why this
> is
> > > happening.
> > >
> > > Again, my specs:
> > >
> > > A sample from top:
> > >
> > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> > > 1401 maildrop 16 0 39744 34m 6840 R 28.3 3.4 3:04.18 spamd
> > >
> > > spamd children average around 30% CPU, but even 50% not too unusual.
> > >
> > > load average is around 15 to 18 during the middle of the day
> > >
> > > And this is how I start spamd:
> > >
> > > LANG=en_US; export LANG; TMPDIR=/tmp/spamassassin; export TMPDIR
> > > spamd -d -q -x --max-children=5 -H /etc/razor -u maildrop -r
> > > /var/run/spamd/spamd.pid
> > >
> > > (also tried with -L to no avail)
> > >
> > > /tmp/spamassassin is mounted with tmpfs
> > >
> > > prefs/bayes/awl all in SQL, but bayes/awl not being used right now
> > >
> > > we also run named on the same machine
> > >
> > > if it's important, this is 3.0.1, downloaded and compiled manually (not
> a
> > > CPAN install)
> > >
> > > I have installed no custom rulesets, nothing extra beside whatever
> comes 100%
> > > stock. This is a Fedora Core 2 machine (2.8P-IV hyperthreaded, 1GB
> RAM)
> > >
> > > spamc is called from maildrop as such:
> > >
> > > if ( $SIZE < 262144 )
> > > {
> > > exception {
> > > xfilter "/usr/bin/spamc -u $LOGNAME"
> > > }
> > > }
> > >
> > > (also tried running inside of amavis to no avail)
> > >
> > > Any advice or even just pointers on any more reading I can do would be
> highly
> > > appreciated!
> > >
> > >
> > >>>What in the world is going on? Isn't it true that spamd (beside DCC)
> > >>
> > >>does
> > >>
> > >>>its thing w/out disk I/O? If so, what else could be chewing up so
> much
> > >>
> > >>CPU?
> > >>
> > >>I don't know - The same thing happens to me a couple of times a day,
> and I
> > >>only get about 350 messages per day. Today it was at 12:25p:
> > >>
> > >>12:25:07 4496 511804 99.13 2532 9420 65088
> > >>432884 86.93
> > >>
> > >>12:25:07 0 91 5.47 2.35 0.89 <<<<<<<
> LA
> > >>
> > >>When this happens, the HDD is constantly active. I'm using v2.64 with
> > >>network checks. The load average for the 21 hrs of this day is about
> 0.1
> > >
> > >
> > >
> > >
> > >
> > > __________________________________
> > > Do you Yahoo!?
> > > Yahoo! Mail - Helps protect you from nasty viruses.
> > > http://promotions.yahoo.com/new_mail
> > >
> >
> > Have you tried doing a force-expire on your bayes db?
> >
> > I found this helped me. Disabling autoexpire, and twice a day running
> > sa-learn --force-expire
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
> Comment: Exmh CVS
>
> iD8DBQFBerAlMJF5cimLx9ARAkSzAJ4ziHgz6iLos/0Obf7OxcFEBxs3gwCfRCvG
> 3pCNDv79pho0WFpZWnvVDEA=
> =6UpA
> -----END PGP SIGNATURE-----
>
>
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail