critical spamc/spamd problem on high volume implementation

[Matthew R Anderson <MA...@iddx.net>]

Greetings-

I'm working on a mailserver with spamassassin installed sitewide.  It
receives some impressive spikes in traffic which seem to go hand in hand
with SA crashing the machine, apparently via libc.

It's a dual PIII 1.26ghz with 2GB of RAM running debian 3.0 with SA 2.63
compiled and installed by hand with no apparent problems in that process.
It's using everything but razor and pyzor at the moment, the bayesean
filtering a must-have.

This machine receives mail via Postfix and has been happily receiving mail
(albeit a lot of junk with SA) for at least 6 months.  It's configured in
master.cf to inject mail into spamassassin directly.

I'm not the quickest with my ninjitsu, but I've been pretty good at solving
problems with this software.  I've tried limiting the spamd child processes
(-m 32) and tinkering with the spamc timeout (-t 5), and nothing at all, and
each time when traffic spikes heavily the machine crashes and requires a
hard reboot.

I'd like some options and/or advice.  Is there a problem with compiling SA
by hand on a debian machine without any compiler tricks?  I can't use the
debian stable spamassassin as its spamc does not include the functionality
to integrate it into the MTA and I don't really like the idea of sending
this much mail through procmail.

I can generate any further specifics as needed.  This is a high priority fix
and I'm getting sick of rebooting this machine! :)  Thanks,

/m

Re: critical spamc/spamd problem on high volume implementation

[Paolo Cravero as2594 <pc...@as2594.net>]

Matthew R Anderson wrote:

Hi Matthew

> It's a dual PIII 1.26ghz with 2GB of RAM running debian 3.0 with SA 2.63
[...]
> This machine receives mail via Postfix and has been happily receiving mail
[...]
> I'm not the quickest with my ninjitsu, but I've been pretty good at solving
> problems with this software.  I've tried limiting the spamd child processes
> (-m 32) and tinkering with the spamc timeout (-t 5), and nothing at all, and
> each time when traffic spikes heavily the machine crashes and requires a
> hard reboot.

On a dual Xeon 2.4GHz here spamd is limited to 10 processes, and I have 
never seen them up at the same time ('ps -ef |grep spamd' reported no 
more than 3 at peak hours).

What is the average message process time? (there's a tool to generate a 
simple report)

What is the inbound traffic (msg/h or peak)?

Did you limit the size of messages to be processed by spamd? We have 250k

> I can generate any further specifics as needed.  This is a high priority fix
> and I'm getting sick of rebooting this machine! :)  Thanks,

What is the kernel of your Debian? 2.4.18 by any chance?

Depending upon your incoming traffic you can tweak Postfix's maximum 
inbound SMTP connections in master.cf.

HTH,
Paolo
(online 7-15z / 8-17 Europe -)