Distributing the processing load

[Micah Anderson <mi...@riseup.net>]

Our poor spamassassin machine is not able to keep up with the mail
load. We are constantly getting "prefork: server reached --max-children
setting, consider raising it" errors, and our max-children are already
set at the max that this machine can handle (50). 

Since we are using spamc/spamd I figured that it would be trivial to
setup a second spamd on another machine and then the load could be
split. I accomplished this by setting my mailfilter to use '-d spamd'
and configured the spamd host in my DNS to be a round-robin between the
two participating IPs. However, this seems to only work as a
'fail-over', and not a load-balancer, as the spamc man page says:

           If host resolves to multiple addresses, then spamc will
           fail-over to the other addresses, if the first one cannot be
           connected to.  It will first try all addresses of one host
           before it tries the next one in the list.  

In fact, looking at my logs, one of the spamd machines is only
processing requests for one of the three mail servers, the other
requests are going to the other spamd. Likely this is because they all
looked up the address, and then have it cached?

I am using -x, and the man page says that the fail-over behaviour is
incompatible with -x; if that switch is used, fail-over will not occur.
Thats fine, I'm not particularly interested in fail-over, but rather
load-balancing, is there any way to do this without having to setup my
different mail servers to query different spamds?

Thanks for any ideas,
micah

Re: Distributing the processing load

[Rick Macdougall <ri...@ummm-beer.com>]

Micah Anderson wrote:
> Our poor spamassassin machine is not able to keep up with the mail
> load. We are constantly getting "prefork: server reached --max-children
> setting, consider raising it" errors, and our max-children are already
> set at the max that this machine can handle (50). 
> 
> Since we are using spamc/spamd I figured that it would be trivial to
> setup a second spamd on another machine and then the load could be
> split. I accomplished this by setting my mailfilter to use '-d spamd'
> and configured the spamd host in my DNS to be a round-robin between the
> two participating IPs. However, this seems to only work as a
> 'fail-over', and not a load-balancer, as the spamc man page says:

Hi,

I use -d server1,server2,server3 and this seems to work for us.

Regards,

Rick

Re: Distributing the processing load

[Matt Kettler <mk...@verizon.net>]

Micah Anderson wrote:
> Our poor spamassassin machine is not able to keep up with the mail
> load. We are constantly getting "prefork: server reached --max-children
> setting, consider raising it" errors, and our max-children are already
> set at the max that this machine can handle (50). 
>
> Since we are using spamc/spamd I figured that it would be trivial to
> setup a second spamd on another machine and then the load could be
> split. I accomplished this by setting my mailfilter to use '-d spamd'
> and configured the spamd host in my DNS to be a round-robin between the
> two participating IPs. However, this seems to only work as a
> 'fail-over', and not a load-balancer, as the spamc man page says:
>
>            If host resolves to multiple addresses, then spamc will
>            fail-over to the other addresses, if the first one cannot be
>            connected to.  It will first try all addresses of one host
>            before it tries the next one in the list.  
>
> In fact, looking at my logs, one of the spamd machines is only
> processing requests for one of the three mail servers, the other
> requests are going to the other spamd. Likely this is because they all
> looked up the address, and then have it cached?
>
> I am using -x, and the man page says that the fail-over behaviour is
> incompatible with -x; if that switch is used, fail-over will not occur.
> Thats fine, I'm not particularly interested in fail-over, but rather
> load-balancing, is there any way to do this without having to setup my
> different mail servers to query different spamds?
>From man spamc:

-H, --randomize
    For TCP/IP sockets, randomize the IP addresses returned for the
hosts given by the -d switch. This provides for a simple kind of load
balancing. It will try only three times though.