Load Balancing with Postfix [and SpamAssassin]

[Alan Fullmer <li...@xnote.com>]

Greetings, 

I am wondering if anyone can direct me to any books/sites/etc regarding
possibly running postfix on one or more servers accepting mail, processing
with SA, then forwarding onto a 3rd machine.

Currently my setup only includes one Postfix box, which does all of the mail
filtering (with spamassassin) then forwards it on to a mailbox holding
server.

The setup works, however I get so backlogged.  For example, running a mailq
as we speak, comes up with:
35918 Kbytes in 5257 Requests.

It eventually gets through, and during the night it catches up.  I am seeing
delays up to 3 hours sometimes.   I tail -f the maillog file and it's
constantly running, so there is definitely activity.   Running top indicates
4 main processes with spamd indicating the slowdown is, indeed, with
SpamAssassin.  I originally thought it might have something to do with the
Bayes database, but for fun, I deleted it to see what would happen.  Same
result.

I am running a dual P3 1ghz machine currently and have plans to upgrade that
to a dual xeon 3ghz.  However, I know this is only a temporary solution and
at this point I'm not sure that it's enough.  

I get an enormous amount of mail coming into this machine and need to plan
accordingly for future growth as well.

So my goal is to possibly balance the incoming mail between two beefy
machines to do the processing, hopefully accomplishing and eliminating this
slow down.

My question is any suggestions on a different way to do this?  Or are there
suggestions on accomplishing a more balanced load between two machines.

Any comments, helpful hints or suggestions would be more than welcome.

Thanks,

Alan Fullmer
www.xnote.com
www.zoobuh.com

Re: Load Balancing with Postfix [and SpamAssassin]

[Mike Jackson <mj...@barking-dog.net>]

> I am wondering if anyone can direct me to any books/sites/etc regarding
> possibly running postfix on one or more servers accepting mail, processing
> with SA, then forwarding onto a 3rd machine.
>
> Currently my setup only includes one Postfix box, which does all of the 
> mail
> filtering (with spamassassin) then forwards it on to a mailbox holding
> server.
>
> The setup works, however I get so backlogged.  For example, running a 
> mailq
> as we speak, comes up with:
> 35918 Kbytes in 5257 Requests.
>
> It eventually gets through, and during the night it catches up.  I am 
> seeing
> delays up to 3 hours sometimes.   I tail -f the maillog file and it's
> constantly running, so there is definitely activity.   Running top 
> indicates
> 4 main processes with spamd indicating the slowdown is, indeed, with
> SpamAssassin.  I originally thought it might have something to do with the
> Bayes database, but for fun, I deleted it to see what would happen.  Same
> result.
>
> I am running a dual P3 1ghz machine currently and have plans to upgrade 
> that
> to a dual xeon 3ghz.  However, I know this is only a temporary solution 
> and
> at this point I'm not sure that it's enough.
>
> I get an enormous amount of mail coming into this machine and need to plan
> accordingly for future growth as well.
>
> So my goal is to possibly balance the incoming mail between two beefy
> machines to do the processing, hopefully accomplishing and eliminating 
> this
> slow down.
>
> My question is any suggestions on a different way to do this?  Or are 
> there
> suggestions on accomplishing a more balanced load between two machines.
>
> Any comments, helpful hints or suggestions would be more than welcome.

My experience comes from Sendmail, not Postfix, invoking SA from procmail, 
but I'm sure the same principles can be applied to your situation.

At one ISP where I worked, we had 5000+ mailboxes and rather older 
equipment. We had all the mail come into one server, then call SA via spamc 
and connect remotely to spam.example.com, where there were multiple IPs for 
spam.example.com. spamc will round-robin through them, then give up if a 
host is not available. You can throw as many machines as you want into the 
round-robin queue. That was back in the SA 2.5, 2.6 days; now with 3.1 and 
SQL-based bayes and AWL, it's much more viable (we had to forgo those 
features, though we did use SQL-based user preferences). You could, say, run 
MySQL on one of your round-robin hosts, then have the rest refer to it. Have 
all the round-robin machines start up and keep idle all the spamd children 
they can possibly run. Add more machines as necessary.