You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Andy Smith <an...@strugglers.net> on 2007/04/30 03:23:23 UTC
problems using haproxy for spamd
Hi,
I'm trying to use haproxy (http://haproxy.1wt.eu/) to load balance 3
spamd servers on the same network.
Here's my haproxy config:
global
log 127.0.0.1 local0 debug
maxconn 100
ulimit-n 512
uid 999
gid 999
daemon
pidfile /var/run/haproxy-spamd.pid
listen spamd
bind 212.13.194.5:783
mode tcp
option tcplog
log global
balance roundrobin
source 212.13.194.5:0
clitimeout 150000
srvtimeout 150000
contimeout 30000
server corona 212.13.194.122:783 weight 5
server curacao 212.13.194.71:783 weight 5
server islay 212.13.194.96:783 weight 6
Unfortunately I seem to be intermittently getting connection
failures. The haproxy log looks like this:
Apr 28 05:13:49 localhost haproxy[11683]: Proxy spamd started.
Apr 28 05:14:57 localhost haproxy[11684]: 212.13.194.70:55827 [28/Apr/2007:05:14:57] spamd corona 0/0/148 765 -- 0/0/0 0/0
Apr 28 05:14:57 localhost haproxy[11684]: 212.13.194.70:55828 [28/Apr/2007:05:14:57] spamd curacao 0/-1/1 0 CC 0/0/0 0/0
Apr 28 05:16:07 localhost haproxy[11684]: 212.13.194.70:55858 [28/Apr/2007:05:16:07] spamd islay 0/-1/0 0 CC 0/0/0 0/0
Apr 28 05:16:08 localhost haproxy[11684]: 212.13.194.70:55859 [28/Apr/2007:05:16:07] spamd corona 0/0/327 4369 -- 0/0/0 0/0
Apr 28 05:17:04 localhost haproxy[11684]: 212.13.194.70:55863 [28/Apr/2007:05:17:02] spamd curacao 0/0/2419 839 -- 0/0/0 0/0
Apr 28 05:17:04 localhost haproxy[11684]: 212.13.194.70:55864 [28/Apr/2007:05:17:04] spamd islay 0/-1/0 0 CC 0/0/0 0/0
Apr 28 05:25:38 localhost haproxy[11684]: 212.13.194.70:54248 [28/Apr/2007:05:25:37] spamd corona 0/0/492 3930 -- 0/0/0 0/0
Apr 28 05:26:12 localhost haproxy[11684]: 212.13.194.70:54254 [28/Apr/2007:05:26:12] spamd islay 0/-1/4 0 CC 0/0/0 0/0
Apr 28 05:26:12 localhost haproxy[11684]: 212.13.194.70:54255 [28/Apr/2007:05:26:12] spamd curacao 0/-1/10 0 CC 0/0/0 0/0
According to http://haproxy.1wt.eu/download/1.2/doc/haproxy-en.txt
state CC means that the client aborted the connection before it
could be passed to any backend server. As you can see above this
does not happen to every connection.
Yet on the connections that aborted with status CC, the server did
actually receive them and deal with them:
Apr 28 05:26:12 islay spamd[861]: spamd: connection from 212.13.194.5 [212.13.194.5] at port 48949
Apr 28 05:26:13 islay spamd[861]: spamd: processing message <20...@dario.dodds.net> aka <5l...@murphy> for Debian-exim:102
Apr 28 05:26:17 islay spamd[861]: spamd: clean message (-2.2/5.0) for Debian-exim:102 in 4.9 seconds, 4055 bytes.
Apr 28 05:26:17 islay spamd[861]: spamd: result: . -2 - AWL,BAYES_00,FORGED_RCVD_HELO scantime=4.9,size=4055,user=Debian-exim,uid=102,required_score=5.0,rhost=212.13.194.5,raddr=212.13.194.5,rport=48949,mid=<20...@murphy>,bayes=0,autolearn=ham
in Exim this was reported as a protocol error though:
2007-04-28 05:26:12 1HhfRA-00065X-P0 spam acl condition: cannot parse spamd output
2007-04-28 05:26:12 1HhfRA-00065X-P0 <= bounce-debian-devel=andy=strugglers.net@lists.debian.org H=murphy.debian.org [70.103.162.31] P=esmtp S=4176 id=5l6urC.A.SaC.HrtMGB@murphy
2007-04-28 05:26:14 1HhfRA-00065X-P0 => andy <an...@strugglers.net> R=procmail T=procmail_pipe
2007-04-28 05:26:14 1HhfRA-00065X-P0 Completed
Seems like Exim must have sent data to spamd, but the saw some
problem and aborted the connection.
I've tried telnetting to the listen address/port over and over and
never see anything other than what I expect. If I give Exim the IPs
of the spamd servers directly then it works fine. I'm using version
3.1.7-1~bpo.1 from Debian backports.
Do anyone have any ideas what I might be doing wrong here? Any tips
for getting more info on what might be going wrong?
Alternatively, can anyone recommend some other open source software
load balancing solution? Preferably one that will let me direct to
least busy server or to set a per-server concurrent connection
limit.
Cheers,
Andy
Re: problems using haproxy for spamd
Posted by Jason Haar <Ja...@trimble.co.nz>.
Andy Smith wrote:
>
> Seems like Exim must have sent data to spamd, but the saw some
> problem and aborted the connection.
>
Could it simply be that the connection was taking too long and Exim
timeout it out? e.g. it sent the data but spamd still hadn't finished
processing the mail in (say) >30sec - so Exim just killed the connection
and carried on?
If so, it's probably configurable...
--
Cheers
Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +64 3 9635 377 Fax: +64 3 9635 417
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
Re: problems using haproxy for spamd
Posted by Andy Smith <an...@strugglers.net>.
On Mon, Apr 30, 2007 at 01:23:23AM +0000, Andy Smith wrote:
> Hi,
>
> I'm trying to use haproxy (http://haproxy.1wt.eu/) to load balance 3
> spamd servers on the same network.
[...]
> Unfortunately I seem to be intermittently getting connection
> failures. The haproxy log looks like this:
>
> Apr 28 05:13:49 localhost haproxy[11683]: Proxy spamd started.
> Apr 28 05:14:57 localhost haproxy[11684]: 212.13.194.70:55827 [28/Apr/2007:05:14:57] spamd corona 0/0/148 765 -- 0/0/0 0/0
> Apr 28 05:14:57 localhost haproxy[11684]: 212.13.194.70:55828 [28/Apr/2007:05:14:57] spamd curacao 0/-1/1 0 CC 0/0/0 0/0
It turned out to be a bug in haproxy. I sent an strace to the
author, Willy Tarreau, and he replied in less than 24 hours with a
full annotation of the strace and a patch to fix it. That's
service!
The bug manifested itself when the client would connect, send all
its data and shutdown before haproxy had successfully established a
connection with the backend server. If haproxy managed to establish
a connection before the client fnished sending then it would work
fine. Here's the simple fix:
diff --git a/haproxy.c b/haproxy.c
index 8e57700..357a37a 100644
--- a/haproxy.c
+++ b/haproxy.c
@@ -5589,7 +5589,7 @@ int process_srv(struct session *t) {
else if (s == SV_STCONN) { /* connection in progress */
if (c == CL_STCLOSE || c == CL_STSHUTW ||
(c == CL_STSHUTR &&
- (t->req->l == 0 || t->proxy->options & PR_O_ABRT_CLOSE))) { /* give up */
+ ((t->req->l == 0 && t->res_sw == RES_SILENT) || t->proxy->options & PR_O_ABRT_CLOSE))) { /* give up */
tv_eternity(&t->cnexpire);
fd_delete(t->srv_fd);
if (t->srv)
Cheers,
Andy