You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Jari Fredriksson <ja...@iki.fi> on 2014/09/17 21:10:42 UTC
Simple question: load balancing spamd
What kind of simple load balancers are you using? I have been using just
DNS multiple address but that does not work any more. Something a *bit*
more intelligent is needed.
--
jarif.bit
Re: Simple question: load balancing spamd
Posted by Axb <ax...@gmail.com>.
On 09/18/2014 09:58 PM, Bob Proulx wrote:
> Jari Fredriksson wrote:
>> haproxy is just a small app capable of working as a proxy for http
>> or plain tcp connections. HA.
>
> What are you using for the Bayes database on the distributed compute
> farm? (Just curious...)
Redis, can't beat it in speed and comfort
Re: Simple question: load balancing spamd
Posted by Reindl Harald <h....@thelounge.net>.
Am 18.09.2014 um 21:58 schrieb Bob Proulx:
> Jari Fredriksson wrote:
>> haproxy is just a small app capable of working as a proxy for http
>> or plain tcp connections. HA.
>
> What are you using for the Bayes database on the distributed compute
> farm? (Just curious...)
something like MySQL is a no-brainer for shared storage
http://www.starbridge.org/spip/spip.php?article15
Re: Simple question: load balancing spamd
Posted by Jari Fredriksson <ja...@iki.fi>.
18.09.2014, 22:58, Bob Proulx kirjoitti:
> Jari Fredriksson wrote:
>> haproxy is just a small app capable of working as a proxy for http
>> or plain tcp connections. HA.
> What are you using for the Bayes database on the distributed compute
> farm? (Just curious...)
>
> Bob
>
MySQL /MariaDB 5.5
Re: Simple question: load balancing spamd
Posted by Bob Proulx <bo...@proulx.com>.
Jari Fredriksson wrote:
> haproxy is just a small app capable of working as a proxy for http
> or plain tcp connections. HA.
What are you using for the Bayes database on the distributed compute
farm? (Just curious...)
Bob
Re: Simple question: load balancing spamd
Posted by Jari Fredriksson <ja...@iki.fi>.
18.09.2014, 20:15, Alex Regan kirjoitti:
>
>>> LVS
>>
>> Implemented haproxy. Took 3 minutes to install and configure ;)
>
> So that basically works the same way as LVS, where you set up one
> server to distribute the load across N number of spamassassin systems
> connected to it?
>
> If you set it up in 3 minutes, how about tuning? Is it as flexible?
>
> Thanks,
> Alex
>
It has tuning options, but can't say yet. As LVS seems to be a virtual
Linux distro or so it felt like an overkill for my purposes. haproxy is
just a small app capable of working as a proxy for http or plain tcp
connections. HA.
--
jarif.bit
Re: Simple question: load balancing spamd
Posted by Alex Regan <my...@gmail.com>.
>> LVS
>
> Implemented haproxy. Took 3 minutes to install and configure ;)
So that basically works the same way as LVS, where you set up one server
to distribute the load across N number of spamassassin systems connected
to it?
If you set it up in 3 minutes, how about tuning? Is it as flexible?
Thanks,
Alex
Re: Simple question: load balancing spamd
Posted by Jari Fredriksson <ja...@iki.fi>.
17.09.2014, 22:22, Axb kirjoitti:
> On 09/17/2014 09:10 PM, Jari Fredriksson wrote:
>> What kind of simple load balancers are you using? I have been using just
>> DNS multiple address but that does not work any more. Something a *bit*
>> more intelligent is needed.
>>
>
> LVS
>
>
Implemented haproxy. Took 3 minutes to install and configure ;)
--
jarif.bit
Re: Simple question: load balancing spamd
Posted by Axb <ax...@gmail.com>.
On 09/17/2014 09:10 PM, Jari Fredriksson wrote:
> What kind of simple load balancers are you using? I have been using just
> DNS multiple address but that does not work any more. Something a *bit*
> more intelligent is needed.
>
LVS
Re: Simple question: load balancing spamd
Posted by Jari Fredriksson <ja...@iki.fi>.
17.09.2014, 22:22, Reindl Harald kirjoitti:
> Am 17.09.2014 um 21:10 schrieb Jari Fredriksson:
>> What kind of simple load balancers are you using? I have been using just
>> DNS multiple address but that does not work any more. Something a *bit*
>> more intelligent is needed
> have you considered how to reduce the amount making it
> to SA at all? 3 weeks production turns out that most
> can be rejected by the MTA and so reduce the need
> of load balancing greatly
I do that + postgrey for email receivers that do not really want to
receive spam.
But personally *I* do want them, and I want them classified with spamd.
I collect spam
for SpamAssassin ruleqa corpus.
>
> in my case Postfix/Postscreen with a bundle of RBL's
> with different weight to avoid false positives and
> a honeypot-mx answering in any case with 450
>
> the honeypot-mx catchs a lot of botnet crap never
> connecting to the real MX and even if i saw enough
> not blocked by RBL's at the first connect but on
> the retry to the primary MX
>
> below some numbers from this week
>
> * per day around 3000 legit mail
> * SA blocked 949 messages
> * 67396 rejected by postscreen
> * 2791 rejected by postfix (making it through postscreen)
> * 66220 RBL rejects out of the 67396 postcreen ones
> * 1942 is crap talking too early (postscreen_greet_wait)
>
> in fact most connections are not making it to smtpd at all
>
> some of the DNSBL/DNSWL are internally ones or mirrored
> on a internal 'dnsrbld' to reduce WAN load, i would suggest
> looking at the postfix-docs for some options below
> _____________________________________________________________________
>
> postscreen_cache_retention_time = 7d
> postscreen_bare_newline_ttl = 7d
> postscreen_greet_ttl = 7d
> postscreen_non_smtp_command_ttl = 7d
> postscreen_pipelining_ttl = 7d
> postscreen_dnsbl_ttl = 15m
> postscreen_dnsbl_threshold = 8
> postscreen_dnsbl_action = enforce
> postscreen_greet_action = enforce
> postscreen_greet_wait = ${stress?2}${stress:10}s
> postscreen_whitelist_interfaces = !<honeypot-ip>, static:all
>
> postscreen_dnsbl_sites = dnsbl.thelounge.net*16
> dnsbl.sorbs.net=127.0.0.10*8
> zen.spamhaus.org=127.0.0.[10;11]*8
> b.barracudacentral.org*7
> dnsbl.inps.de*7
> dnsbl.sorbs.net=127.0.0.5*6
> zen.spamhaus.org=127.0.0.[4..7]*6
> bl.mailspike.net*4
> bl.spamcop.net*4
> bl.spameatingmonkey.net*4
> dnsbl-ix.thelounge.net*4
> dnsrbl.swinog.ch*4
> zen.spamhaus.org=127.0.0.3*4
> dnsbl-surriel.thelounge.net*3
> dnsbl-uce.thelounge.net*3
> zen.spamhaus.org=127.0.0.2*3
> dnsbl.sorbs.net=127.0.0.6*2
> dnsbl.sorbs.net=127.0.0.9*2
> dnsbl-backscatterer.thelounge.net*1
> dnswl-whitelisted-org.thelounge.net*-2
> list.dnswl.org=127.0.[0..255].0*-2
> dnswl-aggregate.thelounge.net=127.0.0.5*-3
> list.dnswl.org=127.0.[0..255].1*-3
> list.dnswl.org=127.0.[0..255].2*-4
> list.dnswl.org=127.0.[0..255].3*-5
> dnswl-aggregate.thelounge.net=127.0.0.4*-8
> dnswl-aggregate.thelounge.net=127.0.0.3*-16
> dnswl-aggregate.thelounge.net=127.0.0.2*-24
> _____________________________________________________________________
>
> spamfilter-general-stats.sh
> Connections: 84415
> Delivered: 9637
> Invalid User: 1427
> Rejected-1: 67396
> Rejected-2: 2791
> Blacklist: 66220
> Pregreet: 1942
> Protocol Error: 809
> Spamfilter: 949
> Virus: 52
> Helo: 152
> Subject: 10
> Attachment: 18
> Sender Blocked: 111
> Sender Invalid: 103
> Sender Spoofed: 509
> PTR Missing: 511
> PTR Generic: 144
> SPF: 1
> _____________________________________________________________________
>
> spamfilter-honeypot-stats.php
> Default-MX: 18535
> Honeypot-MX: 8774
> Honeypot-Only: 7321
> _____________________________________________________________________
>
> dnsblcount.sh
> spamhaus.org 40305
> barracudacentral.org 12764
> sorbs.net 7407
> inps.de 5407
> thelounge.net 185
> manitu.net 63
> mailspike.net 57
> spamcop.net 21
> psbl.org 7
> swinog.ch 4
> spameatingmonkey.net 2
> uceprotect.net 1
> =================================
> Total DNSBL rejections: 66223
>
--
jarif.bit
Re: Simple question: load balancing spamd
Posted by Reindl Harald <h....@thelounge.net>.
Am 17.09.2014 um 21:10 schrieb Jari Fredriksson:
> What kind of simple load balancers are you using? I have been using just
> DNS multiple address but that does not work any more. Something a *bit*
> more intelligent is needed
have you considered how to reduce the amount making it
to SA at all? 3 weeks production turns out that most
can be rejected by the MTA and so reduce the need
of load balancing greatly
in my case Postfix/Postscreen with a bundle of RBL's
with different weight to avoid false positives and
a honeypot-mx answering in any case with 450
the honeypot-mx catchs a lot of botnet crap never
connecting to the real MX and even if i saw enough
not blocked by RBL's at the first connect but on
the retry to the primary MX
below some numbers from this week
* per day around 3000 legit mail
* SA blocked 949 messages
* 67396 rejected by postscreen
* 2791 rejected by postfix (making it through postscreen)
* 66220 RBL rejects out of the 67396 postcreen ones
* 1942 is crap talking too early (postscreen_greet_wait)
in fact most connections are not making it to smtpd at all
some of the DNSBL/DNSWL are internally ones or mirrored
on a internal 'dnsrbld' to reduce WAN load, i would suggest
looking at the postfix-docs for some options below
_____________________________________________________________________
postscreen_cache_retention_time = 7d
postscreen_bare_newline_ttl = 7d
postscreen_greet_ttl = 7d
postscreen_non_smtp_command_ttl = 7d
postscreen_pipelining_ttl = 7d
postscreen_dnsbl_ttl = 15m
postscreen_dnsbl_threshold = 8
postscreen_dnsbl_action = enforce
postscreen_greet_action = enforce
postscreen_greet_wait = ${stress?2}${stress:10}s
postscreen_whitelist_interfaces = !<honeypot-ip>, static:all
postscreen_dnsbl_sites = dnsbl.thelounge.net*16
dnsbl.sorbs.net=127.0.0.10*8
zen.spamhaus.org=127.0.0.[10;11]*8
b.barracudacentral.org*7
dnsbl.inps.de*7
dnsbl.sorbs.net=127.0.0.5*6
zen.spamhaus.org=127.0.0.[4..7]*6
bl.mailspike.net*4
bl.spamcop.net*4
bl.spameatingmonkey.net*4
dnsbl-ix.thelounge.net*4
dnsrbl.swinog.ch*4
zen.spamhaus.org=127.0.0.3*4
dnsbl-surriel.thelounge.net*3
dnsbl-uce.thelounge.net*3
zen.spamhaus.org=127.0.0.2*3
dnsbl.sorbs.net=127.0.0.6*2
dnsbl.sorbs.net=127.0.0.9*2
dnsbl-backscatterer.thelounge.net*1
dnswl-whitelisted-org.thelounge.net*-2
list.dnswl.org=127.0.[0..255].0*-2
dnswl-aggregate.thelounge.net=127.0.0.5*-3
list.dnswl.org=127.0.[0..255].1*-3
list.dnswl.org=127.0.[0..255].2*-4
list.dnswl.org=127.0.[0..255].3*-5
dnswl-aggregate.thelounge.net=127.0.0.4*-8
dnswl-aggregate.thelounge.net=127.0.0.3*-16
dnswl-aggregate.thelounge.net=127.0.0.2*-24
_____________________________________________________________________
spamfilter-general-stats.sh
Connections: 84415
Delivered: 9637
Invalid User: 1427
Rejected-1: 67396
Rejected-2: 2791
Blacklist: 66220
Pregreet: 1942
Protocol Error: 809
Spamfilter: 949
Virus: 52
Helo: 152
Subject: 10
Attachment: 18
Sender Blocked: 111
Sender Invalid: 103
Sender Spoofed: 509
PTR Missing: 511
PTR Generic: 144
SPF: 1
_____________________________________________________________________
spamfilter-honeypot-stats.php
Default-MX: 18535
Honeypot-MX: 8774
Honeypot-Only: 7321
_____________________________________________________________________
dnsblcount.sh
spamhaus.org 40305
barracudacentral.org 12764
sorbs.net 7407
inps.de 5407
thelounge.net 185
manitu.net 63
mailspike.net 57
spamcop.net 21
psbl.org 7
swinog.ch 4
spameatingmonkey.net 2
uceprotect.net 1
=================================
Total DNSBL rejections: 66223