You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by James Lay <jl...@slave-tothe-box.net> on 2006/05/17 15:27:13 UTC
Minimizing spamd's memory footprint
Hello all!
Soo.....yesterday I decided to get gutsy and use just about all the
rules from SARE. Here's my rulesdujour config:
TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS RANDOMVAL
SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML SARE_EVILNUMBERS0
SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD SARE_GENLSUBJ
SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2 SARE_GENLSUBJ3
SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER SARE_HEADER0
SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG SARE_HEADER_X30
SARE_HIGHRISK SARE_HTML SARE_HTML0 SARE_HTML1 SARE_HTML2 SARE_HTML3
SARE_HTML4 SARE_HTML_ENG SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2
SARE_OBFU3 SARE_OEM SARE_RANDOM SARE_RATWARE SARE_REDIRECT
SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF
SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG
SARE_WHITELIST TRIPWIRE"
Now here's the output of ps aux:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 3338 31.6 26.8 287636 277940 ? Ss 07:24 0:39 /usr/bin/spamd -u filter -d -m 10 -r /home/filter/run/spamd.pid --socketpath=/home/filter/run/spamd
filter 3365 19.1 27.1 290940 281204 ? S 07:25 0:14 spamd child
filter 3366 0.0 26.7 287636 276788 ? S 07:25 0:00 spamd child
Is this normal?
James
Re: Minimizing spamd's memory footprint
Posted by Matt Kettler <mk...@evi-inc.com>.
James Lay wrote:
> Hello all!
>
> Soo.....yesterday I decided to get gutsy and use just about all the
> rules from SARE. Here's my rulesdujour config:
>
> TRUSTED_RULESETS="ANTIDRUG
If you have SA 3.0.0 or higher, remove antidrug. These rules are included in SA,
and this ruleset is only for users of SA 2.6x and older.
I am the author of antidrug, so I speak with a solid understanding of the ruleset.
At some point I will create antidrug-pre30.cf, antidrug-30.cf and antidrug-31.cf.
After I've had that config for at least 6 months, I will replace antidrug.cf
with a file that generates a warning for anyone attempting to load it.
BLACKLIST BLACKLIST_URI
Ditch blacklist and blacklist_uri. Those rulesets are MAJOR memory hogs.
(In general, look at the file size of your .cf files. Anything over 128k is
possibly a memory hog, and anything over 256k is quite likely a memory hog.
blacklist and blacklist_uri are both over 512k. blacklist is nearly 2mb.
BOGUSVIRUS RANDOMVAL
> SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML SARE_EVILNUMBERS0
> SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD SARE_GENLSUBJ
> SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2 SARE_GENLSUBJ3
> SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER SARE_HEADER0
> SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG SARE_HEADER_X30
> SARE_HIGHRISK SARE_HTML SARE_HTML0 SARE_HTML1 SARE_HTML2 SARE_HTML3
> SARE_HTML4 SARE_HTML_ENG SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2
> SARE_OBFU3 SARE_OEM SARE_RANDOM SARE_RATWARE SARE_REDIRECT
> SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF
> SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG
> SARE_WHITELIST TRIPWIRE"
>
> Now here's the output of ps aux:
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> root 3338 31.6 26.8 287636 277940 ? Ss 07:24 0:39 /usr/bin/spamd -u filter -d -m 10 -r /home/filter/run/spamd.pid --socketpath=/home/filter/run/spamd
> filter 3365 19.1 27.1 290940 281204 ? S 07:25 0:14 spamd child
> filter 3366 0.0 26.7 287636 276788 ? S 07:25 0:00 spamd child
>
> Is this normal?
If you're using blacklist, yes..
>
> James
>
Re: Minimizing spamd's memory footprint
Posted by Kai Schaetzl <ma...@conactive.com>.
James Lay wrote on Wed, 17 May 2006 07:27:13 -0600:
> yesterday I decided to get gutsy and use just about all the
> rules from SARE.
Be careful with any rulesets that are larger than 100 KB. And you use
rulesets that are not intended to be used with SA 3 at all because there
are better alternatives.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Minimizing spamd's memory footprint
Posted by jdow <jd...@earthlink.net>.
From: "James Lay" <jl...@slave-tothe-box.net>
> Hello all!
>
> Soo.....yesterday I decided to get gutsy and use just about all the
> rules from SARE. Here's my rulesdujour config:
>
> TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS RANDOMVAL
> SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML SARE_EVILNUMBERS0
> SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD SARE_GENLSUBJ
> SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2 SARE_GENLSUBJ3
> SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER SARE_HEADER0
> SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG SARE_HEADER_X30
> SARE_HIGHRISK SARE_HTML SARE_HTML0 SARE_HTML1 SARE_HTML2 SARE_HTML3
> SARE_HTML4 SARE_HTML_ENG SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2
> SARE_OBFU3 SARE_OEM SARE_RANDOM SARE_RATWARE SARE_REDIRECT
> SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF
> SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG
> SARE_WHITELIST TRIPWIRE"
>
> Now here's the output of ps aux:
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> root 3338 31.6 26.8 287636 277940 ? Ss 07:24 0:39 /usr/bin/spamd -u
> filter -d -m 10 -r /home/filter/run/spamd.pid --socketpath=/home/filter/run/spamd
> filter 3365 19.1 27.1 290940 281204 ? S 07:25 0:14 spamd child
> filter 3366 0.0 26.7 287636 276788 ? S 07:25 0:00 spamd child
>
> Is this normal?
Since you used SARE_EVILNUMBERS* without reading that they are deprecated
this is normal. Jettison them and use the BL tools instead.
{^_^}
Re: Minimizing spamd's memory footprint
Posted by Mike Jackson <mj...@mightymerchant.com>.
> Soo.....yesterday I decided to get gutsy and use just about all the
> rules from SARE. Here's my rulesdujour config:
>
> TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS RANDOMVAL
> SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML SARE_EVILNUMBERS0
> SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD SARE_GENLSUBJ
> SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2 SARE_GENLSUBJ3
> SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER SARE_HEADER0
> SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG SARE_HEADER_X30
> SARE_HIGHRISK SARE_HTML SARE_HTML0 SARE_HTML1 SARE_HTML2 SARE_HTML3
> SARE_HTML4 SARE_HTML_ENG SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2
> SARE_OBFU3 SARE_OEM SARE_RANDOM SARE_RATWARE SARE_REDIRECT
> SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF
> SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG
> SARE_WHITELIST TRIPWIRE"
I know I keep harping about this on the list, but you should check which
rulesets are actually triggering on the spam your server receives. These are
the rulesets I'm grabbing with RulesDuJour:
SARE_ADULT
SARE_BAYES_POISON_NXM
SARE_FRAUD
SARE_HTML0
SARE_OBFU0
SARE_OEM
SARE_RANDOM
SARE_REDIRECT_POST300
SARE_SPAMCOP_TOP200
SARE_SPECIFIC
SARE_SPOOF
SARE_WHITELIST_RCVD
SARE_WHITELIST_SPF
SARE_STOCKS
>From looking at my logs, it's mostly SARE_SPECIFIC and SARE_STOCKS that
trigger. Most of the others are wastes of resources for the spam my server
receives. It could be the same for you too. OTOH, Bayes, Razor, and the DNS
tests identify the most spam.
Re: Minimizing spamd's memory footprint
Posted by jdow <jd...@earthlink.net>.
rom: "Matt Kettler" <mk...@evi-inc.com>
> jdow wrote:
>> 70_sare_evilnum0.cf # snap
>> 70_sare_evilnum1.cf # snap
>> 70_sare_evilnum2.cf # snap
>>
>> If you can in ANY WAY use the DNS based tests do so. Those sets
>> are HUGE and lead to incredibly large memory footprints.
>
> Erm, J.. evilnum is NOT replaced by a DNS test.. you're thinking of bigevil.
>
> evilnum works on phone numbers, not URIs.
You've been swamped, I can tell. {^_-} Yeah, I migrained that one good.
However, it prompted me to check out the evil numbers hits around here.
They hit less often than Matt's "useless.cf" rules around here. The mere
existence of these rule sets keeps spammers from using these old tricks
again. So while they are low scoring they are not useless.
{^_-}
Re: Minimizing spamd's memory footprint
Posted by jdow <jd...@earthlink.net>.
From: "Matt Kettler" <mk...@evi-inc.com>
> jdow wrote:
>> 70_sare_evilnum0.cf # snap
>> 70_sare_evilnum1.cf # snap
>> 70_sare_evilnum2.cf # snap
>>
>> If you can in ANY WAY use the DNS based tests do so. Those sets
>> are HUGE and lead to incredibly large memory footprints.
>
> Erm, J.. evilnum is NOT replaced by a DNS test.. you're thinking of bigevil.
>
> evilnum works on phone numbers, not URIs.
Thanks - I should really remember not to post while I am fighting a
migraine. <sigh> They seem to drop my IQ into the bit bucket as the
brains leak out of the hole in my head the migraine opens up.
{o.o}
Re: Minimizing spamd's memory footprint
Posted by Matt Kettler <mk...@evi-inc.com>.
jdow wrote:
> 70_sare_evilnum0.cf # snap
> 70_sare_evilnum1.cf # snap
> 70_sare_evilnum2.cf # snap
>
> If you can in ANY WAY use the DNS based tests do so. Those sets
> are HUGE and lead to incredibly large memory footprints.
Erm, J.. evilnum is NOT replaced by a DNS test.. you're thinking of bigevil.
evilnum works on phone numbers, not URIs.
Re: Minimizing spamd's memory footprint
Posted by jdow <jd...@earthlink.net>.
70_sare_evilnum0.cf # snap
70_sare_evilnum1.cf # snap
70_sare_evilnum2.cf # snap
If you can in ANY WAY use the DNS based tests do so. Those sets
are HUGE and lead to incredibly large memory footprints.
{^_^}
----- Original Message -----
From: "Dermot Paikkos" <de...@sciencephoto.com>
To: "James Lay" <jl...@slave-tothe-box.net>; "Spamassassin" <us...@spamassassin.apache.org>
Sent: Wednesday, May 17, 2006 07:30
Subject: Re: Minimizing spamd's memory footprint
>I am on V3.02.
>
> I certainly would be interesting to know which one of these is
> causing the problem.
> Dp.
>
>
> On 17 May 2006 at 8:19, James Lay wrote:
>
>> On Wed, 17 May 2006 15:10:45 +0100
>> "Dermot Paikkos" <de...@sciencephoto.com> wrote:
>>
>> > I wrote about this yesterday.
>> >
>> > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> > COMMAND
>> >
>> > nobody 17140 1.3 13.1 194984 169432 ? S 09:49 3:58 spamd
>> > child nobody 18656 1.3 10.4 159208 134328 ? R 10:08 3:43
>> > spamd child nobody 21371 1.1 12.7 191072 164440 ? S 10:38
>> > 2:51 spamd child nobody 21372 1.4 15.1 243424 195616 ? S
>> > 10:38 3:34 spamd child nobody 22331 1.4 22.7 327064 293176 ?
>> > S 10:47 3:32 spamd child nobody 22481 1.2 15.6 242200
>> > 201256 ? S 10:49 3:10 spamd child
>> >
>> > I am averaging 200MB per child.
>> >
>> > Here are my other rules:
>> > 70_sare_bayes_poison_nxm.cf # snap
>> > 70_sare_evilnum0.cf # snap
>> > 70_sare_evilnum1.cf # snap
>> > 70_sare_evilnum2.cf # snap
>> > 70_sare_header0.cf # snap
>> > 70_sare_header1.cf # snap
>> > 70_sare_header2.cf # snap
>> > 70_sare_header3.cf # snap
>> > 70_sare_html.cf # snap
>> > 70_sare_obfu0.cf # snap
>> > 70_sare_obfu1.cf # snap
>> > 70_sare_oem.cf # snap
>> > 70_sare_random.cf # snap
>> > 70_sare_specific.cf # snap
>> > 70_sare_unsub.cf # snap
>> > 70_sare_uri0.cf # snap
>> > 72_sare_redirect_post3.0.0.cf # snap
>> > 99_FVGT_Tripwire.cf
>> > 99_sare_fraud_post25x.cf
>> >
>> >
>> > There is a lot of overlap there. What version of SA are you running?
>> > Perhaps we should start removing them one at time and see what
>> > happens to the memory usage.
>> >
>> > Dp.
>> >
>> >
>> Version 3.1.1. I went back to my original list of:
>>
>> TRUSTED_RULESETS="SARE_REDIRECT_POST300 SARE_EVILNUMBERS0
>> SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_BAYES_POISON_NXM SARE_HTML
>> SARE_HEADER SARE_SPECIFIC SARE_ADULT SARE_FRAUD SARE_SPOOF SARE_RANDOM
>> SARE_SPAMCOP_TOP200 SARE_OEM SARE_GENLSUBJ SARE_GENLSUBJ4
>> SARE_GENLSUBJ_ENG SARE_HIGHRISK SARE_UNSUB SARE_URI0 SARE_URI1
>> SARE_URI3 SARE_URI_ENG SARE_WHITELIST SARE_WHITELIST_SPF
>> SARE_WHITELIST_RCVD SARE_OBFU SARE_OBFU2 SARE_OBFU3 SARE_OBFU4
>> TRIPWIRE"
>>
>> with the same effect. I didn't see this issue before, so I suspect
>> I'll simply nuke all sare rules, start and start adding them one by
>> one. I'll let you know how it goes =)
>>
>> James
>>
>> >
>> > On 17 May 2006 at 7:27, James Lay wrote:
>> >
>> > > Hello all!
>> > >
>> > > Soo.....yesterday I decided to get gutsy and use just about all
>> > > the rules from SARE. Here's my rulesdujour config:
>> > >
>> > > TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS
>> > > RANDOMVAL SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML
>> > > SARE_EVILNUMBERS0 SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD
>> > > SARE_GENLSUBJ SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2
>> > > SARE_GENLSUBJ3 SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER
>> > > SARE_HEADER0 SARE_HEADER1 SARE_HEADER2 SARE_HEADER3
>> > > SARE_HEADER_ENG SARE_HEADER_X30 SARE_HIGHRISK SARE_HTML SARE_HTML0
>> > > SARE_HTML1 SARE_HTML2 SARE_HTML3 SARE_HTML4 SARE_HTML_ENG
>> > > SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2 SARE_OBFU3 SARE_OEM
>> > > SARE_RANDOM SARE_RATWARE SARE_REDIRECT SARE_REDIRECT_POST300
>> > > SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF SARE_UNSUB SARE_URI0
>> > > SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG SARE_WHITELIST
>> > > TRIPWIRE"
>> > >
>> > > Now here's the output of ps aux:
>> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> > > COMMAND root 3338 31.6 26.8 287636 277940 ? Ss 07:24
>> > > 0:39 /usr/bin/spamd -u filter -d -m 10 -r
>> > > /home/filter/run/spamd.pid --socketpath=/home/filter/run/spamd
>> > > filter 3365 19.1 27.1 290940 281204 ? S 07:25 0:14
>> > > spamd child filter 3366 0.0 26.7 287636 276788 ? S
>> > > 07:25 0:00 spamd child
>> > >
>> > > Is this normal?
>> > >
>> > > James
>> >
>> >
>
Re: Minimizing spamd's memory footprint
Posted by Dermot Paikkos <de...@sciencephoto.com>.
I am on V3.02.
I certainly would be interesting to know which one of these is
causing the problem.
Dp.
On 17 May 2006 at 8:19, James Lay wrote:
> On Wed, 17 May 2006 15:10:45 +0100
> "Dermot Paikkos" <de...@sciencephoto.com> wrote:
>
> > I wrote about this yesterday.
> >
> > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
> > COMMAND
> >
> > nobody 17140 1.3 13.1 194984 169432 ? S 09:49 3:58 spamd
> > child nobody 18656 1.3 10.4 159208 134328 ? R 10:08 3:43
> > spamd child nobody 21371 1.1 12.7 191072 164440 ? S 10:38
> > 2:51 spamd child nobody 21372 1.4 15.1 243424 195616 ? S
> > 10:38 3:34 spamd child nobody 22331 1.4 22.7 327064 293176 ?
> > S 10:47 3:32 spamd child nobody 22481 1.2 15.6 242200
> > 201256 ? S 10:49 3:10 spamd child
> >
> > I am averaging 200MB per child.
> >
> > Here are my other rules:
> > 70_sare_bayes_poison_nxm.cf # snap
> > 70_sare_evilnum0.cf # snap
> > 70_sare_evilnum1.cf # snap
> > 70_sare_evilnum2.cf # snap
> > 70_sare_header0.cf # snap
> > 70_sare_header1.cf # snap
> > 70_sare_header2.cf # snap
> > 70_sare_header3.cf # snap
> > 70_sare_html.cf # snap
> > 70_sare_obfu0.cf # snap
> > 70_sare_obfu1.cf # snap
> > 70_sare_oem.cf # snap
> > 70_sare_random.cf # snap
> > 70_sare_specific.cf # snap
> > 70_sare_unsub.cf # snap
> > 70_sare_uri0.cf # snap
> > 72_sare_redirect_post3.0.0.cf # snap
> > 99_FVGT_Tripwire.cf
> > 99_sare_fraud_post25x.cf
> >
> >
> > There is a lot of overlap there. What version of SA are you running?
> > Perhaps we should start removing them one at time and see what
> > happens to the memory usage.
> >
> > Dp.
> >
> >
> Version 3.1.1. I went back to my original list of:
>
> TRUSTED_RULESETS="SARE_REDIRECT_POST300 SARE_EVILNUMBERS0
> SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_BAYES_POISON_NXM SARE_HTML
> SARE_HEADER SARE_SPECIFIC SARE_ADULT SARE_FRAUD SARE_SPOOF SARE_RANDOM
> SARE_SPAMCOP_TOP200 SARE_OEM SARE_GENLSUBJ SARE_GENLSUBJ4
> SARE_GENLSUBJ_ENG SARE_HIGHRISK SARE_UNSUB SARE_URI0 SARE_URI1
> SARE_URI3 SARE_URI_ENG SARE_WHITELIST SARE_WHITELIST_SPF
> SARE_WHITELIST_RCVD SARE_OBFU SARE_OBFU2 SARE_OBFU3 SARE_OBFU4
> TRIPWIRE"
>
> with the same effect. I didn't see this issue before, so I suspect
> I'll simply nuke all sare rules, start and start adding them one by
> one. I'll let you know how it goes =)
>
> James
>
> >
> > On 17 May 2006 at 7:27, James Lay wrote:
> >
> > > Hello all!
> > >
> > > Soo.....yesterday I decided to get gutsy and use just about all
> > > the rules from SARE. Here's my rulesdujour config:
> > >
> > > TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS
> > > RANDOMVAL SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML
> > > SARE_EVILNUMBERS0 SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD
> > > SARE_GENLSUBJ SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2
> > > SARE_GENLSUBJ3 SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER
> > > SARE_HEADER0 SARE_HEADER1 SARE_HEADER2 SARE_HEADER3
> > > SARE_HEADER_ENG SARE_HEADER_X30 SARE_HIGHRISK SARE_HTML SARE_HTML0
> > > SARE_HTML1 SARE_HTML2 SARE_HTML3 SARE_HTML4 SARE_HTML_ENG
> > > SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2 SARE_OBFU3 SARE_OEM
> > > SARE_RANDOM SARE_RATWARE SARE_REDIRECT SARE_REDIRECT_POST300
> > > SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF SARE_UNSUB SARE_URI0
> > > SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG SARE_WHITELIST
> > > TRIPWIRE"
> > >
> > > Now here's the output of ps aux:
> > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
> > > COMMAND root 3338 31.6 26.8 287636 277940 ? Ss 07:24
> > > 0:39 /usr/bin/spamd -u filter -d -m 10 -r
> > > /home/filter/run/spamd.pid --socketpath=/home/filter/run/spamd
> > > filter 3365 19.1 27.1 290940 281204 ? S 07:25 0:14
> > > spamd child filter 3366 0.0 26.7 287636 276788 ? S
> > > 07:25 0:00 spamd child
> > >
> > > Is this normal?
> > >
> > > James
> >
> >
Re: Minimizing spamd's memory footprint
Posted by James Lay <jl...@slave-tothe-box.net>.
On Wed, 17 May 2006 15:10:45 +0100
"Dermot Paikkos" <de...@sciencephoto.com> wrote:
> I wrote about this yesterday.
>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
> COMMAND
>
> nobody 17140 1.3 13.1 194984 169432 ? S 09:49 3:58 spamd
> child
> nobody 18656 1.3 10.4 159208 134328 ? R 10:08 3:43 spamd
> child
> nobody 21371 1.1 12.7 191072 164440 ? S 10:38 2:51 spamd
> child
> nobody 21372 1.4 15.1 243424 195616 ? S 10:38 3:34 spamd
> child
> nobody 22331 1.4 22.7 327064 293176 ? S 10:47 3:32 spamd
> child
> nobody 22481 1.2 15.6 242200 201256 ? S 10:49 3:10 spamd
> child
>
> I am averaging 200MB per child.
>
> Here are my other rules:
> 70_sare_bayes_poison_nxm.cf # snap
> 70_sare_evilnum0.cf # snap
> 70_sare_evilnum1.cf # snap
> 70_sare_evilnum2.cf # snap
> 70_sare_header0.cf # snap
> 70_sare_header1.cf # snap
> 70_sare_header2.cf # snap
> 70_sare_header3.cf # snap
> 70_sare_html.cf # snap
> 70_sare_obfu0.cf # snap
> 70_sare_obfu1.cf # snap
> 70_sare_oem.cf # snap
> 70_sare_random.cf # snap
> 70_sare_specific.cf # snap
> 70_sare_unsub.cf # snap
> 70_sare_uri0.cf # snap
> 72_sare_redirect_post3.0.0.cf # snap
> 99_FVGT_Tripwire.cf
> 99_sare_fraud_post25x.cf
>
>
> There is a lot of overlap there. What version of SA are you running?
> Perhaps we should start removing them one at time and see what
> happens to the memory usage.
>
> Dp.
>
>
Version 3.1.1. I went back to my original list of:
TRUSTED_RULESETS="SARE_REDIRECT_POST300 SARE_EVILNUMBERS0
SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_BAYES_POISON_NXM SARE_HTML
SARE_HEADER SARE_SPECIFIC SARE_ADULT SARE_FRAUD SARE_SPOOF SARE_RANDOM
SARE_SPAMCOP_TOP200 SARE_OEM SARE_GENLSUBJ SARE_GENLSUBJ4
SARE_GENLSUBJ_ENG SARE_HIGHRISK SARE_UNSUB SARE_URI0 SARE_URI1
SARE_URI3 SARE_URI_ENG SARE_WHITELIST SARE_WHITELIST_SPF
SARE_WHITELIST_RCVD SARE_OBFU SARE_OBFU2 SARE_OBFU3 SARE_OBFU4 TRIPWIRE"
with the same effect. I didn't see this issue before, so I suspect
I'll simply nuke all sare rules, start and start adding them one by
one. I'll let you know how it goes =)
James
>
> On 17 May 2006 at 7:27, James Lay wrote:
>
> > Hello all!
> >
> > Soo.....yesterday I decided to get gutsy and use just about all the
> > rules from SARE. Here's my rulesdujour config:
> >
> > TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS
> > RANDOMVAL SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML
> > SARE_EVILNUMBERS0 SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD
> > SARE_GENLSUBJ SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2
> > SARE_GENLSUBJ3 SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER
> > SARE_HEADER0 SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG
> > SARE_HEADER_X30 SARE_HIGHRISK SARE_HTML SARE_HTML0 SARE_HTML1
> > SARE_HTML2 SARE_HTML3 SARE_HTML4 SARE_HTML_ENG SARE_OBFU SARE_OBFU0
> > SARE_OBFU1 SARE_OBFU2 SARE_OBFU3 SARE_OEM SARE_RANDOM SARE_RATWARE
> > SARE_REDIRECT SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200
> > SARE_SPECIFIC SARE_SPOOF SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2
> > SARE_URI3 SARE_URI_ENG SARE_WHITELIST TRIPWIRE"
> >
> > Now here's the output of ps aux:
> > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
> > COMMAND root 3338 31.6 26.8 287636 277940 ? Ss 07:24
> > 0:39 /usr/bin/spamd -u filter -d -m 10 -r /home/filter/run/spamd.pid
> > --socketpath=/home/filter/run/spamd filter 3365 19.1 27.1 290940
> > 281204 ? S 07:25 0:14 spamd child filter 3366 0.0
> > 26.7 287636 276788 ? S 07:25 0:00 spamd child
> >
> > Is this normal?
> >
> > James
>
>
Re: Minimizing spamd's memory footprint
Posted by jdow <jd...@earthlink.net>.
Do not use:
70_sare_evilnum0.cf # snap
70_sare_evilnum1.cf # snap
70_sare_evilnum2.cf # snap
{^_^}
----- Original Message -----
From: "Dermot Paikkos" <de...@sciencephoto.com>
To: "Spamassassin" <us...@spamassassin.apache.org>
Sent: Wednesday, May 17, 2006 07:10
Subject: Re: Minimizing spamd's memory footprint
>I wrote about this yesterday.
>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
> COMMAND
>
> nobody 17140 1.3 13.1 194984 169432 ? S 09:49 3:58 spamd
> child
> nobody 18656 1.3 10.4 159208 134328 ? R 10:08 3:43 spamd
> child
> nobody 21371 1.1 12.7 191072 164440 ? S 10:38 2:51 spamd
> child
> nobody 21372 1.4 15.1 243424 195616 ? S 10:38 3:34 spamd
> child
> nobody 22331 1.4 22.7 327064 293176 ? S 10:47 3:32 spamd
> child
> nobody 22481 1.2 15.6 242200 201256 ? S 10:49 3:10 spamd
> child
>
> I am averaging 200MB per child.
>
> Here are my other rules:
> 70_sare_bayes_poison_nxm.cf # snap
> 70_sare_evilnum0.cf # snap
> 70_sare_evilnum1.cf # snap
> 70_sare_evilnum2.cf # snap
> 70_sare_header0.cf # snap
> 70_sare_header1.cf # snap
> 70_sare_header2.cf # snap
> 70_sare_header3.cf # snap
> 70_sare_html.cf # snap
> 70_sare_obfu0.cf # snap
> 70_sare_obfu1.cf # snap
> 70_sare_oem.cf # snap
> 70_sare_random.cf # snap
> 70_sare_specific.cf # snap
> 70_sare_unsub.cf # snap
> 70_sare_uri0.cf # snap
> 72_sare_redirect_post3.0.0.cf # snap
> 99_FVGT_Tripwire.cf
> 99_sare_fraud_post25x.cf
>
>
> There is a lot of overlap there. What version of SA are you running?
> Perhaps we should start removing them one at time and see what
> happens to the memory usage.
>
> Dp.
>
>
>
> On 17 May 2006 at 7:27, James Lay wrote:
>
>> Hello all!
>>
>> Soo.....yesterday I decided to get gutsy and use just about all the
>> rules from SARE. Here's my rulesdujour config:
>>
>> TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS
>> RANDOMVAL SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML SARE_EVILNUMBERS0
>> SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD SARE_GENLSUBJ
>> SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2 SARE_GENLSUBJ3
>> SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER SARE_HEADER0
>> SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG SARE_HEADER_X30
>> SARE_HIGHRISK SARE_HTML SARE_HTML0 SARE_HTML1 SARE_HTML2 SARE_HTML3
>> SARE_HTML4 SARE_HTML_ENG SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2
>> SARE_OBFU3 SARE_OEM SARE_RANDOM SARE_RATWARE SARE_REDIRECT
>> SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF
>> SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG
>> SARE_WHITELIST TRIPWIRE"
>>
>> Now here's the output of ps aux:
>> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> COMMAND root 3338 31.6 26.8 287636 277940 ? Ss 07:24
>> 0:39 /usr/bin/spamd -u filter -d -m 10 -r /home/filter/run/spamd.pid
>> --socketpath=/home/filter/run/spamd filter 3365 19.1 27.1 290940
>> 281204 ? S 07:25 0:14 spamd child filter 3366 0.0 26.7
>> 287636 276788 ? S 07:25 0:00 spamd child
>>
>> Is this normal?
>>
>> James
>
Re: Minimizing spamd's memory footprint
Posted by Dermot Paikkos <de...@sciencephoto.com>.
I wrote about this yesterday.
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
COMMAND
nobody 17140 1.3 13.1 194984 169432 ? S 09:49 3:58 spamd
child
nobody 18656 1.3 10.4 159208 134328 ? R 10:08 3:43 spamd
child
nobody 21371 1.1 12.7 191072 164440 ? S 10:38 2:51 spamd
child
nobody 21372 1.4 15.1 243424 195616 ? S 10:38 3:34 spamd
child
nobody 22331 1.4 22.7 327064 293176 ? S 10:47 3:32 spamd
child
nobody 22481 1.2 15.6 242200 201256 ? S 10:49 3:10 spamd
child
I am averaging 200MB per child.
Here are my other rules:
70_sare_bayes_poison_nxm.cf # snap
70_sare_evilnum0.cf # snap
70_sare_evilnum1.cf # snap
70_sare_evilnum2.cf # snap
70_sare_header0.cf # snap
70_sare_header1.cf # snap
70_sare_header2.cf # snap
70_sare_header3.cf # snap
70_sare_html.cf # snap
70_sare_obfu0.cf # snap
70_sare_obfu1.cf # snap
70_sare_oem.cf # snap
70_sare_random.cf # snap
70_sare_specific.cf # snap
70_sare_unsub.cf # snap
70_sare_uri0.cf # snap
72_sare_redirect_post3.0.0.cf # snap
99_FVGT_Tripwire.cf
99_sare_fraud_post25x.cf
There is a lot of overlap there. What version of SA are you running?
Perhaps we should start removing them one at time and see what
happens to the memory usage.
Dp.
On 17 May 2006 at 7:27, James Lay wrote:
> Hello all!
>
> Soo.....yesterday I decided to get gutsy and use just about all the
> rules from SARE. Here's my rulesdujour config:
>
> TRUSTED_RULESETS="ANTIDRUG BLACKLIST BLACKLIST_URI BOGUSVIRUS
> RANDOMVAL SARE_ADULT SARE_BAYES_POISON_NXM SARE_BML SARE_EVILNUMBERS0
> SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 SARE_FRAUD SARE_GENLSUBJ
> SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2 SARE_GENLSUBJ3
> SARE_GENLSUBJ_ENG SARE_GENLSUBJ_X30 SARE_HEADER SARE_HEADER0
> SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG SARE_HEADER_X30
> SARE_HIGHRISK SARE_HTML SARE_HTML0 SARE_HTML1 SARE_HTML2 SARE_HTML3
> SARE_HTML4 SARE_HTML_ENG SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2
> SARE_OBFU3 SARE_OEM SARE_RANDOM SARE_RATWARE SARE_REDIRECT
> SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200 SARE_SPECIFIC SARE_SPOOF
> SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG
> SARE_WHITELIST TRIPWIRE"
>
> Now here's the output of ps aux:
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
> COMMAND root 3338 31.6 26.8 287636 277940 ? Ss 07:24
> 0:39 /usr/bin/spamd -u filter -d -m 10 -r /home/filter/run/spamd.pid
> --socketpath=/home/filter/run/spamd filter 3365 19.1 27.1 290940
> 281204 ? S 07:25 0:14 spamd child filter 3366 0.0 26.7
> 287636 276788 ? S 07:25 0:00 spamd child
>
> Is this normal?
>
> James