You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Sevrin Robstad <qu...@start.no> on 2008/01/28 00:06:57 UTC
Spamd uses over 1 gigabyte of memory in one child
I have used spamassassin over a year on my mail server, using James as
pop/smtp and a homewritten mailet to connect to spamd through tcp.
Yesterday I suddenly discovered load average peaks over 100 (!!!) on the
server, and soon found this :
22617 spamd 18 0 1830m 1.6g 2156 D 2 54.2 2:00.59 spamd
. Yea, one instance of spamd uses 1,6 *GIGABYTE* of memory. This forces my
server to swap, which makes the server dead slow.
It happens some time after I have started SA, but mails get through fine
before this happens.
information about my system:
cat /etc/sysconfig/spamassassin
# Options to spamd (-D for debug logging)
SPAMDOPTIONS="-D -d -c -m 4 -H -u spamd"
I was using spamassassin v 3.1.9, but have now upgraded to 3.2.6, without
this fixing the problem.
plugins: I don't know which information is important so I list everything :
I'm using URIDNSBL, Hashcash and SPF, ImageInfo,
VBounce,Bayes,BodyEval,DNSEval,HTMLEval,HEaderEval,MIMEEval,RelayEval,URIEval,WLBLEval,HTTPSMismatch,URIDetail
DCC,Pyzor,Razor2,SpamCop,AWL,AutoLearnThreshold,WhiteListSubject,MimeHeader,ReplaceTags
.
No rules over 150k.
The debug log seems to give only "normal" output, but i *think* this is what
it wrote when SA started to eat my memory :
metadata: X-Spam-Relays-Trusted:
Jan 27 23:38:25 s2 spamd[30682]: metadata: X-Spam-Relays-Untrusted: [
ip=80.202.4.58 rdns=osl1smout1.broadpark.no helo=osl1smout1.broadpark.no
by=www.refero.n
Jan 27 23:38:25 s2 spamd[30682]: metadata: X-Spam-Relays-Internal:
Jan 27 23:38:25 s2 spamd[30682]: metadata: X-Spam-Relays-External: [
ip=80.202.4.58 rdns=osl1smout1.broadpark.no helo=osl1smout1.broadpark.no
by=www.refero.ne
Jan 27 23:38:25 s2 spamd[30682]: message: ---- MIME PARSER START ----
Jan 27 23:38:25 s2 spamd[30682]: message: parsing multipart, got boundary:
Boundary_(ID_d1hvhmVn8zITA06RMg8o+g)
Jan 27 23:38:25 s2 spamd[30682]: message: found part of type text/plain,
boundary: Boundary_(ID_d1hvhmVn8zITA06RMg8o+g)
Jan 27 23:38:25 s2 spamd[30682]: message: added part, type: text/plain
Jan 27 23:38:40 s2 spamd[30682]: message: found part of type
application/ms-tnef, boundary: Boundary_(ID_d1hvhmVn8zITA06RMg8o+g)
Jan 27 23:38:40 s2 spamd[30682]: message: added part, type:
application/ms-tnef
Jan 27 23:38:40 s2 spamd[30682]: message: parsing normal part
Jan 27 23:38:40 s2 spamd[30682]: message: parsing normal part
Jan 27 23:38:43 s2 spamd[30675]: prefork: ordered 30683 to accept
Jan 27 23:38:43 s2 spamd[30675]: prefork: sysread(8) not ready, wait max 300
secs
Jan 27 23:38:43 s2 spamd[30675]: prefork: child 30683: entering state 2
Jan 27 23:38:43 s2 spamd[30675]: prefork: new lowest idle kid: none
Anybody have a clue?
--
View this message in context: http://www.nabble.com/Spamd-uses-over-1-gigabyte-of-memory-in-one-child-tp15125515p15125515.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Spamd uses over 1 gigabyte of memory in one child
Posted by Sevrin Robstad <qu...@start.no>.
Matus UHLAR - fantomas wrote:
>
> On 27.01.08 15:06, Sevrin Robstad wrote:
>> I have used spamassassin over a year on my mail server, using James as
>> pop/smtp and a homewritten mailet to connect to spamd through tcp.
>> Yesterday I suddenly discovered load average peaks over 100 (!!!) on the
>> server, and soon found this :
>>
>> 22617 spamd 18 0 1830m 1.6g 2156 D 2 54.2 2:00.59 spamd
>>
>> . Yea, one instance of spamd uses 1,6 *GIGABYTE* of memory. This forces
>> my
>> server to swap, which makes the server dead slow.
>
> this can be caused by old perl or perl module with memory leaks. Try
> upgrading those, if possible
> --
> Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Due to unexpected conditions Windows 2000 will be released
> in first quarter of year 1901
>
>
Well, It turned out I was right - the custom mailet we wrote to use spamd
together with the James MTA didn't use spamc and therefore it didn't limit
the size of messages to be spamchecked. So, When a user had some errors
which resulteted in about 150 150MB mails it's understandable that it
hung....
--
View this message in context: http://www.nabble.com/Spamd-uses-over-1-gigabyte-of-memory-in-one-child-tp15125515p15138542.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Spamd uses over 1 gigabyte of memory in one child
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 27.01.08 15:06, Sevrin Robstad wrote:
> I have used spamassassin over a year on my mail server, using James as
> pop/smtp and a homewritten mailet to connect to spamd through tcp.
> Yesterday I suddenly discovered load average peaks over 100 (!!!) on the
> server, and soon found this :
>
> 22617 spamd 18 0 1830m 1.6g 2156 D 2 54.2 2:00.59 spamd
>
> . Yea, one instance of spamd uses 1,6 *GIGABYTE* of memory. This forces my
> server to swap, which makes the server dead slow.
this can be caused by old perl or perl module with memory leaks. Try
upgrading those, if possible
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Due to unexpected conditions Windows 2000 will be released
in first quarter of year 1901
Re: Spamd uses over 1 gigabyte of memory in one child
Posted by Matt Kettler <mk...@verizon.net>.
Sevrin Robstad wrote:
> I have used spamassassin over a year on my mail server, using James as
> pop/smtp and a homewritten mailet to connect to spamd through tcp.
> Yesterday I suddenly discovered load average peaks over 100 (!!!) on the
> server, and soon found this :
>
> 22617 spamd 18 0 1830m 1.6g 2156 D 2 54.2 2:00.59 spamd
>
> . Yea, one instance of spamd uses 1,6 *GIGABYTE* of memory. This forces my
> server to swap, which makes the server dead slow.
>
> It happens some time after I have started SA, but mails get through fine
> before this happens.
>
>
> information about my system:
>
> cat /etc/sysconfig/spamassassin
> # Options to spamd (-D for debug logging)
> SPAMDOPTIONS="-D -d -c -m 4 -H -u spamd"
>
> I was using spamassassin v 3.1.9, but have now upgraded to 3.2.6, without
> this fixing the problem.
>
> plugins: I don't know which information is important so I list everything :
> I'm using URIDNSBL, Hashcash and SPF, ImageInfo,
> VBounce,Bayes,BodyEval,DNSEval,HTMLEval,HEaderEval,MIMEEval,RelayEval,URIEval,WLBLEval,HTTPSMismatch,URIDetail
> DCC,Pyzor,Razor2,SpamCop,AWL,AutoLearnThreshold,WhiteListSubject,MimeHeader,ReplaceTags
> .
>
> No rules over 150k.
>
>
>
<snip>
> Anybody have a clue?
>
Normally that's a sign of a giant ruleset.. (ie: sa-blacklist)
Have you tried running spamassassin --lint -D, and looking at the debug
output for all the rulesets parsed. Are there any you weren't aware of?
Do you have a lot (thousands) of black/whitelist entries in a user_prefs
somewhere?