You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Bikrant Neupane <bi...@wlink.com.np> on 2005/05/03 11:41:18 UTC
High memory usage
I shifted to Spamassin 3.0.3 on FreeBSD 5.3 yesterday with dcc enabled.
Hardware is Xeon (hyper threading) SC1420 with 1536 MB RAM.
I am using following options to start spamd
-d -i $O1 -m40 -u smtpd
spamd process use up all the RAM + another 1GB swap within just 10-20 Mins. It
is really strange. I had run spamassin 2.6 in Linux for more than a year
without problem.
On further analysis I found following message in log file.
>>>>>
Deep recursion on subroutine "Mail::SpamAssassin::Message::_parse_normal"
at /usr/local/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Message.pm line
446, <GEN643> line 8173.
Deep recursion on subroutine "Mail::SpamAssassin::Message::new"
at /usr/local/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Message.pm line
611, <GEN643> line 8173.
Deep recursion on subroutine "Mail::SpamAssassin::Message::Node::_find_parts"
at /usr/local/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Message/Node.pm
line 122, <GEN643> line 8173.
>>>>>
Is it due to incompatibility between spamassassin and perl?? Similar problem
(Deep recursion) is mentioned at spamassassin site but that one is related
with DB_File.
I had installed perl 5.8 from freebsd package during installation. I think
installing perl 5.8 from ports should fix the problem (if it is because of
incompatibility) .
I would like to know if anyone has run into the same problem before.
thanks a lot
Bikrant
more debug::
Seems all children are busy!! Do i have to increase max child value ??
14498 ?? Is 0:01.22 /usr/local/bin/spamd -d -i spam-scanner-01 -m40
--allowed-ips=127.0.0.1,202.79.32.7
52851 ?? S 1:07.40 spamd child (perl)
53039 ?? S 1:07.19 spamd child (perl)
53165 ?? S 0:57.46 spamd child (perl)
53186 ?? S 1:01.37 spamd child (perl)
53208 ?? S 0:57.25 spamd child (perl)
53307 ?? S 0:57.76 spamd child (perl)
53468 ?? S 1:03.32 spamd child (perl)
53488 ?? S 1:08.63 spamd child (perl)
53511 ?? S 1:02.60 spamd child (perl)
53516 ?? S 1:02.55 spamd child (perl)
53566 ?? S 1:04.30 spamd child (perl)
53615 ?? S 1:08.33 spamd child (perl)
53620 ?? S 1:07.25 spamd child (perl)
53638 ?? S 1:01.89 spamd child (perl)
53644 ?? S 1:10.25 spamd child (perl)
53661 ?? S 1:00.59 spamd child (perl)
53662 ?? S 1:03.87 spamd child (perl)
53668 ?? S 1:04.95 spamd child (perl)
53679 ?? S 1:02.06 spamd child (perl)
53681 ?? S 0:57.79 spamd child (perl)
53698 ?? S 0:56.96 spamd child (perl)
53748 ?? S 0:58.17 spamd child (perl)
53757 ?? S 1:04.80 spamd child (perl)
53766 ?? S 0:56.18 spamd child (perl)
53768 ?? S 1:02.78 spamd child (perl)
53770 ?? S 0:56.63 spamd child (perl)
53777 ?? S 0:55.23 spamd child (perl)
53785 ?? S 0:55.13 spamd child (perl)
53794 ?? S 0:57.76 spamd child (perl)
53808 ?? S 0:58.48 spamd child (perl)
53827 ?? S 1:00.06 spamd child (perl)
53830 ?? S 0:57.52 spamd child (perl)
53840 ?? S 1:07.25 spamd child (perl)
54031 ?? S 0:56.30 spamd child (perl)
54062 ?? S 0:52.49 spamd child (perl)
54070 ?? S 0:54.79 spamd child (perl)
54095 ?? S 0:55.25 spamd child (perl)
54121 ?? S 1:02.73 spamd child (perl)
54132 ?? S 0:59.53 spamd child (perl)
54232 ?? S 0:57.41 spamd child (perl)
Re: High memory usage
Posted by Bikrant Neupane <bi...@wlink.com.np>.
On Wednesday 04 May 2005 01:26, Daryl C. W. O'Shea wrote:
> Bikrant Neupane wrote:
> > I shifted to Spamassin 3.0.3 on FreeBSD 5.3 yesterday with dcc enabled.
> > Hardware is Xeon (hyper threading) SC1420 with 1536 MB RAM.
> >
> > I am using following options to start spamd
> > -d -i $O1 -m40 -u smtpd
>
> <snip>
>
> > more debug::
> > Seems all children are busy!! Do i have to increase max child value ??
> >
> > 14498 ?? Is 0:01.22 /usr/local/bin/spamd -d -i spam-scanner-01 -m40
> > --allowed-ips=127.0.0.1,202.79.32.7
> > 52851 ?? S 1:07.40 spamd child (perl)
> > 53039 ?? S 1:07.19 spamd child (perl)
> > 53165 ?? S 0:57.46 spamd child (perl)
> > 53186 ?? S 1:01.37 spamd child (perl)
>
> <snip>
>
> No, reduce the number of children. At 20-40MB per child it's no wonder
> why you've run out of memory with 40 children. Once you do that and the
> system stops thrashing you should be able to process messages faster.
Ok now I have reduced the number of process to 15. Now I am no more running
out of RAM but still I see spamassassin is using relatively high memory.
from top :
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
12221 root 4 0 95872K 81456K accept 0 1:14 15.14% 15.14% perl
12304 root 4 0 76768K 65012K accept 1 1:13 8.50% 8.50% perl
12276 root 4 0 90240K 77688K accept 1 1:27 7.18% 7.18% perl
12301 root -8 0 95360K 80128K piperd 0 1:06 6.05% 6.05% perl
12176 root 4 0 98780K 82440K accept 1 1:31 5.66% 5.66% perl
12348 root 4 0 63892K 53872K accept 0 1:18 4.74% 4.74% perl
12571 root 4 0 49580K 41252K accept 1 1:06 4.74% 4.74% perl
12640 root 4 0 95256K 77580K accept 1 1:00 4.74% 4.74% perl
12368 root 4 0 65008K 53528K accept 1 1:18 4.30% 4.30% perl
12601 root 4 0 89616K 77548K accept 1 1:17 4.30% 4.30% perl
12429 root 4 0 88864K 74852K accept 1 1:06 3.76% 3.76% perl
13108 root 4 0 83728K 69244K accept 1 0:51 3.52% 3.52% perl
12549 root 4 0 44844K 36636K accept 1 1:01 2.59% 2.59% perl
12584 root 4 0 90304K 76152K accept 1 1:08 2.49% 2.49% perl
12852 root 96 0 45212K 37972K select 1 0:51 2.25% 2.25% perl
Also I am still getting "Deep recursion" message in log file
>>>
May 4 14:44:32 spam-scanner-01 spamd[12221]: Deep recursion on subroutine
"Mail::SpamAssassin::Message::Node::finish"
at /usr/local/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Message/Node.pm
line 659
>>>
Previously I was running perl 5.8.5 which was installed from binary package
from the installation cd. Now I have reinstalled 5.8.5 from ports. But still
I am getting same message.
It will be great if someone could point me to right direction. Next I am going
to sync my ports from latest cvs and recompile perl and other modules again.
thanks a lot,
Bikrant
>
> Daryl
Re: High memory usage
Posted by Bikrant Neupane <bi...@wlink.com.np>.
On Wednesday 04 May 2005 01:26, Daryl C. W. O'Shea wrote:
> Bikrant Neupane wrote:
> > I shifted to Spamassin 3.0.3 on FreeBSD 5.3 yesterday with dcc enabled.
> > Hardware is Xeon (hyper threading) SC1420 with 1536 MB RAM.
> >
> > I am using following options to start spamd
> > -d -i $O1 -m40 -u smtpd
>
> <snip>
>
> > more debug::
> > Seems all children are busy!! Do i have to increase max child value ??
> >
> > 14498 ?? Is 0:01.22 /usr/local/bin/spamd -d -i spam-scanner-01 -m40
> > --allowed-ips=127.0.0.1,202.79.32.7
> > 52851 ?? S 1:07.40 spamd child (perl)
> > 53039 ?? S 1:07.19 spamd child (perl)
> > 53165 ?? S 0:57.46 spamd child (perl)
> > 53186 ?? S 1:01.37 spamd child (perl)
>
> <snip>
>
> No, reduce the number of children. At 20-40MB per child it's no wonder
> why you've run out of memory with 40 children. Once you do that and the
> system stops thrashing you should be able to process messages faster.
Yes you are right. But I started with 5 process first. I observed that the
size of one process keeps increasing upto 450MB (and may be more). I think
this is not normal. Where as other process remains at 20-40 MB. I think the
process that has undergone "Deep recursion" keeps chewing memory all the time
until it reaches the per process max memory limit of FreeBSD system.
regards,
Bikrant
>
> Daryl
Re: High memory usage
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Bikrant Neupane wrote:
> I shifted to Spamassin 3.0.3 on FreeBSD 5.3 yesterday with dcc enabled.
> Hardware is Xeon (hyper threading) SC1420 with 1536 MB RAM.
>
> I am using following options to start spamd
> -d -i $O1 -m40 -u smtpd
>
<snip>
> more debug::
> Seems all children are busy!! Do i have to increase max child value ??
>
> 14498 ?? Is 0:01.22 /usr/local/bin/spamd -d -i spam-scanner-01 -m40
> --allowed-ips=127.0.0.1,202.79.32.7
> 52851 ?? S 1:07.40 spamd child (perl)
> 53039 ?? S 1:07.19 spamd child (perl)
> 53165 ?? S 0:57.46 spamd child (perl)
> 53186 ?? S 1:01.37 spamd child (perl)
<snip>
No, reduce the number of children. At 20-40MB per child it's no wonder
why you've run out of memory with 40 children. Once you do that and the
system stops thrashing you should be able to process messages faster.
Daryl