max-conn-per-child spamd flag?

[Mike Jackson <mj...@barking-dog.net>]

I just noticed the --max-conn-per-child option in the spamd man page. While 
the description is fairly straightforward, I'm curious if anyone else is 
using this, why, and if it's helped with spamd processes consuming RAM 
(which is what I'm trying to fix at the moment). 

Re: max-conn-per-child spamd flag?

[Mike Jackson <mj...@barking-dog.net>]

> Depending on the weather, phase of the moon, amount of ram you have, and
> other things, you may find limits from a couple hundred down to about 5 to
> be appropriate.  I'd probably suggest 100 to 50 as a good starting range 
> to
> play with.
>
> Dont forget the -m option.  If you have more than about 5 children running
> and don't have a huge email flow you might do well to cut the number of
> children down to the 3 to 10 range.

Thanks for the advice. I set max-conn-per-child to 20 - probably needlessly 
low, but I'll see how it does at that level and work my way up. So far, 
working like a charm. max-children is set by default to 5 (at least 
according to the man page), and I've never seen this box hit the limit. SA 
is opt-in on this box and only processes a few thousand messages a day, but 
when I checked this morning the five spamd children were using over 300MB 
apiece; with this option turned on, they're down to about 40-50MB apiece. 

Re: max-conn-per-child spamd flag?

[Loren Wilton <lw...@earthlink.net>]

>    LW> Dont forget the -m option.  If you have more than about 5 children
>    LW> running and don't have a huge email flow you might do well to cut
the
>    LW> number of children down to the 3 to 10 range.
>
> What is considered "huge email flow" and what are appropriate values for
> connections and children?

I'd think 5 children should be good for a few thousand mail/hour at least,
given a decent box (>= 1GHz) to run SA.  A home net like you describe would
probably run perfectly happily with 1 or 2 children at most.  Anything above
that is likely to just be sitting there using resources.  Of course, if you
have the resources to burn, then it probably isn't worth cutting the number
of children down.

Look at it this way: how long does it take a child to process a message, on
average (total time, not processor time)?  Maybe a few seconds at most?
Let's say 5 seconds as an estimate.  Then each child can process 3600/5 =
720 messages/hour.  You are receiving 1K messages/day, which is 1000/24=42
messages/hour.  A single child would then have 17 times the capacity you
need, and would be idle 94% of the time.  Five children taken together will
be idle something like 99% of the time.

Of course these are steady-state estimates, and queuing theory says that
things can get nasty for a while in burst modes if you only have one or two
children.  So having 5 children can result in keeping the throughput time
down in the area of 10-20 seconds total queuing+processing time per mail
when you get a sudden flood of 50 or 100 mails in a short period of time.
Still, more than 3 children is probably overkill under any situation for
your mail rate.


> Does this make sense?  Should I (can I) reduce the numebr of sendmail
> children to better match spamd?

Sendmail I don't know beans about, so hopefully someone else will be able to
answer those questions.  Or possibly you can answer them yourself, if you
just consider the mail process as a long queue with lumps in it, and
determine what the overall linear processing time for a mail item is.  If
you know the single-item processing time, you can assume (for lack of better
measurement information) that you can get a throughput that will be
something like (single item rate * number of parallel processes * 0.8)
before things got to hell and the queue depths start to blow out the top.

        Loren

Re: max-conn-per-child spamd flag?

[Jake Colman <co...@ppllc.com>]

>>>>> "LW" == Loren Wilton <lw...@earthlink.net> writes:

   >> I just noticed the --max-conn-per-child option in the spamd man page.
   >> While the description is fairly straightforward, I'm curious if anyone
   >> else is using this

   LW> Yes, many people

   >> , why, and if it's helped with spamd processes consuming RAM (which is
   >> what I'm trying to fix at the moment).

   LW> Which is why people are using it.  In 3.0.1 and to a lesser extent in
   LW> 3.0.2, spamd children could get real fat and stay that way for a long
   LW> time, eating the machine for dinner.  Setting max-con-per-child to a
   LW> lower number will make the fat kids die quicker, so there is much less
   LW> chance of them all being fat at once.

   LW> Depending on the weather, phase of the moon, amount of ram you have,
   LW> and other things, you may find limits from a couple hundred down to
   LW> about 5 to be appropriate.  I'd probably suggest 100 to 50 as a good
   LW> starting range to play with.

   LW> Dont forget the -m option.  If you have more than about 5 children
   LW> running and don't have a huge email flow you might do well to cut the
   LW> number of children down to the 3 to 10 range.


Loren,

What is considered "huge email flow" and what are appropriate values for
connections and children?

I have a home network and use it to host a mail server for my published
domain.  Almost 90% of the email received by my server is spam.  A huge
number of additional sendmail connections are rejected because of unknown
users are other invalid crap.  In the past 24 hours I received appx 1000
valid messages if which only appx 100 were non-spam.  I received over 45000
invalid smtp connections over the same period.

In order to sopy my server (running on old harware) from being brought to its
knees, I use the following spamd/sendmail configuration:

spamd: default number of children
sendmail: daemon_children 25, rate_throttle 3, rcpt_throttle 3

Does this make sense?  Should I (can I) reduce the numebr of sendmail
children to better match spamd?

Thanks for your help and insight.

...Jake

-- 
Jake Colman
Sr. Applications Developer
Principia Partners LLC
Harborside Financial Center
1001 Plaza Two
Jersey City, NJ 07311
(201) 209-2467
www.principiapartners.com

Re: max-conn-per-child spamd flag?

[Loren Wilton <lw...@earthlink.net>]

> I just noticed the --max-conn-per-child option in the spamd man page.
While
> the description is fairly straightforward, I'm curious if anyone else is
> using this

Yes, many people

, why, and if it's helped with spamd processes consuming RAM
> (which is what I'm trying to fix at the moment).

Which is why people are using it.  In 3.0.1 and to a lesser extent in 3.0.2,
spamd children could get real fat and stay that way for a long time, eating
the machine for dinner.  Setting max-con-per-child to a lower number will
make the fat kids die quicker, so there is much less chance of them all
being fat at once.

Depending on the weather, phase of the moon, amount of ram you have, and
other things, you may find limits from a couple hundred down to about 5 to
be appropriate.  I'd probably suggest 100 to 50 as a good starting range to
play with.

Dont forget the -m option.  If you have more than about 5 children running
and don't have a huge email flow you might do well to cut the number of
children down to the 3 to 10 range.

        Loren