Perl, fork, and copy-on-write (was Re: setup spamassassin without amavisd)

["David F. Skoll" <df...@roaringpenguin.com>]

On Mon, 9 Jul 2012 09:06:39 +0300
Henrik K <he...@hege.li> wrote:

> You can easily run many children since amavisd or spamd forks are
> copy-on-writed pretty well.  So only extra memory used is the per
> scan state and file data etc.

Have you actually measured this?  My experience is that forked Perl
processes end up sharing surprisingly little memory.  The reason is
that Perl uses reference-counting to track liveness (so it can free
memory when the reference count reaches zero.)  This turns a lot of
what should be read-only accesses into writes and severely hurts
memory page sharing.

Be careful.

Regards,

David.

Re: Perl, fork, and copy-on-write (was Re: setup spamassassin without amavisd)

[Henrik K <he...@hege.li>]

On Mon, Jul 09, 2012 at 04:06:48AM -0400, David F. Skoll wrote:
> On Mon, 9 Jul 2012 09:06:39 +0300
> Henrik K <he...@hege.li> wrote:
> 
> > You can easily run many children since amavisd or spamd forks are
> > copy-on-writed pretty well.  So only extra memory used is the per
> > scan state and file data etc.
> 
> Have you actually measured this?  My experience is that forked Perl
> processes end up sharing surprisingly little memory.  The reason is
> that Perl uses reference-counting to track liveness (so it can free
> memory when the reference count reaches zero.)  This turns a lot of
> what should be read-only accesses into writes and severely hurts
> memory page sharing.

Yes I tested it quite a bit with amavis, sadly I don't have my notes
anymore.  I think each active child added around 5-10MB to used system
memory seen with top/free commands.

Since childs can be configured to be shut down after x requests, it also
reduces any "risks".  Of course YMMV and there could be flaws in my testing. 
But if 20 childs happily start and run on my small VPS, it should already
tell something.  ;-)