Re: Help with SA / Procmail regex - Back On-Topic

[Karsten Bräckelmann <gu...@rudersport.de>]

> > > If there is even the slightest chance, your MTA might flood your MDA
> > > with mail during a peek -- add some explicit locking here, even though
> > > this is not a delivery receipt (explicit, because it is a filter, and
> > > procmail can't lock the target file). IIRC the SA docs do have a lock
> > > there, too.
> > > 
> > > :0 fw: spamassassin.lock
> > > * < 512000
> > > | spamc
> > > 
> > > If you don't lock, you might end up with more mail being piped to spamc
> > > consecutively, than your spamd can generate children.
> 
> I've just been doing a little reading...
> http://www.issociate.de/board/post/232336/Lock_failure_on_%22spamc.lock%22.html

Well, Nancy is wrong. :)  Anyway, she claims a lock file is not
necessary, and refers to a page by her, discussing this this:

> http://www.ii.com/internet/robots/procmail/qs/#SA
> 
> which tend to suggest that one should NOT put a lock on for SA processing...

Even though the "discussion" is rather sparse, there are some comments
outlining this. And while she indeed suggest in the post that one
"should not" lock here, her discussion merely mentions that is is "not
needed". The latter is correct, from a procmail POV.


Also, here reasoning is slightly inconsistent. Again, the only comment
regarding locking when filtering through spamc is, that it is not
necessary. However, she *does* use locking with the spamassassin filter
example. The reason being, that this comes with a penalty of drastically
increased CPU usage. Right, again. And right to use a lock.

Now, as far as procmail is concerned, there is no difference between a
slow, CPU intensive filter with a few messages -- and a speedy filter
that is being called *much* more often, because you got more mail to
process.


I stand to what I said. Use locking. See the top-most part of this post.
Locking can be useful, if you expect spikes [1] of messages. Simply to
prevent concurrent calls to spamc from exhausting your spamd max child
limit.

And yes, I do have seen this myself. Mail will go through unfiltered.

  guenther


[1] In particular, running fetchmail will result in spikes. Every
    $interval seconds.

-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Help with SA / Procmail regex - Back On-Topic

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2008-01-31 at 22:57 +0000, Arthur Dent wrote:
> On Thu, Jan 31, 2008 at 11:35:17PM +0100, Karsten Bräckelmann wrote:

> Thanks for this. I appreciate you taking the time to explain this. I respect
> your opinion so I have the locks in place just as you suggest.
> 
> Incidentally, with those locks in place (including on the copy section, and
> incorporating the "numerical" filter just as you wrote it, I have now had a
> range of real mail. This includes some regular mail, some "normal" spam and some
> "numerical" spam. None of them triggered that lock error.
> 
> As well as my test message, I piped an old "real" spam message through and
> they *both* caused the error.
> 
> It seems therefore that there is only a problem when Procmail tries to copy a message 
> that already exists into the backup mbox(?) Could this be?

No.  Procmail does by no means check the content. It merely appends.

Also, the contents of your test messages are entirely irrelevant, as
long as procmail happily processes them and the receipt matches.

> It's strange because there 
> is no such problem with the "IN-Spam" mbox. There can be as many copies as I like in there...
> I have checked and the permissions are identical (and same owner/group too). The *only* 
> difference is that the backup mbox lives on a different partition (ext3 fs mounted with 
> fstab). Could that be significant?

Nope. So this locking issue actually happens with any test message?
Still looks like some kind of permission problem the way you call
procmail for testing.

> Anyway, test messages aside, real mail (spam or otherwise) seems to be
> processed without problem, so, once again...
> 
> Thanks!

You're welcome.

  guenther


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Help with SA / Procmail regex - Back On-Topic

[Arthur Dent <sa...@troodos.demon.co.uk>]

On Thu, Jan 31, 2008 at 11:35:17PM +0100, Karsten Bräckelmann wrote:
> 
> > > > If there is even the slightest chance, your MTA might flood your MDA
> > > > with mail during a peek -- add some explicit locking here, even though
> > > > this is not a delivery receipt (explicit, because it is a filter, and
> > > > procmail can't lock the target file). IIRC the SA docs do have a lock
> > > > there, too.
> > > > 
> > > > :0 fw: spamassassin.lock
> > > > * < 512000
> > > > | spamc
> > > > 
> > > > If you don't lock, you might end up with more mail being piped to spamc
> > > > consecutively, than your spamd can generate children.
> > 
> > I've just been doing a little reading...
> > http://www.issociate.de/board/post/232336/Lock_failure_on_%22spamc.lock%22.html
> 
> Well, Nancy is wrong. :)  Anyway, she claims a lock file is not
> necessary, and refers to a page by her, discussing this this:
> 
> > http://www.ii.com/internet/robots/procmail/qs/#SA
> > 
> > which tend to suggest that one should NOT put a lock on for SA processing...
> 
> Even though the "discussion" is rather sparse, there are some comments
> outlining this. And while she indeed suggest in the post that one
> "should not" lock here, her discussion merely mentions that is is "not
> needed". The latter is correct, from a procmail POV.
> 
> 
> Also, here reasoning is slightly inconsistent. Again, the only comment
> regarding locking when filtering through spamc is, that it is not
> necessary. However, she *does* use locking with the spamassassin filter
> example. The reason being, that this comes with a penalty of drastically
> increased CPU usage. Right, again. And right to use a lock.
> 
> Now, as far as procmail is concerned, there is no difference between a
> slow, CPU intensive filter with a few messages -- and a speedy filter
> that is being called *much* more often, because you got more mail to
> process.
> 
> 
> I stand to what I said. Use locking. See the top-most part of this post.
> Locking can be useful, if you expect spikes [1] of messages. Simply to
> prevent concurrent calls to spamc from exhausting your spamd max child
> limit.
> 
> And yes, I do have seen this myself. Mail will go through unfiltered.
> 
>   guenther
> 
> 
> [1] In particular, running fetchmail will result in spikes. Every
>     $interval seconds.
> 
Hi Guenther,

Thanks for this. I appreciate you taking the time to explain this. I respect
your opinion so I have the locks in place just as you suggest.

Incidentally, with those locks in place (including on the copy section, and
incorporating the "numerical" filter just as you wrote it, I have now had a
range of real mail. This includes some regular mail, some "normal" spam and some
"numerical" spam. None of them triggered that lock error.

As well as my test message, I piped an old "real" spam message through and
they *both* caused the error.

It seems therefore that there is only a problem when Procmail tries to copy a message 
that already exists into the backup mbox(?) Could this be? It's strange because there 
is no such problem with the "IN-Spam" mbox. There can be as many copies as I like in there...
I have checked and the permissions are identical (and same owner/group too). The *only* 
difference is that the backup mbox lives on a different partition (ext3 fs mounted with 
fstab). Could that be significant?

Anyway, test messages aside, real mail (spam or otherwise) seems to be
processed without problem, so, once again...

Thanks!

Mark