How to avoid filtering twice when having mail-groups

[Manu <m_...@fastmail.fm>]

Hi all,

I'm using SpamAssassin 3.0.2 and Qmail on a Debian Sarge Server. 
Administrative hosting panel is Plesk 8.1.

Imagine the following situation:
mailgroup@server.org forwards to user1@server.org, user2@server.org,...

Now if SpamAssassin checks mailgroup@server.org and each user has enabled spam 
filtering too, SpamAssassin will filter the same message once for mailgroup@ 
and then once again for each user. We'll get: 1 + #users scans.
If I disable scanning of mailgroup@, we'll get #user scans for the very same 
message.

What I would like to have: SpamAssassin scans for mailgroup@ and when the very 
same message has to be scanned for each user, SpamAssassin remembers that 
this message has already been scanned seconds ago and doesn't scan it again.

SpamAssassin sometimes needs 15 seconds to process a message, so you can 
imagine that this will save much time and ressources for mailgroups with many 
recipients.

Any chance to get this working?

Thanks in advance.

-- 
Cheers, Manu

Re: How to avoid filtering twice when having mail-groups

[Matthias Haegele <mh...@linuxrocks.dyndns.org>]

Manu schrieb:
> Hi all,
> 
> I'm using SpamAssassin 3.0.2 and Qmail on a Debian Sarge Server. 
> Administrative hosting panel is Plesk 8.1.
> 
> Imagine the following situation:
> mailgroup@server.org forwards to user1@server.org, user2@server.org,...
> 
> Now if SpamAssassin checks mailgroup@server.org and each user has enabled spam 
> filtering too, SpamAssassin will filter the same message once for mailgroup@ 
> and then once again for each user. We'll get: 1 + #users scans.
> If I disable scanning of mailgroup@, we'll get #user scans for the very same 
> message.
> 
> What I would like to have: SpamAssassin scans for mailgroup@ and when the very 
> same message has to be scanned for each user, SpamAssassin remembers that 
> this message has already been scanned seconds ago and doesn't scan it again.
> 
> SpamAssassin sometimes needs 15 seconds to process a message, so you can 
> imagine that this will save much time and ressources for mailgroups with many 
> recipients.
> 
> Any chance to get this working?

I cant imagine this could be done.
Cause SA gets every "single recipient" from MTA, if possible i think you 
would have to put another filter/script in the chain to pass only 1 
address if certain circumstances (aliases for a address are there).
But perhaps a more experienced user here, might know more ...

> Thanks in advance.


-- 
Grüsse/Greetings
MH


Dont send mail to: ubecatcher@linuxrocks.dyndns.org
--

Re: How to avoid filtering twice when having mail-groups

[Manu <m_...@fastmail.fm>]

Hi Kris,

> Well, here's a starting point for procmail-based processing.  Note that
> I haven't checked this to actually be correct procmail syntax, or that
> it's logically correct.  <g>  I'm pretty sure it should work the way
> you're looking for, however.

Okay, I see. Thanks for this example.

> Munge to taste;  man procmail, man procmailrc and man procmailex should
> be useful references.

Sure. ;)

> If your SA calls are happening somewhere else, you'll have to work out
> what sort of syntax you need to add an arbitrary header on messages from
> a certain source, and then not call per-user SA for messages that have
> that header added.

Should be no problem. The .procmailrc-Files can't actually be managed by a 
third-party script, just by something I wrote myself. So I can modify it as 
needed.
The thing I would do, is to check for SpamAssassins header with the name of my 
server. This would be enough and I don't have to use formail to add an extra 
header.

Thanks for the hints, sendmail+procmail guy. ;)

-- 
Regards, Manu

Re: How to avoid filtering twice when having mail-groups

[Kris Deugau <kd...@vianet.ca>]

Manu wrote:
> I'll pipe it into a new file and work with grep from this point on.
> Works nice :)
[snip shell]

IIRC there's suitable syntax for a procmail recipe to do what you want
(I don't think you need formail for anything other than adding a custom
header;  procmail can match content in headers via regex just fine);
but unless you're really running high load it's probably not worth the
effort to find out.  <g>

-kgd

Re: How to avoid filtering twice when having mail-groups

[Manu <m_...@fastmail.fm>]

Hello again,

sorry for writing that often. :)

> Perhaps you - or someone else - could help me with this:
> formail reads from stdin. Okay, so I want to extract a header with -X.
> But spamc also reads from stdin and stdin is empty after formail has read
> the mail. So, how do I get a program to "write to stdin". Hmm, understand
> what I mean?

I'll pipe it into a new file and work with grep from this point on.
Works nice :)

cat > "${tmpfile}orig"
grep -q "^X-Spam-Checker-Version: SpamAssassin [^o]* on 
name.myserver.org$" "${tmpfile}orig"
if ! [ $? = 0 ] ; then
        "$SPAMC" -s "$MAX_SIZE" -u "$MAILNAME" -U "$SOCKETPATH" -E  
< "${tmpfile}orig" > "$tmpfile"
        check=$?
else
        grep -q "X-Spam-Flag: YES" "${tmpfile}orig"
        if [ $? = 0 ] ; then
                check=1
        else
                check=0
        fi
fi

Thank you!

-- 
Regards, Manu

Re: How to avoid filtering twice when having mail-groups

[Manu <m_...@fastmail.fm>]

Kris, it's me again,

> I have no idea.  I'm a sendmail+procmail guy.  <g>

Perhaps you - or someone else - could help me with this:
formail reads from stdin. Okay, so I want to extract a header with -X.
But spamc also reads from stdin and stdin is empty after formail has read the 
mail. So, how do I get a program to "write to stdin". Hmm, understand what I 
mean?

Example:
It now looks like this:

formail -a "X-My-Extra-Header: 
Checked" | "$SPAMC" -s "$MAX_SIZE" -u "$MAILNAME" -U "$SOCKETPATH" -E 
> "$tmpfile"
check=$?

But before this command, there should be a check for that header. This has to 
look like this:

if( formail -X "X-My-Extra-Header: Checked == true ) {
  write to "$tmpfile"
}else{
  "command above"
}

Know what I mean and where my problem is? :)

-- 
Regards, Manu

Re: How to avoid filtering twice when having mail-groups

[Kris Deugau <kd...@vianet.ca>]

Manu wrote:
> Great, that's what I want.
> Currently my .qmail-Files look like this:
> 
> | /opt/psa/bin/psa-spamc accept
> | preline /usr/bin/procmail -m -o .procmailrc
> 
> So I have to insert a line before the first line, right?

I have no idea.  I'm a sendmail+procmail guy.  <g>

> Would you be so kind 
> to help me a bit? :)

Well, here's a starting point for procmail-based processing.  Note that
I haven't checked this to actually be correct procmail syntax, or that
it's logically correct.  <g>  I'm pretty sure it should work the way
you're looking for, however.

In /etc/procmailrc:

:0fw
| spamassassin

:0fw
* ^From:.*groupname@source.tld
| formail -a "X-my-special-header:  PTHBTT"

Then in the individual .procmailrc files:

:0fw
* !^X-my-special-header:  PTHBTT
| spamassassin

Munge to taste;  man procmail, man procmailrc and man procmailex should
be useful references.

If your SA calls are happening somewhere else, you'll have to work out
what sort of syntax you need to add an arbitrary header on messages from
a certain source, and then not call per-user SA for messages that have
that header added.

-kgd

Re: How to avoid filtering twice when having mail-groups

[Manu <m_...@fastmail.fm>]

Kris Deugau wrote:
> > I'm using SpamAssassin 3.0.2 and Qmail on a Debian Sarge Server.
> > Administrative hosting panel is Plesk 8.1.
>
> Between qmail and Plesk, what you're looking for may not be possible
> with the obvious exposed controls;  you may need to manually dig into
> configuration files to get the behaviour you want.

No problem. Filtering is done by my own script which reads the Plesk-data.

> On systems I'm responsible for (all currently SA3.1.8 + sendmail with no
> formal "hosting panel"), I'd add a header during the "systemwide" scan,
> and then check for that header during the per-user processing.  If the
> header was present, don't call SA again.

Great, that's what I want.
Currently my .qmail-Files look like this:

| /opt/psa/bin/psa-spamc accept
| preline /usr/bin/procmail -m -o .procmailrc

So I have to insert a line before the first line, right? Would you be so kind 
to help me a bit? :)

Thanks!

-- 
Regards, Manu

Re: How to avoid filtering twice when having mail-groups

[Kris Deugau <kd...@vianet.ca>]

Manu wrote:
> I'm using SpamAssassin 3.0.2 and Qmail on a Debian Sarge Server. 
> Administrative hosting panel is Plesk 8.1.

Between qmail and Plesk, what you're looking for may not be possible
with the obvious exposed controls;  you may need to manually dig into
configuration files to get the behaviour you want.

> Imagine the following situation:
> mailgroup@server.org forwards to user1@server.org, user2@server.org,...
> 
> Now if SpamAssassin checks mailgroup@server.org and each user has enabled spam 
> filtering too, SpamAssassin will filter the same message once for mailgroup@ 
> and then once again for each user. We'll get: 1 + #users scans.
> If I disable scanning of mailgroup@, we'll get #user scans for the very same 
> message.
> 
> What I would like to have: SpamAssassin scans for mailgroup@ and when the very 
> same message has to be scanned for each user, SpamAssassin remembers that 
> this message has already been scanned seconds ago and doesn't scan it again.

SA doesn't have any sort of capability like that itself;  it scans what
it's been handed.

On systems I'm responsible for (all currently SA3.1.8 + sendmail with no
formal "hosting panel"), I'd add a header during the "systemwide" scan,
and then check for that header during the per-user processing.  If the
header was present, don't call SA again.

I'm pretty sure what you want is *possible*...  but I can't say whether
it's possible via your hosting panel, *easy* but you have to dig into
back-end configuration files, or possible but bury-your-head-in-MTA-guts
*nasty* to set up.

-kgd