You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Martin Gregorie <ma...@gregorie.org> on 2011/11/27 17:52:32 UTC
Has the effect of '__' changed recently?
The SA wiki says: "rules starting with a double undescore are
evaluated with no score, and are intended for use in meta rules where
you don't want the sub-rules to have a score." on this page:
http://wiki.apache.org/spamassassin/WritingRules and yes, the typo
(undescore) is on the web page - thats a cut and paste, not copy typing.
I read this as saying that there is no default score for subrules whose
name starts with a double underscore, i.e., that given the rules:
body __SUB1 /test/
body __SUB2 /message/
meta USTEST1 (__SUB1 + __SUB2 > 1.0)
meta USTEST2 (__SUB1 && __SUB2)
body SUB3 /test/
body SUB4 /message/
meta USTEST3 (SUB3 + SUB4 > 1.0)
meta USTEST4 (SUB3 && SUB4)
and a message containing:
==================================================================
>From envelope_sender@gregorie.org
Date: Wed, 25 Mar 2009 22:00:27 -0100
From: bozo@example.com
To: martin@gregorie.org
Subject: Test message
A test message.
==================================================================
I expected that the meta USTEST1 would not fire while other three rules
(USTEST2, USTEST3, USTEST4) would. When I ran the test, all four metas
fired.
Did I mis-understand the wiki or has something changed in SA since that
wiki page was last updated?
Martin
Re: Has the effect of '__' changed recently?
Posted by RW <rw...@googlemail.com>.
On Sun, 27 Nov 2011 15:27:53 -0600 (CST)
Dave Funk wrote:
> On Sun, 27 Nov 2011, RW wrote:
>
> > If you actually want give a score to a hidden rule (to see whether
> > it's being hit), I would do it this way:
> >
> > meta BAR __FOO
> > score BAR 0.001
> >
>
> Another way to accomplish the same thing is to temporarily change your
> __FOO rules to T_FOO (simple text edit substition). Doesn't require
> any addtional lines in your config files, score added is infinitesimal
> but does show up in score report.
You have to modify the name in several places, and when you put it back
you have to revert all the changes without error and check all the
files where it might be used.
Re: Has the effect of '__' changed recently?
Posted by Martin Gregorie <ma...@gregorie.org>.
On Sun, 2011-11-27 at 15:27 -0600, Dave Funk wrote:
> On Sun, 27 Nov 2011, RW wrote:
>
> > If you actually want give a score to a hidden rule (to see whether
> > it's being hit), I would do it this way:
> >
> > meta BAR __FOO
> > score BAR 0.001
> >
>
> Another way to accomplish the same thing is to temporarily change your
> __FOO rules to T_FOO (simple text edit substition). Doesn't require
> any addtional lines in your config files, score added is infinitesimal
> but does show up in score report.
>
Sure, but the aim of the test was to see if a "score __MYSUBRULE 999"
line affected the score or not. Changing "__" to "T_" would be another
valid test, just nothing to do with what I wanted to know: why the score
line was accepted in the first place, since the dox say a rule with a
name like "__RULE" doesn't have a score.
Martin
Re: Has the effect of '__' changed recently?
Posted by Dave Funk <db...@engineering.uiowa.edu>.
On Sun, 27 Nov 2011, RW wrote:
> If you actually want give a score to a hidden rule (to see whether
> it's being hit), I would do it this way:
>
> meta BAR __FOO
> score BAR 0.001
>
Another way to accomplish the same thing is to temporarily change your
__FOO rules to T_FOO (simple text edit substition). Doesn't require
any addtional lines in your config files, score added is infinitesimal
but does show up in score report.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: Has the effect of '__' changed recently?
Posted by Martin Gregorie <ma...@gregorie.org>.
On Sun, 2011-11-27 at 20:04 +0000, RW wrote:
> On Sun, 27 Nov 2011 19:31:22 +0000
> Martin Gregorie wrote:
>
> I also notice, because I tried it to see what happens, that you can
> > submit a score line for a rule with a __ name prefix without an error
> > being reported. Is that line silently thrown away?
>
> I don't know. You could try it, but since it's a perverse thing to
> do, and not documented, I would regard it as undefined behaviour.
> If you actually want give a score to a hidden rule (to see whether
> it's being hit), I would do it this way:
>
I set up:
body US_MG1 /message/
score US_MG1 5.0
meta US_MGM1 (US_MG1)
and ran it against a message containing 'message'. Both fired, adding 6
to the score compared to the score when these rules were not part of the
configuration, then I added '__' to all three references to US_MG1 and
ran it again. This time only 1 was added to the score.
So, my guess is right: if a message with a '__' name prefix is supplied
with a score line it is silently discarded: would it be better if an
error was output too?
Martin
Re: Has the effect of '__' changed recently?
Posted by jdow <jd...@earthlink.net>.
Even with that, RW, he can't have been running long enough to give that number.
He needs a decent sample of failures before his number is better a figure at
least ten times the figure he gave.
And NO system with that many mails fails to make false positives unless one is
arrogant enough to declare, "What I call spam is by definition spam regardless
of customer complaints." One person's spam is another person's ham. That's why
Loren and I tune our own anti-spam operations to our particular tastes. (Grin,
big boobies do nothing for me, for example. I'd have to wear them and that
would be awesomely uncomfortable. {^_-})
Even gmail mismarks spam on outgoing email. I want to send zip files of test
builds for a program I am maintaining for a customer. Gmail makes me run through
stupid renaming tricks which I could automate in the spam engine if I wished to
send .zip spam. And so far all ISPs I've encountered "mismark spam" by default.
I want to filter and markup my own spam with the spam score in the early part
of the marked up title. Then it is easy to search for mismarked ham email. That
way I don't have to use use-resistant web mail tools and read all the heaeders
to guess which "spams" were really ham.
Noel-boy has not thought through his problem. (And it dawned on me there IS a
way to avoid all false positives. Mark all spam as ham.)
{^_-}
On 2011/11/27 12:04, RW wrote:
> On Sun, 27 Nov 2011 19:31:22 +0000
> Martin Gregorie wrote:
>
> I also notice, because I tried it to see what happens, that you can
>> submit a score line for a rule with a __ name prefix without an error
>> being reported. Is that line silently thrown away?
>
> I don't know. You could try it, but since it's a perverse thing to
> do, and not documented, I would regard it as undefined behaviour.
>
> If you actually want give a score to a hidden rule (to see whether
> it's being hit), I would do it this way:
>
> meta BAR __FOO
> score BAR 0.001
>
Re: Has the effect of '__' changed recently?
Posted by RW <rw...@googlemail.com>.
On Sun, 27 Nov 2011 19:31:22 +0000
Martin Gregorie wrote:
I also notice, because I tried it to see what happens, that you can
> submit a score line for a rule with a __ name prefix without an error
> being reported. Is that line silently thrown away?
I don't know. You could try it, but since it's a perverse thing to
do, and not documented, I would regard it as undefined behaviour.
If you actually want give a score to a hidden rule (to see whether
it's being hit), I would do it this way:
meta BAR __FOO
score BAR 0.001
Re: Has the effect of '__' changed recently?
Posted by Martin Gregorie <ma...@gregorie.org>.
On Sun, 2011-11-27 at 18:40 +0000, RW wrote:
> On Sun, 27 Nov 2011 18:08:55 +0000
> Martin Gregorie wrote:
>
>
> > Yes, thats clear, but what is the Wiki statement I quoted about rules
> > whose name starts with a double underscore meant to mean? Merely that
> > any attempt to add a score line for such a rule will be rejected?
>
> No, point is that rules that are not explicitly scored and which
> don't start with __ are assigned a default score of 1.0.
>
OK.
> Using __ for subrules means that they wont clutter x-spam-status and
> that you wont need to bother with lot of 0.001 scores.
>
OK.
I also notice, because I tried it to see what happens, that you can
submit a score line for a rule with a __ name prefix without an error
being reported. Is that line silently thrown away?
Martin
Re: Has the effect of '__' changed recently?
Posted by RW <rw...@googlemail.com>.
On Sun, 27 Nov 2011 18:08:55 +0000
Martin Gregorie wrote:
> Yes, thats clear, but what is the Wiki statement I quoted about rules
> whose name starts with a double underscore meant to mean? Merely that
> any attempt to add a score line for such a rule will be rejected?
No, point is that rules that are not explicitly scored and which
don't start with __ are assigned a default score of 1.0.
Using __ for subrules means that they wont clutter x-spam-status and
that you wont need to bother with lot of 0.001 scores.
Re: Has the effect of '__' changed recently?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
> Yes, thats clear, but what is the Wiki statement I quoted about rules
> whose name starts with a double underscore meant to mean? Merely that
> any attempt to add a score line for such a rule will be rejected?
Pretty much, yes. Rules starting with __ are intended to be used in
meta rules and never scored independently.
Regards,
KAM
Re: Has the effect of '__' changed recently?
Posted by Martin Gregorie <ma...@gregorie.org>.
On Sun, 2011-11-27 at 17:46 +0000, RW wrote:
> On Sun, 27 Nov 2011 12:14:08 -0500
> Kevin A. McGrail wrote:
>
>
> > The score of a rule has nothing to do with the arithmetic for meta
> > operations in determining if the rule is true. Specifically "An
> > arithmetic meta rule can be used to tell if more than a certain
> > number of sub rules matched."
>
> Most rules are boolean and have a value of 1 or 0. They can have other
> values, firstly through the use of "tflags multiple" which means the
> rule counts the number of matches, and secondly through the use of
> meta rule arithmetic e.g.
>
> meta BAR 1.1*FOO1 + 0.5*FOO2
>
Yes, thats clear, but what is the Wiki statement I quoted about rules
whose name starts with a double underscore meant to mean? Merely that
any attempt to add a score line for such a rule will be rejected?
Martin
Re: Has the effect of '__' changed recently?
Posted by RW <rw...@googlemail.com>.
On Sun, 27 Nov 2011 12:14:08 -0500
Kevin A. McGrail wrote:
> The score of a rule has nothing to do with the arithmetic for meta
> operations in determining if the rule is true. Specifically "An
> arithmetic meta rule can be used to tell if more than a certain
> number of sub rules matched."
Most rules are boolean and have a value of 1 or 0. They can have other
values, firstly through the use of "tflags multiple" which means the
rule counts the number of matches, and secondly through the use of
meta rule arithmetic e.g.
meta BAR 1.1*FOO1 + 0.5*FOO2
Re: Has the effect of '__' changed recently?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
To answer your subject, no, nothing with __ has changed in quite some
time that I can recollect. More information below but I believe you are
misreading the docs.
On 11/27/2011 11:52 AM, Martin Gregorie wrote:
> The SA wiki says: "rules starting with a double undescore are
> evaluated with no score, and are intended for use in meta rules where
> you don't want the sub-rules to have a score." on this page:
> http://wiki.apache.org/spamassassin/WritingRules and yes, the typo
> (undescore) is on the web page - thats a cut and paste, not copy typing.
>
> I read this as saying that there is no default score for subrules whose
> name starts with a double underscore,
The score of a rule has nothing to do with the arithmetic for meta
operations in determining if the rule is true. Specifically "An
arithmetic meta rule can be used to tell if more than a certain number
of sub rules matched."
In pseudo rule code:
Rule __TEST1
Rule __TEST2
Rule TEST3
Score TEST3 0.05
Rule 4 T_TEST4
meta test (__TEST1 + __TEST2 + TEST3 + T_TEST4 >= 4) will be true if all
4 rules hit regardless of their underlying score math which would be 0 +
0 + 0.05 + 0.01 based on the info above.
So the meta is more of a (hit/no hit) check not a check of the score.
I fixed the typo thank you.
> i.e., that given the rules:
>
> body __SUB1 /test/
> body __SUB2 /message/
> meta USTEST1 (__SUB1 + __SUB2> 1.0)
> meta USTEST2 (__SUB1&& __SUB2)
>
> body SUB3 /test/
> body SUB4 /message/
> meta USTEST3 (SUB3 + SUB4> 1.0)
> meta USTEST4 (SUB3&& SUB4)
>
> and a message containing:
>
> ==================================================================
> > From envelope_sender@gregorie.org
> Date: Wed, 25 Mar 2009 22:00:27 -0100
> From: bozo@example.com
> To: martin@gregorie.org
> Subject: Test message
>
> A test message.
> ==================================================================
>
> I expected that the meta USTEST1 would not fire while other three rules
> (USTEST2, USTEST3, USTEST4) would. When I ran the test, all four metas
> fired.
>
> Did I mis-understand the wiki or has something changed in SA since that
> wiki page was last updated?
>
Yes, all four metas should hit. __SUB1 & __SUB2 will be merged (see
below). But yes, the message includes "test" and "message" so all 4
rules will be true.
I use my initials in my rules so I can watch them in a debug situation.
So for example, I added KAM_ to all your rules and created an mbox valid
email from your example above.
As you can see below, there is merger of the duplicate rules and the
meta rules all hit.
spamassassin -t -D < /tmp/2.mbox 2>&1 | grep -i KAM
Nov 27 11:27:03.661 [22589] dbg: config: read file
/etc/mail/spamassassin/KAM-devel.cf
Nov 27 11:27:06.849 [22589] dbg: rules: KAM_SUB4 merged duplicates:
__KAM_SUB2
Nov 27 11:27:06.853 [22589] dbg: rules: KAM_SUB3 merged duplicates:
__KAM_SUB1
Nov 27 11:27:07.781 [22589] dbg: rules: ran body rule KAM_SUB4 ======>
got hit: "message"
Nov 27 11:27:07.782 [22589] dbg: rules: ran body rule KAM_SUB3 ======>
got hit: "test"
Nov 27 11:27:08.637 [22589] dbg: check:
tests=KAM_SUB3,KAM_SUB4,KAM_USTEST1,KAM_USTEST2,KAM_USTEST3,KAM_USTEST4,MISSING_MID,NO_RECEIVED,NO_RELAYS
Nov 27 11:27:08.637 [22589] dbg: check:
subtests=__GATED_THROUGH_RCVD_REMOVER,__HAS_DATE,__HAS_FROM,__HAS_SUBJECT,__KAM_SUB1,__KAM_SUB2,__MISSING_REF,__MISSING_REPLY,__NAKED_TO,__NONEMPTY_BODY,__NOT_SPOOFED,__TOCC_EXISTS,__TO_NO_ARROWS_R,__UNUSABLE_MSGID
X-Spam-Status: Yes, score=6.1 required=5.0
tests=KAM_SUB3,KAM_SUB4,KAM_USTEST1,
KAM_USTEST2,KAM_USTEST3,KAM_USTEST4,MISSING_MID,NO_RECEIVED,NO_RELAYS
1.0 KAM_SUB4 BODY: KAM_SUB4
1.0 KAM_SUB3 BODY: KAM_SUB3
1.0 KAM_USTEST3 KAM_USTEST3
1.0 KAM_USTEST4 KAM_USTEST4
1.0 KAM_USTEST1 KAM_USTEST1
1.0 KAM_USTEST2 KAM_USTEST2
1.0 KAM_SUB4 BODY: KAM_SUB4
1.0 KAM_SUB3 BODY: KAM_SUB3
1.0 KAM_USTEST3 KAM_USTEST3
1.0 KAM_USTEST4 KAM_USTEST4
1.0 KAM_USTEST1 KAM_USTEST1
1.0 KAM_USTEST2 KAM_USTEST2
Regards,
KAM