You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Thomas Barth <tb...@txbweb.de> on 2024/05/09 18:41:27 UTC
Score 0.001
Hello,
I don't understand why there are so many checks where the meaningless
value of 0.001 is assigned. The total score could be much higher. Do I
have to define all the checks myself with a desired value?
X-Spam-Status: No, score=3.999 tagged_above=2 required=6.31
tests=[DMARC_MISSING=0.001, FSL_BULK_SIG=0.001,
HTML_IMAGE_RATIO_02=0.001,
HTML_MESSAGE=0.001, PYZOR_CHECK=1.985, RELAYCOUNTRY_BAD=2,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_TVD_MIME_EPI=0.01]
or
X-Spam-Status: Yes, score=7.281 tagged_above=2 required=6.31
tests=[DMARC_MISSING=0.001, FSL_BULK_SIG=0.001,
HTML_FONT_LOW_CONTRAST=0.001,
HTML_IMAGE_ONLY_24=1.282, HTML_IMAGE_RATIO_02=0.001,
HTML_MESSAGE=0.001,
MIXED_HREF_CASE=1.999, PYZOR_CHECK=1.985, RELAYCOUNTRY_BAD=2,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_TVD_MIME_EPI=0.01]
Debian 12.5
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13)
Re: Score 0.001
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2024-05-10 at 14:15:56 UTC-0400 (Fri, 10 May 2024 14:15:56 -0400)
Bill Cole <sa...@billmail.scconsult.com>
is rumored to have said:
> On 2024-05-09 at 18:19:14 UTC-0400 (Thu, 9 May 2024 15:19:14 -0700)
> jdow <jd...@earthlink.net>
> is rumored to have said:
>
>> On 20240509 15:05:46, Thomas Barth wrote:
>>> Am 2024-05-09 21:41, schrieb Loren Wilton:
>>>> Low-score tests are neither spam nor ham signs by themselves. They can be used in metas in conjunction with other indicators to help determine ham or spam. A zero value indicates that a rule didn't hit and the sign is not present. A small score indicates that the rule did hit, so the sign it is detecting is present.
>>>
>>> 0.001 seems to be the default lowest value. Is it possible to change it to 0.01 or 0.1?
>
> Sure. It's just a number.
Clarifying; You can change any score yourself on your own system locally if you like, but to make no rule ever score 0.001 you'd need to fix the scores for all low-score rules every time that you run sa-update. As John Hardin says, we will not be changing the default to 0.1 in the rules distribution; that would be too significant a value. I also think that there is value in having matched rules showing up in the long form (folded header) of the SA report with "0.0" if they are intended to have no direct impact on the ham/spam decision.
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
Re: Score 0.001
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2024-05-09 at 18:19:14 UTC-0400 (Thu, 9 May 2024 15:19:14 -0700)
jdow <jd...@earthlink.net>
is rumored to have said:
> On 20240509 15:05:46, Thomas Barth wrote:
>> Am 2024-05-09 21:41, schrieb Loren Wilton:
>>> Low-score tests are neither spam nor ham signs by themselves. They can be used in metas in conjunction with other indicators to help determine ham or spam. A zero value indicates that a rule didn't hit and the sign is not present. A small score indicates that the rule did hit, so the sign it is detecting is present.
>>
>> 0.001 seems to be the default lowest value. Is it possible to change it to 0.01 or 0.1?
Sure. It's just a number.
> 1) This cyberunit is unwarrantedly curious, why does this matter to you?
>
> 2) Probably not as it may be related to how perl handles numbers.
Not so much. SA has no need for high-precision floating-point math so there is nothing special about 0.001 or 0.0001 or any other small number.
The reason for such low scores is to assure that the rule is checked, even if no other rule depends on it. Such rules usually are a component in multiple other meta rules that have more significant scores, but are not significantly spam or ham signs on their own.
--
Bill Cole
Re: Score 0.001
Posted by jdow <jd...@earthlink.net>.
On 20240509 15:05:46, Thomas Barth wrote:
> Am 2024-05-09 21:41, schrieb Loren Wilton:
>> Low-score tests are neither spam nor ham signs by themselves. They can be
>> used in metas in conjunction with other indicators to help determine ham or
>> spam. A zero value indicates that a rule didn't hit and the sign is not
>> present. A small score indicates that the rule did hit, so the sign it is
>> detecting is present.
>
> 0.001 seems to be the default lowest value. Is it possible to change it to
> 0.01 or 0.1?
1) This cyberunit is unwarrantedly curious, why does this matter to you?
2) Probably not as it may be related to how perl handles numbers.
{^_^}
Re: Score 0.001
Posted by jdow <jd...@earthlink.net>.
On 20240509 23:57:12, Thomas Barth wrote:
> Am 2024-05-10 06:19, schrieb Reindl Harald (privat):
>> Am 10.05.24 um 00:05 schrieb Thomas Barth:
>>> Am 2024-05-09 21:41, schrieb Loren Wilton:
>>>> Low-score tests are neither spam nor ham signs by themselves. They can be
>>>> used in metas in conjunction with other indicators to help determine ham or
>>>> spam. A zero value indicates that a rule didn't hit and the sign is not
>>>> present. A small score indicates that the rule did hit, so the sign it is
>>>> detecting is present.
>>>
>>> 0.001 seems to be the default lowest value. Is it possible to change it to
>>> 0.01 or 0.1?
>>
>> what do you not understand in meta tests?
>> it's irrelevant if it's 0.001 or 0.01
>>
>> these rules are used in combination with other rules
>>
>> HTML_MESSAGE or SPF_HELO_NONE alone don't mean anything and so it must not
>> score higher - it makes only sense combined with other rules
>
> Most of the messages I receive only have a few hits because they hardly differ
> from a regular e-mail. That's why I want to assign a higher value to the
> individual tests. I don't know how many of the possible tests with a value of
> only 0.001 exist. With this value, theoretically 1000 different tests would
> have to be positive in order to achieve a total value of 1. Therefore, it is
> not irrelevant whether I have a minimum value of 0.001 or 0.1. I would even go
> further and say if there are more than 10 tests with a positive value: Spam!
> Either the strike level is reached or there are more than 10 tests with a
> positive value. So now I repeat my question: is it possible to increase the
> minimum value to 0.1 by default?
Values like .001 are generally used for meta rules. I use meta rules chiefly to
combine multiple relatively benign tests into something significant when they
all appear at once. (Or variations on that theme.) The tiny minimum score
assures the rule gets processed and reported but has minimalist contributions to
the overall score. If you want a minimum score of 0.004, 0.01, 100, whatever
then use a "score" line associated with your rule. That saves you from the
fruits of being too lazy to give a real score to a rule that is important to get
just right.
{o.o}
Re: Score 0.001
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 12.05.24 06:39, Greg Troxel wrote:
>I would suggest that if Debian is modifying the default config from 5 to
>6.31, then
as it was already said, it's not Debian, it's default score in amavis.
Even the original header is in the amavis format:
>> X-Spam-Status: No, score=3.999 tagged_above=2 required=6.31
>> tests=[DMARC_MISSING=0.001, FSL_BULK_SIG=0.001,
Amavis has some more scores than stock SA, of course they can be modified if
your scanner is well trained.
>
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Linux - It's now safe to turn on your computer.
Linux - Teraz mozete pocitac bez obav zapnut.
Re: Score 0.001
Posted by Thomas Barth via users <us...@spamassassin.apache.org>.
Am 2024-05-13 04:33, schrieb jdow:
> Um, "FORGED_SPF_HELO"? Are you sure this message is from MS?
>
> {^_^}
The mail/report is authentic. They already corrected this "error" or
changed the sending server. In today's report FORGED_SPF_HELO is 0.001
and the score is below 5 :)
> On 20240512 06:56:59, Thomas Barth wrote:
>
>> Am 2024-05-12 12:39, schrieb Greg Troxel:
>>
>>> I would suggest that if Debian is modifying the default config
>>> from 5 to
>>> 6.31, then probably they should not be doing that.
>>
>> This is a status of dmarc-report from microsoft today
>>
>> X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
>> tests=[ARC_SIGNED=0.001, ARC_VALID=0.001,
>> BASE64_LENGTH_78_79=0.1,
>> BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001,
>> DKIM_SIGNED=0.1,
>> DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1,
>> HTML_MESSAGE=0.001,
>> MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1,
>> MPART_ALT_DIFF=0.724,
>> PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001,
>> SPF_HELO_PASS=-0.001,
>> T_TVD_MIME_NO_HEADERS=0.01]
>>
>> A strike level of 5 is too low for microsoft mails ;-)
Re: Score 0.001
Posted by jdow <jd...@earthlink.net>.
Um, "FORGED_SPF_HELO"? Are you sure this message is from MS?
{^_^}
On 20240512 06:56:59, Thomas Barth wrote:
> Am 2024-05-12 12:39, schrieb Greg Troxel:
>> I would suggest that if Debian is modifying the default config from 5 to
>> 6.31, then probably they should not be doing that.
>
> This is a status of dmarc-report from microsoft today
>
> X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
> tests=[ARC_SIGNED=0.001, ARC_VALID=0.001, BASE64_LENGTH_78_79=0.1,
> BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1,
> DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1, HTML_MESSAGE=0.001,
> MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1, MPART_ALT_DIFF=0.724,
> PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
> T_TVD_MIME_NO_HEADERS=0.01]
>
> A strike level of 5 is too low for microsoft mails ;-)
Re: Score 0.001
Posted by Benny Pedersen <me...@junc.eu>.
Thomas Barth skrev den 2024-05-12 15:56:
> Am 2024-05-12 12:39, schrieb Greg Troxel:
>> I would suggest that if Debian is modifying the default config from 5
>> to
>> 6.31, then probably they should not be doing that.
>
> This is a status of dmarc-report from microsoft today
>
> X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
> tests=[ARC_SIGNED=0.001, ARC_VALID=0.001, BASE64_LENGTH_78_79=0.1,
> BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1,
> DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1,
> HTML_MESSAGE=0.001,
> MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1, MPART_ALT_DIFF=0.724,
> PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
> T_TVD_MIME_NO_HEADERS=0.01]
>
> A strike level of 5 is too low for microsoft mails ;-)
X-Spam-Status: No, score=-0.719 tagged_above=-999 required=5
tests=[AUTHRES_DKIM_PASS=-0.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001,
KAM_NUMSUBJECT=0.4, MAILING_LIST_MULTI=-0.1, RCVD_IN_MSPIKE_H4=-0.01,
RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.1, SPF_PASS=-0.1,
USER_IN_DEF_SPF_WL=-0.2] autolearn=unavailable autolearn_force=no
AuthRes is nice :)
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Am 2024-05-12 12:39, schrieb Greg Troxel:
> I would suggest that if Debian is modifying the default config from 5
> to
> 6.31, then probably they should not be doing that.
This is a status of dmarc-report from microsoft today
X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
tests=[ARC_SIGNED=0.001, ARC_VALID=0.001, BASE64_LENGTH_78_79=0.1,
BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1,
HTML_MESSAGE=0.001,
MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1, MPART_ALT_DIFF=0.724,
PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
T_TVD_MIME_NO_HEADERS=0.01]
A strike level of 5 is too low for microsoft mails ;-)
Re: Score 0.001
Posted by Greg Troxel <gd...@lexort.com>.
I would suggest that if Debian is modifying the default config from 5 to
6.31, then
probably they should not be doing that. as a packager, I fix bugs
(and file upstream bug reports), but it's usually linuxy
nonportability things that are clearly bugs (test ==, hardcoded lists
of accepted operating systems, etc.). This is a difference in
judgement.
if they are applying a difference in judgement, the package
description should disclose this really clearly. Hard to tell what's
going on, but this appears to be new to most people here.
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Am 2024-05-11 23:49, schrieb Vincent Lefevre:
> The value 6.31 does not even appear in the spamassassin source
> package.
Sorry, the values are overwritten via the Amavis defaults.
cat /etc/debian_version
10.13
egrep -nri "sa_tag_level_deflt|sa_kill_level_deflt" /etc
/etc/amavis/conf.d/20-debian_defaults:36:$sa_tag_level_deflt = 2.0; #
add spam info headers if at, or above that level
/etc/amavis/conf.d/20-debian_defaults:38:$sa_kill_level_deflt = 6.31; #
triggers spam evasive actions
cat /etc/debian_version
12.5
egrep -nri "sa_tag_level_deflt|sa_kill_level_deflt" /etc
/etc/amavis/conf.d/20-debian_defaults:36:$sa_tag_level_deflt = 2.0; #
add spam info headers if at, or above that level
/etc/amavis/conf.d/20-debian_defaults:38:$sa_kill_level_deflt = 6.31; #
triggers spam evasive actions
Re: Score 0.001
Posted by Vincent Lefevre <vi...@vinc17.net>.
On 2024-05-11 20:26:59 +0200, Thomas Barth wrote:
> Am 2024-05-11 19:24, schrieb Loren Wilton:
[...]
> > > found in
> > >
> > > X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
> > > tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
> > > DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
> > > HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43,
> > > RAZOR2_CHECK=1.729,
> > > SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]
> >
> > Why is your score threshold for spam 6.31? By default it is 5, and that
> > message would have been spam.
>
> 6.31 has been the default value on a Debian system for ages and is based on
> the experience of the “spam analysts”.
No, Debian has not changed the default, which is 5.0, as I can see
for your message (here, on a Debian/stable machine):
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on joooj.vinc17.net
X-Spam-Status: No, score=-17.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,DMARC_PASS,HEADER_FROM_DIFFERENT_DOMAINS,
MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H4,
RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_SPF_WL
autolearn=ham autolearn_force=no version=4.0.0
The value 6.31 does not even appear in the spamassassin source
package.
--
Vincent Lefèvre <vi...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Am 2024-05-12 01:08, schrieb jdow:
> Methinks this is a perfect example of "one man's spam is another man's
> ham." Or in my case, "A woman's spam is often a man's ham."
I like spam when it's well designed. That's why I no longer reject it on
my newly set up mail server. I just want them all to be saved in the
junk folder. I sometimes admire the creativity of spammers to attract
attention to a ridiculous product :)
Re: Score 0.001
Posted by jdow <jd...@earthlink.net>.
On 20240511 14:56:51, Greg Troxel wrote:
> Thomas Barth<tb...@txbweb.de> writes:
>
>> Am 2024-05-11 21:54, schrieb Bill Cole:
>>> I have no idea who the Debian "spam analysts" are but I am certain
>>> that they are not doing any sort of data-driven dynamic adjustments
>>> of scores based on a threshold of 6.3 nor are they (obviously)
>>> adjusting that threshold daily based on current scores.
>> I found the passage in my old Postfix book. The author writes: "It is
>> recommended not to carelessly set the value of $sa_kill_level_deflt to
>> any fantasy values. The score of 6.31 is not arbitrarily chosen, but
>> the statistically calculated optimum for the best possible spam filter
>> rate with as few false positives as possible. If you increase the
>> value, more spam will get through; if you lower it, your false
>> positives will increase."
> The comments about adjustments are true, but the idea that it is optimum
> is flat-out nonsensical.
>
> The key question is how you weight a false positive compared to a false
> negative. Only after you decided that can you pick an optimium, for a
> given corpus of already-received mail.
>
>> It may be that the value is outdated, but that is for the maintainers
>> of the relevant Debian package to decide. I'll just adapt my rules to
>> this one value.
> That is an odd position. It is very easy to set the threshold in a
> local config. Deciding instead to adjust scores to an oddball
> threshold seems bizarre to me.
>
> Personally, I don't use the 5, but instead have shades of grey, where
>> =1 is binned into mailboxes that are "maybe spam" through "very likely"
> spam, and at some score, I reject at the MTA level.
>
> I find that legit mail shows up in e.g. spam.2 (>= 2 and < 3), but it is
> almost never mail that I would be upset to have missed (but I don't) or
> mail that I would be upset to not get in a timely manner (I only see it
> every day or so). However, this really drops the FN rate of spam in my
> INBOX, which matters a lot to me. Basically I consider a FP into my
> "spam.1" mailbox, as long as it isn't really important to me, to be not
> a big deal at all, and I'd rather have 10 or those than 1 FN in my
> INBOX. But, actually MTA-rejecting mail that I shouldn't, a FP at that
> level, is a big deal, and I avoid it. I think it's about one message a
> year -- and while it's ham, it's very spammy ham.
Methinks this is a perfect example of "one man's spam is another man's ham." Or
in my case, "A woman's spam is often a man's ham."
{^_-} <- hopelessly square and antiquarian by today's standards. e.g. the
XXXlist wars.
Re: Score 0.001
Posted by Greg Troxel <gd...@lexort.com>.
Thomas Barth <tb...@txbweb.de> writes:
> Am 2024-05-11 21:54, schrieb Bill Cole:
>> I have no idea who the Debian "spam analysts" are but I am certain
>> that they are not doing any sort of data-driven dynamic adjustments
>> of scores based on a threshold of 6.3 nor are they (obviously)
>> adjusting that threshold daily based on current scores.
>
> I found the passage in my old Postfix book. The author writes: "It is
> recommended not to carelessly set the value of $sa_kill_level_deflt to
> any fantasy values. The score of 6.31 is not arbitrarily chosen, but
> the statistically calculated optimum for the best possible spam filter
> rate with as few false positives as possible. If you increase the
> value, more spam will get through; if you lower it, your false
> positives will increase."
The comments about adjustments are true, but the idea that it is optimum
is flat-out nonsensical.
The key question is how you weight a false positive compared to a false
negative. Only after you decided that can you pick an optimium, for a
given corpus of already-received mail.
> It may be that the value is outdated, but that is for the maintainers
> of the relevant Debian package to decide. I'll just adapt my rules to
> this one value.
That is an odd position. It is very easy to set the threshold in a
local config. Deciding instead to adjust scores to an oddball
threshold seems bizarre to me.
Personally, I don't use the 5, but instead have shades of grey, where
>=1 is binned into mailboxes that are "maybe spam" through "very likely"
spam, and at some score, I reject at the MTA level.
I find that legit mail shows up in e.g. spam.2 (>= 2 and < 3), but it is
almost never mail that I would be upset to have missed (but I don't) or
mail that I would be upset to not get in a timely manner (I only see it
every day or so). However, this really drops the FN rate of spam in my
INBOX, which matters a lot to me. Basically I consider a FP into my
"spam.1" mailbox, as long as it isn't really important to me, to be not
a big deal at all, and I'd rather have 10 or those than 1 FN in my
INBOX. But, actually MTA-rejecting mail that I shouldn't, a FP at that
level, is a big deal, and I avoid it. I think it's about one message a
year -- and while it's ham, it's very spammy ham.
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Am 2024-05-11 21:54, schrieb Bill Cole:
> I have no idea who the Debian "spam analysts" are but I am certain that
> they are not doing any sort of data-driven dynamic adjustments of
> scores based on a threshold of 6.3 nor are they (obviously) adjusting
> that threshold daily based on current scores.
I found the passage in my old Postfix book. The author writes: "It is
recommended not to carelessly set the value of $sa_kill_level_deflt to
any fantasy values. The score of 6.31 is not arbitrarily chosen, but the
statistically calculated optimum for the best possible spam filter rate
with as few false positives as possible. If you increase the value, more
spam will get through; if you lower it, your false positives will
increase."
It may be that the value is outdated, but that is for the maintainers of
the relevant Debian package to decide. I'll just adapt my rules to this
one value.
Re: Score 0.001
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2024-05-11 at 14:26:59 UTC-0400 (Sat, 11 May 2024 20:26:59 +0200)
Thomas Barth <tb...@txbweb.de>
is rumored to have said:
> Hello
>
> Am 2024-05-11 19:24, schrieb Loren Wilton:
>>> Can I just take the names of the rules?
>>>
>>> e.g. at least two checks should fire:
>>>
>>> meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK +
>>> URIBL_ABUSE_SURBL) > 1)
>>> score MULTIPLE_TESTS 1
>>>
>>> found in
>>>
>>> X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
>>> tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
>>> DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
>>> HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43,
>>> RAZOR2_CHECK=1.729,
>>> SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]
>>
>> Why is your score threshold for spam 6.31? By default it is 5, and
>> that message would have been spam.
>
> 6.31 has been the default value on a Debian system for ages and is
> based on the experience of the “spam analysts”. That's how I
> remember it. I have therefore retained this value. Who introduced the
> default value of 5? Spamassassin itself, because spam is getting
> better and better and fewer rules apply?
5.0 has been the default threshold in the distribution forever and that
value is an assumption in the dynamic scoring and RuleQA service which
adjusts scores to their optimal values daily based on the latest results
submitted by masscheck contributors.
I have no idea who the Debian "spam analysts" are but I am certain that
they are not doing any sort of data-driven dynamic adjustments of scores
based on a threshold of 6.3 nor are they (obviously) adjusting that
threshold daily based on current scores. The only reason I can see for
boosting the threshold is if there is an additional set of rules being
used with a significant number of the non-standard low-S/O rules. For
example, if you use KAM rules (which are not part of the RuleQA process)
you will have a lot of rule hits on legit mail and you can either boost
the threshold or do a lot of local-specific FP mitigation.
On systems I manage I mostly use a *lower* threshold, because I apply
more active site-specific rule management (and FP avoidance) than most
systems ever receive.
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com
addresses)
Not Currently Available For Hire
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Hello
Am 2024-05-11 19:24, schrieb Loren Wilton:
>> Can I just take the names of the rules?
>>
>> e.g. at least two checks should fire:
>>
>> meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK +
>> URIBL_ABUSE_SURBL) > 1)
>> score MULTIPLE_TESTS 1
>>
>> found in
>>
>> X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
>> tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
>> DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
>> HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43,
>> RAZOR2_CHECK=1.729,
>> SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]
>
> Why is your score threshold for spam 6.31? By default it is 5, and that
> message would have been spam.
6.31 has been the default value on a Debian system for ages and is based
on the experience of the “spam analysts”. That's how I remember it. I
have therefore retained this value. Who introduced the default value of
5? Spamassassin itself, because spam is getting better and better and
fewer rules apply?
>
> The meta you suggest would have fired and added a point, but only
> because the combined score for the rules it mentions added up to > 1.0.
> Since every single one has a score > 1.0, the rule would have fired if
> any single one or any combination of those rules fired. Is that what
> you intended? It would not have fired if you had picked SPF_HELO_NONE,
> SPF_PASS, and FSL_BULK_SIG, as that only adds up to 0.003.
>
> Most commonly a meta for "multiple rules fired" would have used the &&
> operator, for instance:
>
> meta MY_MANY_RULES SPF_HELO_NONE && SPF_PASS && FSL_BULK_SIG
> describe MY_MANY_RULES Several random rules hit
> score MY_MANY_RULES 1
Your metarule says: all three subrules must match. My rule says, that at
least 2 subrules must match. Here it can be A+B+C, A+B, A+C, B+C. In my
rule the matches (trues) are counted. I have taken this rule from the
wiki. Please look here at meta rules
https://cwiki.apache.org/confluence/display/SPAMASSASSIN/writingrules
Re: Score 0.001
Posted by Loren Wilton <lw...@earthlink.net>.
> Can I just take the names of the rules?
>
> e.g. at least two checks should fire:
>
> meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK +
> URIBL_ABUSE_SURBL) > 1)
> score MULTIPLE_TESTS 1
>
> found in
>
> X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
> tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
> DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
> HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43, RAZOR2_CHECK=1.729,
> SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]
Why is your score threshold for spam 6.31? By default it is 5, and that
message would have been spam.
The meta you suggest would have fired and added a point, but only because
the combined score for the rules it mentions added up to > 1.0. Since every
single one has a score > 1.0, the rule would have fired if any single one or
any combination of those rules fired. Is that what you intended? It would
not have fired if you had picked SPF_HELO_NONE, SPF_PASS, and FSL_BULK_SIG,
as that only adds up to 0.003.
Most commonly a meta for "multiple rules fired" would have used the &&
operator, for instance:
meta MY_MANY_RULES SPF_HELO_NONE && SPF_PASS && FSL_BULK_SIG
describe MY_MANY_RULES Several random rules hit
score MY_MANY_RULES 1
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Hi guys,
thank you all for your advice!
Am 2024-05-10 22:39, schrieb Bowie Bailey:
> The rules with the low scores are not intended to contribute to the
> spam score for the email. They only have a defined score at all
> because if the score is 0, SA will not run the rule.
>
> It works like this:
>
> Rule A has a score of 0.001
> Rule B has a score of 0.001
> Rule C is a meta that matches if both A and B match, and has a score of
> 5
>
> It doesn't matter how small the scores are for rule A and B. The only
> thing that matters is the score for rule C. If only A matches, then it
> adds 0.001 to the score and the email is not spam. If only B matches,
> then you get the same result. But if they both match, then you get a
> score of 5.002. The entire point of the 0.001 score is that you could
> match 100 of these rules and not affect the spam score.
>
> These rules are generally things like, "the email has HTML", "there is
> an SPF check", "there is a google drive link", etc. On their own, they
> do not mean anything, but metas can combine these low-scored rules into
> meaningful patterns that are then given larger scores.
Can I just take the names of the rules?
e.g. at least two checks should fire:
meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK +
URIBL_ABUSE_SURBL) > 1)
score MULTIPLE_TESTS 1
found in
X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43, RAZOR2_CHECK=1.729,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]
Re: Score 0.001
Posted by Bowie Bailey <Bo...@BUC.com>.
On 5/10/2024 2:57 AM, Thomas Barth wrote:
> Am 2024-05-10 06:19, schrieb Reindl Harald (privat):
>> Am 10.05.24 um 00:05 schrieb Thomas Barth:
>>> Am 2024-05-09 21:41, schrieb Loren Wilton:
>>>> Low-score tests are neither spam nor ham signs by themselves. They
>>>> can be used in metas in conjunction with other indicators to help
>>>> determine ham or spam. A zero value indicates that a rule didn't
>>>> hit and the sign is not present. A small score indicates that the
>>>> rule did hit, so the sign it is detecting is present.
>>>
>>> 0.001 seems to be the default lowest value. Is it possible to change
>>> it to 0.01 or 0.1?
>>
>> what do you not understand in meta tests?
>> it's irrelevant if it's 0.001 or 0.01
>>
>> these rules are used in combination with other rules
>>
>> HTML_MESSAGE or SPF_HELO_NONE alone don't mean anything and so it
>> must not score higher - it makes only sense combined with other rules
>
> Most of the messages I receive only have a few hits because they
> hardly differ from a regular e-mail. That's why I want to assign a
> higher value to the individual tests. I don't know how many of the
> possible tests with a value of only 0.001 exist. With this value,
> theoretically 1000 different tests would have to be positive in order
> to achieve a total value of 1. Therefore, it is not irrelevant whether
> I have a minimum value of 0.001 or 0.1. I would even go further and
> say if there are more than 10 tests with a positive value: Spam!
> Either the strike level is reached or there are more than 10 tests
> with a positive value. So now I repeat my question: is it possible to
> increase the minimum value to 0.1 by default?
Going through this thread, I note that a few people have said "they are
used in metas", but no one has actually given an example of how that works.
The rules with the low scores are not intended to contribute to the spam
score for the email. They only have a defined score at all because if
the score is 0, SA will not run the rule.
It works like this:
Rule A has a score of 0.001
Rule B has a score of 0.001
Rule C is a meta that matches if both A and B match, and has a score of 5
It doesn't matter how small the scores are for rule A and B. The only
thing that matters is the score for rule C. If only A matches, then it
adds 0.001 to the score and the email is not spam. If only B matches,
then you get the same result. But if they both match, then you get a
score of 5.002. The entire point of the 0.001 score is that you could
match 100 of these rules and not affect the spam score.
These rules are generally things like, "the email has HTML", "there is
an SPF check", "there is a google drive link", etc. On their own, they
do not mean anything, but metas can combine these low-scored rules into
meaningful patterns that are then given larger scores.
--
Bowie
Re: Score 0.001
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2024-05-10 at 11:00:45 UTC-0400 (Fri, 10 May 2024 08:00:45 -0700 (PDT))
John Hardin <jh...@impsec.org>
is rumored to have said:
> Note that poorly-performing rules may get a score that looks informational, but that may change over time based on the corpora.
IOW: rules that in themselves are not good enough performers to get included in the daily active list will still be pulled into the active list with a trivial score if derivative meta rules which are good enough for real scores depend on them.
--
Bill Cole
Re: Score 0.001
Posted by John Hardin <jh...@impsec.org>.
On Fri, 10 May 2024, Thomas Barth wrote:
> So now I repeat my question: is it possible to increase the minimum
> value to 0.1 by default?
Not really.
The score for a rule is either a fixed value assigned by the rule
developer or a dynamic value calculated by masscheck nightly. There isn't
a "macro" for informational scores that would affect them all at once;
each informational rule would have to be updated individually.
And they are considered *informational* - they should not by themselves
contribute to the ham/spam score, so a request to globally change the
informational score from 0.0001 or 0.001 to 0.1 would not be approved.
For example, there is a rule that matches large monetary quantities in
multiple formats and languages. That rule is used in combination with
other rules to look for spam signs. It's scored as informational simply to
expose the fact that the message has content like that, but by itself it
doesn't indicate hammy or spammy content - the message could be a 419
spam, or it could be a news article about the deficit.
Note that poorly-performing rules may get a score that looks
informational, but that may change over time based on the corpora.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
You do not examine legislation in the light of the benefits it
will convey if properly administered, but in the light of the
wrongs it would do and the harms it would cause if improperly
administered. -- Lyndon B. Johnson
-----------------------------------------------------------------------
4 days until the 76th anniversary of Israel's independence
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Am 2024-05-10 06:19, schrieb Reindl Harald (privat):
> Am 10.05.24 um 00:05 schrieb Thomas Barth:
>> Am 2024-05-09 21:41, schrieb Loren Wilton:
>>> Low-score tests are neither spam nor ham signs by themselves. They
>>> can be used in metas in conjunction with other indicators to help
>>> determine ham or spam. A zero value indicates that a rule didn't hit
>>> and the sign is not present. A small score indicates that the rule
>>> did hit, so the sign it is detecting is present.
>>
>> 0.001 seems to be the default lowest value. Is it possible to change
>> it to 0.01 or 0.1?
>
> what do you not understand in meta tests?
> it's irrelevant if it's 0.001 or 0.01
>
> these rules are used in combination with other rules
>
> HTML_MESSAGE or SPF_HELO_NONE alone don't mean anything and so it must
> not score higher - it makes only sense combined with other rules
Most of the messages I receive only have a few hits because they hardly
differ from a regular e-mail. That's why I want to assign a higher value
to the individual tests. I don't know how many of the possible tests
with a value of only 0.001 exist. With this value, theoretically 1000
different tests would have to be positive in order to achieve a total
value of 1. Therefore, it is not irrelevant whether I have a minimum
value of 0.001 or 0.1. I would even go further and say if there are more
than 10 tests with a positive value: Spam! Either the strike level is
reached or there are more than 10 tests with a positive value. So now I
repeat my question: is it possible to increase the minimum value to 0.1
by default?
Re: Score 0.001
Posted by Thomas Barth <tb...@txbweb.de>.
Am 2024-05-09 21:41, schrieb Loren Wilton:
> Low-score tests are neither spam nor ham signs by themselves. They can
> be used in metas in conjunction with other indicators to help determine
> ham or spam. A zero value indicates that a rule didn't hit and the sign
> is not present. A small score indicates that the rule did hit, so the
> sign it is detecting is present.
0.001 seems to be the default lowest value. Is it possible to change it
to 0.01 or 0.1?
Re: Score 0.001
Posted by Loren Wilton <lw...@earthlink.net>.
Low-score tests are neither spam nor ham signs by themselves. They can be
used in metas in conjunction with other indicators to help determine ham or
spam. A zero value indicates that a rule didn't hit and the sign is not
present. A small score indicates that the rule did hit, so the sign it is
detecting is present.
Re: Score 0.001
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 09.05.24 20:41, Thomas Barth wrote:
>I don't understand why there are so many checks where the meaningless
>value of 0.001 is assigned.
Those rules may be tested in the present.
They also may be informative, e.g. DMARC_MISSING or SPF_PASS
rules with score 0 are not used so using 0 is not possible in these cases.
Those rules may have different scores with diffent rulesets
(bayes/non-bayes, network/non-netwotk)
And they can be used in metas, e.g:
score HTML_MESSAGE 0.001
meta OBFUSCATING_COMMENT ((__OBFUSCATING_COMMENT_A && HTML_MESSAGE) || (__OBFUSCATING_COMMENT_B && MIME_HTML_ONLY)) && !__ISO_2022_JP_DELIM
score OBFUSCATING_COMMENT 0.000 0.000 0.001 0.723
> The total score could be much higher. Do I
>have to define all the checks myself with a desired value?
you can redefine values if you think, but you should be careful about it.
>X-Spam-Status: No, score=3.999 tagged_above=2 required=6.31
> tests=[DMARC_MISSING=0.001, FSL_BULK_SIG=0.001,
>HTML_IMAGE_RATIO_02=0.001,
> HTML_MESSAGE=0.001, PYZOR_CHECK=1.985, RELAYCOUNTRY_BAD=2,
> SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_TVD_MIME_EPI=0.01]
>
>or
>
>X-Spam-Status: Yes, score=7.281 tagged_above=2 required=6.31
> tests=[DMARC_MISSING=0.001, FSL_BULK_SIG=0.001,
>HTML_FONT_LOW_CONTRAST=0.001,
> HTML_IMAGE_ONLY_24=1.282, HTML_IMAGE_RATIO_02=0.001,
>HTML_MESSAGE=0.001,
> MIXED_HREF_CASE=1.999, PYZOR_CHECK=1.985, RELAYCOUNTRY_BAD=2,
> SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_TVD_MIME_EPI=0.01]
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Chernobyl was an Windows 95 beta test site.