You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Reindl Harald <h....@thelounge.net> on 2014/08/25 10:57:42 UTC
drop of score after update tonight
Hi
i am at setup a new mailgateway and playing around
with spamassassin-3.4.0 and spamass-milter which
looks both well - but after the update tonight my
testmessage goes down from 7.5 to 5.3
that's one of the very high rated on a Barracuda
appliance downloaded to a folder and only posting
the plaintext part in a web-formmailer
not sure if it is a good idea to post the content :-)
25-Aug-2014 06:04:43: SpamAssassin: Update processed successfully
04:49:
X-Spam-Status: Yes, score=7.5 required=5.0 tests=ADVANCE_FEE_4_NEW,
ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUST
ED, BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,
LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
10:33:
X-Spam-Status: Yes, score=5.3 required=5.0 tests=ADVANCE_FEE_4_NEW,
ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUST
ED, BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,
LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
found it - look at the bottom
the other thread where i try to find out why spam messages don't
get [SPAM] in the subject (still unsolved) turned out that
"sa-update" obviously changed the permissions of the folder
"updates_spamassassin_org" to 750 instead 755
after fixing that it is again above 7
identified spam (7.9/1.0)
7.5 versus 7.9 likely more training messages
in the meantime both (spamd and milter) are running
with the miler user, using a port above 1024 so it
can bind and make sure in the systemd-units that
permissions are never wrong in the future
ExecStartPre=/usr/bin/find /var/lib/spamassassin/ -type d -exec /bin/chmod 0755 "{}" \;
ExecStartPre=/usr/bin/find /var/lib/spamassassin/ -type f -exec /bin/chmod 0644 "{}" \;
ExecStart=/usr/sbin/spamass-milter -g sa-milt -r 7.5 -- -s 1048576 --port=10027
User=sa-milt
Group=sa-milt
PermissionsStartOnly=true
ExecStartPre=/usr/bin/find /var/lib/spamassassin/ -type d -exec /bin/chmod 0755 "{}" \;
ExecStartPre=/usr/bin/find /var/lib/spamassassin/ -type f -exec /bin/chmod 0644 "{}" \;
ExecStart=/usr/bin/spamd -c -H --port=10027
ExecReload=/usr/bin/kill -HUP $MAINPID
User=sa-milt
Group=sa-milt
Am 25.08.2014 um 10:57 schrieb Reindl Harald:
> i am at setup a new mailgateway and playing around
> with spamassassin-3.4.0 and spamass-milter which
> looks both well - but after the update tonight my
> testmessage goes down from 7.5 to 5.3
>
> that's one of the very high rated on a Barracuda
> appliance downloaded to a folder and only posting
> the plaintext part in a web-formmailer
>
> not sure if it is a good idea to post the content :-)
>
> 25-Aug-2014 06:04:43: SpamAssassin: Update processed successfully
>
> 04:49:
> X-Spam-Status: Yes, score=7.5 required=5.0 tests=ADVANCE_FEE_4_NEW,
> ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUST
> ED, BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,
> LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
>
> 10:33:
> X-Spam-Status: Yes, score=5.3 required=5.0 tests=ADVANCE_FEE_4_NEW,
> ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUST
> ED, BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,
> LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
-------- Weitergeleitete Nachricht --------
Betreff: Re: no subject tagging in case of "X-Spam-Status: Yes"
Datum: Mon, 25 Aug 2014 19:43:29 +0200
Von: Reindl Harald <h....@thelounge.net>
Organisation: the lounge interactive design
An: users@spamassassin.apache.org
Am 25.08.2014 um 19:13 schrieb Karsten Bräckelmann:
> On Mon, 2014-08-25 at 18:55 +0200, Reindl Harald wrote:
>> Am 25.08.2014 um 18:00 schrieb Karsten Bräckelmann:
>> X-Spam-Status: Yes, score=3.7 required=1.0 tests=MISSING_DATE,MISSING_FROM,
>> MISSING_HEADERS,MISSING_MID,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS
>> Subject: [SPAM] Foo
>> X-Spam-Prev-Subject: Foo
>
> Exactly as expected. Subject tagging works.
yes
>> [root@mail-gw:~]$ su - sa-milt
>> [sa-milt@mail-gw:~]$ echo -e "Subject: Foo\n" | spamassassin --cf="required_score 1"
>
>> X-Spam-Status: No, score=0.0 required=1.0 tests=none
>> Subject: Foo
>
> No tests at all. I doubt the milter generated all those missing headers
> including From and Date, instead of a Received one only. So it seems the
> restricted sa-milt user has no read permissions on the SA config.
>
> As that user, have a close look at the -D debug output.
>
> spamassassin -D --lint
bingo - only a snippet below
thank you so much for setp in that thread
_______________________________________________________
the files inside exept one have correct permissions (0644)
but "/var/lib/spamassassin/3.004000/updates_spamassassin_org" not
that was pretty sure one of the first "sa-update" cronjobs because
as i started to play around the tagging was fine and i needed to
read manuals how to configure reject above a specific score and
later found out "well, and now the tagging don't work"
_______________________________________________________
on the shell now it looks fine, mail still not tagged, all
services hard restarted and as said at the begin of play
around one time it worked - strange
Subject: Test
X-Spam-Status: Yes, score=1.7 required=1.0 tests=ADVANCE_FEE_4_NEW,
ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUST
ED, DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,LOTS_OF_MONEY,
T_MONEY_PERCENT,URG_BIZ
i guess i will setup a cronjob to make sure the permissions
below "/var/lib/spamassassin/" are 755 and 644 for any item
[root@mail-gw:~]$ cat /usr/local/bin/sa-permissions.sh
#!/usr/bin/bash
/usr/bin/find /var/lib/spamassassin/ -type d -exec /bin/chmod 0755 "{}" \;
/usr/bin/find /var/lib/spamassassin/ -type f -exec /bin/chmod 0644 "{}" \;
[root@mail-gw:~]$ sa-permissions.sh
_______________________________________________________
[sa-milt@mail-gw:~]$ echo -e "Subject: Foo\n" | spamassassin --cf="required_score 1"
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
mail-gw.thelounge.net
X-Spam-Status: Yes, score=3.7 required=1.0 tests=MISSING_DATE,MISSING_FROM,
MISSING_HEADERS,MISSING_MID,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS
Subject: [SPAM] Foo
X-Spam-Prev-Subject: Foo
_______________________________________________________
Aug 25 19:18:58.225 [32610] dbg: config: file or directory
/var/lib/spamassassin/3.004000/updates_spamassassin_org/local.cf not accessible: Permission denied
Aug 25 19:18:58.226 [32610] dbg: config: file or directory
/var/lib/spamassassin/3.004000/updates_spamassassin_org/regression_tests.cf not accessible: Permission denied
[sa-milt@mail-gw:~]$ stat /var/lib/spamassassin/3.004000/updates_spamassassin_org/regression_tests.cf
stat: cannot stat '/var/lib/spamassassin/3.004000/updates_spamassassin_org/regression_tests.cf': Permission denied
_______________________________________________________
[root@mail-gw:~]$ stat /var/lib/spamassassin/3.004000/updates_spamassassin_org
File: '/var/lib/spamassassin/3.004000/updates_spamassassin_org'
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 811h/2065d Inode: 41664 Links: 2
Access: (0750/drwxr-x---) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2014-08-14 19:25:43.022151858 +0200
Modify: 2014-08-25 06:04:43.425632505 +0200
Change: 2014-08-25 06:04:43.425632505 +0200
Birth: -
[root@mail-gw:~]$ chmod 755 /var/lib/spamassassin/3.004000/updates_spamassassin_org
mode of '/var/lib/spamassassin/3.004000/updates_spamassassin_org' changed from 0750 (rwxr-x---) to 0755 (rwxr-xr-x)
[root@mail-gw:~]$ ls /var/lib/spamassassin/3.004000/updates_spamassassin_org/
total 920K
-rw-r--r-- 1 root root 100K 2014-08-25 06:04 languages
-rw-r----- 1 root root 718 2014-08-25 06:04 MIRRORED.BY
-rw-r--r-- 1 root root 8.5K 2014-08-25 06:04 10_default_prefs.cf
-rw-r--r-- 1 root root 2.4K 2014-08-25 06:04 10_hasbase.cf
-rw-r--r-- 1 root root 7.5K 2014-08-25 06:04 20_advance_fee.cf
-rw-r--r-- 1 root root 8.9K 2014-08-25 06:04 20_aux_tlds.cf
-rw-r--r-- 1 root root 6.9K 2014-08-25 06:04 20_body_tests.cf
-rw-r--r-- 1 root root 1.9K 2014-08-25 06:04 20_compensate.cf
-rw-r--r-- 1 root root 9.6K 2014-08-25 06:04 20_dnsbl_tests.cf
-rw-r--r-- 1 root root 15K 2014-08-25 06:04 20_drugs.cf
-rw-r--r-- 1 root root 12K 2014-08-25 06:04 20_dynrdns.cf
-rw-r--r-- 1 root root 8.4K 2014-08-25 06:04 20_fake_helo_tests.cf
-rw-r--r-- 1 root root 3.0K 2014-08-25 06:04 20_freemail.cf
-rw-r--r-- 1 root root 42K 2014-08-25 06:04 20_freemail_domains.cf
-rw-r--r-- 1 root root 26K 2014-08-25 06:04 20_head_tests.cf
-rw-r--r-- 1 root root 11K 2014-08-25 06:04 20_html_tests.cf
-rw-r--r-- 1 root root 5.2K 2014-08-25 06:04 20_imageinfo.cf
-rw-r--r-- 1 root root 2.6K 2014-08-25 06:04 20_mailspike.cf
-rw-r--r-- 1 root root 3.3K 2014-08-25 06:04 20_meta_tests.cf
-rw-r--r-- 1 root root 1.9K 2014-08-25 06:04 20_net_tests.cf
-rw-r--r-- 1 root root 8.0K 2014-08-25 06:04 20_phrases.cf
-rw-r--r-- 1 root root 2.1K 2014-08-25 06:04 20_porn.cf
-rw-r--r-- 1 root root 16K 2014-08-25 06:04 20_ratware.cf
-rw-r--r-- 1 root root 5.7K 2014-08-25 06:04 20_uri_tests.cf
-rw-r--r-- 1 root root 19K 2014-08-25 06:04 20_vbounce.cf
-rw-r--r-- 1 root root 2.7K 2014-08-25 06:04 23_bayes.cf
-rw-r--r-- 1 root root 1.6K 2014-08-25 06:04 25_accessdb.cf
-rw-r--r-- 1 root root 1.6K 2014-08-25 06:04 25_antivirus.cf
-rw-r--r-- 1 root root 1.6K 2014-08-25 06:04 25_asn.cf
-rw-r--r-- 1 root root 2.3K 2014-08-25 06:04 25_dcc.cf
-rw-r--r-- 1 root root 4.4K 2014-08-25 06:04 25_dkim.cf
-rw-r--r-- 1 root root 2.9K 2014-08-25 06:04 25_hashcash.cf
-rw-r--r-- 1 root root 1.3K 2014-08-25 06:04 25_pyzor.cf
-rw-r--r-- 1 root root 3.4K 2014-08-25 06:04 25_razor2.cf
-rw-r--r-- 1 root root 9.6K 2014-08-25 06:04 25_replace.cf
-rw-r--r-- 1 root root 3.4K 2014-08-25 06:04 25_spf.cf
-rw-r--r-- 1 root root 1.8K 2014-08-25 06:04 25_textcat.cf
-rw-r--r-- 1 root root 13K 2014-08-25 06:04 25_uribl.cf
-rw-r--r-- 1 root root 28K 2014-08-25 06:04 30_text_de.cf
-rw-r--r-- 1 root root 20K 2014-08-25 06:04 30_text_fr.cf
-rw-r--r-- 1 root root 1.9K 2014-08-25 06:04 30_text_it.cf
-rw-r--r-- 1 root root 22K 2014-08-25 06:04 30_text_nl.cf
-rw-r--r-- 1 root root 18K 2014-08-25 06:04 30_text_pl.cf
-rw-r--r-- 1 root root 45K 2014-08-25 06:04 30_text_pt_br.cf
-rw-r--r-- 1 root root 38K 2014-08-25 06:04 50_scores.cf
-rw-r--r-- 1 root root 9.1K 2014-08-25 06:04 60_adsp_override_dkim.cf
-rw-r--r-- 1 root root 1.3K 2014-08-25 06:04 60_awl.cf
-rw-r--r-- 1 root root 2.2K 2014-08-25 06:04 60_shortcircuit.cf
-rw-r--r-- 1 root root 4.9K 2014-08-25 06:04 60_whitelist.cf
-rw-r--r-- 1 root root 6.1K 2014-08-25 06:04 60_whitelist_dkim.cf
-rw-r--r-- 1 root root 3.6K 2014-08-25 06:04 60_whitelist_spf.cf
-rw-r--r-- 1 root root 1.9K 2014-08-25 06:04 60_whitelist_subject.cf
-rw-r--r-- 1 root root 202K 2014-08-25 06:04 72_active.cf
-rw-r--r-- 1 root root 11K 2014-08-25 06:04 72_scores.cf
-rw-r--r-- 1 root root 2.7K 2014-08-25 06:04 73_sandbox_manual_scores.cf
-rw-r--r-- 1 root root 2.2K 2014-08-25 06:04 local.cf
-rw-r--r-- 1 root root 2.8K 2014-08-25 06:04 regression_tests.cf
-rw-r--r-- 1 root root 1.9K 2014-08-25 06:04 user_prefs.template
-rw-r--r-- 1 root root 1.5K 2014-08-25 06:04 STATISTICS-set0-72_scores.cf.txt
-rw-r--r-- 1 root root 1.5K 2014-08-25 06:04 STATISTICS-set1-72_scores.cf.txt
-rw-r--r-- 1 root root 0 2014-08-25 06:04 STATISTICS-set2-72_scores.cf.txt
-rw-r--r-- 1 root root 0 2014-08-25 06:04 STATISTICS-set3-72_scores.cf.txt
-rw-r--r-- 1 root root 4.7K 2014-08-25 06:04 sa-update-pubkey.txt
RE: drop of score after update tonight
Posted by David Jones <dj...@ena.com>.
> ________________________________________
> From: Ian Zimmerman <it...@buug.org>
> Sent: Monday, August 25, 2014 5:02 PM
> To: users@spamassassin.apache.org
> Subject: Re: drop of score after update tonight
> On Mon, 25 Aug 2014 19:50:20 +0000,
> David Jones <dj...@ena.com> wrote:
> Ian> I definitely have FNs today (about 10 by now today, normally 0).
> Ian> Looks like some/all RBLs tests are not working. I have not changed
> Ian> my configuration at all.
> Ian> Sample here:
> Ian> http://pastebin.com/dsqaVA9Z
> David> This hit DCC_CHECK, BAYES_50, CRM114, BOGOFILTER and KAM_EU rules
> David> and would have been blocked on my SA 3.4.0 servers.
> Isn't it a bit odd that SA has rules for all these other Bayes powered
> backends? Why not give a bit more weight to its own Bayes instead,
> rather than make users forage for other tools that do essentially the
> same thing?
Based on my testing, I have found that having BAYES, CRM114, and BOGOFILTER
together provides very good "checks and balances". Spammers pay "spam shops"
for new spam campaigns that get through standard spam systems including SA.
I have often seen new spam score low on BAYES but CRM114 and/or BOGOFILTER
correctly score it as spam. So they do similar things but aren't identical.
If you don't have other indicators like DCC, RAZOR, CRM114, BOGOFILTER, RBLs,
etc., how else will you hit the bayes_auto_learn_threshold_nonspam and
bayes_auto_learn_threshold_spam levels to make BAYES more accurate and
detect new spam campaigns quickly?
> David> (I understand that the DCC_CHECK hit could have also hit on your
> David> mail server too after time had passed if you have DCC enabled.)
> Don't you need non-free software for DCC?
DCC is free for your own filtering or if you are an ISP that participates in
the DCC network. We participate in the DCC network.
http://www.rhyolite.com/dcc/
> (Meanwhile, more spam came in. This is definitely a crisis for me.)
> --
> Please *no* private copies of mailing list or newsgroup messages.
> Local Variables:
> mode:claws-external
> End:
Re: drop of score after update tonight
Posted by Christian Laußat <us...@spamassassin.shambhu.info>.
Am 26.08.2014 09:30, schrieb Ian Zimmerman:
> Apparently not. So, I have to rephrase: Isn't it a bit odd to use
> these external rules? :)
No, I don't think that its odd to use other statistical filters than the
SA Bayes.
CRM114 uses a completely different algorithem, building statistics not
just on single words but on phrases up to five words. It has less FPs
and FNs then SA Bayes. CRM114 has no autolearning mechanism, but if you
run it as a SA plugin you can combine the strenght of both.
I'm not sure about the recognition rate of Bogofilter compared to CRM114
and SA Bayes, but I also run all three (CRM114 and Bogofiler as SA
plugins).
--
Christian Laußat
https://kvm.laussat.info/
Re: drop of score after update tonight
Posted by Ian Zimmerman <it...@buug.org>.
On Tue, 26 Aug 2014 08:10:23 +0200,
Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
Ian> Isn't it a bit odd that SA has rules for all these other Bayes
Ian> powered backends? Why not give a bit more weight to its own Bayes
Ian> instead, rather than make users forage for other tools that do
Ian> essentially the same thing?
Matus> are they part of stock 3.4.0?
Apparently not. So, I have to rephrase: Isn't it a bit odd to use
these external rules? :)
Ian> Don't you need non-free software for DCC?
Matus> non-free in Debian definition.
Matus> (you need own server if you process ofer 100k messages daily, and
Matus> license if you have internal checksum database)
Matus> you can get the source, build and run in most of cases freely.
But that presents difficulties even apart from the religious ones. For
instance, it means installing development tools on the target server, or
else cross-compiling (and we know how easy that is with average C code).
The good news is the bout of spam seems to have calmed down.
_Something_ must have been wrong earlier today. The RBLs and Razor and
Pyzor all seemed to be out to lunch. Maybe a connectivity problem on my
side.
> Christian Science Programming: "Let God Debug It!".
May I quote this? :-)
--
Please *no* private copies of mailing list or newsgroup messages.
Local Variables:
mode:claws-external
End:
Re: drop of score after update tonight
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>Ian> http://pastebin.com/dsqaVA9Z
>On Mon, 25 Aug 2014 19:50:20 +0000,
>David Jones <dj...@ena.com> wrote:
>David> This hit DCC_CHECK, BAYES_50, CRM114, BOGOFILTER and KAM_EU rules
>David> and would have been blocked on my SA 3.4.0 servers.
On 25.08.14 15:02, Ian Zimmerman wrote:
>Isn't it a bit odd that SA has rules for all these other Bayes powered
>backends? Why not give a bit more weight to its own Bayes instead,
>rather than make users forage for other tools that do essentially the
>same thing?
are they part of stock 3.4.0?
>Don't you need non-free software for DCC?
non-free in Debian definition.
(you need own server if you process ofer 100k messages daily, and license if
you have internal checksum database)
you can get the source, build and run in most of cases freely.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Christian Science Programming: "Let God Debug It!".
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 26.08.2014 um 11:30 schrieb Axb:
> On 08/26/2014 11:23 AM, Reindl Harald wrote:
>> i am at building the new MTA which will replace a commercial
>> spamfilter appliance and currently i am at training byes and
>> building admin backends
>>
>> * postscreen with RBL/DNSWL weight
>> * PTR filters
>> * subject filters
>> * attachemnt extensions
>> * ClamAV milter
>> * spamassasin milter
>>
>> so what i am doing now is send testmessages from my MUA
>> or a webform in the LAN, look how things behave and get
>> some picture about sorces which needs to be adjusted
>> later after real mailflow starts with the first live
>> domain
>
> just in case you're not nailed to spamass-milter and using Postfix,
> http://fuglu.org/ works great as a pre-queue filter
thanks for the hint but already nailed because written
webinterfaces to configure milter options and re-create
systemd-units from templates which works too good for
switching now
Re: drop of score after update tonight
Posted by Axb <ax...@gmail.com>.
On 08/26/2014 11:23 AM, Reindl Harald wrote:
> i am at building the new MTA which will replace a commercial
> spamfilter appliance and currently i am at training byes and
> building admin backends
>
> * postscreen with RBL/DNSWL weight
> * PTR filters
> * subject filters
> * attachemnt extensions
> * ClamAV milter
> * spamassasin milter
>
> so what i am doing now is send testmessages from my MUA
> or a webform in the LAN, look how things behave and get
> some picture about sorces which needs to be adjusted
> later after real mailflow starts with the first live
> domain
>
just in case you're not nailed to spamass-milter and using Postfix,
http://fuglu.org/ works great as a pre-queue filter
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 26.08.2014 um 10:52 schrieb Matthias Leisi:
> On Tue, Aug 26, 2014 at 10:16 AM, Reindl Harald <h....@thelounge.net> wrote:
>
> ADVANCE_FEE_4_NEW,ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUSTED,BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
>>>> scantime=0.3,size=4760,user=sa-milt,uid=189,required_score=1.0,rhost=localhost,raddr=127.0.0.1,rport=29317,mid=<*********>,bayes=1.000000,autolearn=disabled
>>>
>>> ALL_TRUSTED? Are you sure you have set up your trustpath the right
>>> way? Do the Received: headers make their way to SpamAssassin or are
>>> they possibly stripped/altered by some "glue" software which calls SA?
>>>
>>> https://wiki.apache.org/spamassassin/TrustPath
>>
>> sure - the message comes directly from my MUA
>> in the same LAN as the mail machine and only
>> a transparent VPN between
>
> The spam comes from your MUA to SpamAssassin? I would expect the
> mailflow to be something like
>
> [actual source] => [your gateway/MTA] => [mailstore] => [your MUA]
>
> and I would expect SpamAssassin to sit in the MTA?
that's just testing
i am at building the new MTA which will replace a commercial
spamfilter appliance and currently i am at training byes and
building admin backends
* postscreen with RBL/DNSWL weight
* PTR filters
* subject filters
* attachemnt extensions
* ClamAV milter
* spamassasin milter
so what i am doing now is send testmessages from my MUA
or a webform in the LAN, look how things behave and get
some picture about sorces which needs to be adjusted
later after real mailflow starts with the first live
domain
Re: drop of score after update tonight
Posted by Matthias Leisi <ma...@leisi.net>.
On Tue, Aug 26, 2014 at 10:16 AM, Reindl Harald <h....@thelounge.net> wrote:
ADVANCE_FEE_4_NEW,ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUSTED,BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
>>> scantime=0.3,size=4760,user=sa-milt,uid=189,required_score=1.0,rhost=localhost,raddr=127.0.0.1,rport=29317,mid=<*********>,bayes=1.000000,autolearn=disabled
>>
>> ALL_TRUSTED? Are you sure you have set up your trustpath the right
>> way? Do the Received: headers make their way to SpamAssassin or are
>> they possibly stripped/altered by some "glue" software which calls SA?
>>
>> https://wiki.apache.org/spamassassin/TrustPath
>
> sure - the message comes directly from my MUA
> in the same LAN as the mail machine and only
> a transparent VPN between
The spam comes from your MUA to SpamAssassin? I would expect the
mailflow to be something like
[actual source] => [your gateway/MTA] => [mailstore] => [your MUA]
and I would expect SpamAssassin to sit in the MTA?
-- Matthias
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 26.08.2014 um 08:54 schrieb Matthias Leisi:
> On Tue, Aug 26, 2014 at 12:08 AM, Reindl Harald <h....@thelounge.net> wrote:
>
>> Aug 26 00:01:32 mail-gw spamd[6836]: spamd: result: Y 5 -
>> ADVANCE_FEE_4_NEW,ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUSTED,BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
>> scantime=0.3,size=4760,user=sa-milt,uid=189,required_score=1.0,rhost=localhost,raddr=127.0.0.1,rport=29317,mid=<*********>,bayes=1.000000,autolearn=disabled
>
> ALL_TRUSTED? Are you sure you have set up your trustpath the right
> way? Do the Received: headers make their way to SpamAssassin or are
> they possibly stripped/altered by some "glue" software which calls SA?
>
> https://wiki.apache.org/spamassassin/TrustPath
sure - the message comes directly from my MUA
in the same LAN as the mail machine and only
a transparent VPN between
Re: drop of score after update tonight
Posted by Matthias Leisi <ma...@leisi.net>.
On Tue, Aug 26, 2014 at 12:08 AM, Reindl Harald <h....@thelounge.net> wrote:
> Aug 26 00:01:32 mail-gw spamd[6836]: spamd: result: Y 5 -
> ADVANCE_FEE_4_NEW,ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUSTED,BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
> scantime=0.3,size=4760,user=sa-milt,uid=189,required_score=1.0,rhost=localhost,raddr=127.0.0.1,rport=29317,mid=<*********>,bayes=1.000000,autolearn=disabled
ALL_TRUSTED? Are you sure you have set up your trustpath the right
way? Do the Received: headers make their way to SpamAssassin or are
they possibly stripped/altered by some "glue" software which calls SA?
https://wiki.apache.org/spamassassin/TrustPath
-- Matthias
Re: drop of score after update tonight
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2014-08-26 at 00:08 +0200, Reindl Harald wrote:
> the "bayes=1.000000" below makes me wonder because around 1000 careful
> selected ham/spam messages for training - IMHO that should be more in
> such clear cases
Please do read the docs or at least the rule's description (hint, see
the BAYES_99 one) before venting such opinion.
The Bayesian Classifier returns a probability of the mail being ham or
spam, in a range between 0 and 1. Zero being ham, 1 spam, and a value of
0.5 being neutral, kind of undecided.
A bayes value of 1.0000 is as high as it gets, and the rules'
descriptions also clearly state the spam probability being 99.9 to 100%.
> however, i admit that i am a beginner with SA!
>
> Aug 26 00:01:32 mail-gw spamd[6836]: spamd: result: Y 5 -
> ADVANCE_FEE_4_NEW,ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUSTED,BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
> scantime=0.3,size=4760,user=sa-milt,uid=189,required_score=1.0,rhost=localhost,raddr=127.0.0.1,rport=29317,mid=<*********>,bayes=1.000000,autolearn=disabled
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 26.08.2014 um 00:02 schrieb Ian Zimmerman:
> On Mon, 25 Aug 2014 19:50:20 +0000,
> David Jones <dj...@ena.com> wrote:
>
> Ian> I definitely have FNs today (about 10 by now today, normally 0).
>
> Ian> Looks like some/all RBLs tests are not working. I have not changed
> Ian> my configuration at all.
>
> Ian> Sample here:
>
> Ian> http://pastebin.com/dsqaVA9Z
>
> David> This hit DCC_CHECK, BAYES_50, CRM114, BOGOFILTER and KAM_EU rules
> David> and would have been blocked on my SA 3.4.0 servers.
>
> Isn't it a bit odd that SA has rules for all these other Bayes powered
> backends? Why not give a bit more weight to its own Bayes instead,
> rather than make users forage for other tools that do essentially the
> same thing?
+1
the "bayes=1.000000" below makes me wonder because around 1000 careful
selected ham/spam messages for training - IMHO that should be more in
such clear cases
however, i admit that i am a beginner with SA!
Aug 26 00:01:32 mail-gw spamd[6836]: spamd: result: Y 5 -
ADVANCE_FEE_4_NEW,ADVANCE_FEE_4_NEW_MONEY,ADVANCE_FEE_5_NEW,ADVANCE_FEE_5_NEW_MONEY,ALL_TRUSTED,BAYES_99,BAYES_999,DEAR_SOMETHING,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,LOTS_OF_MONEY,T_MONEY_PERCENT,URG_BIZ
scantime=0.3,size=4760,user=sa-milt,uid=189,required_score=1.0,rhost=localhost,raddr=127.0.0.1,rport=29317,mid=<*********>,bayes=1.000000,autolearn=disabled
Re: drop of score after update tonight
Posted by Ian Zimmerman <it...@buug.org>.
On Mon, 25 Aug 2014 19:50:20 +0000,
David Jones <dj...@ena.com> wrote:
Ian> I definitely have FNs today (about 10 by now today, normally 0).
Ian> Looks like some/all RBLs tests are not working. I have not changed
Ian> my configuration at all.
Ian> Sample here:
Ian> http://pastebin.com/dsqaVA9Z
David> This hit DCC_CHECK, BAYES_50, CRM114, BOGOFILTER and KAM_EU rules
David> and would have been blocked on my SA 3.4.0 servers.
Isn't it a bit odd that SA has rules for all these other Bayes powered
backends? Why not give a bit more weight to its own Bayes instead,
rather than make users forage for other tools that do essentially the
same thing?
David> (I understand that the DCC_CHECK hit could have also hit on your
David> mail server too after time had passed if you have DCC enabled.)
Don't you need non-free software for DCC?
(Meanwhile, more spam came in. This is definitely a crisis for me.)
--
Please *no* private copies of mailing list or newsgroup messages.
Local Variables:
mode:claws-external
End:
RE: drop of score after update tonight
Posted by David Jones <dj...@ena.com>.
> ________________________________________
> From: Ian Zimmerman <it...@buug.org>
> Sent: Monday, August 25, 2014 2:28 PM
> To: users@spamassassin.apache.org
> Subject: Re: drop of score after update tonight
> I definitely have FNs today (about 10 by now today, normally 0).
> Looks like some/all RBLs tests are not working. I have not changed my
> configuration at all.
> Sample here:
> http://pastebin.com/dsqaVA9Z
> --
> Please *no* private copies of mailing list or newsgroup messages.
> Local Variables:
> mode:claws-external
> End:
This hit DCC_CHECK, BAYES_50, CRM114, BOGOFILTER and KAM_EU
rules and would have been blocked on my SA 3.4.0 servers.
(I understand that the DCC_CHECK hit could have also hit on your mail
server too after time had passed if you have DCC enabled.)
Re: drop of score after update tonight
Posted by Ian Zimmerman <it...@buug.org>.
I definitely have FNs today (about 10 by now today, normally 0).
Looks like some/all RBLs tests are not working. I have not changed my
configuration at all.
Sample here:
http://pastebin.com/dsqaVA9Z
--
Please *no* private copies of mailing list or newsgroup messages.
Local Variables:
mode:claws-external
End:
Re: drop of score after update tonight
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2014-08-25 at 17:47 +0200, Reindl Harald wrote:
> yes and that is one which the currently existing
> Barracuda Spamfirewall scored with around 20 and
> grabbed from the backend there for testings
> the plain content i attached as ZIP (what made it to the listg)
> is used for testing by just copy the content to a formmailer or
> in a new plaintext message in TB point directly to the test MX
Given (a) you disabled RBL checks in SA, (b) that sample is a plain
body without any headers, and (c) your method of sending the sample
even hits ALL_TRUSTED, SA still does a pretty decent job in comparison.
The Barracuda appliance you're comparing results to did not have those
disadvantages.
Anyway, changing scores after a successful sa-update are to be expected.
The re-scoring algorithm only uses the default threshold of 5.0, it does
not know the concept of a second "reject" score.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: drop of score after update tonight
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>> On 25.08.14 19:06, Reindl Harald wrote:
>you misunderstood me - whatever i did before and how the config looks
>like is not the problem - i was alerted by the dramatic change after
>sa-update last night wwith no other changes
Scores usually drop when many FPs appear. When some scores lower, other
often increase, or new tests appear. This can
If you want to exclude some sort of rules, you _must_ to be prepared for
higher number of FNs, not only for score drops.
What I think you should do it, you should either
- enable RBLs in SA and solve potential problems with it
or just live with the fact SA is not as effective
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
WinError #99999: Out of error messages.
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 25.08.2014 um 20:41 schrieb Matus UHLAR - fantomas:
>>>> Given (a) you disabled RBL checks in SA
>
> On 25.08.14 19:06, Reindl Harald wrote:
>> the reason for that is that postfix in front already does a damned
>> good job with RBL's
>
> since SA uses deep header scanning in many times, which postfix does not
> (afaik), it's always better to NOT disable RBL's as SA level.
coming from Barracuda Networks devices where "deep header inspection"
was the root of all evil like breaking PTR checks
>> and especially uses internal whitelists (rbldnsd)
>> and a honeypot RBL and what i want to avoid is that that SA beats
>> out the whitelists
>
> ...and SA can use whitelists too
i am still about dig in the oppurtunities and did not found something
to specify used whitelists and blacklists and what is most important
to reflect the postscreen setup weight them different
if the is some doc which i did not found by "spamassassin whitelist"
and "spamassassin RBL" i would be grateful for a link!
>> - i replay the data of the internal ones to
>> "local.conf" into "trusted_networks" because until now i did not
>> find a way to reflect the postscreen scoring below in SA
>
> putting IPS/ranges into trusted_networks is NOT whitelisting.
> It just pushes RBL checking to next headers.
as said - i need to control the RBL/DNSWL sources and be
happy since it would take away the additional work replay
database content feeding the RBL daemon instead just point
to the existing DNSWL/DNSBL lists
there is a reason why that machine don't serve in public
for now and for having first touch spamassassin two weeks
ago that all looks not too bad but has for sure improvements
> you have done "good" job by preventing SA from hitting rules and increasing
> score, and now you are complaining about low scores... silly
you misunderstood me - whatever i did before and how the config looks
like is not the problem - i was alerted by the dramatic change after
sa-update last night wwith no other changes
Re: drop of score after update tonight
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>>> Given (a) you disabled RBL checks in SA
On 25.08.14 19:06, Reindl Harald wrote:
>the reason for that is that postfix in front already does a damned
>good job with RBL's
since SA uses deep header scanning in many times, which postfix does not
(afaik), it's always better to NOT disable RBL's as SA level.
> and especially uses internal whitelists (rbldnsd)
>and a honeypot RBL and what i want to avoid is that that SA beats
>out the whitelists
...and SA can use whitelists too.
> - i replay the data of the internal ones to
>"local.conf" into "trusted_networks" because until now i did not
>find a way to reflect the postscreen scoring below in SA
putting IPS/ranges into trusted_networks is NOT whitelisting.
It just pushes RBL checking to next headers.
you have done "good" job by preventing SA from hitting rules and increasing
score, and now you are complaining about low scores... silly.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Silvester Stallone: Father of the RISC concept.
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
first - thank you for your feedback
SA is a new beat to me
Am 25.08.2014 um 22:00 schrieb Daniel Staal:
> --As of August 25, 2014 7:49:39 PM +0200, Reindl Harald is alleged to have said:
>> Am 25.08.2014 um 19:35 schrieb Daniel Staal:
>>> --As of August 25, 2014 7:06:32 PM +0200, Reindl Harald is alleged to
>>> have said:
>>>
>>>>> masscheck ties to ensure spams score at least 5 points, but doesn't
>>>>> care beyond that
>>>>
>>>> yes, but given that the intention is to flag message above
>>>> 5 with [SPAM] and reject messages above 7 which is the
>>>> intention running SA as milter the reduced score matters
>>>
>>> Who sets that policy? Is it something you could think about
>>> changing (if it's a problem).
>>
>> finally i do that - which values needs to be found out and honestly
>> seeing that change i am unsure how to set score limits for both
>> (flag and reject) to prevent too mach messages passing through
>> and at the same time if such a large change happens introduce
>> false positives from one day to another
>
> Based on a quick check of my email, if you consider 'flagged' as non-spam (but possible), then I'd probably set
> flag at 3 or 4, and reject (as spam) at 5. Personally I use a 'probably spam' and 'definitely spam' system (both
> are set aside), with cutoffs at 5 and 10, respectively.
that is indeed something i had in mind as valid possibility
after looking what happens with real mail flow on my personal
domain
frankly, i am building up config tools for the whole system
the last few days until around 5:00 AM each night and can't
await to see it doing something but need some postfix
configurations since it will be a inbound-only for
multiple targets with and without rcpt-list :-)
currently i am at implement postfix "rcpt verification"
for the domains with no access to RCPT databases....
> But part of the point is that 7.5 to 5.3 is *not* a large change, as far a
> spamassassin is concerned. 5.1 to 4.9 would be a large change. ;)
i didn't get the joke completly but fine :-)
> I have rarely ever had a false positive with spamassassin - I get maybe two-three a year. I get that in false
> negatives a day, when things are working well. (Which amounts to about 1% of the spam I get as false negative.)
sounds good
>> i admit not have that much expierience but want to avoid
>> major mistakes in the setup as good as possible before
>> going live
>
> My advice: Don't over-think it. Spamassassin normally does a good job, with base settings and things turned on.
> Train your bayes well, and watch for new things, but in general don't try messing with a lot of settings unless you
> have problems with a live mail stream.
agreed - what i currently try is to implement a webinterface based
on other existing inhouse solutions to adjust params but feeded
with defaults so that later if all goes well anybody without
touching the device itself can adjust things in case some aeroplane
kills me from on day to another :-)
they bayes is trained well i think based on a few testmessages which are spam
without: score around 1.4
with: score around 5
since with milter there is only one userhome and so i feeded
the folders below with 1000 messages ham as well as spam
[root@mail-gw:~]$ cat /scripts/sa-learn.sh
#!/usr/bin/bash
chown root:sa-milt -R /var/lib/spamass-milter/training/ham/
chown root:sa-milt -R /var/lib/spamass-milter/training/spam/
chmod 750 /var/lib/spamass-milter/training/ham/
chmod 750 /var/lib/spamass-milter/training/spam/
chmod 640 /var/lib/spamass-milter/training/ham/*.eml
chmod 640 /var/lib/spamass-milter/training/spam/*.eml
fdupes -r -f /var/lib/spamass-milter/training/ham/ | grep -v '^$' | xargs rm -v 2> /dev/null
fdupes -r -f /var/lib/spamass-milter/training/spam/ | grep -v '^$' | xargs rm -v 2> /dev/null
/usr/bin/su -c "/var/lib/spamass-milter/training/learn.sh" sa-milt
[root@mail-gw:~]$ cat /var/lib/spamass-milter/training/learn.sh
#!/usr/bin/bash
akt_user=`whoami`
if test $akt_user = "sa-milt"
then
/bin/echo "" > /dev/null
else
/bin/echo "Das Script 'learn.sh' muss als Benutzer 'sa-milt' aufgerufen werden"
exit
fi
MY_TIME=$(/bin/date "+%d-%m-%Y %H:%M:%S")
echo "$MY_TIME: Verarbeite SPAM Samples"
/usr/bin/sa-learn --progress --spam /var/lib/spamass-milter/training/spam/*.eml
echo ""
MY_TIME=$(/bin/date "+%d-%m-%Y %H:%M:%S")
echo "$MY_TIME: Verarbeite HAM Samples"
/usr/bin/sa-learn --progress --ham /var/lib/spamass-milter/training/ham/*.eml
echo ""
MY_TIME=$(/bin/date "+%d-%m-%Y %H:%M:%S")
echo "$MY_TIME: Done"
>>> Did the percentage of spam flagged vs. rejected change overall?
>>
>> i am at early testing of SA and there is no active mail flow
>> since i am about finsish admin backends and how to generate
>> config files for SA/ClamAV/Postfix which is now at a nearly
>> "well, for my private doamin as public test good enough"
>>
>>> Every time the rules update some rules will be scored higher and
>>> some lower, so figuring out each individual case is going to be
>>> pointless, but if the overall percentages remain stable your system
>>> hasn't actually changed how it operates
>>
>> as said - i am about implement SA, saw the message from the
>> update cronjob the first time for some days and looked a
>> bit deeper if things changed
>
> And I think you ended up over-thinking it. It was marked as spam before, it's marked as spam now. Some other
> emails would probably have scored higher than they used to. We've actually had a long break in updates - usually
> they are multiple times a week, if not every day, but it's been around a month since they last updated. Rules
> probably changed scores more than normal - but it still scored the mail as spam
i will see and also think even if there are bad impacts there would
be another update soon
Re: drop of score after update tonight
Posted by Daniel Staal <DS...@usa.net>.
--As of August 25, 2014 7:49:39 PM +0200, Reindl Harald is alleged to have
said:
>
>
> Am 25.08.2014 um 19:35 schrieb Daniel Staal:
>> --As of August 25, 2014 7:06:32 PM +0200, Reindl Harald is alleged to
>> have said:
>>
>>>> masscheck ties to ensure spams score at least 5 points, but doesn't
>>>> care beyond that
>>>
>>> yes, but given that the intention is to flag message above
>>> 5 with [SPAM] and reject messages above 7 which is the
>>> intention running SA as milter the reduced score matters
>>
>> Who sets that policy? Is it something you could think about
>> changing (if it's a problem).
>
> finally i do that - which values needs to be found out and honestly
> seeing that change i am unsure how to set score limits for both
> (flag and reject) to prevent too mach messages passing through
> and at the same time if such a large change happens introduce
> false positives from one day to another
Based on a quick check of my email, if you consider 'flagged' as non-spam
(but possible), then I'd probably set flag at 3 or 4, and reject (as spam)
at 5. Personally I use a 'probably spam' and 'definitely spam' system
(both are set aside), with cutoffs at 5 and 10, respectively.
But part of the point is that 7.5 to 5.3 is *not* a large change, as far a
spamassassin is concerned. 5.1 to 4.9 would be a large change. ;)
I have rarely ever had a false positive with spamassassin - I get maybe
two-three a year. I get that in false negatives a day, when things are
working well. (Which amounts to about 1% of the spam I get as false
negative.)
> i admit not have that much expierience but want to avoid
> major mistakes in the setup as good as possible before
> going live
My advice: Don't over-think it. Spamassassin normally does a good job,
with base settings and things turned on. Train your bayes well, and watch
for new things, but in general don't try messing with a lot of settings
unless you have problems with a live mail stream.
>> Did the percentage of spam flagged vs. rejected change overall?
>
> i am at early testing of SA and there is no active mail flow
> since i am about finsish admin backends and how to generate
> config files for SA/ClamAV/Postfix which is now at a nearly
> "well, for my private doamin as public test good enough"
>
>> Every time the rules update some rules will be scored higher and
>> some lower, so figuring out each individual case is going to be
>> pointless, but if the overall percentages remain stable your system
>> hasn't actually changed how it operates
>
> as said - i am about implement SA, saw the message from the
> update cronjob the first time for some days and looked a
> bit deeper if things changed
And I think you ended up over-thinking it. It was marked as spam before,
it's marked as spam now. Some other emails would probably have scored
higher than they used to. We've actually had a long break in updates -
usually they are multiple times a week, if not every day, but it's been
around a month since they last updated. Rules probably changed scores more
than normal - but it still scored the mail as spam.
Daniel T. Staal
---------------------------------------------------------------
This email copyright the author. Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes. This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---------------------------------------------------------------
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 25.08.2014 um 19:35 schrieb Daniel Staal:
> --As of August 25, 2014 7:06:32 PM +0200, Reindl Harald is alleged to have said:
>
>>> masscheck ties to ensure spams score at least 5 points, but doesn't
>>> care beyond that
>>
>> yes, but given that the intention is to flag message above
>> 5 with [SPAM] and reject messages above 7 which is the
>> intention running SA as milter the reduced score matters
>
> --As for the rest, it is mine.
>
> Who sets that policy? Is it something you could think about
> changing (if it's a problem).
finally i do that - which values needs to be found out and honestly
seeing that change i am unsure how to set score limits for both
(flag and reject) to prevent too mach messages passing through
and at the same time if such a large change happens introduce
false positives from one day to another
i admit not have that much expierience but want to avoid
major mistakes in the setup as good as possible before
going live
> Did the percentage of spam flagged vs. rejected change overall?
i am at early testing of SA and there is no active mail flow
since i am about finsish admin backends and how to generate
config files for SA/ClamAV/Postfix which is now at a nearly
"well, for my private doamin as public test good enough"
> Every time the rules update some rules will be scored higher and
> some lower, so figuring out each individual case is going to be
> pointless, but if the overall percentages remain stable your system
> hasn't actually changed how it operates
as said - i am about implement SA, saw the message from the
update cronjob the first time for some days and looked a
bit deeper if things changed
Re: drop of score after update tonight
Posted by Daniel Staal <DS...@usa.net>.
--As of August 25, 2014 7:06:32 PM +0200, Reindl Harald is alleged to have
said:
>> masscheck ties to ensure spams score at least 5 points, but doesn't
>> care beyond that
>
> yes, but given that the intention is to flag message above
> 5 with [SPAM] and reject messages above 7 which is the
> intention running SA as milter the reduced score matters
--As for the rest, it is mine.
Who sets that policy? Is it something you could think about changing (if
it's a problem).
Did the percentage of spam flagged vs. rejected change overall? Every time
the rules update some rules will be scored higher and some lower, so
figuring out each individual case is going to be pointless, but if the
overall percentages remain stable your system hasn't actually changed how
it operates.
Daniel T. Staal
---------------------------------------------------------------
This email copyright the author. Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes. This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---------------------------------------------------------------
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 25.08.2014 um 18:28 schrieb John Hardin:
> On Mon, 25 Aug 2014, Reindl Harald wrote:
>
>> the plain content i attached as ZIP (what made it to the listg) is used for testing by just copy the content to a
>> formmailer or in a new plaintext message in TB point directly to the test MX
>
> The massage body by itself usually isn't enough to tell much. Looking at it, it's obviously a 419 spam, and your
> rule hits support that.
correct
> When troubleshooting stuff, we generally need the complete message including all the original headers, saved as an
> RFC-822-format text file.
that's clear but it's a matter of how to reprocude what i did
short before the update given a very high score and doing
the same after the update and that is in all cases just
past that plaintext into a MUA and send it to SA
> In this case the question isn't "why aren't rules hitting", it's
> "why are the rules now scoring less?"
that's exactly my question
> the same rules are hitting now as were before the update
that's why i am wondering
> Karsten covered that
>> Given (a) you disabled RBL checks in SA
the reason for that is that postfix in front already does a damned
good job with RBL's and especially uses internal whitelists (rbldnsd)
and a honeypot RBL and what i want to avoid is that that SA beats
out the whitelists - i replay the data of the internal ones to
"local.conf" into "trusted_networks" because until now i did not
find a way to reflect the postscreen scoring below in SA
postscreen_dnsbl_sites = dnsbl.thelounge.net*16
dul.dnsbl.sorbs.net*8
b.barracudacentral.org*7
dnsbl.inps.de*7
zen.spamhaus.org=127.0.0.[10;11]*6
zen.spamhaus.org=127.0.0.[4..7]*5
bl.spamcop.net*4
ix.dnsbl.manitu.net*4
zen.spamhaus.org=127.0.0.3*4
dnsbl-1.uceprotect.net*3
zen.spamhaus.org=127.0.0.2*3
bl.spameatingmonkey.net*2
dnsrbl.swinog.ch*2
psbl.surriel.com*2
spam.dnsbl.sorbs.net*2
ips.backscatterer.org*1
dnswl-low.thelounge.net*-3
list.dnswl.org=127.0.[0..255].0*-3
list.dnswl.org=127.0.[0..255].1*-4
list.dnswl.org=127.0.[0..255].2*-5
list.dnswl.org=127.0.[0..255].3*-6
dnswl-medium.thelounge.net*-8
dnswl-high.thelounge.net*-16
dnswl.thelounge.net*-24
> masscheck ties to ensure spams score at least 5 points, but doesn't
> care beyond that
yes, but given that the intention is to flag message above
5 with [SPAM] and reject messages above 7 which is the
intention running SA as milter the reduced score matters
Re: drop of score after update tonight
Posted by John Hardin <jh...@impsec.org>.
On Mon, 25 Aug 2014, Reindl Harald wrote:
> the plain content i attached as ZIP (what made it to the listg) is used
> for testing by just copy the content to a formmailer or in a new
> plaintext message in TB point directly to the test MX
The massage body by itself usually isn't enough to tell much. Looking at
it, it's obviously a 419 spam, and your rule hits support that.
When troubleshooting stuff, we generally need the complete message
including all the original headers, saved as an RFC-822-format text file.
In this case the question isn't "why aren't rules hitting", it's "why are
the rules now scoring less?" - the same rules are hitting now as were
before the update. Karsten covered that - masscheck ties to ensure spams
score at least 5 points, but doesn't care beyond that.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The first time I saw a bagpipe, I thought the player was torturing
an octopus. I was amazed they could scream so loudly.
-- cat_herder_5263 on Y! SCOX
-----------------------------------------------------------------------
Today: the 1935th anniversary of the destruction of Pompeii
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 25.08.2014 um 17:43 schrieb John Hardin:
> On Mon, 25 Aug 2014, Reindl Harald wrote:
>> Am 25.08.2014 um 16:06 schrieb John Hardin:
>>> On Mon, 25 Aug 2014, Reindl Harald wrote:
>>>
>>>> i am at setup a new mailgateway and playing around
>>>> with spamassassin-3.4.0 and spamass-milter which
>>>> looks both well - but after the update tonight my
>>>> testmessage goes down from 7.5 to 5.3
>>>
>>> 5.0 is still the score for "spam". Rather than focusing on the specific
>>> score, has the update caused FNs to increase?
>>
>> i know that it is "still spam" but before the update the current
>> testing setup would reject that message with spamass-milter and
>> "-r 7.5" while after the update it would slip through and only
>> flagged in the subject which don't work at all (see other thread)
>
> OK, so they've become user-visible.
yes and that is one which the currently existing
Barracuda Spamfirewall scored with around 20 and
grabbed from the backend there for testings
>> so that change has indeed impact on the amount of rejected spam
>>
>> i attached the plain content as ZIP which hopefully makes it
>> to the list :-)
>
> Best practice is to post the entire message to something like pastebin,
> then send the URL for that to the list
OK, did not know because on most other lists it's disapproved
the plain content i attached as ZIP (what made it to the listg)
is used for testing by just copy the content to a formmailer or
in a new plaintext message in TB point directly to the test MX
Re: drop of score after update tonight
Posted by John Hardin <jh...@impsec.org>.
On Mon, 25 Aug 2014, Reindl Harald wrote:
>
>
> Am 25.08.2014 um 16:06 schrieb John Hardin:
>> On Mon, 25 Aug 2014, Reindl Harald wrote:
>>
>>> i am at setup a new mailgateway and playing around
>>> with spamassassin-3.4.0 and spamass-milter which
>>> looks both well - but after the update tonight my
>>> testmessage goes down from 7.5 to 5.3
>>
>> 5.0 is still the score for "spam". Rather than focusing on the specific
>> score, has the update caused FNs to increase?
>
> i know that it is "still spam" but before the update the current
> testing setup would reject that message with spamass-milter and
> "-r 7.5" while after the update it would slip through and only
> flagged in the subject which don't work at all (see other thread)
OK, so they've become user-visible.
> so that change has indeed impact on the amount of rejected spam
>
> i attached the plain content as ZIP which hopefully makes it
> to the list :-)
Best practice is to post the entire message to something like pastebin,
then send the URL for that to the list.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
False is the idea of utility that sacrifices a thousand real
advantages for one imaginary or trifling inconvenience; that would
take fire from men because it burns, and water because one may drown
in it; that has no remedy for evils except destruction. The laws
that forbid the carrying of arms are laws of such a nature. They
disarm only those who are neither inclined nor determined to commit
crime. -- Cesare Beccaria, quoted by Thomas Jefferson
-----------------------------------------------------------------------
Today: the 1935th anniversary of the destruction of Pompeii
Re: drop of score after update tonight
Posted by Reindl Harald <h....@thelounge.net>.
Am 25.08.2014 um 16:06 schrieb John Hardin:
> On Mon, 25 Aug 2014, Reindl Harald wrote:
>
>> i am at setup a new mailgateway and playing around
>> with spamassassin-3.4.0 and spamass-milter which
>> looks both well - but after the update tonight my
>> testmessage goes down from 7.5 to 5.3
>
> 5.0 is still the score for "spam". Rather than focusing on the specific
> score, has the update caused FNs to increase?
i know that it is "still spam" but before the update the current
testing setup would reject that message with spamass-milter and
"-r 7.5" while after the update it would slip through and only
flagged in the subject which don't work at all (see other thread)
so that change has indeed impact on the amount of rejected spam
i attached the plain content as ZIP which hopefully makes it
to the list :-)
Re: drop of score after update tonight
Posted by John Hardin <jh...@impsec.org>.
On Mon, 25 Aug 2014, Reindl Harald wrote:
> Hi
>
> i am at setup a new mailgateway and playing around
> with spamassassin-3.4.0 and spamass-milter which
> looks both well - but after the update tonight my
> testmessage goes down from 7.5 to 5.3
5.0 is still the score for "spam". Rather than focusing on the specific
score, has the update caused FNs to increase?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Activist: Someone who gets involved.
Unregistered Lobbyist: Someone who gets involved with something
the MSM doesn't approve of. -- WizardPC
-----------------------------------------------------------------------
Today: the 1935th anniversary of the destruction of Pompeii