You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Asai <as...@globalchangemusic.org> on 2015/04/07 05:08:44 UTC
Mail Filter Recommendations
Greetings,
We've been using Amavis for a number of years, but it seems to not be
doing what we need it to be doing regarding spam filtering. e.g. I
can't seem to get it to learn bayes data on a per user basis. We have
our spam filters turned up so high ( kill level 3 ) for some users it
just seems like we're doing something wrong. I have deleted the Bayes
data before and let it rebuild, but it doesn't seem to make sense that
we've got all this crazy spam, some of which gets caught and some
doesn't even though it's the exact same spam (or nearly the same). Is
Amavis to blame here? Does it get in the way of Spamassassin running as
it should, or is it more just configuration problems on our part?
Thanks.
Re: Mail Filter Recommendations
Posted by Larry Rosenman <le...@lerctr.org>.
On 2015-04-07 17:35, Alex Regan wrote:
> Hi,
>
>
> I think the reason it didn't match on anything useful for the OP is
> because he doesn't have the latest RegisterBoundaries.pm.
>
> If he had the latest, it would have at least matched the MSGID and
> MALFORMED rules.
>
> Select the download link here:
>
> http://svn.apache.org/viewvc/spamassassin/trunk/lib/Mail/SpamAssassin/Util/RegistrarBoundaries.pm?view=log
>
> I believe this works with at least 3.4.0 or is it only 3.4.1?
I'm using it successfully with 3.4.0. I believe that this is being
heavily modified for 3.4.1 to be in
a .cf file.
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 108 Turvey Cove, Hutto, TX 78634-5688
Re: Mail Filter Recommendations
Posted by Axb <ax...@gmail.com>.
On 04/09/2015 12:14 AM, Kevin A. McGrail wrote:
> if you are trying to retrofit registrarboundaries.pm onto older SA
> releases, you'll need to follow the instructions on the older PM file
> from your install or a previous SVN commit.
>
> In 3.4.1, registrarboundaries will be updatable via a cf file delivered
> by sa-update so we no longer need to maintain that list in the PM.
please note the difference:
SA 3.3.x uses = RegistrarBoundaries
SA 3.4.0 uses = RegistrarBoundaries
SA 3.4.1 uses = RegistryBoundaries
new installs SA will use RegistryBoundaries and the tld .cf file
Updated setups will probably have both (RegistryBoundaries & older
RegistrarBoundaries) but rules and standard SA plugins will use
RegistryBoundaries
For backward compatiblity I'll try to mantain RegistrarBoundaries for
some time (SA 3.3.x and possible custom plugins) in case anybody still
needs it.
After aprox 6 months, RegistrarBoundaries will become unmantained. IF
someone still needs it, instructions to update have always been included
in the module.
RegistrarBoundaries cannot be updated via sa-update
RegistryBoundaries uses 20_aux_tlds.cf for tld lists which will be
updated via sa-update.
Axb
RE: Mail Filter Recommendations
Posted by Kevin Miller <ke...@juneau.org>.
I installed 3.4.0 from source some time back. IIRC, I just downloaded and did the stock configure, make, make install routine.
Can I just overwrite the existing version at (in my system at least)
"/usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/Util/RegistrarBoundaries.pm"" dating from Feb 6, 2014?
Does sa-compile get involved or is that just for rules?
Thanks...
...Kevin
--
Kevin Miller
Network/email Administrator, CBJ MIS Dept.
155 South Seward Street
Juneau, Alaska 99801
Phone: (907) 586-0242, Fax: (907) 586-4500
Registered Linux User No: 307357
> -----Original Message-----
> From: Kevin A. McGrail [mailto:KMcGrail@PCCC.com]
> Sent: Wednesday, April 08, 2015 2:14 PM
> To: Kevin Miller; 'Alex Regan'; users@spamassassin.apache.org
> Subject: Re: Mail Filter Recommendations
>
> if you are trying to retrofit registrarboundaries.pm onto older SA
> releases, you'll need to follow the instructions on the older PM file
> from your install or a previous SVN commit.
>
> In 3.4.1, registrarboundaries will be updatable via a cf file delivered
> by sa-update so we no longer need to maintain that list in the PM.
>
> regards,
> KAM
Re: Mail Filter Recommendations
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
if you are trying to retrofit registrarboundaries.pm onto older SA
releases, you'll need to follow the instructions on the older PM file
from your install or a previous SVN commit.
In 3.4.1, registrarboundaries will be updatable via a cf file delivered
by sa-update so we no longer need to maintain that list in the PM.
regards,
KAM
RE: Mail Filter Recommendations
Posted by Kevin Miller <ke...@juneau.org>.
> -----Original Message-----
> From: Alex Regan [mailto:mysqlstudent@gmail.com]
> Sent: Tuesday, April 07, 2015 2:35 PM
> To: users@spamassassin.apache.org
> Subject: Re: Mail Filter Recommendations
> Select the download link here:
>
> http://svn.apache.org/viewvc/spamassassin/trunk/lib/Mail/SpamAssassin/Ut
> il/RegistrarBoundaries.pm?view=log
>
> I believe this works with at least 3.4.0 or is it only 3.4.1?
>
> Regards,
> Alex
The latest shows this:
=======================================================================
DEPRECATED AND REPLACED WITH Mail::SpamAssassin::RegistryBoundaries !!
DO NOT USE. This is left as fallback for third party plugins.
=======================================================================
Is that just because it's being revamped in 3.4.2?
How does one go about installing it?
...Kevin
--
Kevin Miller
Network/email Administrator, CBJ MIS Dept.
155 South Seward Street
Juneau, Alaska 99801
Phone: (907) 586-0242, Fax: (907) 586-4500
Registered Linux User No: 307357
Re: Mail Filter Recommendations
Posted by RW <rw...@googlemail.com>.
On Tue, 07 Apr 2015 18:35:14 -0400
Alex Regan wrote:
> I think the reason it didn't match on anything useful for the OP is
> because he doesn't have the latest RegisterBoundaries.pm.
>
> If he had the latest, it would have at least matched the MSGID and
> MALFORMED rules.
These two rules don't appear to rely on RegisterBoundaries.pm, although
they do rely on 3.4.0+.
Re: Mail Filter Recommendations
Posted by Alex Regan <my...@gmail.com>.
Hi,
>>> Here's a couple of example spams that are the kind which are slipping
>>> through constantly. Some of the them get caught, others do not.
>>>
>>> http://pastebin.com/UH5BA6zs
>>> http://pastebin.com/esEz1a4J
>>
>> Neither of those is matching on much of anything useful
>
> a well trained bayes would catch both (our milter-reject score is 8.0)
>
> http://pastebin.com/UH5BA6zs:
> Content analysis details: (17.1 points, 5.5 required)
>
> pts rule name description
> ---- ----------------------
> --------------------------------------------------
> 5.0 NO_DNS_FOR_FROM DNS: Envelope sender has no MX or A DNS
> records
> 2.1 TO_MALFORMED To: has a malformed address
> -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
> domain
> 5.0 BAYES_80 BODY: Bayes spam probability is 80 to 95%
> [score: 0.9337]
> 4.0 MSGID_NOFQDN1 Message-ID with no domain name
> 1.0 INVALID_MSGID Message-Id is not valid, according to RFC 2822
I think the reason it didn't match on anything useful for the OP is
because he doesn't have the latest RegisterBoundaries.pm.
If he had the latest, it would have at least matched the MSGID and
MALFORMED rules.
Select the download link here:
http://svn.apache.org/viewvc/spamassassin/trunk/lib/Mail/SpamAssassin/Util/RegistrarBoundaries.pm?view=log
I believe this works with at least 3.4.0 or is it only 3.4.1?
Regards,
Alex
>
>
> http://pastebin.com/esEz1a4J
> Content analysis details: (32.0 points, 5.5 required)
>
> pts rule name description
> ---- ----------------------
> --------------------------------------------------
> 7.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%
> [score: 1.0000]
> 1.5 FROM_STARTS_WITH_NUMS From: starts with several numbers
> 2.1 TO_MALFORMED To: has a malformed address
> 4.5 CUST_DNSBL_7 RBL: b.barracudacentral.org
> [209.61.252.171 listed in
> b.barracudacentral.org]
> 3.0 DKIM_ADSP_NXDOMAIN No valid author signature and domain not in
> DNS
> 5.0 NO_DNS_FOR_FROM DNS: Envelope sender has no MX or A DNS
> records
> 0.4 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
> [score: 1.0000]
> 0.5 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/)
> 2.5 RDNS_NONE Delivered to internal network by a host
> with no rDNS
> 4.0 MSGID_NOFQDN1 Message-ID with no domain name
> 1.0 INVALID_MSGID Message-Id is not valid, according to RFC 2822
>
Re: Mail Filter Recommendations
Posted by Reindl Harald <h....@thelounge.net>.
Am 07.04.2015 um 22:35 schrieb Bowie Bailey:
> On 4/7/2015 3:07 PM, Asai wrote:
>> Thanks, Bowie and Noel,
>>
>> Here's a couple of example spams that are the kind which are slipping
>> through constantly. Some of the them get caught, others do not.
>>
>> http://pastebin.com/UH5BA6zs
>> http://pastebin.com/esEz1a4J
>
> Neither of those is matching on much of anything useful
a well trained bayes would catch both (our milter-reject score is 8.0)
http://pastebin.com/UH5BA6zs:
Content analysis details: (17.1 points, 5.5 required)
pts rule name description
---- ----------------------
--------------------------------------------------
5.0 NO_DNS_FOR_FROM DNS: Envelope sender has no MX or A DNS records
2.1 TO_MALFORMED To: has a malformed address
-0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
5.0 BAYES_80 BODY: Bayes spam probability is 80 to 95%
[score: 0.9337]
4.0 MSGID_NOFQDN1 Message-ID with no domain name
1.0 INVALID_MSGID Message-Id is not valid, according to RFC 2822
http://pastebin.com/esEz1a4J
Content analysis details: (32.0 points, 5.5 required)
pts rule name description
---- ----------------------
--------------------------------------------------
7.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[score: 1.0000]
1.5 FROM_STARTS_WITH_NUMS From: starts with several numbers
2.1 TO_MALFORMED To: has a malformed address
4.5 CUST_DNSBL_7 RBL: b.barracudacentral.org
[209.61.252.171 listed in
b.barracudacentral.org]
3.0 DKIM_ADSP_NXDOMAIN No valid author signature and domain not in DNS
5.0 NO_DNS_FOR_FROM DNS: Envelope sender has no MX or A DNS records
0.4 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
[score: 1.0000]
0.5 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/)
2.5 RDNS_NONE Delivered to internal network by a host
with no rDNS
4.0 MSGID_NOFQDN1 Message-ID with no domain name
1.0 INVALID_MSGID Message-Id is not valid, according to RFC 2822
Re: Mail Filter Recommendations
Posted by Bowie Bailey <Bo...@BUC.com>.
On 4/7/2015 3:07 PM, Asai wrote:
> Thanks, Bowie and Noel,
>
> Here's a couple of example spams that are the kind which are slipping
> through constantly. Some of the them get caught, others do not.
>
> http://pastebin.com/UH5BA6zs
> http://pastebin.com/esEz1a4J
Neither of those is matching on much of anything useful. I ran them
through my systems and they both hit on quite a few network tests
(blacklists, DCC, etc), but not much else. The network tests may or may
not have hit when you first received the message. Are you running with
the network tests active? Are you using DCC and Razor?
The second message hit on the KAM_MEDICARE rule from KAM.cf. You may
want to add that that rule set to your SA config if you are not using it
already.
http://www.pccc.com/downloads/SpamAssassin/contrib/KAM.cf
Also, make sure you are running the latest SA (3.4.0) and that you are
running sa-update on a regular basis to keep the rules updated.
--
Bowie
Re: Mail Filter Recommendations
Posted by Noel <no...@gmail.com>.
On 4/7/2015 2:07 PM, Asai wrote:
> Thanks, Bowie and Noel,
>
> Here's a couple of example spams that are the kind which are
> slipping through constantly. Some of the them get caught, others
> do not.
>
> http://pastebin.com/UH5BA6zs
> http://pastebin.com/esEz1a4J
Do you have a reasonably current version of SpamAssassin installed?
Updated the ruleset recently?
>
> Also, could you offer some guidance on how to set up scanning /per
> user/ for Bayes data with Amavis running at the same time?
Regardless of whether you use a site-wide or per-user bayes, it must
be trained. A well trained site-wide bayes is usually quite
effective if your users (mostly) agree on what is and isn't spam.
If per-user bayes is a requirement, something other than amavisd-new
may be more appropriate, but you'll lose the features and
efficiencies of amavisd-new. You get to decide that for yourself.
Re: Mail Filter Recommendations
Posted by Asai <as...@globalchangemusic.org>.
Thanks, Bowie and Noel,
Here's a couple of example spams that are the kind which are slipping
through constantly. Some of the them get caught, others do not.
http://pastebin.com/UH5BA6zs
http://pastebin.com/esEz1a4J
Also, could you offer some guidance on how to set up scanning /per user/
for Bayes data with Amavis running at the same time?
Thanks,
Asai
On 4/7/15 6:10 AM, Bowie Bailey wrote:
> On 4/6/2015 11:47 PM, Noel wrote:
>> On 4/6/2015 10:08 PM, Asai wrote:
>>> Greetings,
>>>
>>> We've been using Amavis for a number of years, but it seems to not
>>> be doing what we need it to be doing regarding spam filtering.
>>> e.g. I can't seem to get it to learn bayes data on a per user
>>> basis. We have our spam filters turned up so high ( kill level 3
>>> ) for some users it just seems like we're doing something wrong.
>>> I have deleted the Bayes data before and let it rebuild, but it
>>> doesn't seem to make sense that we've got all this crazy spam,
>>> some of which gets caught and some doesn't even though it's the
>>> exact same spam (or nearly the same). Is Amavis to blame here?
>>> Does it get in the way of Spamassassin running as it should, or is
>>> it more just configuration problems on our part?
>>>
>>> Thanks.
>> Amavis is normally configured for site-wide bayes as the "vscan" or
>> "amavis" user, not per-user.
>> If you're training bayes per-user, but SA is running with a
>> site-wide bayes, you'll get poor results.
>>
>> This is operator error, not the fault of Amavis or SpamAssassin.
>> You must train the proper bayes database.
>
> Also, please post a few spam samples (with SA headers) that did not
> get caught to pastebin.com and give us the links. If we can see the
> spam and what rules hit, we may be able to give you some more
> suggestions.
>
--
--asai
Re: Mail Filter Recommendations
Posted by Bowie Bailey <Bo...@BUC.com>.
On 4/6/2015 11:47 PM, Noel wrote:
> On 4/6/2015 10:08 PM, Asai wrote:
>> Greetings,
>>
>> We've been using Amavis for a number of years, but it seems to not
>> be doing what we need it to be doing regarding spam filtering.
>> e.g. I can't seem to get it to learn bayes data on a per user
>> basis. We have our spam filters turned up so high ( kill level 3
>> ) for some users it just seems like we're doing something wrong.
>> I have deleted the Bayes data before and let it rebuild, but it
>> doesn't seem to make sense that we've got all this crazy spam,
>> some of which gets caught and some doesn't even though it's the
>> exact same spam (or nearly the same). Is Amavis to blame here?
>> Does it get in the way of Spamassassin running as it should, or is
>> it more just configuration problems on our part?
>>
>> Thanks.
> Amavis is normally configured for site-wide bayes as the "vscan" or
> "amavis" user, not per-user.
> If you're training bayes per-user, but SA is running with a
> site-wide bayes, you'll get poor results.
>
> This is operator error, not the fault of Amavis or SpamAssassin.
> You must train the proper bayes database.
Also, please post a few spam samples (with SA headers) that did not get
caught to pastebin.com and give us the links. If we can see the spam
and what rules hit, we may be able to give you some more suggestions.
--
Bowie
Re: Mail Filter Recommendations
Posted by Noel <no...@gmail.com>.
On 4/6/2015 10:08 PM, Asai wrote:
> Greetings,
>
> We've been using Amavis for a number of years, but it seems to not
> be doing what we need it to be doing regarding spam filtering.
> e.g. I can't seem to get it to learn bayes data on a per user
> basis. We have our spam filters turned up so high ( kill level 3
> ) for some users it just seems like we're doing something wrong.
> I have deleted the Bayes data before and let it rebuild, but it
> doesn't seem to make sense that we've got all this crazy spam,
> some of which gets caught and some doesn't even though it's the
> exact same spam (or nearly the same). Is Amavis to blame here?
> Does it get in the way of Spamassassin running as it should, or is
> it more just configuration problems on our part?
>
> Thanks.
Amavis is normally configured for site-wide bayes as the "vscan" or
"amavis" user, not per-user.
If you're training bayes per-user, but SA is running with a
site-wide bayes, you'll get poor results.
This is operator error, not the fault of Amavis or SpamAssassin.
You must train the proper bayes database.
-- Noel Jones