You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2016/10/03 12:28:02 UTC

[Bug 7354] New: Allow custom addrlists for WLBLEval

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

            Bug ID: 7354
           Summary: Allow custom addrlists for WLBLEval
           Product: Spamassassin
           Version: 3.4 SVN branch
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: Plugins
          Assignee: dev@spamassassin.apache.org
          Reporter: paul.stead@gmail.com

Created attachment 5405
  --> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5405&action=edit
patch from trunk

On 30/09/16 01:07, Paul Stead wrote:
> Hi guys,
>
> Whilst looking through WLBLEval eval rules I noticed the likes of the
> following:
>
> -- 
>   $self->register_eval_rule("check_from_in_list");
>   $self->register_eval_rule("check_to_in_list");
> -- 
>
> Cool! I can check from and to lists, except I can't find a way to
> add/remove to addrlists like we can by using enlist_uri_host for hosts.
>

This patch adds a new directive "enlist_addrlist" to Conf.pm - after asking on
the Dev list I got no answer as to whether this new directive would be best
here or within WLBLEval.pm itself.

I've also added a patch to WLBLEval to add the eval check_replyto_in_list which
compliments the patch to Conf.pm

Example new rules which could come from this:

---8<---
enlist_addrlist (SUSTLDS) *@*.top
enlist_addrlist (SUSTLDS) *@*.download
enlist_addrlist (SUSTLDS) *@*.science

header __FROM_SUS_TLD eval:check_from_in_list('SUSTLDS') 
---8<---

Another example for something posted to the SA list earlier today:

---8<---
enlist_addrlist (PAYPAL_DOMAINS) *@paypal.com *@paypal.co.uk
header __FROM_PAYPAL eval:check_from_in_list('PAYPAL_DOMAINS')
---8<---

Thoughts?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Kevin A. McGrail <km...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kmcgrail@apache.org

--- Comment #4 from Kevin A. McGrail <km...@apache.org> ---
How would this change help you?  What is the goal?  What's a rule(s) that uses
it?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Kevin A. McGrail <km...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|Undefined                   |3.4.2
           Severity|enhancement                 |blocker

--- Comment #10 from Kevin A. McGrail <km...@apache.org> ---
Never mind, I see I had show obsolete turned on.  

Adding this patch but Paul, can you add more examples to the man page, please? 
Marking it as a blocker to get that done for 3.4.2

I'll update KAM.cf to use this eval too after it's in 3.4.2

Also add this to your list of potential rules to add to rulesrc like bug 7606.

3.4:
Committed revision 1839390.
trunk:
Committed revision 1839391.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Henrik Krohns <he...@hege.li> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.4.3                       |4.0.0
           Severity|blocker                     |enhancement

--- Comment #18 from Henrik Krohns <he...@hege.li> ---
Isn't this already committed? Targeting 4.0.0, add_to_addrlist() should be
optimized to use hashes when possible..

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #5 from Steadramon <pa...@gmail.com> ---
(In reply to Kevin A. McGrail from comment #4)
> How would this change help you?  What is the goal?  What's a rule(s) that
> uses it?

Hi,

I find this extra function quite handy when adding rules which match headers or
body uris, quite a bit easier to set up and cleaner looking that a regex
expression.

I have reffered to this BZ a few times on the mailing list:

http://spamassassin.1065346.n5.nabble.com/Blacklist-for-reply-to-td150498.html
http://spamassassin.1065346.n5.nabble.com/Ends-with-string-td139774.html
http://spamassassin.1065346.n5.nabble.com/top-and-other-spammy-TLDs-td124063.html

I have mostly used this to have a list of suspicious TLDs example in first
commit shows :

enlist_addrlist (SUSTLDS) *@*.top
enlist_addrlist (SUSTLDS) *@*.download
enlist_addrlist (SUSTLDS) *@*.science

header __FROM_SUS_TLD eval:check_from_in_list('SUSTLDS') 

In KAM.cf we have a similar rule:

header          __KAM_SOMETLD_ARE_BAD_TLD_FROM          From:addr =~
/\.(pw|stream|trade|bid|press|top|date)$/i

---

When this comes to matching more complex patterns in an email address I've seen
people on the mailing list with long regexes where a more simple
enlist_addrlist could be used, an example I have seen, but can't find at the
moment is where someone was wanting to match their own domains, with several
TLD variants...

enlist_addrlist (MYDOMAIN) *@example.com
enlist_addrlist (MYDOMAIN) *@example.org
enlist_addrlist (MYDOMAIN) *@example.co.uk

header __FROM_MYDOMAIN eval:check_from_in_list('MYDOMAIN') 
header __REPLYTO_MYDOMAIN eval:check_replyto_in_list('MYDOMAIN') 

---

I don't think it totally replaces From:addr regex but I do think adds a
completeness to the WLBLEvals eval ruleset and add an easily understandable,
readable set of features for SA.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #6 from Steadramon <pa...@gmail.com> ---
Just to note this BZ adds both enlist_addrlist and eval:check_replyto_in_list

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Henrik Krohns <ap...@hege.li> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #19 from Henrik Krohns <ap...@hege.li> ---
Addrlists work fast enough with Revision 1862688, closing.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Steadramon <pa...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #5405|0                           |1
        is obsolete|                            |
                 CC|                            |paul.stead@gmail.com

--- Comment #1 from Steadramon <pa...@gmail.com> ---
Created attachment 5406
  --> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5406&action=edit
patch from trunk

Fix paths and slight adjust on readme

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #8 from Kevin A. McGrail <km...@apache.org> ---
Paul, there are three patches attached.  Are the 1st two obsoleted by the 3rd?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #16 from Steadramon <pa...@gmail.com> ---

> I don't understand. I don't see anything like that.


Apologies - it replicates the enlist_uri_host syntax

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Henrik Krohns <he...@hege.li> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hege@hege.li

--- Comment #12 from Henrik Krohns <he...@hege.li> ---
Isn't 3.4 supposed to be bug fixes only?

Also I really don't like the whole (GROUP) syntax.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #13 from Steadramon <pa...@gmail.com> ---
(In reply to Henrik Krohns from comment #12)
> Isn't 3.4 supposed to be bug fixes only?
> 
> Also I really don't like the whole (GROUP) syntax.

The (GROUP) syntax follows the same syntax that was already established in this
plugin

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #3 from Steadramon <pa...@gmail.com> ---
(In reply to Steadramon from comment #0)

> Another example for something posted to the SA list earlier today:
> 
> ---8<---
> enlist_addrlist (PAYPAL_DOMAINS) *@paypal.com *@paypal.co.uk
> header __FROM_PAYPAL eval:check_from_in_list('PAYPAL_DOMAINS')
> ---8<---

The person wasn't interested if the email was validly from PayPal, just that
the fingerprint was there

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

John Hardin <jh...@impsec.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jhardin@impsec.org

--- Comment #9 from John Hardin <jh...@impsec.org> ---
(In reply to Steadramon from comment #7)
> Another example on the SA user list which has a "messy" regex which would be
> a lot easier to create, maintain and read in the future:
> 
> http://mail-archives.apache.org/mod_mbox/spamassassin-users/201711.mbox/
> %3Calpine.LNX.2.00.1711271736050.30613@athena.impsec.org%3E
> 
> ---8<---
> 
> header     FROM_RARE_TLD    From:addr =~
> /\.(?:wor(?:
> k|ld)|space|club|science|pub|red|blue|green|link|ninja|lol|xyz|faith|review|d
> ownload|top|global|(?:
> web)?site|tech|party|pro|bid|trade|win|moda|news|online|biz|host|loan|study|c
> lick|stream|xxx|date)$/i
> describe   FROM_RARE_TLD    From address in rarely-nonspam TLD
> 
> ---8<---

Any comparative performance data for the different approaches?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #7 from Steadramon <pa...@gmail.com> ---
Another example on the SA user list which has a "messy" regex which would be a
lot easier to create, maintain and read in the future:

http://mail-archives.apache.org/mod_mbox/spamassassin-users/201711.mbox/%3Calpine.LNX.2.00.1711271736050.30613@athena.impsec.org%3E

---8<---

header     FROM_RARE_TLD    From:addr =~
/\.(?:wor(?:k|ld)|space|club|science|pub|red|blue|green|link|ninja|lol|xyz|faith|review|download|top|global|(?:web)?site|tech|party|pro|bid|trade|win|moda|news|online|biz|host|loan|study|click|stream|xxx|date)$/i
describe   FROM_RARE_TLD    From address in rarely-nonspam TLD

---8<---

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #14 from Kevin A. McGrail <km...@apache.org> ---
3.4.3 is for bug fixes, yes.  The plugin is in 3.4.2 so I think it's ok for
3.4.3.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #15 from Henrik Krohns <he...@hege.li> ---
(In reply to Steadramon from comment #13)
> (In reply to Henrik Krohns from comment #12)
> > Isn't 3.4 supposed to be bug fixes only?
> > 
> > Also I really don't like the whole (GROUP) syntax.
> 
> The (GROUP) syntax follows the same syntax that was already established in
> this plugin

I don't understand. I don't see anything like that.

There is call for unifying all these addrlist stuff, so I vote to keep all
enhacements to 4.0 and think about the formats carefully. There are already so
many "internal regex addrlists" around that I'd also like to optimize the
lookups etc.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #2 from Steadramon <pa...@gmail.com> ---
Created attachment 5407
  --> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5407&action=edit
patch from trunk

*actually* adjusted readme text

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Steadramon <pa...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #5406|0                           |1
        is obsolete|                            |

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

--- Comment #17 from Henrik Krohns <he...@hege.li> ---
(In reply to Steadramon from comment #16)
> > I don't understand. I don't see anything like that.
> 
> 
> Apologies - it replicates the enlist_uri_host syntax

Thanks, I had no idea about that. I guess it's too late to change the syntax.
:-) Doesn't matter.

I'll look a bit into these.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7354] Allow custom addrlists for WLBLEval

Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7354

Kevin A. McGrail <km...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.4.2                       |3.4.3

--- Comment #11 from Kevin A. McGrail <km...@apache.org> ---
Pushing to 3.4.3 where Paul can commit this himself :-)

-- 
You are receiving this mail because:
You are the assignee for the bug.