You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@issues.apache.org on 2011/01/21 10:03:53 UTC
[Bug 6537] New: Update the uridnsbl_skip_domain list
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
Summary: Update the uridnsbl_skip_domain list
Product: Spamassassin
Version: SVN Trunk (Latest Devel Version)
Platform: PC
OS/Version: Linux
Status: NEW
Severity: normal
Priority: P2
Component: Rules
AssignedTo: dev@spamassassin.apache.org
ReportedBy: wtogami@gmail.com
rules/25_uribl.cf contains a list of roughly 200 uridnsbl_skip_domain. The
purpose of this list is to avoid trillions of lookups for domains that will
never be blacklisted. Unfortunately the last time it was updated was so long
ago that it was lacking common URL's like facebook or twitter.
SURBL has provided the PMC with a list of recent most frequently queried
whitelisted domains along with frequency counts. PMC should decide which among
that list are worthwhile to add to our skip list.
SURBL has requested that we do not post the raw data publicly because that list
could be used by spammers to learn of the list of whitelisted domains.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
Kevin A. McGrail <km...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |kmcgrail@pccc.com
--- Comment #2 from Kevin A. McGrail <km...@pccc.com> 2011-01-21 09:31:13 UTC ---
> SURBL has provided the PMC with a list of recent most frequently queried
> whitelisted domains along with frequency counts. PMC should decide which among
> that list are worthwhile to add to our skip list.
This isn't a PMC issue. This is a developer issue and I am +1 for
expanding/revising the list.
Submit a patch of the suggested list, please. If you feel it may need to be
private, I guess you can submit a proposed patch to private@.
Regards,
KAM
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
AXB <ax...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |RESOLVED
Resolution|--- |FIXED
--- Comment #25 from AXB <ax...@gmail.com> ---
closing this after having reviewed/added domains from WT's list
" wtogami's most frequent known good URIDNSBL lookups (1/1/2011)"
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #3 from Warren Togami <wt...@gmail.com> 2011-01-26 19:14:46 EST ---
nbebout is working on the list cleaning while I am away. He will post a
proposal to private@.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #11 from AXB <ax...@gmail.com> 2011-05-03 06:43:16 UTC ---
(In reply to comment #10)
> Bug #6531 made it possible to override the skip list. Is this not good enough?
ya.. that's ok. I had missed that one.
Pls do not include ESPs and massmailers.
Social networking, banks, hospitals, airlines, hotels & such are OK.
Web hosters can be a problem if skipped as there is a huge abundancy of hacked
servers/phish/malware spreading and thesde should not be given a SA "blank
check"
Same applies to many corps.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #23 from AXB <ax...@gmail.com> 2012-01-18 22:08:23 UTC ---
(In reply to comment #22)
> This seems like a good idea but missing someone to propose an actual list, I
> think. The whiteboard says "ready to commit" which I've removed.
>
> Warren, any luck on a revised list? Did it get committed and I missed it?
> Sent on a private list?
>
> Moving this to a blocker for 3.4.0. We should get this in the code.
This is low prio an can be updated anytime and pushed via sa-update.
Hardly a reason for a blocker.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #10 from Warren Togami <wt...@gmail.com> 2011-05-03 06:37:02 UTC ---
Bug #6531 made it possible to override the skip list. Is this not good enough?
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #12 from Warren Togami <wt...@gmail.com> 2011-05-03 06:46:12 UTC ---
(In reply to comment #11)
>
> Pls do not include ESPs and massmailers.
> Social networking, banks, hospitals, airlines, hotels & such are OK.
>
> Web hosters can be a problem if skipped as there is a huge abundancy of hacked
> servers/phish/malware spreading and thesde should not be given a SA "blank
> check"
> Same applies to many corps.
I am mostly in agreement. I'll hand pick changes to the list and propose it
here before committing.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #17 from Mark Martinec <Ma...@ijs.si> 2011-05-06 13:42:04 UTC ---
> Bug #6531 made it possible to override the skip list.
> > That one is targeted to 3.4.
> > Would it be better to retarget it to 3.3.2?
> > (it's a fairly straightforward patch)
> > +1 for 3.3.2
The Bug 6531 (clear_uridnsbl_skip_domain) is now in 3.3.2.
> Aren't we waiting on a skip list from Warren to vote on anything? Though in
> spirit I am +1 and I have every confidence that Warren's list will be well
> researched.
Sure. Back to waiting on skip list updates. Can't hurt to have both,
an updated list and some additional user-controlled flexibility.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
Jeff Chan <je...@surbl.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jeffc@surbl.org
--- Comment #1 from Jeff Chan <je...@surbl.org> 2011-01-21 04:59:06 UTC ---
+1 :)
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #7 from AXB <ax...@gmail.com> 2011-05-03 06:03:24 UTC ---
(In reply to comment #6)
> We have a hit frequency list from SURBL. I will personally hand-pick a
> reasonable addition for the skip list.
IMO uridnsbl_skip_domain list should be treated as a whitelist and move to a
dedicated file.
Updates should be pushed via a dedicated SA channel as not everyone requires or
wishes to be ruled as to what they can query or not.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #9 from AXB <ax...@gmail.com> 2011-05-03 06:32:26 UTC ---
(In reply to comment #8)
> > IMO uridnsbl_skip_domain list should be treated as a whitelist and move to a
> > dedicated file.
> > Updates should be pushed via a dedicated SA channel as not everyone requires or
> > wishes to be ruled as to what they can query or not.
>
> The list should contain stuff like facebook.com, where there is never any
> productive reason to query it. Thus I continue to be puzzled by this desire to
> remove the entire skip list from the default rules. In reality, how many
> people will ever need this?
you are right BUT: as there is no "unskip" feature built into
uridnsbl_skip_domain, it's not user reversible so you impose you choice on ppl.
SA is a framework and not our personal
I object to any whitelistings unless they are are reversible by a user.
> I feel that this resistance is not productive.
not relevant.
IMO, SA should remain a framework which gives users options based on scoring.
uridnsbl_skip_domain doesn't offer this.
As soon as someone includes a switch to reverse this, I'm 100% +1 to any
additions to your list.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #21 from AXB <ax...@gmail.com> 2011-05-06 20:45:46 UTC ---
(In reply to comment #20)
> (In reply to comment #19)
> > I'd be willing to review the list and add/comment on its content.
>
> Beware that the SURBL list is heavily biased because of spamassassin's
> preexisting skip list. So some educating guessing will be involved.
I have good insight into what is being frequently and "pointlessly" queried.
Would get in contact with Jeff Chan if necessary.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #18 from Warren Togami <wt...@gmail.com> 2011-05-06 19:32:17 UTC ---
Sorry, a bit too busy today to make a hand selected list. I might be able to
get to it next week. Meanwhile, PMC should have the frequency list generated
by SURBL, and someone else should be able to pick domains from that list.
NOTE: DO NOT SHARE THE RAW FREQUENCY LIST WITH THE PUBLIC.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
Kevin A. McGrail <km...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard| |ready to commit
--- Comment #15 from Kevin A. McGrail <km...@pccc.com> 2011-05-06 13:12:04 UTC ---
(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #10)
> > > Bug #6531 made it possible to override the skip list. Is this not good enough?
> >
> > That one is targeted to 3.4.
> > Would it be better to retarget it to 3.3.2?
> > (it's a fairly straightforward patch)
>
> +1 for 3.3.2
Aren't we waiting on a skip list from Warren to vote on anything? Though in
spirit I am +1 and I have every confidence that Warren's list will be well
researched.
KAM
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #4 from Karsten Bräckelmann <gu...@rudersport.de> 2011-01-26 19:35:50 EST ---
As I mentioned someplace else, SA is not the only user of URIBLs. Milters are
too, and in the list is plenty of evidence for that -- URIs SA never would have
queried for.
Rather than manually working on a list to add, and do it again later, and
again... here's a (possibly crack) idea.
How hard would it be for SURBL (or URIBL) to generate such a list periodically?
Maybe on a monthly basis, maybe 6 months. Whatever.
I guess it should not be too hard to write a script, that *automatically*
generates a fresh list, to be included in new releases and even updates. Given
a history, the script could add raising URIs, and (eventually) would drop
fading out URIs, that have been below a certain threshold for N iterations.
That way, there is no need for manual work later. There will be no need for
bugs like this. And, given URIs will drop off eventually, the list will not
only grow, but include those that actually DO cause queries for the mirrors.
Yes, I would volunteer to write that script...
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #5 from Warren Togami <wt...@gmail.com> 2011-01-26 22:14:08 EST ---
> As I mentioned someplace else, SA is not the only user of URIBLs. Milters are
> too, and in the list is plenty of evidence for that -- URIs SA never would have
> queried for.
> How hard would it be for SURBL (or URIBL) to generate such a list periodically?
> Maybe on a monthly basis, maybe 6 months. Whatever.
I agree this is a good idea, but the existing skip list in spamassassin might
be causing significant bias in the hit frequencies. It might even be the
majority of the SURBL users?
Are the milters really operating without their own skip lists?
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #8 from Warren Togami <wt...@gmail.com> 2011-05-03 06:25:33 UTC ---
> IMO uridnsbl_skip_domain list should be treated as a whitelist and move to a
> dedicated file.
> Updates should be pushed via a dedicated SA channel as not everyone requires or
> wishes to be ruled as to what they can query or not.
The list should contain stuff like facebook.com, where there is never any
productive reason to query it. Thus I continue to be puzzled by this desire to
remove the entire skip list from the default rules. In reality, how many
people will ever need this?
I feel that this resistance is not productive.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
Kevin A. McGrail <km...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.3.2 |3.4.0
Status Whiteboard|ready to commit |
Severity|normal |blocker
--- Comment #22 from Kevin A. McGrail <km...@pccc.com> 2012-01-18 22:02:10 UTC ---
This seems like a good idea but missing someone to propose an actual list, I
think. The whiteboard says "ready to commit" which I've removed.
Warren, any luck on a revised list? Did it get committed and I missed it?
Sent on a private list?
Moving this to a blocker for 3.4.0. We should get this in the code.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #20 from Warren Togami <wt...@gmail.com> 2011-05-06 20:27:57 UTC ---
(In reply to comment #19)
> I'd be willing to review the list and add/comment on its content.
Beware that the SURBL list is heavily biased because of spamassassin's
preexisting skip list. So some educating guessing will be involved.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #14 from AXB <ax...@gmail.com> 2011-05-06 12:09:05 UTC ---
(In reply to comment #13)
> (In reply to comment #10)
> > Bug #6531 made it possible to override the skip list. Is this not good enough?
>
> That one is targeted to 3.4.
> Would it be better to retarget it to 3.3.2?
> (it's a fairly straightforward patch)
+1 for 3.3.2
(I'd love to see the blacklist_uri stuff you made for 3.4 be ported to 3.3.x
but that's a major, I'd assume)
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #13 from Mark Martinec <Ma...@ijs.si> 2011-05-06 12:00:34 UTC ---
(In reply to comment #10)
> Bug #6531 made it possible to override the skip list. Is this not good enough?
That one is targeted to 3.4.
Would it be better to retarget it to 3.3.2?
(it's a fairly straightforward patch)
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #19 from AXB <ax...@gmail.com> 2011-05-06 20:20:56 UTC ---
I'd be willing to review the list and add/comment on its content.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
Kevin A. McGrail <km...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Priority|P2 |P1
Target Milestone|3.4.0 |Undefined
Severity|blocker |major
--- Comment #24 from Kevin A. McGrail <km...@pccc.com> 2012-01-18 22:19:33 UTC ---
> This is low prio an can be updated anytime and pushed via sa-update.
> Hardly a reason for a blocker.
Good point but this should have some priority to it.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
Warren Togami <wt...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |ASSIGNED
Target Milestone|Undefined |3.3.2
--- Comment #6 from Warren Togami <wt...@gmail.com> 2011-05-03 05:55:39 UTC ---
We have a hit frequency list from SURBL. I will personally hand-pick a
reasonable addition for the skip list.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6537] Update the uridnsbl_skip_domain list
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6537
--- Comment #16 from AXB <ax...@gmail.com> 2011-05-06 13:23:09 UTC ---
(In reply to comment #15)
> (In reply to comment #14)
> > (In reply to comment #13)
> > > (In reply to comment #10)
> > > > Bug #6531 made it possible to override the skip list. Is this not good enough?
> > >
> > > That one is targeted to 3.4.
> > > Would it be better to retarget it to 3.3.2?
> > > (it's a fairly straightforward patch)
> >
> > +1 for 3.3.2
>
> Aren't we waiting on a skip list from Warren to vote on anything? Though in
> spirit I am +1 and I have every confidence that Warren's list will be well
> researched.
>
> KAM
in 25_uribl.cf I see an unsorted list of domains
# wtogami's most frequent known good URIDNSBL lookups (1/1/2011)
(?)
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.