You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Jeff Chan <je...@surbl.org> on 2010/03/12 22:17:20 UTC
xs.surbl.org
Hi Guys,
Would there be time for us to add xs.surbl.org into the 128th bit
of multi and get it into 3.3.1? We would add much more data into
XS than currently is there.
Cheers,
Jeff C.
Re: xs.surbl.org
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 13/03/2010 7:58 AM, Kevin A. McGrail wrote:
>
>> Well, maybe not quite. As recently mentioned we need to be careful
>> about how we go about releasing new network rules. If a rule causes a
>> new lookup to be done (including if, like in this case, other existing
>> lookups shared by this rule are/could be disabled) we need to have a way
>> to notify users of the rule being added. I suggested only adding new
>> net rules in new code releases and ensuring they don't appear in old
>> version updates by using "if version" lines.
>>
>> Daryl
>>
>
> Agree on being careful but not sure I agree that net rules only get
> released in new code. Net rules are a big part of an effective
> anti-spam system but I see your point.
So, just release some code, whenever. Theo and I released 3.1 versions
every month for a while. Somebody just needs the tuits to do it. We
have to have a way to give people heads up about new net rules.
Releasing net rules without a heads up is unfair to both the users and
DNSBL operators. Imagine releasing a new net rule to 100 sites that
process 50 million messages a day. That could be un-good for an
unsuspecting DNSBL operator. Releasing a new net rule with a new
version allows us to give users a heads up and DNSBL operators time to
slowly ramp up query volume capacity.
> However, in light of your position, I believe I am right that this is a
> "simple" matter of testing a new net rule for 3.3.X branch from my
> original response?
Yeah, I'd just use whatever rules get generated by the weekly score gen
for net rules.
Do note that if the rule has been committed it had better have "if
version" lines around it. Otherwise it's probably going to get released
in an update in about 4 hours.
Daryl
Re: xs.surbl.org
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
> Well, maybe not quite. As recently mentioned we need to be careful
> about how we go about releasing new network rules. If a rule causes a
> new lookup to be done (including if, like in this case, other existing
> lookups shared by this rule are/could be disabled) we need to have a way
> to notify users of the rule being added. I suggested only adding new
> net rules in new code releases and ensuring they don't appear in old
> version updates by using "if version" lines.
>
> Daryl
>
Agree on being careful but not sure I agree that net rules only get
released in new code. Net rules are a big part of an effective
anti-spam system but I see your point.
However, in light of your position, I believe I am right that this is a
"simple" matter of testing a new net rule for 3.3.X branch from my
original response?
Regards,
KAM
Re: xs.surbl.org
Posted by Mark Martinec <Ma...@ijs.si>.
Please followups to:
Bug 6374 - adding xs.surbl.org into the 128th bit of multi.surbl.org
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6374
Mark
Re: xs.surbl.org
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 13/03/2010 10:30 PM, John Hardin wrote:
> On Sat, 13 Mar 2010, Daryl C. W. O'Shea wrote:
>
>> On 13/03/2010 8:00 AM, Jeff Chan wrote:
>>> This brings up a couple questions:
>>>
>>> 1. How often are the scores for existing rules re-calculated? If
>>> we shuffled some of data for the existing SURBL lists internally,
>>> they'd presumably need to be rescored. Does that only happen at
>>> release time?
>>
>> To date we've only done score generation for all rules for major
>> releases (3.0.0, 3.1.0, 3.2.0, 3.3.0).
>
> I've been planning on updating the ADVANCE_FEE ruleset for a while now,
> but I'm concerned about how the scores would for my changes would be
> generated given this model. Any suggestions?
If the changes are drastic enough that you think they'll significantly
alter what the rules hit on, I would retire the old rules and create new
rules. The new rules will have scores generated for them by the
weekly/nightly score gen.
Daryl
Re: xs.surbl.org
Posted by John Hardin <jh...@impsec.org>.
On Sat, 13 Mar 2010, Daryl C. W. O'Shea wrote:
> On 13/03/2010 8:00 AM, Jeff Chan wrote:
>> This brings up a couple questions:
>>
>> 1. How often are the scores for existing rules re-calculated? If
>> we shuffled some of data for the existing SURBL lists internally,
>> they'd presumably need to be rescored. Does that only happen at
>> release time?
>
> To date we've only done score generation for all rules for major
> releases (3.0.0, 3.1.0, 3.2.0, 3.3.0).
I've been planning on updating the ADVANCE_FEE ruleset for a while now,
but I'm concerned about how the scores would for my changes would be
generated given this model. Any suggestions?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Failure to plan ahead on someone else's part does not constitute
an emergency on my part. -- David W. Barts in a.s.r
-----------------------------------------------------------------------
Tomorrow: Daylight Saving Time begins in U.S. - Spring Forward
Re: xs.surbl.org
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 13/03/2010 8:00 AM, Jeff Chan wrote:
> This brings up a couple questions:
>
> 1. How often are the scores for existing rules re-calculated? If
> we shuffled some of data for the existing SURBL lists internally,
> they'd presumably need to be rescored. Does that only happen at
> release time?
To date we've only done score generation for all rules for major
releases (3.0.0, 3.1.0, 3.2.0, 3.3.0).
Any major shuffling would affect how things should be scored... and
would affect people running not-the-latest-version no matter if we did a
new score generation exercise or not.
> 2. If we did want to add XS, when would 3.3.1 be likely to get
> frozen? (Partially depends on question 1.)
Stable releases are frozen when they're ready for release. It's a quick
process... somebody decides they want to do a release (hopefully with
others' agreement), they propose a release tarball, wait three days, and
release it.
If you want it in a 3.3 version it's not a big deal for us to release
3.3.2 a couple of weeks after 3.3.1. There's no need to rush it into 3.3.1.
Daryl
Re: xs.surbl.org
Posted by Jeff Chan <je...@surbl.org>.
On Friday, March 12, 2010, 4:51:13 PM, Daryl O'Shea wrote:
> On 12/03/2010 4:29 PM, Kevin A. McGrail wrote:
>>
>>> As far as I know, all that really requires would be adding a rule and
>>> a masscheck to confirm the scoring:
>>>
>>> urirhssub URIBL_XS_SURBL multi.surbl.org. A 128
>>> body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL')
>>> describe URIBL_XS_SURBL Contains an URL listed in the XS SURBL
>>> blocklist
>>> tflags URIBL_XS_SURBL net
>>> #reuse URIBL_XS_SURBL
>>>
>>
>> Following up my previous test, this rule does not require consideration
>> for a specific release as it's a rule only and would work with 3.3.0,
>> for example. So all it needs to do is be approved as a rule added to
>> sa-update for 3.3.X.
> Well, maybe not quite. As recently mentioned we need to be careful
> about how we go about releasing new network rules. If a rule causes a
> new lookup to be done (including if, like in this case, other existing
> lookups shared by this rule are/could be disabled) we need to have a way
> to notify users of the rule being added. I suggested only adding new
> net rules in new code releases and ensuring they don't appear in old
> version updates by using "if version" lines.
Thanks Daryl,
This brings up a couple questions:
1. How often are the scores for existing rules re-calculated? If
we shuffled some of data for the existing SURBL lists internally,
they'd presumably need to be rescored. Does that only happen at
release time?
2. If we did want to add XS, when would 3.3.1 be likely to get
frozen? (Partially depends on question 1.)
Cheers,
Jeff C.
Re: xs.surbl.org
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 12/03/2010 4:29 PM, Kevin A. McGrail wrote:
>
>> As far as I know, all that really requires would be adding a rule and
>> a masscheck to confirm the scoring:
>>
>> urirhssub URIBL_XS_SURBL multi.surbl.org. A 128
>> body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL')
>> describe URIBL_XS_SURBL Contains an URL listed in the XS SURBL
>> blocklist
>> tflags URIBL_XS_SURBL net
>> #reuse URIBL_XS_SURBL
>>
>
> Following up my previous test, this rule does not require consideration
> for a specific release as it's a rule only and would work with 3.3.0,
> for example. So all it needs to do is be approved as a rule added to
> sa-update for 3.3.X.
Well, maybe not quite. As recently mentioned we need to be careful
about how we go about releasing new network rules. If a rule causes a
new lookup to be done (including if, like in this case, other existing
lookups shared by this rule are/could be disabled) we need to have a way
to notify users of the rule being added. I suggested only adding new
net rules in new code releases and ensuring they don't appear in old
version updates by using "if version" lines.
Daryl
Re: xs.surbl.org
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
> As far as I know, all that really requires would be adding a rule and
> a masscheck to confirm the scoring:
>
> urirhssub URIBL_XS_SURBL multi.surbl.org. A 128
> body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL')
> describe URIBL_XS_SURBL Contains an URL listed in the XS SURBL
> blocklist
> tflags URIBL_XS_SURBL net
> #reuse URIBL_XS_SURBL
>
Following up my previous test, this rule does not require consideration
for a specific release as it's a rule only and would work with 3.3.0,
for example. So all it needs to do is be approved as a rule added to
sa-update for 3.3.X.
I think it could also be added for 3.2.X if we are still maintaining
those rules as well.
This is the key point of separating the rules from the code. If no code
is needed, there is no need to wait for a release to add a good and
valuable rule.
Regards,
KAM
Re: xs.surbl.org
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 3/12/2010 4:17 PM, Jeff Chan wrote:
> Hi Guys,
> Would there be time for us to add xs.surbl.org into the 128th bit
> of multi and get it into 3.3.1? We would add much more data into
> XS than currently is there.
>
> Cheers,
>
> Jeff C.
>
Jeff,
As far as I know, all that really requires would be adding a rule and a
masscheck to confirm the scoring:
urirhssub URIBL_XS_SURBL multi.surbl.org. A 128
body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL')
describe URIBL_XS_SURBL Contains an URL listed in the XS SURBL
blocklist
tflags URIBL_XS_SURBL net
#reuse URIBL_XS_SURBL
I've been testing with xs with a score of 2.0 since october of 2009 and
I have no problem supporting this for consideration for the next
release. Here's the exact rule I've been testing:
urirhsbl URIBL_XS_SURBL xs.surbl.org. A
body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL')
describe URIBL_XS_SURBL Has URI in XS - Testing
tflags URIBL_XS_SURBL net
score URIBL_XS_SURBL 2.0
Regards,
KAM