You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Justin Mason <jm...@jmason.org> on 2010/03/15 15:41:57 UTC

proposed 3.3.1 tarballs

http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
http://people.apache.org/~jm/devel/

please vote!

md5sum of archive files:

  2290490889b2d91f71a3104eaf9c5cd3  Mail-SpamAssassin-3.3.1.tar.bz2
  e70096d6baa695371b413e6691a49038  Mail-SpamAssassin-3.3.1.tar.gz
  915e22168642997f168f5678f66d8def  Mail-SpamAssassin-3.3.1.zip
  e004b3172650f003c2e9f15e5b2f6ac4  Mail-SpamAssassin-rules-3.3.1.r923257.tgz

sha1sum of archive files:

  9a84b175241c52cf6e1bf4cfcf0719cf65f5dc18  Mail-SpamAssassin-3.3.1.tar.bz2
  4160be9c795573d8208a81665614387797136030  Mail-SpamAssassin-3.3.1.tar.gz
  8f8ba7c69283bec99a91a7d266f8e5a1477c3b41  Mail-SpamAssassin-3.3.1.zip
  e211a58e0d4cf7d30fdaf6b9ff8f797086f96312
Mail-SpamAssassin-rules-3.3.1.r923257.tgz


cheers,

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by Doc Schneider <ma...@maddoc.net>.
Justin Mason wrote:
> http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
> http://people.apache.org/~jm/devel/
> 
> please vote!
> 
> md5sum of archive files:
> 
>   2290490889b2d91f71a3104eaf9c5cd3  Mail-SpamAssassin-3.3.1.tar.bz2
>   e70096d6baa695371b413e6691a49038  Mail-SpamAssassin-3.3.1.tar.gz
>   915e22168642997f168f5678f66d8def  Mail-SpamAssassin-3.3.1.zip
>   e004b3172650f003c2e9f15e5b2f6ac4  Mail-SpamAssassin-rules-3.3.1.r923257.tgz
> 
> sha1sum of archive files:
> 
>   9a84b175241c52cf6e1bf4cfcf0719cf65f5dc18  Mail-SpamAssassin-3.3.1.tar.bz2
>   4160be9c795573d8208a81665614387797136030  Mail-SpamAssassin-3.3.1.tar.gz
>   8f8ba7c69283bec99a91a7d266f8e5a1477c3b41  Mail-SpamAssassin-3.3.1.zip
>   e211a58e0d4cf7d30fdaf6b9ff8f797086f96312
> Mail-SpamAssassin-rules-3.3.1.r923257.tgz
> 
> 
> cheers,
> 

Crap I sent my +1 only to Justin.

But a +1 here.

-- 

  -Doc

  Penguins: Do it on the ice.
  Cairns: Do it underground

Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
On Tue, Mar 16, 2010 at 13:18, Mark Martinec <Ma...@ijs.si> wrote:
>> A diff between the proposed 3.3.1 and trunk reveals a couple of
>> trivialities which should go into 3.3.2 (or into 3.3.1 if there will be
>> a re-cut).
>
> 3.3:
> Backported docs spelling fixes and CREDITS update from trunk,
> one trivial fix to URIDNSBL.pm to avoid undef warnings
> Sending        CREDITS
> Sending        lib/Mail/SpamAssassin/BayesStore/PgSQL.pm
> Sending        lib/Mail/SpamAssassin/Client.pm
> Sending        lib/Mail/SpamAssassin/PerMsgLearner.pm
> Sending        lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
> Sending        lib/Mail/SpamAssassin/Plugin.pm
> Sending        lib/Mail/SpamAssassin.pm
> Sending        lib/spamassassin-run.pod
> Sending        sa-learn.raw
> Transmitting file data .........
> Committed revision 923725.
>
> I'll leave Justin to decide whether t/uribl_domains_only.t
> and t/uribl_ips_only.t need to be backported from trunk.

let's not; spurious test failures can really cause hassle for users.
perhaps open a bug to remind us to backport those changes post 3.3.1.

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by Mark Martinec <Ma...@ijs.si>.
> A diff between the proposed 3.3.1 and trunk reveals a couple of
> trivialities which should go into 3.3.2 (or into 3.3.1 if there will be
> a re-cut).

3.3:
Backported docs spelling fixes and CREDITS update from trunk,                                                  
one trivial fix to URIDNSBL.pm to avoid undef warnings                                                         
Sending        CREDITS
Sending        lib/Mail/SpamAssassin/BayesStore/PgSQL.pm
Sending        lib/Mail/SpamAssassin/Client.pm
Sending        lib/Mail/SpamAssassin/PerMsgLearner.pm
Sending        lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
Sending        lib/Mail/SpamAssassin/Plugin.pm
Sending        lib/Mail/SpamAssassin.pm
Sending        lib/spamassassin-run.pod
Sending        sa-learn.raw
Transmitting file data .........
Committed revision 923725.

I'll leave Justin to decide whether t/uribl_domains_only.t
and t/uribl_ips_only.t need to be backported from trunk.


  Mark

Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
On Wed, Mar 17, 2010 at 00:23, Daryl C. W. O'Shea
<sp...@dostech.ca> wrote:
> On 16/03/2010 10:36 AM, Justin Mason wrote:
>> On Tue, Mar 16, 2010 at 12:52, Justin Mason <jm...@jmason.org> wrote:
>>> For long term use, though, we'll need some way to cut a rules tarball
>>> using what's in SVN right now, rather than what was there on the previous
>>> night.   in my opinion it's unsafe to risk differences between what's
>>> live in svn and what we're releasing, particularly if changes went in
>>> during that window.
>>>
>>> We can, of course, use the scores that were generated then (but with
>>> changes since then included too).
>>
>> simple way to do this -- can we just SVN merge from trunk/rules back
>> to 3.3/rules?
>> that'd be quite elegant and easy to do.
>
> I'm not sure what we're really merging in this scenario.  If we're not
> going to maintain more than one set of rules (ie. no back-porting, which
> I think you agreed with previously today) is there any benefit to
> essentially just creating a copy of the rules directory?

The benefit is that when we tag a release, we'll also be tagging a
ruleset known to work with that release.  We also allow the test suite
to test against that ruleset in SVN.  It simplifies the build scripts
(I think).

> If there is, I
> think it's a little more complicated than just merging rules/.

indeed, we may need to "svn rm" / "svn cp" the directory instead
of attempting to merge it.

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 16/03/2010 10:36 AM, Justin Mason wrote:
> On Tue, Mar 16, 2010 at 12:52, Justin Mason <jm...@jmason.org> wrote:
>> For long term use, though, we'll need some way to cut a rules tarball
>> using what's in SVN right now, rather than what was there on the previous
>> night.   in my opinion it's unsafe to risk differences between what's
>> live in svn and what we're releasing, particularly if changes went in
>> during that window.
>>
>> We can, of course, use the scores that were generated then (but with
>> changes since then included too).
> 
> simple way to do this -- can we just SVN merge from trunk/rules back
> to 3.3/rules?
> that'd be quite elegant and easy to do.

I'm not sure what we're really merging in this scenario.  If we're not
going to maintain more than one set of rules (ie. no back-porting, which
I think you agreed with previously today) is there any benefit to
essentially just creating a copy of the rules directory?  If there is, I
think it's a little more complicated than just merging rules/.

Daryl


Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
On Tue, Mar 16, 2010 at 12:52, Justin Mason <jm...@jmason.org> wrote:
> On Tue, Mar 16, 2010 at 00:36, Daryl C. W. O'Shea
> <sp...@dostech.ca> wrote:
>> If we're publishing rule updates for 3.3 from trunk I don't see why we'd
>> generate a rule tarball from the branch (with sandbox rules, sans
>> scores, anyway).  If you install 3.3 using sa-update to get the rules
>> you're getting the trunk version of the rules; the tarball should give
>> you the same sort of thing.
>>
>>> if so, what script should we use to do so?
>>
>> Just grab a recent nightly update, rename it, and use that.  The safest
>> one to use would be a weekly one right after the net enabled mass-check
>> results (Saturday night's update around 10:30 PM ET -- Sunday, 2:30 AM
>> UTC).  Although, any update should be safe, the differences should be
>> minor.  922507 is the most recent.
>
> OK, I'll do that for 3.3.1.

I think I have this working btw.

> For long term use, though, we'll need some way to cut a rules tarball
> using what's in SVN right now, rather than what was there on the previous
> night.   in my opinion it's unsafe to risk differences between what's
> live in svn and what we're releasing, particularly if changes went in
> during that window.
>
> We can, of course, use the scores that were generated then (but with
> changes since then included too).

simple way to do this -- can we just SVN merge from trunk/rules back
to 3.3/rules?
that'd be quite elegant and easy to do.

--j.

Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
On Wed, Mar 17, 2010 at 00:14, Daryl C. W. O'Shea
<sp...@dostech.ca> wrote:
> First, I still agree that we need a way to generate a rule update using
> the latest svn versions of rules for *emergency updates*.
>
> On 16/03/2010 8:52 AM, Justin Mason wrote:
>> On Tue, Mar 16, 2010 at 00:36, Daryl C. W. O'Shea
>>> Just grab a recent nightly update, rename it, and use that.  The safest
>>> one to use would be a weekly one right after the net enabled mass-check
>>> results (Saturday night's update around 10:30 PM ET -- Sunday, 2:30 AM
>>> UTC).  Although, any update should be safe, the differences should be
>>> minor.  922507 is the most recent.
>>
>> OK, I'll do that for 3.3.1.
>>
>> For long term use, though, we'll need some way to cut a rules tarball
>> using what's in SVN right now, rather than what was there on the previous
>> night.   in my opinion it's unsafe to risk differences between what's
>> live in svn and what we're releasing, particularly if changes went in
>> during that window.
>
> Sorry, I don't follow.  If the rule tarball lints on the proposed code
> release version, what sort of risks might we encounter that we wouldn't
> also encounter doing our day-to-day rule updates after the code release
> happens.

Here's the scenario.  Imagine that we have a tree that's nearing release,
but which has a broken rule (cough URIBL_DBL cough ;).
we've manually concocted scores for it, so the GA output
is not relevant for that rule.  We fix it, and within hours want to
generate a new release tarball.

If we don't have a way to push current SVN as the rules package,
then we have to wait until the next day (possibly longer?)
for the rescored sa-update package which includes the fixed
rule, AFAICT.

does that make sense?

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
First, I still agree that we need a way to generate a rule update using
the latest svn versions of rules for *emergency updates*.

On 16/03/2010 8:52 AM, Justin Mason wrote:
> On Tue, Mar 16, 2010 at 00:36, Daryl C. W. O'Shea
>> Just grab a recent nightly update, rename it, and use that.  The safest
>> one to use would be a weekly one right after the net enabled mass-check
>> results (Saturday night's update around 10:30 PM ET -- Sunday, 2:30 AM
>> UTC).  Although, any update should be safe, the differences should be
>> minor.  922507 is the most recent.
> 
> OK, I'll do that for 3.3.1.
> 
> For long term use, though, we'll need some way to cut a rules tarball
> using what's in SVN right now, rather than what was there on the previous
> night.   in my opinion it's unsafe to risk differences between what's
> live in svn and what we're releasing, particularly if changes went in
> during that window.

Sorry, I don't follow.  If the rule tarball lints on the proposed code
release version, what sort of risks might we encounter that we wouldn't
also encounter doing our day-to-day rule updates after the code release
happens.

I'm concerned about using generated scores from the day before on
current rules when there are changes to rules since, well, the rules
could hit different things after the change.  If the active.list changes
in this time period we'll also have the issue of extra or missing rule
scores.

> We can, of course, use the scores that were generated then (but with
> changes since then included too).

I think the safest, most paranoid, way is to schedule the code tarball
cutting around our rule update schedule.  Cut the tarball using the
nightly mass-check code revision, then wait for the update tarball to be
generated.  This probably isn't necessary though, as long as the latest
update tarball lints OK on a (any) proposed release.  If there are other
necessary sanity checks we should be doing them for all rule updates,
not just for the release tarballs.

Daryl


Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
On Tue, Mar 16, 2010 at 00:36, Daryl C. W. O'Shea
<sp...@dostech.ca> wrote:
> If we're publishing rule updates for 3.3 from trunk I don't see why we'd
> generate a rule tarball from the branch (with sandbox rules, sans
> scores, anyway).  If you install 3.3 using sa-update to get the rules
> you're getting the trunk version of the rules; the tarball should give
> you the same sort of thing.
>
>> if so, what script should we use to do so?
>
> Just grab a recent nightly update, rename it, and use that.  The safest
> one to use would be a weekly one right after the net enabled mass-check
> results (Saturday night's update around 10:30 PM ET -- Sunday, 2:30 AM
> UTC).  Although, any update should be safe, the differences should be
> minor.  922507 is the most recent.

OK, I'll do that for 3.3.1.

For long term use, though, we'll need some way to cut a rules tarball
using what's in SVN right now, rather than what was there on the previous
night.   in my opinion it's unsafe to risk differences between what's
live in svn and what we're releasing, particularly if changes went in
during that window.

We can, of course, use the scores that were generated then (but with
changes since then included too).

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 15/03/2010 6:59 PM, Justin Mason wrote:
> 2010/3/15 John Hardin <jh...@impsec.org>:
>> On Mon, 15 Mar 2010, Karsten Bräckelmann wrote:
>>
>>> The following 30 rules appear to have NOT assigned a score in the
>>> tarball. :(
>>>
>>>  DEAR_BENEFICIARY
>>>  DEAR_EMAIL
>>>  FROM_MISSP_DYNIP
>>>  FROM_MISSP_MSFT
>>>  HDRS_MISSP
>>>  IMG_DIRECT_TO_MX
>>>  LOTTO_AGENT
>>>  MANY_GOOG_PROXY
>>>  MANY_SPAN_IN_TEXT
>>>  MANY_TINY_FLOAT
>>>  MONEY_FROM_MISSP
>>>  TO_NO_BRKTS_DYNIP
>>
>> I'd expect those sandbox rules to have their scores assigned by the nightly
>> masscheck evaluation process. Daryl?

Yes, those "sandbox rules" would have had their scores generated and put
in 72_scores.cf.  My current update script should be generating updates
without the T_ rules, too.

> as I said -- the rules tarball is being built from the 3.3 branch,
> whereas the nightly evaluation process is running off trunk.  that's
> why they're not matching.
> 
> so the question is: should we build the rules tarball from trunk as
> well?

If we're publishing rule updates for 3.3 from trunk I don't see why we'd
generate a rule tarball from the branch (with sandbox rules, sans
scores, anyway).  If you install 3.3 using sa-update to get the rules
you're getting the trunk version of the rules; the tarball should give
you the same sort of thing.

> if so, what script should we use to do so?

Just grab a recent nightly update, rename it, and use that.  The safest
one to use would be a weekly one right after the net enabled mass-check
results (Saturday night's update around 10:30 PM ET -- Sunday, 2:30 AM
UTC).  Although, any update should be safe, the differences should be
minor.  922507 is the most recent.

NEXT PROMISE: Documentation for this stuff.

Daryl


Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
On Tue, Mar 16, 2010 at 00:56, Daryl C. W. O'Shea
<sp...@dostech.ca> wrote:
> On 15/03/2010 7:13 PM, Karsten Bräckelmann wrote:
>> On Mon, 2010-03-15 at 22:59 +0000, Justin Mason wrote:
>>> 2010/3/15 John Hardin <jh...@impsec.org>:
>>>> On Mon, 15 Mar 2010, Karsten Bräckelmann wrote:
>>>>
>>>>> The following 30 rules appear to have NOT assigned a score in the
>>>>> tarball. :(
>>
>>>> I'd expect those sandbox rules to have their scores assigned by the nightly
>>>> masscheck evaluation process. Daryl?
>>>
>>> as I said -- the rules tarball is being built from the 3.3 branch,
>>> whereas the nightly evaluation process is running off trunk.  that's
>>> why they're not matching.
>>
>> I might be confused, but why would that result in rules without scores?
>> Unless rules are removed from trunk, shouldn't it be the other way
>> round?
>
> The only script that pays any attention to the scores in 72_active.cf is
> the script I've got running to publish the stable branch updates (which
> also generates those scores).  The scores are based on a mass-check for
> a specific svn revision.  I can't say that they'll be any good for
> future, or past, revisions... who knows how the rules will change
> between mass-checks (and score gen runs).
>
> I suppose another script could try using those scores for another
> revision of the rules, but user beware, changes in rules could make the
> scores quite invalid.
>
>>> so the question is: should we build the rules tarball from trunk as
>>> well?  if so, what script should we use to do so?
>>
>> Just a gut feeling, but shouldn't both be built from the branch?
>
> Currently, for the update rule tarballs, I use the rules/ and rulesrc/
> from trunk.  It's the only version of the rules that we have current
> mass-check statistics for since we don't do stable version mass-checks.
>
> Using the branch version of the rules/ directory would be OK, I guess.
> I think that the only advantage might be more thorough review of changes
> to rules in rules/.  However, since we've apparently opened up those
> rules to CTR, it probably doesn't make a difference there either.  The
> downside would be the stagnation that the stable branch always falls
> into.  At some point nobody will bother to backport the rules and
> they'll never get updated.

Or worse, we'll repeat the FH_DATE_IN_FUTURE fiasco again. :(

>> Is the update tarball (like the nightly evaluation) built from trunk? In
>> that case, the dist tarball probably should, too. It would be what the
>> users get after an sa-update anyway...
>
> Yep.
>
>> But if we distribute off trunk in sa-update, why the distinction and
>> need to backport sandbox rules in the first place?
>
> We used to backport the *sandbox* rules because all the rule updates we
> manual and trunk and branches were kept completely seperate.  That's no
> longer the case.  Now that we're using trunk rules for stable branches
> I'm not sure that we even need to backport any rules now.

true.

--j.

Re: proposed 3.3.1 tarballs

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 15/03/2010 7:13 PM, Karsten Bräckelmann wrote:
> On Mon, 2010-03-15 at 22:59 +0000, Justin Mason wrote:
>> 2010/3/15 John Hardin <jh...@impsec.org>:
>>> On Mon, 15 Mar 2010, Karsten Bräckelmann wrote:
>>>
>>>> The following 30 rules appear to have NOT assigned a score in the
>>>> tarball. :(
> 
>>> I'd expect those sandbox rules to have their scores assigned by the nightly
>>> masscheck evaluation process. Daryl?
>>
>> as I said -- the rules tarball is being built from the 3.3 branch,
>> whereas the nightly evaluation process is running off trunk.  that's
>> why they're not matching.
> 
> I might be confused, but why would that result in rules without scores?
> Unless rules are removed from trunk, shouldn't it be the other way
> round?

The only script that pays any attention to the scores in 72_active.cf is
the script I've got running to publish the stable branch updates (which
also generates those scores).  The scores are based on a mass-check for
a specific svn revision.  I can't say that they'll be any good for
future, or past, revisions... who knows how the rules will change
between mass-checks (and score gen runs).

I suppose another script could try using those scores for another
revision of the rules, but user beware, changes in rules could make the
scores quite invalid.

>> so the question is: should we build the rules tarball from trunk as
>> well?  if so, what script should we use to do so?
> 
> Just a gut feeling, but shouldn't both be built from the branch?

Currently, for the update rule tarballs, I use the rules/ and rulesrc/
from trunk.  It's the only version of the rules that we have current
mass-check statistics for since we don't do stable version mass-checks.

Using the branch version of the rules/ directory would be OK, I guess.
I think that the only advantage might be more thorough review of changes
to rules in rules/.  However, since we've apparently opened up those
rules to CTR, it probably doesn't make a difference there either.  The
downside would be the stagnation that the stable branch always falls
into.  At some point nobody will bother to backport the rules and
they'll never get updated.

> Is the update tarball (like the nightly evaluation) built from trunk? In
> that case, the dist tarball probably should, too. It would be what the
> users get after an sa-update anyway...

Yep.

> But if we distribute off trunk in sa-update, why the distinction and
> need to backport sandbox rules in the first place?

We used to backport the *sandbox* rules because all the rule updates we
manual and trunk and branches were kept completely seperate.  That's no
longer the case.  Now that we're using trunk rules for stable branches
I'm not sure that we even need to backport any rules now.

> They definitely, absolutely need to match. Score generation with an
> alternated rule-set will skew results, if trunk-only rules are missing.

To be honest, and without looking at stats, I don't think that the
"skew" would be too drastic (towards the harmful side, anyway).  Without
scores the rules would get default scores of 1.0.  Most of the rules
have scores greater that 1.0, so without the scores those rules would be
safe.  A smaller set of new sandbox rules have scores less than 1.0...
those might drive the ham FP-as-spam rate up a little bit.  If anyone
cares enough I'm sure you could come up with numbers, however, since we
do have scores we can use I won't bother myself.

One things for sure, though, is that with the generated scores the new
rules will catch more spam since a bulk of them have higher than default
1.0 scores.

Daryl


Re: proposed 3.3.1 tarballs

Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-15 at 22:59 +0000, Justin Mason wrote:
> 2010/3/15 John Hardin <jh...@impsec.org>:
> > On Mon, 15 Mar 2010, Karsten Bräckelmann wrote:
> >
> >> The following 30 rules appear to have NOT assigned a score in the
> >> tarball. :(

> > I'd expect those sandbox rules to have their scores assigned by the nightly
> > masscheck evaluation process. Daryl?
> 
> as I said -- the rules tarball is being built from the 3.3 branch,
> whereas the nightly evaluation process is running off trunk.  that's
> why they're not matching.

I might be confused, but why would that result in rules without scores?
Unless rules are removed from trunk, shouldn't it be the other way
round?

> so the question is: should we build the rules tarball from trunk as
> well?  if so, what script should we use to do so?

Just a gut feeling, but shouldn't both be built from the branch?

Is the update tarball (like the nightly evaluation) built from trunk? In
that case, the dist tarball probably should, too. It would be what the
users get after an sa-update anyway...

But if we distribute off trunk in sa-update, why the distinction and
need to backport sandbox rules in the first place?


They definitely, absolutely need to match. Score generation with an
alternated rule-set will skew results, if trunk-only rules are missing.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
2010/3/15 John Hardin <jh...@impsec.org>:
> On Mon, 15 Mar 2010, Karsten Bräckelmann wrote:
>
>> The following 30 rules appear to have NOT assigned a score in the
>> tarball. :(
>>
>>  DEAR_BENEFICIARY
>>  DEAR_EMAIL
>>  FROM_MISSP_DYNIP
>>  FROM_MISSP_MSFT
>>  HDRS_MISSP
>>  IMG_DIRECT_TO_MX
>>  LOTTO_AGENT
>>  MANY_GOOG_PROXY
>>  MANY_SPAN_IN_TEXT
>>  MANY_TINY_FLOAT
>>  MONEY_FROM_MISSP
>>  TO_NO_BRKTS_DYNIP
>
> I'd expect those sandbox rules to have their scores assigned by the nightly
> masscheck evaluation process. Daryl?

as I said -- the rules tarball is being built from the 3.3 branch,
whereas the nightly evaluation process is running off trunk.  that's
why they're not matching.

so the question is: should we build the rules tarball from trunk as
well?  if so, what script should we use to do so?

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by John Hardin <jh...@impsec.org>.
On Mon, 15 Mar 2010, Karsten Br�ckelmann wrote:

> The following 30 rules appear to have NOT assigned a score in the
> tarball. :(
>
>  DEAR_BENEFICIARY
>  DEAR_EMAIL
>  FROM_MISSP_DYNIP
>  FROM_MISSP_MSFT
>  HDRS_MISSP
>  IMG_DIRECT_TO_MX
>  LOTTO_AGENT
>  MANY_GOOG_PROXY
>  MANY_SPAN_IN_TEXT
>  MANY_TINY_FLOAT
>  MONEY_FROM_MISSP
>  TO_NO_BRKTS_DYNIP

I'd expect those sandbox rules to have their scores assigned by the 
nightly masscheck evaluation process. Daryl?

-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   We are hell-bent and determined to allocate the talent, the
   resources, the money, the innovation to absolutely become a
   powerhouse in the ad business.       -- Microsoft CEO Steve Ballmer
   ...because allocating talent to securing Windows isn't profitable?
-----------------------------------------------------------------------
  157 days since President Obama won the Nobel "Not George W. Bush" prize

Re: proposed 3.3.1 tarballs

Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-15 at 14:41 +0000, Justin Mason wrote:
> http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
> http://people.apache.org/~jm/devel/
>   e004b3172650f003c2e9f15e5b2f6ac4  Mail-SpamAssassin-rules-3.3.1.r923257.tgz

The following 30 rules appear to have NOT assigned a score in the
tarball. :(

Two of them are T_ testing rules. Shouldn't those be excluded anyway?

  AWL
  AXB_HELO_HOME_UN
  DEAR_BENEFICIARY
  DEAR_EMAIL
  DOS_HIGHBIT_HDRS_BODY
  FORGED_RELAY_MUA_TO_MX
  FORGED_TBIRD_IMG_ARROW
  FORGED_TBIRD_IMG_TO_MX
  FROM_MISSP_DYNIP
  FROM_MISSP_MSFT
  FRT_PHARMAC
  FSL_LSPACES_ABUSE
  HDRS_MISSP
  HELO_NO_DOMAIN
  HK_NAME_FREE
  HK_SCAM_N2
  IMG_DIRECT_TO_MX
  LOTTO_AGENT
  MANY_GOOG_PROXY
  MANY_SPAN_IN_TEXT
  MANY_TINY_FLOAT
  MONEY_FROM_MISSP
  S25R_4
  S25R_6
  TAB_IN_FROM
  TO_NO_BRKTS_DYNIP
  TVD_PCT_OFF
  TWO_IPS_RCVD
  T_DOS_OUTLOOK_TO_MX_IMAGE
  T_FROM_MISSPACED


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


Re: proposed 3.3.1 tarballs

Posted by Mark Martinec <Ma...@ijs.si>.
Bugzilla admin: added target 3.3.2 and version 3.3.1

  Mark

Re: proposed 3.3.1 tarballs

Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-15 at 18:11 +0000, Justin Mason wrote:
> 2010/3/15 Karsten Bräckelmann <gu...@rudersport.de>:
> > On Mon, 2010-03-15 at 18:21 +0100, Mark Martinec wrote:
> >> --- 3.003001.TAR/updates_spamassassin_org.cf
> >> +++ 3.003001.NET/updates_spamassassin_org.cf

> >> @@ -50 +50,4 @@
> >>  include updates_spamassassin_org/72_active.cf
> >> +include updates_spamassassin_org/72_scores.cf
> >                                    ^^^^^^^^^^^^
> >> +include updates_spamassassin_org/local.cf
> >> +include updates_spamassassin_org/regression_tests.cf
> >
> > Not in the tarball!?

I'm much more worried about the missing 72_scores.cf.

> > A quick glimpse seems mostly harmless otherwise. Just a few rules that
> > have been removed recently, like RATWARE_GECKO_BUILD, which are still in
> > the tar but won't be in the sa-update rule-set. Wonder why...
> 
> they haven't been backported to the 3.3 branch.  hmm.  do we need to
> redo our release procedures to take this into account?

But sa-update would get rid of them? Yes, that's an inconsistency.
Though probably not a blocker.

-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
2010/3/15 Karsten Bräckelmann <gu...@rudersport.de>:
> On Mon, 2010-03-15 at 18:21 +0100, Mark Martinec wrote:
>> --- 3.003001.TAR/updates_spamassassin_org.cf    2010-03-15
>> 17:55:03.000000000 +0100
>> +++ 3.003001.NET/updates_spamassassin_org.cf    2010-03-15
>> 17:53:33.000000000 +0100
>> @@ -1,2 +1,2 @@
>> -# UPDATE version 923257
>> +# UPDATE version 922507
>>  include updates_spamassassin_org/10_default_prefs.cf
>> @@ -50 +50,4 @@
>>  include updates_spamassassin_org/72_active.cf
>> +include updates_spamassassin_org/72_scores.cf
>                                    ^^^^^^^^^^^^
>> +include updates_spamassassin_org/local.cf
>> +include updates_spamassassin_org/regression_tests.cf
>
> Not in the tarball!?
>
> A quick glimpse seems mostly harmless otherwise. Just a few rules that
> have been removed recently, like RATWARE_GECKO_BUILD, which are still in
> the tar but won't be in the sa-update rule-set. Wonder why...

they haven't been backported to the 3.3 branch.  hmm.  do we need to
redo our release procedures to take this into account?

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 15/03/2010 3:36 PM, Karsten Bräckelmann wrote:
> On Mon, 2010-03-15 at 18:47 +0100, Karsten Bräckelmann wrote:
>>> --- 3.003001.TAR/updates_spamassassin_org.cf    2010-03-15 17:55:03.000000000 +0100
>>> +++ 3.003001.NET/updates_spamassassin_org.cf    2010-03-15 17:53:33.000000000 +0100
> 
>>> @@ -50 +50,4 @@
>>>  include updates_spamassassin_org/72_active.cf
>>> +include updates_spamassassin_org/72_scores.cf
>>                                     ^^^^^^^^^^^^
>> Not in the tarball!?
> 
> Wait. That cf file with the includes is for the update channel only. It
> isn't even in the rules tarball -- despite the above diff looking as if
> it where.

Yes, 72_scores.cf is in the nightly sa-update rule update tarball that
Mark diffed.  The "Only in" line for 72_scores.cf was just slightly
above the line you quoted.

updates_spamassassin_org.cf is generated by sa-update (listing all the
cf files it found in the update) on the local machine running sa-update,
it's not included in the update itself.

> This appears to be a red herring. 50_scores.cf is in the tarball.

50_scores.cf != 72_scores.cf.  50_scores.cf is from rules/50_scores.cf.
 72_scores.cf is are the scores generated nightly for the rules in
72_active.cf (those rules are the ones from the sandboxes listed in the
current active.list file).

Confuseded, yet? :)

Daryl


Re: proposed 3.3.1 tarballs

Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-15 at 18:47 +0100, Karsten Bräckelmann wrote:
> > --- 3.003001.TAR/updates_spamassassin_org.cf    2010-03-15 17:55:03.000000000 +0100
> > +++ 3.003001.NET/updates_spamassassin_org.cf    2010-03-15 17:53:33.000000000 +0100

> > @@ -50 +50,4 @@
> >  include updates_spamassassin_org/72_active.cf
> > +include updates_spamassassin_org/72_scores.cf
>                                     ^^^^^^^^^^^^
> Not in the tarball!?

Wait. That cf file with the includes is for the update channel only. It
isn't even in the rules tarball -- despite the above diff looking as if
it where.

This appears to be a red herring. 50_scores.cf is in the tarball.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


Re: proposed 3.3.1 tarballs

Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-15 at 18:21 +0100, Mark Martinec wrote:
> --- 3.003001.TAR/updates_spamassassin_org.cf    2010-03-15
> 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org.cf    2010-03-15
> 17:53:33.000000000 +0100
> @@ -1,2 +1,2 @@
> -# UPDATE version 923257
> +# UPDATE version 922507
>  include updates_spamassassin_org/10_default_prefs.cf
> @@ -50 +50,4 @@
>  include updates_spamassassin_org/72_active.cf
> +include updates_spamassassin_org/72_scores.cf
                                    ^^^^^^^^^^^^
> +include updates_spamassassin_org/local.cf
> +include updates_spamassassin_org/regression_tests.cf

Not in the tarball!?

A quick glimpse seems mostly harmless otherwise. Just a few rules that
have been removed recently, like RATWARE_GECKO_BUILD, which are still in
the tar but won't be in the sa-update rule-set. Wonder why...


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


Re: proposed 3.3.1 tarballs

Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
+1 here including with/without trivialities noted by Mark.

On 3/15/2010 1:21 PM, Mark Martinec wrote:
> On 2010-03-15 15:41, Justin Mason wrote:
>    
>> http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
>> http://people.apache.org/~jm/devel/
>>
>> please vote!
>>      
> +1
> Looks good enough to me.
>
>
> A diff between the proposed 3.3.1 and trunk reveals a couple of
> trivialities which should go into 3.3.2 (or into 3.3.1 if there will be
> a re-cut).
>
> Further down there is a diff between 3.3.1 rules as installed by sa-update
> from a tarball, vs. the ones installed by sa-update from net. Seems like
> some more recent edits did not make it into a tarball, please review if
> there is any need for concern.
>
>    Mark
>
>
> --- Mail-SpamAssassin-3.3.1/CREDITS	2010-03-15 15:29:46.000000000 +0100
> +++ spamassassin-trunk/CREDITS	2010-03-11 16:04:32.000000000 +0100
> @@ -29,2 +29,3 @@
>      - John Hardin
> +   - Adam Katz
>      - Warren Togami        http://www.amazon.com/wishlist/1RJO0CGTPEDQT
> --- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/BayesStore/PgSQL.pm	2010-03-15 15:29:42.000000000 +0100
> +++ spamassassin-trunk/lib/Mail/SpamAssassin/BayesStore/PgSQL.pm	2010-02-05 20:19:38.000000000 +0100
> @@ -25,3 +25,3 @@
>
> -This module implementes a PostgresSQL specific bayesian storage module.
> +This module implements a PostgreSQL specific bayesian storage module.
>
> --- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/Client.pm	2010-03-15 15:29:43.000000000 +0100
> +++ spamassassin-trunk/lib/Mail/SpamAssassin/Client.pm	2010-01-27 01:49:50.000000000 +0100
> @@ -474,3 +474,3 @@
>
> -We have this as a seperate method in case we ever decide to get fancy
> +We have this as a separate method in case we ever decide to get fancy
>   with the response line.
> Mail/SpamAssassin/PerMsgLearner.pm
> --- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/PerMsgLearner.pm	2010-03-15 15:29:43.000000000 +0100
> +++ spamassassin-trunk/lib/Mail/SpamAssassin/PerMsgLearner.pm	2010-01-27 01:49:50.000000000 +0100
> @@ -165,3 +165,3 @@
>
> -Returns C<1>  if the message was learned from or forgotten succesfully.
> +Returns C<1>  if the message was learned from or forgotten successfully.
>
> b/Mail/SpamAssassin/Plugin/URIDNSBL.pm
> --- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm	2010-03-15 15:29:43.000000000 +0100
> +++ spamassassin-trunk/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm	2010-03-10 02:38:52.000000000 +0100
> @@ -314,2 +314,3 @@
>       my $tflags = $scanner->{conf}->{tflags}->{$rulename};
> +    $tflags = ''  if !defined $tflags;
>
> --- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/Plugin.pm	2010-03-15 15:29:43.000000000 +0100
> +++ spamassassin-trunk/lib/Mail/SpamAssassin/Plugin.pm	2010-02-15 18:50:35.000000000 +0100
> @@ -678,3 +678,3 @@
>   are a single string containing the raw token value.  You can test for
> -backwards compatability by checking to see if the value for a key is a
> +backwards compatibility by checking to see if the value for a key is a
>   reference to a perl HASH, for instance:
> --- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin.pm	2010-03-15 15:29:43.000000000 +0100
> +++ spamassassin-trunk/lib/Mail/SpamAssassin.pm	2010-03-11 16:04:32.000000000 +0100
> @@ -231,3 +231,3 @@
>   If set to 1, DNS tests will not attempt to use IPv6. Use if the existing tests
> -for IPv6 availablity produce incorrect results or crashes.
> +for IPv6 availability produce incorrect results or crashes.
>
> --- Mail-SpamAssassin-3.3.1/lib/spamassassin-run.pod	2010-03-15 15:29:43.000000000 +0100
> +++ spamassassin-trunk/lib/spamassassin-run.pod	2010-02-15 18:50:35.000000000 +0100
> @@ -208,3 +208,3 @@
>   IPv6 is available, using only IPv4 if it is not. Use if the existing tests
> -for IPv6 availablity produce incorrect results or crashes.
> +for IPv6 availability produce incorrect results or crashes.
>
> --- Mail-SpamAssassin-3.3.1/sa-learn	2010-03-15 17:23:25.000000000 +0100
> +++ spamassassin-trunk/sa-learn	2010-03-03 00:15:43.000000000 +0100
> @@ -613,3 +613,3 @@
>    --use-ignores         Use bayes_ignore_from and bayes_ignore_to
> - --sync                Syncronize the database and the journal if needed
> + --sync                Synchronize the database and the journal if needed
>    --force-expire        Force a database sync and expiry run
> @@ -729,3 +729,3 @@
>
> -Syncronize the journal and databases.  Upon successfully syncing the
> +Synchronize the journal and databases.  Upon successfully syncing the
>   database with the entries in the journal, the journal file is removed.
> @@ -1237,3 +1237,3 @@
>   =item - if no expire has been done before, or the last expire looks
> -"wierd", do an estimation pass.  The definition of "wierd" is:
> +"weird", do an estimation pass.  The definition of "weird" is:
>
> --- Mail-SpamAssassin-3.3.1/sa-learn.raw	2010-03-15 15:29:46.000000000 +0100
> +++ spamassassin-trunk/sa-learn.raw	2010-01-27 01:49:50.000000000 +0100
> @@ -614,3 +614,3 @@
>    --use-ignores         Use bayes_ignore_from and bayes_ignore_to
> - --sync                Syncronize the database and the journal if needed
> + --sync                Synchronize the database and the journal if needed
>    --force-expire        Force a database sync and expiry run
> @@ -730,3 +730,3 @@
>
> -Syncronize the journal and databases.  Upon successfully syncing the
> +Synchronize the journal and databases.  Upon successfully syncing the
>   database with the entries in the journal, the journal file is removed.
> @@ -1238,3 +1238,3 @@
>   =item - if no expire has been done before, or the last expire looks
> -"wierd", do an estimation pass.  The definition of "wierd" is:
> +"weird", do an estimation pass.  The definition of "weird" is:
>
> --- Mail-SpamAssassin-3.3.1/t/uribl_domains_only.t	2010-03-15 15:29:45.000000000 +0100
> +++ spamassassin-trunk/t/uribl_domains_only.t	2010-03-03 19:10:36.000000000 +0100
> @@ -11,3 +11,3 @@
>   BEGIN {
> -  plan tests =>  (DO_RUN ? 2 : 0),
> +  plan tests =>  (DO_RUN ? 4 : 0),
>   };
> @@ -18,5 +18,3 @@
>
> -%anti_patterns = (
> - q{ X_URIBL_DOMSONLY } =>  'A',
> -);
> +%anti_patterns = ( q{ X_URIBL_DOMSONLY } =>  'A' );
>
> @@ -38 +36,8 @@
>
> +%patterns = ( q{ X_URIBL_DOMSONLY } =>  'A' );
> +%anti_patterns = ();
> +
> +clear_pattern_counters();
> +ok sarun ("-t<  data/spam/dnsbl_ipsonly.eml 2>&1", \&patterns_run_cb);
> +ok_all_patterns();
> +
> --- Mail-SpamAssassin-3.3.1/t/uribl_ips_only.t	2010-03-15 15:29:45.000000000 +0100
> +++ spamassassin-trunk/t/uribl_ips_only.t	2010-03-03 19:10:36.000000000 +0100
> @@ -11,3 +11,3 @@
>   BEGIN {
> -  plan tests =>  (DO_RUN ? 2 : 0),
> +  plan tests =>  (DO_RUN ? 4 : 0),
>   };
> @@ -38 +38,8 @@
>
> +%patterns = ( q{ X_URIBL_IPSONLY } =>  'A' );
> +%anti_patterns = ();
> +
> +clear_pattern_counters();
> +ok sarun ("-t<  data/spam/dnsbl_domsonly.eml 2>&1", \&patterns_run_cb);
> +ok_all_patterns();
> +
>
>
>
>
>
>
>
>
> --- 3.003001.TAR/updates_spamassassin_org/20_freemail_domains.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/20_freemail_domains.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -25,2 +25,3 @@
>
> +#Last Update AXB 3/4/2010
>   ifplugin Mail::SpamAssassin::Plugin::FreeMail
> @@ -112,3 +113,3 @@
>   freemail_domains earthling.net eastmail.com eastrolog.com easy-pages.com easy.com
> -freemail_domains easyinfomail.co.za easypeasy.com echina.com ecn.org ecplaza.net
> +freemail_domains easyinfomail.co.za easypeasy.com echina.com ecn.org ecplaza.net eircom.net
>   freemail_domains edsamail.com.ph educacao.te.pt edumail.co.za eeism.com ego.co.th
> @@ -163,3 +164,3 @@
>   freemail_domains happycounsel.com hawaii.com hawaii.usa.com hayahaya.tg hedgeai.com
> -freemail_domains heesun.net heremail.com highveldmail.co.za hilarious.com hildebrands.de
> +freemail_domains heesun.net heremail.com hetnet.nl highveldmail.co.za hilarious.com hildebrands.de
>   freemail_domains hingis.org hispavista.com hitmanrecords.com hockeyghiaccio.com
> @@ -167,3 +168,3 @@
>   freemail_domains homemail.co.za homenetmail.com homestead.com homosexual.net hongkong.com
> -freemail_domains hopthu.com hot-shot.com hot.ee hotbot.com hotcoolmail.com hotdak.com
> +freemail_domains hopthu.com hot-shot.com hot.ee hotbot.com hotbox.ru hotcoolmail.com hotdak.com
>   freemail_domains hotfire.net hotinbox.com hotmail.* hotmail.co*.*
> @@ -185,3 +186,3 @@
>   freemail_domains internetmailing.net inwind.it iobox.com iobox.fi iol.it iol.pt iowa.usa.com
> -freemail_domains ip3.com ipermitmail.com iqemail.com iran.com irangate.net irelandmail.com
> +freemail_domains ip3.com ipermitmail.com iqemail.com iquebec.com iran.com irangate.net irelandmail.com
>   freemail_domains iscool.net islandmama.com ismart.net isonews2.com isonfire.com isp9.net
> @@ -197,3 +198,3 @@
>   freemail_domains kuronowish.com kyokodate.com kyokofukada.net ladymail.cz lagoon.nc
> -freemail_domains lahaonline.com lamalla.net lancsmail.com laposte.net latinmail.com
> +freemail_domains lahaonline.com lamalla.net lancsmail.com land.ru laposte.net latinmail.com
>   freemail_domains lawyer.com lawyersmail.com lawyerzone.com lebanonatlas.com leehom.net
> @@ -202,3 +203,3 @@
>   freemail_domains libertysurf.net libre.net lightwines.org linkmaster.com linuxfreemail.com
> -freemail_domains linuxmail.org lionsfan.com.au live.com livedoor.com llandudno.com
> +freemail_domains linuxmail.org lionsfan.com.au live.* livedoor.com llandudno.com
>   freemail_domains llangollen.com lmxmail.sk lobbyist.com loggain.net loggain.nu lolnetwork.net
> @@ -214,3 +215,3 @@
>   freemail_domains mail.goo.ne.jp mail.gr mail.lawguru.com mail.md mail.mn mail.org mail.pf
> -freemail_domains mail.pt mail.ru mail.yahoo.co.jp mail2*.com mail3000.com
> +freemail_domains mail.pt mail.ru mail.yahoo.co.jp mail15.com mail2*.com mail3000.com mail333.com
>   freemail_domains mail8.com mailandftp.com mailandnews.com mailas.com mailasia.com mailbg.com
> @@ -238,3 +239,3 @@
>   freemail_domains ms*.hinet.net mscold.com msn.com mundo-r.com munich.com musician.org muslim.com
> -freemail_domains muslimsonline.com mustangs.com mxs.de mycabin.com mycity.com mycommail.com
> +freemail_domains muslimsonline.com mustangs.com mxs.de myblue.cc mycabin.com mycity.com mycommail.com
>   freemail_domains mycool.com mydomain.com myeweb.com myfastmail.com myfunnymail.com
> @@ -246,3 +247,3 @@
>   freemail_domains nanaseaikawa.com nandomail.com naseej.com nastything.com national-champs.com
> -freemail_domains nativeweb.net naveganas.com nebraska.usa.com nemra1.com nenter.com
> +freemail_domains nativeweb.net narod.ru naveganas.com nebraska.usa.com nemra1.com nenter.com
>   freemail_domains nerdshack.com nervhq.org net.hr net4b.pt net4jesus.com net4you.at
> @@ -255,3 +256,3 @@
>   freemail_domains newyork.usa.com newyorkcity.com nfmail.com nicegal.com nightimeuk.com
> -freemail_domains nightly.com nightmail.com noavar.com noemail.com nonomail.com
> +freemail_domains nightly.com nightmail.com nightmail.ru noavar.com noemail.com nonomail.com
>   freemail_domains nonpartisan.com noolhar.com northcarolina.usa.com northdakota.usa.com
> @@ -263,3 +264,3 @@
>   freemail_domains online.ru onlinewiz.com onobox.com openbg.com openforyou.com
> -freemail_domains opentransfer.com operamail.com oplusnet.com optician.com orangehome.co.uk
> +freemail_domains opentransfer.com operamail.com oplusnet.com optician.com orange.* orangehome.co.uk
>   freemail_domains orbitel.bg orcon.net.nz oregon.usa.com oreka.com organizer.net orgio.net
> @@ -275,5 +276,5 @@
>   freemail_domains planetout.com plasa.com playersodds.com playful.com pluno.com
> -freemail_domains plusmail.com.br pmail.net pnetmail.co.za pobox.ru pobox.sk pochta.ru
> +freemail_domains plusmail.com.br pmail.net pnetmail.co.za pobox.ru pobox.sk pochtamt.ru pochta.ru
>   freemail_domains poczta.fm poetic.com pogowave.com polandmail.com polbox.com politician.com
> -freemail_domains pop.co.th popmail.com poppymail.com popsmail.com popstar.com portafree.com
> +freemail_domains pop3.ru pop.co.th popmail.com poppymail.com popsmail.com popstar.com portafree.com
>   freemail_domains portaldosalunos.com portugalmail.com portugalmail.pt post.com post.cz
> @@ -286,3 +287,3 @@
>   freemail_domains qatar.io qlmail.com qrio.com qsl.net qudsmail.com queerplaces.com quepasa.com
> -freemail_domains quick.cz quickwebmail.com r-o-o-t.com r320.hu raakim.com racingseat.com
> +freemail_domains quick.cz quickwebmail.com r-o-o-t.com r320.hu raakim.com rbcmail.ru racingseat.com
>   freemail_domains radicalz.com radiojobbank.com radiologist.net ragingbull.com
> @@ -310,3 +311,3 @@
>   freemail_domains sistersbrothers.com sizzling.com skim.com slamdunkfan.com slickriffs.co.uk
> -freemail_domains slingshot.com slo.net slomusic.net smartemail.co.uk snail-mail.net
> +freemail_domains slingshot.com slo.net slomusic.net smartemail.co.uk smtp.ru snail-mail.net
>   freemail_domains snakebite.com sndt.net sneakemail.com snoopymail.com snowboarding.com
> @@ -329,3 +330,3 @@
>   freemail_domains taxcutadvice.com teachers.org techemail.com techie.com technisamail.co.za
> -freemail_domains technologist.com teenmail.co.uk teenmail.co.za tejary.com telebot.com
> +freemail_domains technologist.com teenmail.co.uk teenmail.co.za tejary.com telebot.com telefonica.net
>   freemail_domains telegraf.by teleline.es telinco.net telkom.net telpage.net telstra.com
> @@ -373,3 +374,3 @@
>   freemail_domains yaho.com yahoo.* yahoo.co*.* yalla.com.lb
> -freemail_domains yam.com yamal.info yapost.com yawmail.com yebox.com yehey.com
> +freemail_domains yam.com yamal.info yandex.ru yapost.com yawmail.com yebox.com yehey.com
>   freemail_domains yellow-jackets.com yellowstone.net yenimail.com yepmail.net yifan.net
> --- 3.003001.TAR/updates_spamassassin_org/20_head_tests.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/20_head_tests.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -508,3 +508,4 @@
>
> -header DATE_IN_FUTURE_96_XX	eval:check_for_shifted_date('96', 'undef')
> +#header DATE_IN_FUTURE_96_XX	eval:check_for_shifted_date('96', 'undef')
> +meta DATE_IN_FUTURE_96_XX 	(0)
>   describe DATE_IN_FUTURE_96_XX	Date: is 96 hours or more after Received: date
> --- 3.003001.TAR/updates_spamassassin_org/20_ratware.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/20_ratware.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -208,9 +208,2 @@
>
> -# spammer tool, sometimes has "netIP with HTTP;" in Received: header
> -
> -# this is really badly faked.  Also the spammer who uses "25250101"
> -# for the build is a total hippie.
> -header RATWARE_GECKO_BUILD	User-Agent =~ /Gecko\/(?!200\d\d\d\d\d)\d/
> -describe RATWARE_GECKO_BUILD	Bulk email fingerprint (Gecko faked) found
> -
>   ########################################################################
> --- 3.003001.TAR/updates_spamassassin_org/20_uri_tests.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/20_uri_tests.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -65,3 +65,3 @@
>   # though, which actually doesn't have a weird port in it.
> -uri WEIRD_PORT			m{https?://[^/\s]+?:\d+(?<!:80)(?<!:443)(?<!:8080)(?:/|\s|$)}
> +uri WEIRD_PORT			m{https?://[^/?\s]+?:\d+(?<!:80)(?<!:443)(?<!:8080)(?:/|\s|$)}
>   describe WEIRD_PORT		Uses non-standard port number for HTTP
> --- 3.003001.TAR/updates_spamassassin_org/30_text_de.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/30_text_de.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -314,3 +314,2 @@
>   lang de describe RATWARE_HASH_DASH Enthält Abwehrmaßnahme gegen Anti-Spam-Software ("hashbuster")
> -lang de describe RATWARE_GECKO_BUILD Kopfzeilen enthalten gefälschte Hinweise auf Mozilla/Gecko
>   lang de describe RATWARE_ZERO_TZ Seltsame Zeitzone (+0000)
> --- 3.003001.TAR/updates_spamassassin_org/30_text_nl.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/30_text_nl.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -236,3 +236,2 @@
>   lang nl describe RATWARE_HASH_DASH               Bevat een hashbuster in Send-Safe opmaak
> -lang nl describe RATWARE_GECKO_BUILD             Bulk email signatuur (vervalste Gecko) gevonden
>   lang nl describe NUMERIC_HTTP_ADDR               Gebruikt een numeriek IP adres in een URL
> --- 3.003001.TAR/updates_spamassassin_org/50_scores.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/50_scores.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -458,3 +458,2 @@
>   score RATWARE_EGROUPS 1.898 1.258 1.406 1.621
> -score RATWARE_GECKO_BUILD 0 # n=0 n=1 n=2 n=3
>   score RATWARE_HASH_DASH 0 # n=0 n=1 n=2 n=3
> @@ -799,3 +798,4 @@
>   score DATE_IN_FUTURE_48_96 2.384 0.813 1.078 2.181
> -score DATE_IN_FUTURE_96_XX 2.614 3.028 2.851 3.087
> +#score DATE_IN_FUTURE_96_XX 2.614 3.028 2.851 3.087
> +score DATE_IN_FUTURE_96_XX 0
>   score DATE_IN_PAST_03_06 2.399 1.076 1.200 1.592
> Only in 3.003001.NET/updates_spamassassin_org: 72_scores.cf
> Only in 3.003001.NET/updates_spamassassin_org: MIRRORED.BY
> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set0-72_scores.cf.txt
> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set1-72_scores.cf.txt
> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set2-72_scores.cf.txt
> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set3-72_scores.cf.txt
> Only in 3.003001.NET/updates_spamassassin_org: languages
> Only in 3.003001.NET/updates_spamassassin_org: local.cf
> Only in 3.003001.NET/updates_spamassassin_org: regression_tests.cf
> Only in 3.003001.NET/updates_spamassassin_org: sa-update-pubkey.txt
> Only in 3.003001.NET/updates_spamassassin_org: user_prefs.template
> --- 3.003001.TAR/updates_spamassassin_org.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org.cf	2010-03-15 17:53:33.000000000 +0100
> @@ -1,2 +1,2 @@
> -# UPDATE version 923257
> +# UPDATE version 922507
>   include updates_spamassassin_org/10_default_prefs.cf
> @@ -50 +50,4 @@
>   include updates_spamassassin_org/72_active.cf
> +include updates_spamassassin_org/72_scores.cf
> +include updates_spamassassin_org/local.cf
> +include updates_spamassassin_org/regression_tests.cf
> --- 3.003001.TAR/updates_spamassassin_org/72_active.cf	2010-03-15 17:55:03.000000000 +0100
> +++ 3.003001.NET/updates_spamassassin_org/72_active.cf	2010-03-15 17:53:33.000000000 +0100
> [lots of diff]
>    


Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
On Tue, Mar 16, 2010 at 01:04, Daryl C. W. O'Shea
<sp...@dostech.ca> wrote:
>
>> Only in 3.003001.NET/updates_spamassassin_org: languages
>> Only in 3.003001.NET/updates_spamassassin_org: local.cf
>> Only in 3.003001.NET/updates_spamassassin_org: regression_tests.cf
>
> These, or some of them, probably shouldn't be in the nightly rule update
> tarballs.  Do these get installed as part of the code package?  I know
> 'languages' needs to get on a system... I don't know how any more.  Do
> the latter need to ship at all (I would guess not)?
>
>> Only in 3.003001.NET/updates_spamassassin_org: sa-update-pubkey.txt
>> Only in 3.003001.NET/updates_spamassassin_org: user_prefs.template
>
> Same for these.  Do they need to be in the update tarballs?

I'm not sure.  can you open a ticket so we remember to investigate this?
some of them may be indeed still required....

-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 15/03/2010 1:21 PM, Mark Martinec wrote:
> Only in 3.003001.NET/updates_spamassassin_org: 72_scores.cf

We should definitely include scores.  As I just mentioned in another
message, I think that we should just ship a renamed nightly update as
the rule tarball for the release.

> Only in 3.003001.NET/updates_spamassassin_org: MIRRORED.BY

sa-update will just get this from DNS, so that's OK.

> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set0-72_scores.cf.txt
> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set1-72_scores.cf.txt
> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set2-72_scores.cf.txt
> Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set3-72_scores.cf.txt

OK.  They're just some quasi-stats for the generated rules in 72_scores.cf.

> Only in 3.003001.NET/updates_spamassassin_org: languages
> Only in 3.003001.NET/updates_spamassassin_org: local.cf
> Only in 3.003001.NET/updates_spamassassin_org: regression_tests.cf

These, or some of them, probably shouldn't be in the nightly rule update
tarballs.  Do these get installed as part of the code package?  I know
'languages' needs to get on a system... I don't know how any more.  Do
the latter need to ship at all (I would guess not)?

> Only in 3.003001.NET/updates_spamassassin_org: sa-update-pubkey.txt
> Only in 3.003001.NET/updates_spamassassin_org: user_prefs.template

Same for these.  Do they need to be in the update tarballs?

Daryl


Re: proposed 3.3.1 tarballs

Posted by Mark Martinec <Ma...@ijs.si>.
On 2010-03-15 15:41, Justin Mason wrote:
> http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
> http://people.apache.org/~jm/devel/
>
> please vote!

+1
Looks good enough to me.


A diff between the proposed 3.3.1 and trunk reveals a couple of
trivialities which should go into 3.3.2 (or into 3.3.1 if there will be
a re-cut).

Further down there is a diff between 3.3.1 rules as installed by sa-update
from a tarball, vs. the ones installed by sa-update from net. Seems like
some more recent edits did not make it into a tarball, please review if
there is any need for concern.

  Mark


--- Mail-SpamAssassin-3.3.1/CREDITS	2010-03-15 15:29:46.000000000 +0100
+++ spamassassin-trunk/CREDITS	2010-03-11 16:04:32.000000000 +0100
@@ -29,2 +29,3 @@
    - John Hardin
+   - Adam Katz
    - Warren Togami        http://www.amazon.com/wishlist/1RJO0CGTPEDQT
--- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/BayesStore/PgSQL.pm	2010-03-15 15:29:42.000000000 +0100
+++ spamassassin-trunk/lib/Mail/SpamAssassin/BayesStore/PgSQL.pm	2010-02-05 20:19:38.000000000 +0100
@@ -25,3 +25,3 @@
 
-This module implementes a PostgresSQL specific bayesian storage module.
+This module implements a PostgreSQL specific bayesian storage module.
 
--- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/Client.pm	2010-03-15 15:29:43.000000000 +0100
+++ spamassassin-trunk/lib/Mail/SpamAssassin/Client.pm	2010-01-27 01:49:50.000000000 +0100
@@ -474,3 +474,3 @@
 
-We have this as a seperate method in case we ever decide to get fancy
+We have this as a separate method in case we ever decide to get fancy
 with the response line.
Mail/SpamAssassin/PerMsgLearner.pm
--- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/PerMsgLearner.pm	2010-03-15 15:29:43.000000000 +0100
+++ spamassassin-trunk/lib/Mail/SpamAssassin/PerMsgLearner.pm	2010-01-27 01:49:50.000000000 +0100
@@ -165,3 +165,3 @@
 
-Returns C<1> if the message was learned from or forgotten succesfully.
+Returns C<1> if the message was learned from or forgotten successfully.
 
b/Mail/SpamAssassin/Plugin/URIDNSBL.pm
--- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm	2010-03-15 15:29:43.000000000 +0100
+++ spamassassin-trunk/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm	2010-03-10 02:38:52.000000000 +0100
@@ -314,2 +314,3 @@
     my $tflags = $scanner->{conf}->{tflags}->{$rulename};
+    $tflags = ''  if !defined $tflags;
 
--- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin/Plugin.pm	2010-03-15 15:29:43.000000000 +0100
+++ spamassassin-trunk/lib/Mail/SpamAssassin/Plugin.pm	2010-02-15 18:50:35.000000000 +0100
@@ -678,3 +678,3 @@
 are a single string containing the raw token value.  You can test for
-backwards compatability by checking to see if the value for a key is a
+backwards compatibility by checking to see if the value for a key is a
 reference to a perl HASH, for instance:
--- Mail-SpamAssassin-3.3.1/lib/Mail/SpamAssassin.pm	2010-03-15 15:29:43.000000000 +0100
+++ spamassassin-trunk/lib/Mail/SpamAssassin.pm	2010-03-11 16:04:32.000000000 +0100
@@ -231,3 +231,3 @@
 If set to 1, DNS tests will not attempt to use IPv6. Use if the existing tests
-for IPv6 availablity produce incorrect results or crashes.
+for IPv6 availability produce incorrect results or crashes.
 
--- Mail-SpamAssassin-3.3.1/lib/spamassassin-run.pod	2010-03-15 15:29:43.000000000 +0100
+++ spamassassin-trunk/lib/spamassassin-run.pod	2010-02-15 18:50:35.000000000 +0100
@@ -208,3 +208,3 @@
 IPv6 is available, using only IPv4 if it is not. Use if the existing tests
-for IPv6 availablity produce incorrect results or crashes.
+for IPv6 availability produce incorrect results or crashes.
 
--- Mail-SpamAssassin-3.3.1/sa-learn	2010-03-15 17:23:25.000000000 +0100
+++ spamassassin-trunk/sa-learn	2010-03-03 00:15:43.000000000 +0100
@@ -613,3 +613,3 @@
  --use-ignores         Use bayes_ignore_from and bayes_ignore_to
- --sync                Syncronize the database and the journal if needed
+ --sync                Synchronize the database and the journal if needed
  --force-expire        Force a database sync and expiry run
@@ -729,3 +729,3 @@
 
-Syncronize the journal and databases.  Upon successfully syncing the
+Synchronize the journal and databases.  Upon successfully syncing the
 database with the entries in the journal, the journal file is removed.
@@ -1237,3 +1237,3 @@
 =item - if no expire has been done before, or the last expire looks
-"wierd", do an estimation pass.  The definition of "wierd" is:
+"weird", do an estimation pass.  The definition of "weird" is:
 
--- Mail-SpamAssassin-3.3.1/sa-learn.raw	2010-03-15 15:29:46.000000000 +0100
+++ spamassassin-trunk/sa-learn.raw	2010-01-27 01:49:50.000000000 +0100
@@ -614,3 +614,3 @@
  --use-ignores         Use bayes_ignore_from and bayes_ignore_to
- --sync                Syncronize the database and the journal if needed
+ --sync                Synchronize the database and the journal if needed
  --force-expire        Force a database sync and expiry run
@@ -730,3 +730,3 @@
 
-Syncronize the journal and databases.  Upon successfully syncing the
+Synchronize the journal and databases.  Upon successfully syncing the
 database with the entries in the journal, the journal file is removed.
@@ -1238,3 +1238,3 @@
 =item - if no expire has been done before, or the last expire looks
-"wierd", do an estimation pass.  The definition of "wierd" is:
+"weird", do an estimation pass.  The definition of "weird" is:
 
--- Mail-SpamAssassin-3.3.1/t/uribl_domains_only.t	2010-03-15 15:29:45.000000000 +0100
+++ spamassassin-trunk/t/uribl_domains_only.t	2010-03-03 19:10:36.000000000 +0100
@@ -11,3 +11,3 @@
 BEGIN {
-  plan tests => (DO_RUN ? 2 : 0),
+  plan tests => (DO_RUN ? 4 : 0),
 };
@@ -18,5 +18,3 @@
 
-%anti_patterns = (
- q{ X_URIBL_DOMSONLY } => 'A',
-);
+%anti_patterns = ( q{ X_URIBL_DOMSONLY } => 'A' );
 
@@ -38 +36,8 @@
 
+%patterns = ( q{ X_URIBL_DOMSONLY } => 'A' );
+%anti_patterns = ();
+
+clear_pattern_counters();
+ok sarun ("-t < data/spam/dnsbl_ipsonly.eml 2>&1", \&patterns_run_cb);
+ok_all_patterns();
+
--- Mail-SpamAssassin-3.3.1/t/uribl_ips_only.t	2010-03-15 15:29:45.000000000 +0100
+++ spamassassin-trunk/t/uribl_ips_only.t	2010-03-03 19:10:36.000000000 +0100
@@ -11,3 +11,3 @@
 BEGIN {
-  plan tests => (DO_RUN ? 2 : 0),
+  plan tests => (DO_RUN ? 4 : 0),
 };
@@ -38 +38,8 @@
 
+%patterns = ( q{ X_URIBL_IPSONLY } => 'A' );
+%anti_patterns = ();
+
+clear_pattern_counters();
+ok sarun ("-t < data/spam/dnsbl_domsonly.eml 2>&1", \&patterns_run_cb);
+ok_all_patterns();
+








--- 3.003001.TAR/updates_spamassassin_org/20_freemail_domains.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/20_freemail_domains.cf	2010-03-15 17:53:33.000000000 +0100
@@ -25,2 +25,3 @@
 
+#Last Update AXB 3/4/2010
 ifplugin Mail::SpamAssassin::Plugin::FreeMail
@@ -112,3 +113,3 @@
 freemail_domains earthling.net eastmail.com eastrolog.com easy-pages.com easy.com
-freemail_domains easyinfomail.co.za easypeasy.com echina.com ecn.org ecplaza.net
+freemail_domains easyinfomail.co.za easypeasy.com echina.com ecn.org ecplaza.net eircom.net
 freemail_domains edsamail.com.ph educacao.te.pt edumail.co.za eeism.com ego.co.th
@@ -163,3 +164,3 @@
 freemail_domains happycounsel.com hawaii.com hawaii.usa.com hayahaya.tg hedgeai.com
-freemail_domains heesun.net heremail.com highveldmail.co.za hilarious.com hildebrands.de
+freemail_domains heesun.net heremail.com hetnet.nl highveldmail.co.za hilarious.com hildebrands.de
 freemail_domains hingis.org hispavista.com hitmanrecords.com hockeyghiaccio.com
@@ -167,3 +168,3 @@
 freemail_domains homemail.co.za homenetmail.com homestead.com homosexual.net hongkong.com
-freemail_domains hopthu.com hot-shot.com hot.ee hotbot.com hotcoolmail.com hotdak.com
+freemail_domains hopthu.com hot-shot.com hot.ee hotbot.com hotbox.ru hotcoolmail.com hotdak.com
 freemail_domains hotfire.net hotinbox.com hotmail.* hotmail.co*.*
@@ -185,3 +186,3 @@
 freemail_domains internetmailing.net inwind.it iobox.com iobox.fi iol.it iol.pt iowa.usa.com
-freemail_domains ip3.com ipermitmail.com iqemail.com iran.com irangate.net irelandmail.com
+freemail_domains ip3.com ipermitmail.com iqemail.com iquebec.com iran.com irangate.net irelandmail.com
 freemail_domains iscool.net islandmama.com ismart.net isonews2.com isonfire.com isp9.net
@@ -197,3 +198,3 @@
 freemail_domains kuronowish.com kyokodate.com kyokofukada.net ladymail.cz lagoon.nc
-freemail_domains lahaonline.com lamalla.net lancsmail.com laposte.net latinmail.com
+freemail_domains lahaonline.com lamalla.net lancsmail.com land.ru laposte.net latinmail.com
 freemail_domains lawyer.com lawyersmail.com lawyerzone.com lebanonatlas.com leehom.net
@@ -202,3 +203,3 @@
 freemail_domains libertysurf.net libre.net lightwines.org linkmaster.com linuxfreemail.com
-freemail_domains linuxmail.org lionsfan.com.au live.com livedoor.com llandudno.com
+freemail_domains linuxmail.org lionsfan.com.au live.* livedoor.com llandudno.com
 freemail_domains llangollen.com lmxmail.sk lobbyist.com loggain.net loggain.nu lolnetwork.net
@@ -214,3 +215,3 @@
 freemail_domains mail.goo.ne.jp mail.gr mail.lawguru.com mail.md mail.mn mail.org mail.pf
-freemail_domains mail.pt mail.ru mail.yahoo.co.jp mail2*.com mail3000.com
+freemail_domains mail.pt mail.ru mail.yahoo.co.jp mail15.com mail2*.com mail3000.com mail333.com
 freemail_domains mail8.com mailandftp.com mailandnews.com mailas.com mailasia.com mailbg.com
@@ -238,3 +239,3 @@
 freemail_domains ms*.hinet.net mscold.com msn.com mundo-r.com munich.com musician.org muslim.com
-freemail_domains muslimsonline.com mustangs.com mxs.de mycabin.com mycity.com mycommail.com
+freemail_domains muslimsonline.com mustangs.com mxs.de myblue.cc mycabin.com mycity.com mycommail.com
 freemail_domains mycool.com mydomain.com myeweb.com myfastmail.com myfunnymail.com
@@ -246,3 +247,3 @@
 freemail_domains nanaseaikawa.com nandomail.com naseej.com nastything.com national-champs.com
-freemail_domains nativeweb.net naveganas.com nebraska.usa.com nemra1.com nenter.com
+freemail_domains nativeweb.net narod.ru naveganas.com nebraska.usa.com nemra1.com nenter.com
 freemail_domains nerdshack.com nervhq.org net.hr net4b.pt net4jesus.com net4you.at
@@ -255,3 +256,3 @@
 freemail_domains newyork.usa.com newyorkcity.com nfmail.com nicegal.com nightimeuk.com
-freemail_domains nightly.com nightmail.com noavar.com noemail.com nonomail.com
+freemail_domains nightly.com nightmail.com nightmail.ru noavar.com noemail.com nonomail.com
 freemail_domains nonpartisan.com noolhar.com northcarolina.usa.com northdakota.usa.com
@@ -263,3 +264,3 @@
 freemail_domains online.ru onlinewiz.com onobox.com openbg.com openforyou.com
-freemail_domains opentransfer.com operamail.com oplusnet.com optician.com orangehome.co.uk
+freemail_domains opentransfer.com operamail.com oplusnet.com optician.com orange.* orangehome.co.uk
 freemail_domains orbitel.bg orcon.net.nz oregon.usa.com oreka.com organizer.net orgio.net
@@ -275,5 +276,5 @@
 freemail_domains planetout.com plasa.com playersodds.com playful.com pluno.com
-freemail_domains plusmail.com.br pmail.net pnetmail.co.za pobox.ru pobox.sk pochta.ru
+freemail_domains plusmail.com.br pmail.net pnetmail.co.za pobox.ru pobox.sk pochtamt.ru pochta.ru
 freemail_domains poczta.fm poetic.com pogowave.com polandmail.com polbox.com politician.com
-freemail_domains pop.co.th popmail.com poppymail.com popsmail.com popstar.com portafree.com
+freemail_domains pop3.ru pop.co.th popmail.com poppymail.com popsmail.com popstar.com portafree.com
 freemail_domains portaldosalunos.com portugalmail.com portugalmail.pt post.com post.cz
@@ -286,3 +287,3 @@
 freemail_domains qatar.io qlmail.com qrio.com qsl.net qudsmail.com queerplaces.com quepasa.com
-freemail_domains quick.cz quickwebmail.com r-o-o-t.com r320.hu raakim.com racingseat.com
+freemail_domains quick.cz quickwebmail.com r-o-o-t.com r320.hu raakim.com rbcmail.ru racingseat.com
 freemail_domains radicalz.com radiojobbank.com radiologist.net ragingbull.com
@@ -310,3 +311,3 @@
 freemail_domains sistersbrothers.com sizzling.com skim.com slamdunkfan.com slickriffs.co.uk
-freemail_domains slingshot.com slo.net slomusic.net smartemail.co.uk snail-mail.net
+freemail_domains slingshot.com slo.net slomusic.net smartemail.co.uk smtp.ru snail-mail.net
 freemail_domains snakebite.com sndt.net sneakemail.com snoopymail.com snowboarding.com
@@ -329,3 +330,3 @@
 freemail_domains taxcutadvice.com teachers.org techemail.com techie.com technisamail.co.za
-freemail_domains technologist.com teenmail.co.uk teenmail.co.za tejary.com telebot.com
+freemail_domains technologist.com teenmail.co.uk teenmail.co.za tejary.com telebot.com telefonica.net
 freemail_domains telegraf.by teleline.es telinco.net telkom.net telpage.net telstra.com
@@ -373,3 +374,3 @@
 freemail_domains yaho.com yahoo.* yahoo.co*.* yalla.com.lb
-freemail_domains yam.com yamal.info yapost.com yawmail.com yebox.com yehey.com
+freemail_domains yam.com yamal.info yandex.ru yapost.com yawmail.com yebox.com yehey.com
 freemail_domains yellow-jackets.com yellowstone.net yenimail.com yepmail.net yifan.net
--- 3.003001.TAR/updates_spamassassin_org/20_head_tests.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/20_head_tests.cf	2010-03-15 17:53:33.000000000 +0100
@@ -508,3 +508,4 @@
 
-header DATE_IN_FUTURE_96_XX	eval:check_for_shifted_date('96', 'undef')
+#header DATE_IN_FUTURE_96_XX	eval:check_for_shifted_date('96', 'undef')
+meta DATE_IN_FUTURE_96_XX 	(0)
 describe DATE_IN_FUTURE_96_XX	Date: is 96 hours or more after Received: date
--- 3.003001.TAR/updates_spamassassin_org/20_ratware.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/20_ratware.cf	2010-03-15 17:53:33.000000000 +0100
@@ -208,9 +208,2 @@
 
-# spammer tool, sometimes has "netIP with HTTP;" in Received: header
-
-# this is really badly faked.  Also the spammer who uses "25250101"
-# for the build is a total hippie.
-header RATWARE_GECKO_BUILD	User-Agent =~ /Gecko\/(?!200\d\d\d\d\d)\d/
-describe RATWARE_GECKO_BUILD	Bulk email fingerprint (Gecko faked) found
-
 ########################################################################
--- 3.003001.TAR/updates_spamassassin_org/20_uri_tests.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/20_uri_tests.cf	2010-03-15 17:53:33.000000000 +0100
@@ -65,3 +65,3 @@
 # though, which actually doesn't have a weird port in it.
-uri WEIRD_PORT			m{https?://[^/\s]+?:\d+(?<!:80)(?<!:443)(?<!:8080)(?:/|\s|$)}
+uri WEIRD_PORT			m{https?://[^/?\s]+?:\d+(?<!:80)(?<!:443)(?<!:8080)(?:/|\s|$)}
 describe WEIRD_PORT		Uses non-standard port number for HTTP
--- 3.003001.TAR/updates_spamassassin_org/30_text_de.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/30_text_de.cf	2010-03-15 17:53:33.000000000 +0100
@@ -314,3 +314,2 @@
 lang de describe RATWARE_HASH_DASH Enthält Abwehrmaßnahme gegen Anti-Spam-Software ("hashbuster")
-lang de describe RATWARE_GECKO_BUILD Kopfzeilen enthalten gefälschte Hinweise auf Mozilla/Gecko
 lang de describe RATWARE_ZERO_TZ Seltsame Zeitzone (+0000)
--- 3.003001.TAR/updates_spamassassin_org/30_text_nl.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/30_text_nl.cf	2010-03-15 17:53:33.000000000 +0100
@@ -236,3 +236,2 @@
 lang nl describe RATWARE_HASH_DASH               Bevat een hashbuster in Send-Safe opmaak
-lang nl describe RATWARE_GECKO_BUILD             Bulk email signatuur (vervalste Gecko) gevonden
 lang nl describe NUMERIC_HTTP_ADDR               Gebruikt een numeriek IP adres in een URL
--- 3.003001.TAR/updates_spamassassin_org/50_scores.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/50_scores.cf	2010-03-15 17:53:33.000000000 +0100
@@ -458,3 +458,2 @@
 score RATWARE_EGROUPS 1.898 1.258 1.406 1.621
-score RATWARE_GECKO_BUILD 0 # n=0 n=1 n=2 n=3
 score RATWARE_HASH_DASH 0 # n=0 n=1 n=2 n=3
@@ -799,3 +798,4 @@
 score DATE_IN_FUTURE_48_96 2.384 0.813 1.078 2.181
-score DATE_IN_FUTURE_96_XX 2.614 3.028 2.851 3.087
+#score DATE_IN_FUTURE_96_XX 2.614 3.028 2.851 3.087
+score DATE_IN_FUTURE_96_XX 0
 score DATE_IN_PAST_03_06 2.399 1.076 1.200 1.592
Only in 3.003001.NET/updates_spamassassin_org: 72_scores.cf
Only in 3.003001.NET/updates_spamassassin_org: MIRRORED.BY
Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set0-72_scores.cf.txt
Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set1-72_scores.cf.txt
Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set2-72_scores.cf.txt
Only in 3.003001.NET/updates_spamassassin_org: STATISTICS-set3-72_scores.cf.txt
Only in 3.003001.NET/updates_spamassassin_org: languages
Only in 3.003001.NET/updates_spamassassin_org: local.cf
Only in 3.003001.NET/updates_spamassassin_org: regression_tests.cf
Only in 3.003001.NET/updates_spamassassin_org: sa-update-pubkey.txt
Only in 3.003001.NET/updates_spamassassin_org: user_prefs.template
--- 3.003001.TAR/updates_spamassassin_org.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org.cf	2010-03-15 17:53:33.000000000 +0100
@@ -1,2 +1,2 @@
-# UPDATE version 923257
+# UPDATE version 922507
 include updates_spamassassin_org/10_default_prefs.cf
@@ -50 +50,4 @@
 include updates_spamassassin_org/72_active.cf
+include updates_spamassassin_org/72_scores.cf
+include updates_spamassassin_org/local.cf
+include updates_spamassassin_org/regression_tests.cf
--- 3.003001.TAR/updates_spamassassin_org/72_active.cf	2010-03-15 17:55:03.000000000 +0100
+++ 3.003001.NET/updates_spamassassin_org/72_active.cf	2010-03-15 17:53:33.000000000 +0100
[lots of diff]

Re: proposed 3.3.1 tarballs

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 15/03/2010 12:17 PM, Yet Another Ninja wrote:
> On 2010-03-15 17:13, Justin Mason wrote:
>> so that file no longer needs to be used?
> 
> nope... Its still mantained, for ppl using older SA versions, but will
> disappear next year...
> 
> When SA 3.1.1 is released I'll make some noise so ppl stip using the
> Sare version.

If you wanted, you could probably add a "if (version <= 3.003000)"
conditional at some point.  Not that it makes a big difference if the
config gets loaded twice.

Daryl


Re: proposed 3.3.1 tarballs

Posted by Yet Another Ninja <sa...@alexb.ch>.
On 2010-03-15 17:13, Justin Mason wrote:
> so that file no longer needs to be used?

nope... Its still mantained, for ppl using older SA versions, but will 
disappear next year...

When SA 3.1.1 is released I'll make some noise so ppl stip using the 
Sare version.


>  what happens if both are in
> place?  (presumably nothing)
> I'll add that.
> 
> --j.
> 
> On Mon, Mar 15, 2010 at 15:23, Yet Another Ninja <sa...@alexb.ch> wrote:
>> bug 6361: list 2tld and 3tld sub-domain hosters for URIBL/SURBL/DBL queries
>>
>> should it state that this deprecates
>> http://www.rulesemporium.com/rules/90_2tld.cf
>>
>> for those who actually read any release notes :-)
>>
>>
>>
>> On 2010-03-15 15:41, Justin Mason wrote:
>>> http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
>>> http://people.apache.org/~jm/devel/
>>>
>>> please vote!
>>>
>>> md5sum of archive files:
>>>
>>>  2290490889b2d91f71a3104eaf9c5cd3  Mail-SpamAssassin-3.3.1.tar..bz2
>>>  e70096d6baa695371b413e6691a49038  Mail-SpamAssassin-3.3.1.tar..gz
>>>  915e22168642997f168f5678f66d8def  Mail-SpamAssassin-3.3.1.zip
>>>  e004b3172650f003c2e9f15e5b2f6ac4
>>>  Mail-SpamAssassin-rules-3.3.1.r923257.tgz
>>>
>>> sha1sum of archive files:
>>>
>>>  9a84b175241c52cf6e1bf4cfcf0719cf65f5dc18  Mail-SpamAssassin-3..3.1.tar.bz2
>>>  4160be9c795573d8208a81665614387797136030  Mail-SpamAssassin-3..3.1.tar.gz
>>>  8f8ba7c69283bec99a91a7d266f8e5a1477c3b41  Mail-SpamAssassin-3..3.1.zip
>>>  e211a58e0d4cf7d30fdaf6b9ff8f797086f96312
>>> Mail-SpamAssassin-rules-3.3.1.r923257.tgz
>>>
>>>
>>> cheers,
>>>
>>
> 
> 
> 

Re: proposed 3.3.1 tarballs

Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-15 at 16:13 +0000, Justin Mason wrote:
> so that file no longer needs to be used?  what happens if both are in
> place?  (presumably nothing)

Yup, nothing -- but an increased startup time for conf. ;)  Hardly
measurable. Registrar Boundaries 2tld and 3tld data is kept in a hash.
Adding them twice doesn't make any difference.

  $Mail::SpamAssassin::Util::RegistrarBoundaries::TWO_LEVEL_DOMAINS{lc $_} = 1;


> On Mon, Mar 15, 2010 at 15:23, Yet Another Ninja <sa...@alexb.ch> wrote:
> > bug 6361: list 2tld and 3tld sub-domain hosters for URIBL/SURBL/DBL queries
> >
> > should it state that this deprecates
> > http://www.rulesemporium.com/rules/90_2tld.cf
> >
> > for those who actually read any release notes :-)

-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


Re: proposed 3.3.1 tarballs

Posted by Justin Mason <jm...@jmason.org>.
so that file no longer needs to be used?  what happens if both are in
place?  (presumably nothing)

I'll add that.

--j.

On Mon, Mar 15, 2010 at 15:23, Yet Another Ninja <sa...@alexb.ch> wrote:
> bug 6361: list 2tld and 3tld sub-domain hosters for URIBL/SURBL/DBL queries
>
> should it state that this deprecates
> http://www.rulesemporium.com/rules/90_2tld.cf
>
> for those who actually read any release notes :-)
>
>
>
> On 2010-03-15 15:41, Justin Mason wrote:
>>
>> http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
>> http://people.apache.org/~jm/devel/
>>
>> please vote!
>>
>> md5sum of archive files:
>>
>>  2290490889b2d91f71a3104eaf9c5cd3  Mail-SpamAssassin-3.3.1.tar.bz2
>>  e70096d6baa695371b413e6691a49038  Mail-SpamAssassin-3.3.1.tar.gz
>>  915e22168642997f168f5678f66d8def  Mail-SpamAssassin-3.3.1.zip
>>  e004b3172650f003c2e9f15e5b2f6ac4
>>  Mail-SpamAssassin-rules-3.3.1.r923257.tgz
>>
>> sha1sum of archive files:
>>
>>  9a84b175241c52cf6e1bf4cfcf0719cf65f5dc18  Mail-SpamAssassin-3.3.1.tar.bz2
>>  4160be9c795573d8208a81665614387797136030  Mail-SpamAssassin-3.3.1.tar.gz
>>  8f8ba7c69283bec99a91a7d266f8e5a1477c3b41  Mail-SpamAssassin-3.3.1.zip
>>  e211a58e0d4cf7d30fdaf6b9ff8f797086f96312
>> Mail-SpamAssassin-rules-3.3.1.r923257.tgz
>>
>>
>> cheers,
>>
>
>



-- 
--j.

Re: proposed 3.3.1 tarballs

Posted by Yet Another Ninja <sa...@alexb.ch>.
bug 6361: list 2tld and 3tld sub-domain hosters for URIBL/SURBL/DBL queries

should it state that this deprecates 
http://www.rulesemporium.com/rules/90_2tld.cf

for those who actually read any release notes :-)



On 2010-03-15 15:41, Justin Mason wrote:
> http://people.apache.org/~jm/devel/PROPOSED-3.3.1.txt
> http://people.apache.org/~jm/devel/
> 
> please vote!
> 
> md5sum of archive files:
> 
>   2290490889b2d91f71a3104eaf9c5cd3  Mail-SpamAssassin-3.3.1.tar.bz2
>   e70096d6baa695371b413e6691a49038  Mail-SpamAssassin-3.3.1.tar.gz
>   915e22168642997f168f5678f66d8def  Mail-SpamAssassin-3.3.1.zip
>   e004b3172650f003c2e9f15e5b2f6ac4  Mail-SpamAssassin-rules-3.3.1.r923257.tgz
> 
> sha1sum of archive files:
> 
>   9a84b175241c52cf6e1bf4cfcf0719cf65f5dc18  Mail-SpamAssassin-3.3.1.tar.bz2
>   4160be9c795573d8208a81665614387797136030  Mail-SpamAssassin-3.3.1.tar.gz
>   8f8ba7c69283bec99a91a7d266f8e5a1477c3b41  Mail-SpamAssassin-3.3.1.zip
>   e211a58e0d4cf7d30fdaf6b9ff8f797086f96312
> Mail-SpamAssassin-rules-3.3.1.r923257.tgz
> 
> 
> cheers,
>