You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Robert Palmer <ro...@greetin.gs> on 2010/05/21 00:00:27 UTC
Rules updates
I am running spamassassin version 3.2.4 and notice my rules have not
updated (sa-update) for many months and I have started getting a lot of
nasty spam coming through.
Is it the case that the default rules are no longer being updated and
are there any other recommended sources for anti-spam rules?
Thanks
Re: Rules updates
Posted by Bill Landry <bi...@inetmsg.com>.
On Thu, May 20, 2010 4:26 pm, Benny Pedersen wrote:
> On fre 21 maj 2010 00:05:26 CEST, Michael Scheidell wrote
>> On 5/20/10 6:00 PM, Robert Palmer wrote:
>>> I am running spamassassin version 3.2.4 and notice my rules have
>>> not updated (sa-update) for many months and I have started getting
>>> a lot of nasty spam coming through.
>> just upgrade to SA 3.3.1
>> only current versions of SA have current rule updates.
>
> imho 3.2.5 is still latest stable
>
> and some have posted on maillist even 3.3.1 is not being updated
> longer, 3.3.2 is
Hmmm, 3.3.2? Who is hosting future releases?
Bill
Re: Rules updates
Posted by John Hardin <jh...@impsec.org>.
On Wed, 9 Jun 2010, Matt Kettler wrote:
> On 6/9/2010 12:11 PM, LuKreme wrote:
>> On 8-Jun-2010, at 19:34, Matt Kettler wrote:
>>
>>> Legacy version, 3.2.5 (rarely updated)
>>>
>> Even better:
>>
>> Unsupported version 3.2.5 (critical updates only)
>>
>> or
>>
>> Deprecated version: 3.2.5 (critical updates only, if at all)
>
> Well, unsupported is an overstatement. Support is not absent, it is just
> minimalist.
>
> I'm fine with Deprecated, Legacy, Retired, Ancient, Geriatric,
> Out-To-Pasture, Over-The-Hill, Past-its-prime, or many similar variants
> that imply this version is still running, but on its last legs.
Zombie?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
A well educated Electorate, being necessary to the liberty of a
free State, the Right of the People to Keep and Read Books,
shall not be infringed.
-----------------------------------------------------------------------
244 days since President Obama won the Nobel "Not George W. Bush" prize
Re: Rules updates
Posted by Matt Kettler <mk...@verizon.net>.
On 6/9/2010 12:11 PM, LuKreme wrote:
> On 8-Jun-2010, at 19:34, Matt Kettler wrote:
>
>> Legacy version, 3.2.5 (rarely updated)
>>
> Even better:
>
> Unsupported version 3.2.5 (critical updates only)
>
> or
>
> Deprecated version: 3.2.5 (critical updates only, if at all)
>
>
Well, unsupported is an overstatement. Support is not absent, it is just
minimalist.
I'm fine with Deprecated, Legacy, Retired, Ancient, Geriatric,
Out-To-Pasture, Over-The-Hill, Past-its-prime, or many similar variants
that imply this version is still running, but on its last legs.
Re: Rules updates
Posted by LuKreme <kr...@kreme.com>.
On 8-Jun-2010, at 19:34, Matt Kettler wrote:
>
> Legacy version, 3.2.5 (rarely updated)
Even better:
Unsupported version 3.2.5 (critical updates only)
or
Deprecated version: 3.2.5 (critical updates only, if at all)
--
I collect blondes and bottles. ~Marlowe
Re: Rules updates
Posted by Matt Kettler <mk...@verizon.net>.
On 6/8/2010 11:22 PM, Alex wrote:
> Hi,
>
>
>> We also very loudly repeatedly state on the list that if you want to
>> keep abreast of the latest spam, you need to be running the latest
>> version of the codebase (can't take advantage of new features without
>> it!), but don't have that clearly documented either.
>>
> It would be great if you could document exactly what features are
> exclusively available in 3.3.x? In other words, can you quantify how
> much is being missed by continuing to use v3.2.5?
>
> I've read the release notes for v3.3, but I see very little in the
> "MAIN NEW FEATURES" of the release notes that I can see would itself
> result in a marked improvement in catch ratio.
>
Yes, that's largely focused on the things that will matter to
configuration, comparability, etc.
> I know it's obviously a good idea to upgrade, but in a nutshell, how
> much different is one version from the other?
>
Historically a change in the second number of the SA release (ie: 3.2.x
to 3.3.x) is generally where we introduce radical changes to the
ruleset. Right before a x.y.0 release, a large batch of new rules are
added in, and all rules are up on the chopping block for elimination.
This is also when a whole new scoreset gets generated (historically a
long, slow process that took a lot of CPU time). Sa-updates in 3.1.x
will add and remove some rules, change a few scores, etc, but typically
only as needed. 3.2.x expanded that a bit, and we did a few rescoring
runs. 3.3 is taking a newer model where we're regenerating the scores
much more often (thanks in part to the faster perceptron scoring tool
introduced in 3.2).
So if you really want a big shift in rules, versions are traditionally
where that happens. Sa-updates will tend to push the best and the
brightest new rules, but many of the "good but not great" rules wait
until a version update (where they can vie for space in the ruleset in a
competitive deathmatch with other rules... :-)
As for the release notes, these bits would readily affect accuracy:
Main New Features section:
(ipv6, dkim and AWL changes may matter if present in your environment)
Rules section:
- new scores were generated by a genetic algorithm (GA) and then manually
tweaked based on cleaned datasets supplied by a dozen volunteers;
- dropped redundant rules or rules causing too many false positives;
- added or updated many rules;
Plugin section:
- new plugins: FreeMail, PhishTag, Reuse;
Bug fixes:
- fixed some cases where :addr headers were parsed incorrectly
- fixed leakage of 'whitelist_from_rcvd' entries between spamd users;
- the 'exists:' evaluator in HEADER rules now works as documented
and tests for existence of a header field, instead of testing for
a header field body being nonempty; internally, the pms->get can
also now distinguish between empty and nonexistent header fields;
- applied fixes to header fields parsing in several places: header field
names are case-insensitive, whitespace is not required after a colon,
obsolete rfc822 syntax allowed whitespace before a colon;
VBounce: match "Received:" only at the beginning of a line;
- fixed parsing of multi-line Received header fields for
BOUNCE_MESSAGE/VBOUNCE_MESSAGE et al
> Thanks,
> Alex
>
>
Re: Rules updates
Posted by LuKreme <kr...@kreme.com>.
On 9-Jun-2010, at 10:25, Alex wrote:
>
> Hi,
>
>>> It would be great if you could document exactly what features are
>>> exclusively available in 3.3.x? In other words, can you quantify how
>>> much is being missed by continuing to use v3.2.5?
>>
>> All new rules. All current spam-fighting measures.
>
> Yes, I realize that. I was hoping for specifics. Which new rules?
The entire rule database was redone, so asking which new rules doesn't make a lot of sense. Many rules were added, but many were removed as well.
All new rues also means all-new rules, I guess.
> Is it a matter of patterns that are being used that can't be implemented
> in v3.2.5? What are those current spam-fighting measures?
It's an entirely new ruleset. You cannot take the 3.3 rules and drop then into 3.2.5 and have it work, for example.
> It seems the FEATURE LIST describes the features that have been
> developed to implement the "current spam-fighting measures", but
> doesn't actually describe those measures, unless I'm missing it.
I think if you think of it more like an anti-virus tool it will make more sense; it's not really about the new features as much as it is about being effective. running 3.2.5 is a lot like running a year-old Norton.
--
Han : You said you wanted to be around when I made a mistake, well, this
could be it, sweetheart. Leia: I take it back.
Re: Rules updates
Posted by Alex <my...@gmail.com>.
Hi,
>> It would be great if you could document exactly what features are
>> exclusively available in 3.3.x? In other words, can you quantify how
>> much is being missed by continuing to use v3.2.5?
>
> All new rules. All current spam-fighting measures.
Yes, I realize that. I was hoping for specifics. Which new rules? Is
it a matter of patterns that are being used that can't be implemented
in v3.2.5? What are those current spam-fighting measures?
It seems the FEATURE LIST describes the features that have been
developed to implement the "current spam-fighting measures", but
doesn't actually describe those measures, unless I'm missing it.
I hope it doesn't seem like I'm making a bigger deal of this than
necessary, but I'm really just curious what developments have been
made. It's of course my intention to upgrade as soon as possible, but
it would be good to know as specifically as possible what to expect so
I know it's working :-)
I believe there is a new RBL that's implemented in v3.3? Has SPF
changed in some way? Is bayes more effective because of a change in
the algorithm? Maybe it can handle more volume on the same server?
Maybe someone else wants to ring in on their experience with
performance after an upgrade?
Thanks,
Alex
Re: Rules updates
Posted by LuKreme <kr...@kreme.com>.
On 8-Jun-2010, at 21:22, Alex wrote:
>
> Hi,
>
>> We also very loudly repeatedly state on the list that if you want to
>> keep abreast of the latest spam, you need to be running the latest
>> version of the codebase (can't take advantage of new features without
>> it!), but don't have that clearly documented either.
>
> It would be great if you could document exactly what features are
> exclusively available in 3.3.x? In other words, can you quantify how
> much is being missed by continuing to use v3.2.5?
All new rules. All current spam-fighting measures.
> I've read the release notes for v3.3, but I see very little in the
> "MAIN NEW FEATURES" of the release notes that I can see would itself
> result in a marked improvement in catch ratio.
See above.
> I know it's obviously a good idea to upgrade, but in a nutshell, how
> much different is one version from the other?
3.2.5 will let in a lot more spam in a default configuration than 3.3.x
This is not like upgrading Postfix from 2.5 to 2.7 where the gains may or may not be relevant to you. If your anti-spam (or anti-virus) software is not up-to-date, it is not worth running.
--
'If you sow dragons' teeth, you should get dragons. Not fighting
skeletons. What did it say on the packet?' 'I don't know! The myth never
said anything about them coming in a packet!' 'Should have said "Comes
up Dragons" on the packet.' --Interesting Times
Re: Rules updates
Posted by Alex <my...@gmail.com>.
Hi,
> We also very loudly repeatedly state on the list that if you want to
> keep abreast of the latest spam, you need to be running the latest
> version of the codebase (can't take advantage of new features without
> it!), but don't have that clearly documented either.
It would be great if you could document exactly what features are
exclusively available in 3.3.x? In other words, can you quantify how
much is being missed by continuing to use v3.2.5?
I've read the release notes for v3.3, but I see very little in the
"MAIN NEW FEATURES" of the release notes that I can see would itself
result in a marked improvement in catch ratio.
I know it's obviously a good idea to upgrade, but in a nutshell, how
much different is one version from the other?
Thanks,
Alex
Re: Rules updates
Posted by Matt Kettler <mk...@verizon.net>.
On 6/8/2010 5:48 PM, James Ralston wrote:
> On 2010-05-21 at 03:09+02 Karsten Bräckelmann <gu...@rudersport.de> wrote:
>
>
>> 3.2.x is in maintenance, and gets emergency rule updates
>> *exclusively*. As it has been for quite a long time.
>>
>> 3.3.x uses a new rule update model, and gets frequent updates. IFF
>> the mass-check corpus is large enough.
>>
> And exactly where is this ("3.2.x gets emergency rule updates only")
> documented?
>
> It's not mentioned in the 3.3.0 or 3.3.1 release notes.
>
>
We probably should document this, as far as I know, we haven't
documented it anywhere other than various list postings (like this one).
We also very loudly repeatedly state on the list that if you want to
keep abreast of the latest spam, you need to be running the latest
version of the codebase (can't take advantage of new features without
it!), but don't have that clearly documented either.
Perhaps a pair of notes on the download page would be a good start:
Released version, 3.3.1 (recommended release for accuracy)
Previous Released version, 3.2.5 (rarely updated)
or better:
Legacy version, 3.2.5 (rarely updated)
We could also go further by creating a wiki page for the product
lifecycle, similar to the one for the rule lifecycle:
http://wiki.apache.org/spamassassin/RuleLifeCycle
That said it is extraordinarily common practice for open source projects
to have "legacy" releases (or as we term it on the download page
"Previous Released Version") enter a mature stage of its lifecycle with
few changes or updates. SpamAssassin conforms to this model. This may
not be the ideal version to use for spam catching, but it is there for
those who want it. I guess we've always "assumed it was obvious" that
the older codebase was in a mature lifecycle stage.
Re: Rules updates
Posted by James Ralston <qr...@andrew.cmu.edu>.
On 2010-05-21 at 03:09+02 Karsten Bräckelmann <gu...@rudersport.de> wrote:
> 3.2.x is in maintenance, and gets emergency rule updates
> *exclusively*. As it has been for quite a long time.
>
> 3.3.x uses a new rule update model, and gets frequent updates. IFF
> the mass-check corpus is large enough.
And exactly where is this ("3.2.x gets emergency rule updates only")
documented?
It's not mentioned in the 3.3.0 or 3.3.1 release notes.
--
"There is no means of avoiding the final collapse of a boom brought
about by credit expansion. The alternative is only whether the
crisis should come sooner as the result of a voluntary abandonment of
further credit expansion or later as a final and total catastrophe of
the currency involved." - Ludwig von Mises, http://mises.org/
Re: Rules updates
Posted by Benny Pedersen <me...@junc.org>.
On fre 21 maj 2010 03:09:05 CEST, Karsten Bräckelmann wrote
> Ignoring your (humble or not) opinion for a second... 3.3.x is the
> latest stable.
thanks for clearing up this mess :)
3.3.1 is not being stable here on gentoo, there is a few problems with
spf check, and i will try to find where its is, running pypolicyd-spf
in mta sometimes says pass and spamassassin test result being neotral :(
Mail-SPF 2.007
NetAddr-IP 4.027
--
xpoint http://www.unicom.com/pw/reply-to-harmful.html
Re: Rules updates
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2010-05-20 at 21:34 -0400, Robert Palmer wrote:
> yum install insisted I have current version so I used cpan which got me
> to 3.3.1. Should I stop there or consider 3.3.2 or 3.4.x?
http://spamassassin.apache.org/
Did you have a look there, yet? 3.3.1 is the latest stable release.
3.3.2 has not been released, still under development and available in
the SVN repo only.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Rules updates
Posted by Robert Palmer <ro...@greetin.gs>.
yum install insisted I have current version so I used cpan which got me
to 3.3.1. Should I stop there or consider 3.3.2 or 3.4.x?
Thanks
On 5/20/2010 9:09 PM, Karsten Bräckelmann wrote:
> On Fri, 2010-05-21 at 01:26 +0200, Benny Pedersen wrote:
>
>>>> I am running spamassassin version 3.2.4 and notice my rules have
>>>> not updated (sa-update) for many months and I have started getting
>>>> a lot of nasty spam coming through.
>>>>
>
>> imho 3.2.5 is still latest stable
>>
> Ignoring your (humble or not) opinion for a second... 3.3.x is the
> latest stable.
>
>
>> and some have posted on maillist even 3.3.1 is not being updated
>> longer, 3.3.2 is
>>
> Nope. Rule updates for all 3.3.x versions have been lacking recently,
> due to limited mass-check corpora below the (high for sanity reasons)
> threshold.
>
> However, 3.4.x (read that again, FOUR, the current unstable dev tree)
> have not been affected and have had updates -- because there are no
> thresholds, and actually NO rule's scores.
>
>
>> so rules are fuzzy now :(
>>
> 3.2.x is in maintenance, and gets emergency rule updates *exclusively*.
> As it has been for quite a long time.
>
> 3.3.x uses a new rule update model, and gets frequent updates. IFF the
> mass-check corpus is large enough.
>
>
>
Re: Rules updates
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2010-05-21 at 01:26 +0200, Benny Pedersen wrote:
> > > I am running spamassassin version 3.2.4 and notice my rules have
> > > not updated (sa-update) for many months and I have started getting
> > > a lot of nasty spam coming through.
> imho 3.2.5 is still latest stable
Ignoring your (humble or not) opinion for a second... 3.3.x is the
latest stable.
> and some have posted on maillist even 3.3.1 is not being updated
> longer, 3.3.2 is
Nope. Rule updates for all 3.3.x versions have been lacking recently,
due to limited mass-check corpora below the (high for sanity reasons)
threshold.
However, 3.4.x (read that again, FOUR, the current unstable dev tree)
have not been affected and have had updates -- because there are no
thresholds, and actually NO rule's scores.
> so rules are fuzzy now :(
3.2.x is in maintenance, and gets emergency rule updates *exclusively*.
As it has been for quite a long time.
3.3.x uses a new rule update model, and gets frequent updates. IFF the
mass-check corpus is large enough.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Rules updates
Posted by Benny Pedersen <me...@junc.org>.
On fre 21 maj 2010 00:05:26 CEST, Michael Scheidell wrote
> On 5/20/10 6:00 PM, Robert Palmer wrote:
>> I am running spamassassin version 3.2.4 and notice my rules have
>> not updated (sa-update) for many months and I have started getting
>> a lot of nasty spam coming through.
> just upgrade to SA 3.3.1
> only current versions of SA have current rule updates.
imho 3.2.5 is still latest stable
and some have posted on maillist even 3.3.1 is not being updated
longer, 3.3.2 is
so rules are fuzzy now :(
--
xpoint http://www.unicom.com/pw/reply-to-harmful.html
Re: Rules updates
Posted by Michael Scheidell <sc...@secnap.net>.
On 5/20/10 6:00 PM, Robert Palmer wrote:
> I am running spamassassin version 3.2.4 and notice my rules have not
> updated (sa-update) for many months and I have started getting a lot
> of nasty spam coming through.
>
just upgrade to SA 3.3.1
only current versions of SA have current rule updates.
--
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
> *| *SECNAP Network Security Corporation
* Certified SNORT Integrator
* 2008-9 Hot Company Award Winner, World Executive Alliance
* Five-Star Partner Program 2009, VARBusiness
* Best Anti-Spam Product 2008, Network Products Guide
* King of Spam Filters, SC Magazine 2008
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________