You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Robert Menschel <Ro...@Menschel.net> on 2005/02/12 23:26:02 UTC

[SARE] header rules updated

Just a quick note that SARE's header rules files have been updated.

Information concerning these rules files can be found at
http://www.rulesemporium.com/rules.htm#header

Bob Menschel




Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
Didn't realize this was a on list with my private reply....

On Thu, Feb 17, 2005 at 04:51:08PM -0600, Chris Thielen wrote:
>George,
>
>>Maybe the way RDJ does the roll back needs be addressed? I know version
>>2 is nearing release, and this wouldn't be difficult to add:  It could
>>check the cf file for a grep-able, commented, "this release" changes
>>entry, which may include a rules.htm#ChangesVerX url.
>> 
>>
>RDJ has always reported the version line of the updated ruleset.  
>Ruleset authors can use this feature to get a one-line change 
>description to RDJ users when they are emailed.  Based on my RDJ run 
>logs, it seems that this used to happen more than it does now, but of 
>course that's up to the Ruleset authors...
>
>I suppose I could broaden the scope of the text returned for version 
>reporting to allow for multiple lines, or explicitly code for changelog 
>support.

The main work needs to come from the ruleset authors. I propose they
put a changes log in cf files, with special #@@# (look) comments.
I would expect changes to take a few to a few dozen lines... A url
for complete change log would be good to include here too.


>>Then if some change broke your site, you get a likely indicator why,
>>right there beside the roll back commands, near the lint output. And if
>>your update is multiple revisions behind, you have a url to get started
>>on finding changes at the relevant revision.
>> 
>>
>When I get time I will revisit --lint.  I also want to process linting 
>each new file one at a time to try to isolate the broken files.

Not sure if that would work for anything but testing for valid ruleset,
a local cf might score a missing rule which is an error, and lint has
limited use if you somehow just check the downloaded file.  I wouldn't
make it a high priority.


>>Actually, I like this proposed way of reporting changes a lot. I've
>>always wondered the point of email notifications, "ruleset x has
>>changed.." They kinda suggest I should do a diff and figure out if
>>everything is really okay. I'd just assume see a change log as part of
>>the notification that new rules have been loaded (or that lint prevented
>>those changes from happening).
>>
>I don't understand... I assume you would prefer the notification to not 
>knowing it was updated at all...?

The behavior is fine, just add the output of "grep '#@@#' newfile.cf"
to the ${MESSAGES} whether lint failed or not. (Once changes get that
tag.) I can send you a diff to try unless you want to put it in...


So I guess the question is, do any of the ninja's want to create
comments for their changes?

// George


-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re: [SARE] header rules updated

Posted by Chris Thielen <cm...@someone.dhs.org>.
George,

>Maybe the way RDJ does the roll back needs be addressed? I know version
>2 is nearing release, and this wouldn't be difficult to add:  It could
>check the cf file for a grep-able, commented, "this release" changes
>entry, which may include a rules.htm#ChangesVerX url.
>  
>
RDJ has always reported the version line of the updated ruleset.  
Ruleset authors can use this feature to get a one-line change 
description to RDJ users when they are emailed.  Based on my RDJ run 
logs, it seems that this used to happen more than it does now, but of 
course that's up to the Ruleset authors...

I suppose I could broaden the scope of the text returned for version 
reporting to allow for multiple lines, or explicitly code for changelog 
support.

>Then if some change broke your site, you get a likely indicator why,
>right there beside the roll back commands, near the lint output. And if
>your update is multiple revisions behind, you have a url to get started
>on finding changes at the relevant revision.
>  
>
When I get time I will revisit --lint.  I also want to process linting 
each new file one at a time to try to isolate the broken files.

>Actually, I like this proposed way of reporting changes a lot. I've
>always wondered the point of email notifications, "ruleset x has
>changed.." They kinda suggest I should do a diff and figure out if
>everything is really okay. I'd just assume see a change log as part of
>the notification that new rules have been loaded (or that lint prevented
>those changes from happening).
>
I don't understand... I assume you would prefer the notification to not 
knowing it was updated at all...?

Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
On Thu, Feb 17, 2005 at 07:57:54PM -0800, Robert Menschel wrote:
>Hello George,
>
>Thursday, February 17, 2005, 7:43:27 PM, you wrote:
>
>GG> I count approximately 35 active cf files in rulesemporium. Of course the
>GG> changes aren't evenly distributed, but that's an average of 3 changes
>GG> per file, which is more what I was thinking with the request.
>
>No, that's 100 changes among 7 header* files. 40 were adds, 40 were
>moves (which would put a change line in both the old and the new
>files), and 10 were changes. I calculate then 130 change lines / 7
>files averages 18.5 lines each, with more header0, header1, header3,
>and fewer in the others.


just for consideration, this would suit me fine:

#@@# moved to 70_other.cf RULEa RULEb RULEc RULEd RULEe RULEf
#@@# moved from 70_another.cf RULEx RULEy RULEz
#@@# new RULE1 RULE2 RULE3 RULE4 RULE5 RULE6
#@@# removed OLDRULEa OLDRULEb OLDRULEc OLDRULEd OLDRULEe
#@@# changed FAVORITEa FAVORITEb FAVORITEc FAVORITEd FAVORITEe

// George


-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello George,

Thursday, February 17, 2005, 7:43:27 PM, you wrote:

GG> Hi Bob,

>>And as I said, the equivalent change log for this last update to the
>>header* files would have been close to 100 lines long. Is that what
>>you would like to see?

GG> I count approximately 35 active cf files in rulesemporium. Of course the
GG> changes aren't evenly distributed, but that's an average of 3 changes
GG> per file, which is more what I was thinking with the request.

No, that's 100 changes among 7 header* files. 40 were adds, 40 were
moves (which would put a change line in both the old and the new
files), and 10 were changes. I calculate then 130 change lines / 7
files averages 18.5 lines each, with more header0, header1, header3,
and fewer in the others.

GG> It it turns out to be not that useful, stopping is easy enough.

Useful is as useful does.  It's hard to judge just from this
discussion. If others would speak up and let us (SARE) know how
important it would be to them, we'd be better able to judge how much
work and detail to put into our change history.

GG> btw - I don't feel good about askin. I appreciate the rules being
GG> made available in the first place. And would be happy to send a
GG> case of real beer to a deserving fridge, in any event. Feel free
GG> to send address offlist.

Asking is good. Beer is good, at least to other Ninjas. Me, I have
different vices.

Bob Menschel




Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
Hi Bob,

On Thu, Feb 17, 2005 at 06:11:49PM -0800, Robert Menschel wrote:
>Hello George,
>
>Thursday, February 17, 2005, 8:16:21 AM, you wrote:
>
>>>Fair enough.  In this most recent publication of updates to the
>>>70_sare_header*.cf family, 40 rules were added, and 50 rules were
>>>moved from one file to another, including 8 or 9 that were moved from
>>>various files to the archive file.
>>>
>>>How would you structure a change history for those changes?
>
>GG> Are the files in cvs? I would suggest using the log keyword: ...
>
>Nope, no cvs, no svn. They're manually maintained, with generally only
>one SARE Ninja maintaining any single rules files.
>
>GG> #@@# $Log: README,v $
>GG> #@@# Revision 1.7  2005/02/17 15:42:54  geo
>GG> #@@#
>GG> #@@# Changed MSGID_WRONG due to high false negatives
>GG> #@@# Added a log keyword. And noted changes made to this file.
>GG> #@@# Added BROKE_SPAMWARE from Adam
>GG> #@@# Added IDIOT_SPAMER from Jim Hoan
>GG> #@@# Added FREE_MONEY from Joe John
>GG> #@@# Added TEN_QUESTIONS from 70_sare_freefriends.cf
>GG> #@@# Moved MAGIC_WORKS to 70_sare_oem.cf
>GG> #@@# Removed TRIES_HARD due to false positives
>
>And as I said, the equivalent change log for this last update to the
>header* files would have been close to 100 lines long. Is that what
>you would like to see?

I count approximately 35 active cf files in rulesemporium. Of course the
changes aren't evenly distributed, but that's an average of 3 changes
per file, which is more what I was thinking with the request.

Simply what I'm looking for is a one line note describing the rule
change right in the same file, made at the same time as the change. On a
subsequent release, delete the changes from the prior release and just
leave the current ones. Might be good to grep the change notes into an
accumulating log file for historical reference.

It it turns out to be not that useful, stopping is easy enough.

btw - I don't feel good about askin. I appreciate the rules being made
available in the first place. And would be happy to send a case of real
beer to a deserving fridge, in any event. Feel free to send address
offlist.

// George

-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello George,

Thursday, February 17, 2005, 8:16:21 AM, you wrote:

>>Fair enough.  In this most recent publication of updates to the
>>70_sare_header*.cf family, 40 rules were added, and 50 rules were
>>moved from one file to another, including 8 or 9 that were moved from
>>various files to the archive file.
>>
>>How would you structure a change history for those changes?

GG> Are the files in cvs? I would suggest using the log keyword: ...

Nope, no cvs, no svn. They're manually maintained, with generally only
one SARE Ninja maintaining any single rules files.

GG> #@@# $Log: README,v $
GG> #@@# Revision 1.7  2005/02/17 15:42:54  geo
GG> #@@#
GG> #@@# Changed MSGID_WRONG due to high false negatives
GG> #@@# Added a log keyword. And noted changes made to this file.
GG> #@@# Added BROKE_SPAMWARE from Adam
GG> #@@# Added IDIOT_SPAMER from Jim Hoan
GG> #@@# Added FREE_MONEY from Joe John
GG> #@@# Added TEN_QUESTIONS from 70_sare_freefriends.cf
GG> #@@# Moved MAGIC_WORKS to 70_sare_oem.cf
GG> #@@# Removed TRIES_HARD due to false positives

And as I said, the equivalent change log for this last update to the
header* files would have been close to 100 lines long. Is that what
you would like to see?

Bob Menschel




Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
On Wed, Feb 16, 2005 at 10:27:09PM -0800, Robert Menschel wrote:
>Hello George,
>
>Wednesday, February 16, 2005, 9:38:41 PM, you wrote:
>
>GG> Even if someone doesn't use RDJ, isn't a 2-10 line commented change log
>GG> in the cf file worthwhile?
>
>GG> RDJ is not just for people who want to submit full trust. It can also
>GG> be used to help automate distribution of fully validated enterprise
>GG> rulesets, which happens to be the way I use it to distribute rulesets to
>GG> my clients and is why I've been working with Chris Thielen to develop
>GG> RDJ2, for better enterprise support and features that will help RDJ
>GG> scale to support enterprise, and more clients and ruleset servers.
>
>GG> #@@# A note specifying the recent changes made, with a comment designed
>GG> #@@# for grep-ing would be useful for the casual RDJ user to monitor
>GG> #@@# what automatic updates are doing, as well as function to notify the
>GG> #@@# enterprise admin of the changes he needs to validate, and integrate
>GG> #@@# with his private rules.
>
>GG> If you are going through the trouble of publishing your rulesets for
>GG> the ease of others to use, I don't see the point of forcing them to
>GG> submit trust or manually discover changes when you publish them.
>
>Fair enough.  In this most recent publication of updates to the
>70_sare_header*.cf family, 40 rules were added, and 50 rules were
>moved from one file to another, including 8 or 9 that were moved from
>various files to the archive file.
>
>How would you structure a change history for those changes?

Are the files in cvs? I would suggest using the log keyword:

http://cvsbook.red-bean.com/cvsbook.html#Using%20Keyword%20Expansion
http://cvsbook.red-bean.com/cvsbook.html#Keyword%20Substitution%20(RCS%20Keywords)

and record only the changes for a particular file with the commit for
that file.  Completely new files can be advertised on the website or an
announce list.

For example, with 
KeywordExpand=iLocker,Log
in your CVSROOT/config file
Your next commit will expand 

#@@# $Log$

To the entry you make when you commit, eg:

#@@# $Log: README,v $
#@@# Revision 1.7  2005/02/17 15:42:54  geo
#@@#
#@@# Changed MSGID_WRONG due to high false negatives
#@@# Added a log keyword. And noted changes made to this file.
#@@# Added BROKE_SPAMWARE from Adam
#@@# Added IDIOT_SPAMER from Jim Hoan
#@@# Added FREE_MONEY from Joe John
#@@# Added TEN_QUESTIONS from 70_sare_freefriends.cf
#@@# Moved MAGIC_WORKS to 70_sare_oem.cf
#@@# Removed TRIES_HARD due to false positives
#@@#

And that can be grep-ed and reported by various programs.

The log will grow with each commit, but only the recent entry will be
prepended. So any excessive/old lines can be removed right before your
next commit.

If you're not using cvs, it is defiantly worth taking up (nobody ever
stops using it). Subversion (svn) may be a better choice, but I've not
migrated to it yet. SVN is a re-implementation of cvs with CVS limitations
addressed; not as widely used but I understand there is good discussion
list support.

// George

(BTW cvsup is advertised as the most efficient way to distribute files,
it is widely used to distribute incremental changes to OS source in the
BSD world, and allows easy checkout of particular revisions or revision
sets. That should be straight forward with a cvs ruleset repository in
place. I'm not sure if SVN has a cvsup equivalent.)

-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello George,

Wednesday, February 16, 2005, 9:38:41 PM, you wrote:

GG> Even if someone doesn't use RDJ, isn't a 2-10 line commented change log
GG> in the cf file worthwhile?

GG> RDJ is not just for people who want to submit full trust. It can also
GG> be used to help automate distribution of fully validated enterprise
GG> rulesets, which happens to be the way I use it to distribute rulesets to
GG> my clients and is why I've been working with Chris Thielen to develop
GG> RDJ2, for better enterprise support and features that will help RDJ
GG> scale to support enterprise, and more clients and ruleset servers.

GG> #@@# A note specifying the recent changes made, with a comment designed
GG> #@@# for grep-ing would be useful for the casual RDJ user to monitor
GG> #@@# what automatic updates are doing, as well as function to notify the
GG> #@@# enterprise admin of the changes he needs to validate, and integrate
GG> #@@# with his private rules.

GG> If you are going through the trouble of publishing your rulesets for
GG> the ease of others to use, I don't see the point of forcing them to
GG> submit trust or manually discover changes when you publish them.

Fair enough.  In this most recent publication of updates to the
70_sare_header*.cf family, 40 rules were added, and 50 rules were
moved from one file to another, including 8 or 9 that were moved from
various files to the archive file.

How would you structure a change history for those changes?

Bob Menschel




Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
On Wed, Feb 16, 2005 at 07:34:44PM -0800, Robert Menschel wrote:
>Hello George,
>
>Wednesday, February 16, 2005, 7:06:16 PM, you wrote:
>
>>>GG> Rather than squelching custom site rules, I think it more
>>>GG> appropriate to verbosely report why rules become obsoleted (not
>>>GG> necessarily in the new ruleset). Maybe a changes file for each cf
>>>GG> file is appropriate? You cannot guarantee what or how anything is
>>>GG> done in a local config, let the admin who created it address the
>>>GG> changes too.
>>>
>>>The question then becomes, where to verbosely report these. Putting
>>>into the config file probably doesn't help, since if RDJ sees the
>>>--lint failure, it wipes out the config file that has the change
>>>report, and if RDJ doesn't see a --lint failure, then there's no
>>>reason for an admin to go looking for it.
>>>
>>>I'm open to suggestions.
>
>GG> I ended up looking but not finding a change log at:
>GG> http://www.rulesemporium.com/rules.htm
>
>You'll find them inside the file(s) changed.  In this specific case,
>you'll find them inside 70_sare_header0.cf.
>
>However, compare the previous file's change history to the current
>one.  You'll see that I removed a whole bunch of detail, because the
>change history was simply getting too long.  I'm open to disagreement,
>but for these types of files, I don't think it's appropriate to have a
>change history longer than some of the files themselves.

No, I was only suggesting a single change log _entry_ be in each file,
and a link to the the historical change logs. cf "this release" Per next
paragraph.

>GG> Maybe the way RDJ does the roll back needs be addressed? I know version
>GG> 2 is nearing release, and this wouldn't be difficult to add:  It could
>GG> check the cf file for a grep-able, commented, "this release" changes
>GG> entry, which may include a rules.htm#ChangesVerX url.
>
>GG> Then if some change broke your site, you get a likely indicator why,
>GG> right there beside the roll back commands, near the lint output. And if
>GG> your update is multiple revisions behind, you have a url to get started
>GG> on finding changes at the relevant revision.
>
>But that's assuming the reason --lint fails is explained in the change
>history. Past experience suggests that need not be the case -- --lint
>fails more often because of typo errors rather than because of
>intentional changes.

In this case lint failed because of a change. Though I looked I couldn't
find record of the change that made my configuration broken. That is the
reason for my suggestion, I'm not looking for a better lint, that is
what debug is for.

If I break a config, there is a very good chance I know I did it. If the
published ruleset changes, I'd like a change log entry presented or at
least readily available.



>GG> Actually, I like this proposed way of reporting changes a lot. I've
>GG> always wondered the point of email notifications, "ruleset x has
>GG> changed.." They kinda suggest I should do a diff and figure out if
>GG> everything is really okay. I'd just assume see a change log as part of
>GG> the notification that new rules have been loaded (or that lint prevented
>GG> those changes from happening).
>
>Not everyone uses RDJ.  I personally prefer to do my rule changes by
>hand, after I've validated themselves against my own corpus. The email
>notifications are more for people like me than the people who use RDJ
>and want rule updates to be completely automatic.

Even if someone doesn't use RDJ, isn't a 2-10 line commented change log
in the cf file worthwhile?

RDJ is not just for people who want to submit full trust. It can also
be used to help automate distribution of fully validated enterprise
rulesets, which happens to be the way I use it to distribute rulesets to
my clients and is why I've been working with Chris Thielen to develop
RDJ2, for better enterprise support and features that will help RDJ
scale to support enterprise, and more clients and ruleset servers.

#@@# A note specifying the recent changes made, with a comment designed
#@@# for grep-ing would be useful for the casual RDJ user to monitor
#@@# what automatic updates are doing, as well as function to notify the
#@@# enterprise admin of the changes he needs to validate, and integrate
#@@# with his private rules.

If you are going through the trouble of publishing your rulesets for
the ease of others to use, I don't see the point of forcing them to
submit trust or manually discover changes when you publish them.

Regards,
// George


-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello George,

Wednesday, February 16, 2005, 7:06:16 PM, you wrote:

>>GG> Rather than squelching custom site rules, I think it more
>>GG> appropriate to verbosely report why rules become obsoleted (not
>>GG> necessarily in the new ruleset). Maybe a changes file for each cf
>>GG> file is appropriate? You cannot guarantee what or how anything is
>>GG> done in a local config, let the admin who created it address the
>>GG> changes too.
>>
>>The question then becomes, where to verbosely report these. Putting
>>into the config file probably doesn't help, since if RDJ sees the
>>--lint failure, it wipes out the config file that has the change
>>report, and if RDJ doesn't see a --lint failure, then there's no
>>reason for an admin to go looking for it.
>>
>>I'm open to suggestions.

GG> I ended up looking but not finding a change log at:
GG> http://www.rulesemporium.com/rules.htm

You'll find them inside the file(s) changed.  In this specific case,
you'll find them inside 70_sare_header0.cf.

However, compare the previous file's change history to the current
one.  You'll see that I removed a whole bunch of detail, because the
change history was simply getting too long.  I'm open to disagreement,
but for these types of files, I don't think it's appropriate to have a
change history longer than some of the files themselves.

GG> Maybe the way RDJ does the roll back needs be addressed? I know version
GG> 2 is nearing release, and this wouldn't be difficult to add:  It could
GG> check the cf file for a grep-able, commented, "this release" changes
GG> entry, which may include a rules.htm#ChangesVerX url.

GG> Then if some change broke your site, you get a likely indicator why,
GG> right there beside the roll back commands, near the lint output. And if
GG> your update is multiple revisions behind, you have a url to get started
GG> on finding changes at the relevant revision.

But that's assuming the reason --lint fails is explained in the change
history. Past experience suggests that need not be the case -- --lint
fails more often because of typo errors rather than because of
intentional changes.

GG> Actually, I like this proposed way of reporting changes a lot. I've
GG> always wondered the point of email notifications, "ruleset x has
GG> changed.." They kinda suggest I should do a diff and figure out if
GG> everything is really okay. I'd just assume see a change log as part of
GG> the notification that new rules have been loaded (or that lint prevented
GG> those changes from happening).

Not everyone uses RDJ.  I personally prefer to do my rule changes by
hand, after I've validated themselves against my own corpus. The email
notifications are more for people like me than the people who use RDJ
and want rule updates to be completely automatic.

Bob Menschel




Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
On Wed, Feb 16, 2005 at 05:19:38PM -0800, Robert Menschel wrote:
>Hello George,
>
>Wednesday, February 16, 2005, 7:02:58 AM, you wrote:
>
>GG> Rather than squelching custom site rules, I think it more
>GG> appropriate to verbosely report why rules become obsoleted (not
>GG> necessarily in the new ruleset). Maybe a changes file for each cf
>GG> file is appropriate? You cannot guarantee what or how anything is
>GG> done in a local config, let the admin who created it address the
>GG> changes too.
>
>The question then becomes, where to verbosely report these. Putting
>into the config file probably doesn't help, since if RDJ sees the
>--lint failure, it wipes out the config file that has the change
>report, and if RDJ doesn't see a --lint failure, then there's no
>reason for an admin to go looking for it.
>
>I'm open to suggestions.

I ended up looking but not finding a change log at:
http://www.rulesemporium.com/rules.htm

Maybe the way RDJ does the roll back needs be addressed? I know version
2 is nearing release, and this wouldn't be difficult to add:  It could
check the cf file for a grep-able, commented, "this release" changes
entry, which may include a rules.htm#ChangesVerX url.

Then if some change broke your site, you get a likely indicator why,
right there beside the roll back commands, near the lint output. And if
your update is multiple revisions behind, you have a url to get started
on finding changes at the relevant revision.

Actually, I like this proposed way of reporting changes a lot. I've
always wondered the point of email notifications, "ruleset x has
changed.." They kinda suggest I should do a diff and figure out if
everything is really okay. I'd just assume see a change log as part of
the notification that new rules have been loaded (or that lint prevented
those changes from happening).

// George


-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello George,

Wednesday, February 16, 2005, 7:02:58 AM, you wrote:

>>GG> I had custom scores for them but they seem gone now...
>>They've been archived -- moved to the 70_sare_header_arc.cf file.

GG> okay next time I'll look at raising my rule count rather than
GG> raising scores to block spam.

not sure what you mean by that. IMO if you like a rule, and you're
confident it won't hit ham on your system (after testing it, and after
reviewing our mass-check results), then increasing the score IS the
correct way to make this adjustment.

GG> I've not looked at what meta does, but I'll add this comment: My
GG> local.cf file is parsed after all other cf files, so my first
GG> thought is it's going to override anything set before it (as
GG> intended).

If you were to copy the rule to your local.cf, then yes, you'd
override what we have or don't have.  That's intentional, and why all
SARE rules files are supposed to be prefixed with 7* -- to ensure that
your local.cf does just that.

However, if you have only a score in your local.cf, then what you
override is our score, which would be set to 0. So if we have three
lines in our 70_sare_header0.cf file:
> meta     rulename    0  # always false
> describe rulename    Archived Feb 2005 because it hit too much hame
> score    rulename    0
then a) if you have no overrides, our score 0 applies, and the rule
is ignored. b) if you override the score, then our meta definition
applies, the rule hits nothing, and there's no damage. c) if you
override score and rule, your rule and score apply, and your
description if any.

GG> Rather than squelching custom site rules, I think it more
GG> appropriate to verbosely report why rules become obsoleted (not
GG> necessarily in the new ruleset). Maybe a changes file for each cf
GG> file is appropriate? You cannot guarantee what or how anything is
GG> done in a local config, let the admin who created it address the
GG> changes too.

The question then becomes, where to verbosely report these. Putting
into the config file probably doesn't help, since if RDJ sees the
--lint failure, it wipes out the config file that has the change
report, and if RDJ doesn't see a --lint failure, then there's no
reason for an admin to go looking for it.

I'm open to suggestions.

Bob Menschel




Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
Hi,

On Tue, Feb 15, 2005 at 09:03:14PM -0800, Robert Menschel wrote:

>GG> Lint output: warning: score set for non-existent rule SARE_MSGID_IP
>GG> warning: score set for non-existent rule SARE_TOCC_NONE
>GG> lint: 2 issues detected.  please rerun with debug enabled for more information.
>
>GG> I had custom scores for them but they seem gone now...
>
>They've been archived -- moved to the 70_sare_header_arc.cf file.
>
>In our latest mass-check runs, SARE_MSGID_IP hit over 600 ham, and
>SARE_TOCC_NONE hit over 200 ham. Neither rule hit spam/ham at a 90%
>ratio. I archive rules when they hit so much ham and too few spam to
>make it worth while.

okay next time I'll look at raising my rule count rather than raising
scores to block spam.

>Conflicting requirements:
>
>1) We need to be able to archive rules that no longer benefit the
>filters.
>
>2) We don't want to break systems that have custom scores for these
>rules and use RDJ to update them.
>
>An "easy" solution would be to replace the archived rules with a quick
>"no-op" rule which is always false, maybe something like
>> meta     SARE_MSGID_IP     HTML_MESSAGE && !HTML_MESSAGE

I've not looked at what meta does, but I'll add this comment:
My local.cf file is parsed after all other cf files, so my first
thought is it's going to override anything set before it (as intended).

Rather than squelching custom site rules, I think it more appropriate
to verbosely report why rules become obsoleted (not necessarily in the
new ruleset). Maybe a changes file for each cf file is appropriate? You
cannot guarantee what or how anything is done in a local config, let the
admin who created it address the changes too.

// George

-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello George,

Tuesday, February 15, 2005, 7:29:07 PM, you wrote:

GG> On Sat, Feb 12, 2005 at 02:26:02PM -0800, Robert Menschel wrote:
>>Just a quick note that SARE's header rules files have been updated.
>>Information concerning these rules files can be found at
>>http://www.rulesemporium.com/rules.htm#header

GG> BTW - know anything about SARE_MSGID_IP and SARE_TOCC_NONE from
GG> 70_sare_header.cf?

Yep.

GG> Lint output: warning: score set for non-existent rule SARE_MSGID_IP
GG> warning: score set for non-existent rule SARE_TOCC_NONE
GG> lint: 2 issues detected.  please rerun with debug enabled for more information.

GG> I had custom scores for them but they seem gone now...

They've been archived -- moved to the 70_sare_header_arc.cf file.

In our latest mass-check runs, SARE_MSGID_IP hit over 600 ham, and
SARE_TOCC_NONE hit over 200 ham. Neither rule hit spam/ham at a 90%
ratio. I archive rules when they hit so much ham and too few spam to
make it worth while.

Conflicting requirements:

1) We need to be able to archive rules that no longer benefit the
filters.

2) We don't want to break systems that have custom scores for these
rules and use RDJ to update them.

An "easy" solution would be to replace the archived rules with a quick
"no-op" rule which is always false, maybe something like
> meta     SARE_MSGID_IP     HTML_MESSAGE && !HTML_MESSAGE

Does anyone have any better ideas how to handle this type of
situation, and/or any suggests as the best/fastest/efficientest false
noop rule to use?

Bob Menschel




Re: [SARE] header rules updated

Posted by George Georgalis <ge...@galis.org>.
On Sat, Feb 12, 2005 at 02:26:02PM -0800, Robert Menschel wrote:
>Just a quick note that SARE's header rules files have been updated.
>
>Information concerning these rules files can be found at
>http://www.rulesemporium.com/rules.htm#header
>

BTW - know anything about SARE_MSGID_IP and SARE_TOCC_NONE from
70_sare_header.cf?

***WARNING***: spamassassin --lint failed.
Rolling configuration files back, not restarting SpamAssassin.
Rollback command is:  mv -f /etc/spamassassin/70_sare_header.cf
/etc/spamassassin/RulesDuJour/70_sare_header.cf.2; mv -f
/etc/spamassassin/RulesDuJour/70_sare_header.cf.20050214-1634 /etc/spamassassin/70_sare_header.cf;

Lint output: warning: score set for non-existent rule SARE_MSGID_IP
warning: score set for non-existent rule SARE_TOCC_NONE
lint: 2 issues detected.  please rerun with debug enabled for more information.

I had custom scores for them but they seem gone now...

// George


-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org

Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello Jim,

Saturday, February 12, 2005, 5:12:05 PM, you wrote:

JK> danke für die Email vom 13.02.2005 um 00:48
JK> Robert Menschel schrieb - wrote:

JK>>> is this only for SA 3.*? Or as well for 2.64?
RM>> There is one file that does not apply to 2.64
JK> and which is it? I can`t see.

Head to http://www.rulesemporium.com/rules.htm#header and read the
information in the "Note" section. You'll find that
> 70_sare_header_x264_x30.cf contains header rules which have been
> merged into the SpamAssassin distribution rule set in version 2.64.
> These rules are also part of the distribution rule set in version
> 3.0.0. Systems which have upgraded to either of these two versions
> should not use this file. Systems running any 2.5x or 2.60 through
> 2.63 version should benefit from these rules.

Bob Menschel




Re: [SARE] header rules updated

Posted by Jim Knuth <jk...@jkart.de>.
Hallo und guten Morgen Robert,

danke für die Email vom 13.02.2005 um 00:48
Robert Menschel schrieb - wrote:


JK>> is this only for SA 3.*? Or as well for 2.64?

> There is one file that does not apply to 2.64

and which is it? I can`t see.


-- 
Viele Grüße, Kind regards,
 Jim Knuth
 jk@jkart.de
 ICQ #277289867
 Skype: callto://jimknuth
----------
Zufalls-Zitat
----------
Es ist möglich, eine Kuh eine Treppe heraufzuführen. Unmöglich ist es
jedoch, sie die Treppe wieder herunterzuführen.
----------
Dieser Text hat nichts mit dem Empfänger der Mail zu tun
----------
    
virengeprüft mit NOD32 Version 1.998 Update 12.02.2005


Re[2]: [SARE] header rules updated

Posted by Robert Menschel <Ro...@Menschel.net>.
Hello Jim,

Saturday, February 12, 2005, 2:57:06 PM, you wrote:

JK> danke für die Email vom 12.02.2005 um 23:26
JK> Robert Menschel schrieb - wrote:

>> Just a quick note that SARE's header rules files have been updated.

>> Information concerning these rules files can be found at
>> http://www.rulesemporium.com/rules.htm#header

JK> is this only for SA 3.*? Or as well for 2.64?

There are multiple files in this family of files. Almost all apply to
all version of SA 2.5x and up.

There is one file that does not apply to 2.64 (its rules were
incorporated into 2.64 when released), and another that does not apply
to 3.0 (same reason, but this file should be very useful to a 2.64
system), but all other files can be used with any 2.5x+ version.

In Germany you'll also want to stay away from the
70_sare_header_eng.cf file, since those rules work well only for
English language emails.

Bob Menschel




Re: [SARE] header rules updated

Posted by Jim Knuth <jk...@jkart.de>.
Hallo und guten Abend Robert,

danke für die Email vom 12.02.2005 um 23:26
Robert Menschel schrieb - wrote:

> Just a quick note that SARE's header rules files have been updated.

> Information concerning these rules files can be found at
> http://www.rulesemporium.com/rules.htm#header


is this only for SA 3.*? Or as well for 2.64?

-- 
Viele Grüße, Kind regards,
 Jim Knuth
 jk@jkart.de
 ICQ #277289867
 Skype: callto://jimknuth
----------
Zufalls-Zitat
----------
"wo kämen wir hin, wenn alle sagen würden wo kämen wir hin,
und niemand gehen würde um zu schauen, wohin wir kommen,
wenn wir gehen" (Kurt Marti *1921)
----------
Dieser Text hat nichts mit dem Empfänger der Mail zu tun
----------
    
virengeprüft mit NOD32 Version 1.998 Update 12.02.2005