You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Jezz <je...@fairbrass.co.nz> on 2011/06/10 18:07:40 UTC
Sought rules
Hi all,
I recently upgraded SpamAssassin from 3.2.5 to 3.3.1, and I discovered that
the JM_SOUGHT_FRAUD_x rules are now included within the official ruleset,
within the 72_active.cf file.
However, as far as I can tell, these rules seem to be different to the
same-named rules that are within the latest copy of 20_sought_fraud.cf which
is downloaded from the sought.rules.yerp.org channel. Which is to say, the
contents of the 'meta' rule is different between these two files. My guess
is that the version of these rules contained inside 72_active.cf is perhaps
an older version than the ones inside 20_sought_fraud.cf. Is that the case?
What's more, I also see that these three FRAUD rules all have a score of 0
inside 50_scores.cf. My first question then is why they are zeroed out?
Secondly, I'm wondering how I can enable these rules again if I do want to
use them. In other words, if I want to use the latest version contained
within 20_sought_fraud.cf - I don't see how this could be possible.
Certainly I can add 'score' values for those three rules into my local.cf
file, which will override the zeroed-out scores in 50_scores.cf file.
However, because 72_active.cf comes numerically after 20_sought_fraud.cf,
that means the (assumedly older) FRAUD rules inside 72_active.cf will
override the (assumedly newer) FRAUD rules inside 20_sought_fraud.cf -
right? If so, there's no way for me to use the rules from 20_sought_fraud.cf
at all?
I hope that makes sense, and sorry if I'm completely overlooking something
here.
- Jezz
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2011-06-13 at 10:56 -0400, Bowie Bailey wrote:
> On 6/10/2011 8:01 PM, Karsten Bräckelmann wrote:
> > IFF you use the sought channel with SA 3.3.x, you will need the reorder
> > hack to bend the alphabet.
>
> Would it be worthwhile to change the main code instead? Rather than
> making a change to ensure that the sought channel is processed after
> updates, why not ensure that the main updates file is always processed
> first regardless of the naming of any other third party rules? You
> could either make it a special case in the code that processes the .cf
> files, or you could change the filename to 0_updates_spamassassin_org.cf.
>
> This would not only fix the current issue with the sought rules, it
> would also avoid future problems like this.
Thought about a "stock always comes first" code change myself, but this
won't help immediately. It will not get into the upcoming 3.3.2 release,
and moreover is very unlikely to make it into any version before 3.4.0.
The problem remains for 3.3.x -- until one of the next stock rules
updates, which will have the Sought FRAUD patterns dropped.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by Bowie Bailey <Bo...@BUC.com>.
On 6/10/2011 8:01 PM, Karsten Bräckelmann wrote:
>
> Until the frequent re-scoring and stock rules updates are working
> properly, you will have to use the dedicated channel, if you want the
> SOUGHT rules. And even after that, if you want the freshest patterns,
> you still need the sought channel -- unless stock rules are updated once
> a day.
>
> IFF you use the sought channel with SA 3.3.x, you will need the reorder
> hack to bend the alphabet.
Would it be worthwhile to change the main code instead? Rather than
making a change to ensure that the sought channel is processed after
updates, why not ensure that the main updates file is always processed
first regardless of the naming of any other third party rules? You
could either make it a special case in the code that processes the .cf
files, or you could change the filename to 0_updates_spamassassin_org.cf.
This would not only fix the current issue with the sought rules, it
would also avoid future problems like this.
--
Bowie
Remove Sought Rules from sa-update channel
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 6/10/2011 9:38 PM, Yet Another Ninja wrote:
> On 2011-06-11 3:38, Warren Togami Jr. wrote:
>> On 6/10/2011 3:34 PM, John Hardin wrote:
>>> On Fri, 10 Jun 2011, Lawrence @ Rogers wrote:
>>>
>>>> On 10/06/2011 10:24 PM, Warren Togami Jr. wrote:
>>>>> On 6/10/2011 2:01 PM, Karsten Bräckelmann wrote:
>>>>> > > IFF you use the sought channel with SA 3.3.x, you will need the
>>>>> reorder
>>>>> > hack to bend the alphabet.
>>>>>
>>>>> It is not entirely clear to me, what exactly are you supposed to
>>>>> rename
>>>>> for the reorder hack? You have to do it every time you sa-update?
>>>>
>>>> Would renaming 20_sought_fraud.cf to 99_sought_fraud.cf, putting
>>>> 20_sought_fraud.cf (from the yelp.org channel) after 72_active.cf (the
>>>> default and assumed older SA rules) solve this problem?
>>>
>>> Or symlinks from your local configs directory to the SOUGHT channel
>>> directory files. That would probably be easier to not forget about when
>>> things get fixed.
>>>
>>
>> Is Lawrence's suggestion something we can do upstream to fix this
>> problem?
>>
>> Alternatively, I think it is a mistake for us to ship SOUGHT rules at
>> all in the standard sa-update channel. That is, unless we plan on
>> updating the patterns and scores of SOUGHT on a daily basis. I highly
>> doubt we will do that.
>
> Agreed, I'm +1 with removing SOUGHT rules from mainstream sa update
> channel. and keep thems separate.
>
+1
I guess we should put that into the script that generates the
auto-updates? Where is that script, and on what host does it run? Daryl?
Warren Togami
warren@togami.com
Re: Sought rules
Posted by Yet Another Ninja <ax...@gmail.com>.
On 2011-06-11 3:38, Warren Togami Jr. wrote:
> On 6/10/2011 3:34 PM, John Hardin wrote:
>> On Fri, 10 Jun 2011, Lawrence @ Rogers wrote:
>>
>>> On 10/06/2011 10:24 PM, Warren Togami Jr. wrote:
>>>> On 6/10/2011 2:01 PM, Karsten Bräckelmann wrote:
>>>> > > IFF you use the sought channel with SA 3.3.x, you will need the
>>>> reorder
>>>> > hack to bend the alphabet.
>>>>
>>>> It is not entirely clear to me, what exactly are you supposed to rename
>>>> for the reorder hack? You have to do it every time you sa-update?
>>>
>>> Would renaming 20_sought_fraud.cf to 99_sought_fraud.cf, putting
>>> 20_sought_fraud.cf (from the yelp.org channel) after 72_active.cf (the
>>> default and assumed older SA rules) solve this problem?
>>
>> Or symlinks from your local configs directory to the SOUGHT channel
>> directory files. That would probably be easier to not forget about when
>> things get fixed.
>>
>
> Is Lawrence's suggestion something we can do upstream to fix this problem?
>
> Alternatively, I think it is a mistake for us to ship SOUGHT rules at
> all in the standard sa-update channel. That is, unless we plan on
> updating the patterns and scores of SOUGHT on a daily basis. I highly
> doubt we will do that.
Agreed, I'm +1 with removing SOUGHT rules from mainstream sa update
channel. and keep thems separate.
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-06-10 at 15:38 -1000, Warren Togami Jr. wrote:
> > > Would renaming 20_sought_fraud.cf to 99_sought_fraud.cf, putting
> > > 20_sought_fraud.cf (from the yelp.org channel) after 72_active.cf (the
> > > default and assumed older SA rules) solve this problem?
> Is Lawrence's suggestion something we can do upstream to fix this problem?
No. Please carefully re-read all my previous posts. As I already
explained, the names (and numbers) are meaningless in this context,
because they are in different directories.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 6/10/2011 3:34 PM, John Hardin wrote:
> On Fri, 10 Jun 2011, Lawrence @ Rogers wrote:
>
>> On 10/06/2011 10:24 PM, Warren Togami Jr. wrote:
>>> On 6/10/2011 2:01 PM, Karsten Bräckelmann wrote:
>>> > > IFF you use the sought channel with SA 3.3.x, you will need the
>>> reorder
>>> > hack to bend the alphabet.
>>>
>>> It is not entirely clear to me, what exactly are you supposed to rename
>>> for the reorder hack? You have to do it every time you sa-update?
>>
>> Would renaming 20_sought_fraud.cf to 99_sought_fraud.cf, putting
>> 20_sought_fraud.cf (from the yelp.org channel) after 72_active.cf (the
>> default and assumed older SA rules) solve this problem?
>
> Or symlinks from your local configs directory to the SOUGHT channel
> directory files. That would probably be easier to not forget about when
> things get fixed.
>
Is Lawrence's suggestion something we can do upstream to fix this problem?
Alternatively, I think it is a mistake for us to ship SOUGHT rules at
all in the standard sa-update channel. That is, unless we plan on
updating the patterns and scores of SOUGHT on a daily basis. I highly
doubt we will do that.
Warren Togami
warren@togami.com
Re: Sought rules
Posted by John Hardin <jh...@impsec.org>.
On Fri, 10 Jun 2011, Lawrence @ Rogers wrote:
> On 10/06/2011 10:24 PM, Warren Togami Jr. wrote:
>> On 6/10/2011 2:01 PM, Karsten Bräckelmann wrote:
>> >
>> > IFF you use the sought channel with SA 3.3.x, you will need the reorder
>> > hack to bend the alphabet.
>>
>> It is not entirely clear to me, what exactly are you supposed to rename
>> for the reorder hack? You have to do it every time you sa-update?
>
> Would renaming 20_sought_fraud.cf to 99_sought_fraud.cf, putting
> 20_sought_fraud.cf (from the yelp.org channel) after 72_active.cf (the
> default and assumed older SA rules) solve this problem?
Or symlinks from your local configs directory to the SOUGHT channel
directory files. That would probably be easier to not forget about when
things get fixed.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
It is not the place of government to make right every tragedy and
woe that befalls every resident of the nation.
-----------------------------------------------------------------------
183 days since the first successful private orbital launch (SpaceX)
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-06-10 at 22:40 -0230, Lawrence @ Rogers wrote:
> Would renaming 20_sought_fraud.cf to 99_sought_fraud.cf, putting
> 20_sought_fraud.cf (from the yelp.org channel) after 72_active.cf (the
> default and assumed older SA rules) solve this problem?
No, because they are in sub-directories. Not parsed directly, but
included. The order of the cf files including them matters. See my
previous post, as well as the older ones.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by "Lawrence @ Rogers" <la...@nl.rogers.com>.
On 10/06/2011 10:24 PM, Warren Togami Jr. wrote:
> On 6/10/2011 2:01 PM, Karsten Bräckelmann wrote:
>>
>> IFF you use the sought channel with SA 3.3.x, you will need the reorder
>> hack to bend the alphabet.
>>
>
> It is not entirely clear to me, what exactly are you supposed to
> rename for the reorder hack? You have to do it every time you sa-update?
>
> Warren
>
Would renaming 20_sought_fraud.cf to 99_sought_fraud.cf, putting
20_sought_fraud.cf (from the yelp.org channel) after 72_active.cf (the
default and assumed older SA rules) solve this problem?
Regards,
Lawrence
Re: Sought rules
Posted by Michael Scheidell <mi...@secnap.com>.
On 6/11/11 6:45 AM, Michael Scheidell wrote:
> On 6/10/11 9:56 PM, Karsten Bräckelmann wrote:
>> spamassassin -D config --lint 2>&1 | less
> so, one MORE option, we don't need to add the symlink to crontab?
>
> Jun 11 06:39:13.419 [71425] dbg: config: read file
> /var/db/spamassassin/3.003001/sought_rules_yerp_org.cf
> Jun 11 06:39:13.419 [71425] dbg: config: read file
> /var/db/spamassassin/3.003001/updates_spamassassin_org.cf
> Jun 11 06:39:13.420 [71425] dbg: config: using
> "/usr/local/etc/mail/spamassassin" for site rules dir
>
>
> we could put it into out ../etc/mail/spamassassin dir?
>
> to summarize:
>
>
>
> On 6/10/11 1:14 PM, Karsten Bräckelmann wrote:
>> This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
>> in your rule updates dir, pointing at the channel generated cf should
(ie directory?)
>> do. These channel cf files only hold include statements, to pull in the
>> actual cf files in the per-channel dir.
>>
> (or freebsd, your MMV)
>
> cd ../updates/spamassassin_org.cf (or ../etc/mail/spamassassin)
> ln -s /var/db/spamassassin/{ver}/sought_rules_yerp_org.cf
> /var/db/spamassassin/{ver}/updates_spamassassin_org.cf/zzz_sought.cf
>
no, because (at least on freebsd) you can't symlink a file to a directory.
ln:
/var/db/spamassassin/3.003001/updates_spamassassin_org.cf/zzz_sought.cf:
Not a directory
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Sat, 2011-06-11 at 12:50 +0100, Arthur Dent wrote:
> drwxrwxr-x. 2 root root 4096 Jun 11 12:19 sought_rules_yerp_org
> -rw-rw-r--. 1 root root 120 Jun 11 12:19 sought_rules_yerp_org.cf
> drwxr-xr-x. 2 root root 4096 Jun 11 12:17 updates_spamassassin_org
> -rw-r--r--. 1 root root 2599 Jun 11 12:17 updates_spamassassin_org.cf
> lrwxrwxrwx. 1 root root 24 Jun 11 12:40 zzz_sought_rules_yerp_org.cf -> sought_rules_yerp_org.cf
>
> I am now sorted - Is that right?
Yes.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by Arthur Dent <mi...@blueyonder.co.uk>.
On Sat, 2011-06-11 at 06:45 -0400, Michael Scheidell wrote:
> On 6/10/11 9:56 PM, Karsten Bräckelmann wrote:
> > spamassassin -D config --lint 2>&1 | less
> so, one MORE option, we don't need to add the symlink to crontab?
>
> Jun 11 06:39:13.419 [71425] dbg: config: read file
> /var/db/spamassassin/3.003001/sought_rules_yerp_org.cf
> Jun 11 06:39:13.419 [71425] dbg: config: read file
> /var/db/spamassassin/3.003001/updates_spamassassin_org.cf
> Jun 11 06:39:13.420 [71425] dbg: config: using
> "/usr/local/etc/mail/spamassassin" for site rules dir
>
>
> we could put it into out ../etc/mail/spamassassin dir?
>
> to summarize:
>
>
>
> On 6/10/11 1:14 PM, Karsten Bräckelmann wrote:
> > This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
> > in your rule updates dir, pointing at the channel generated cf should
> > do. These channel cf files only hold include statements, to pull in the
> > actual cf files in the per-channel dir.
> >
> (or freebsd, your MMV)
>
> cd ../updates/spamassassin_org.cf (or ../etc/mail/spamassassin)
> ln -s /var/db/spamassassin/{ver}/sought_rules_yerp_org.cf
> /var/db/spamassassin/{ver}/updates_spamassassin_org.cf/zzz_sought.cf
>
> right? scores are ok now?
OK - Forgive my ignorance, I just want to be sure about this...
On my F15 system, in my /var/lib/spamassassin/3.003002/ directory I have
the following:
# ls -l /var/lib/spamassassin/3.003002/
drwxrwxr-x. 2 root root 4096 Jun 11 12:19 sought_rules_yerp_org
-rw-rw-r--. 1 root root 120 Jun 11 12:19 sought_rules_yerp_org.cf
drwxr-xr-x. 2 root root 4096 Jun 11 12:17 updates_spamassassin_org
-rw-r--r--. 1 root root 2599 Jun 11 12:17 updates_spamassassin_org.cf
Inside sought_rules_yerp_org.cf is the following:
# cat /var/lib/spamassassin/3.003002/sought_rules_yerp_org.cf
# UPDATE version 3301124446
include sought_rules_yerp_org/20_sought.cf
include sought_rules_yerp_org/20_sought_fraud.cf
I create a symlink of the sought_rules cf file to a file called
zzz.something.cf
ln -s sought_rules_yerp_org.cf zzz_sought_rules_yerp_org.cf
I now have this:
drwxrwxr-x. 2 root root 4096 Jun 11 12:19 sought_rules_yerp_org
-rw-rw-r--. 1 root root 120 Jun 11 12:19 sought_rules_yerp_org.cf
drwxr-xr-x. 2 root root 4096 Jun 11 12:17 updates_spamassassin_org
-rw-r--r--. 1 root root 2599 Jun 11 12:17 updates_spamassassin_org.cf
lrwxrwxrwx. 1 root root 24 Jun 11 12:40 zzz_sought_rules_yerp_org.cf -> sought_rules_yerp_org.cf
I am now sorted - Is that right?
Re: Sought rules
Posted by John Hardin <jh...@impsec.org>.
On Sat, 11 Jun 2011, Michael Scheidell wrote:
> On 6/10/11 9:56 PM, Karsten Bräckelmann wrote:
>> spamassassin -D config --lint 2>&1 | less
> so, one MORE option, we don't need to add the symlink to crontab?
>
> Jun 11 06:39:13.419 [71425] dbg: config: read file
> /var/db/spamassassin/3.003001/sought_rules_yerp_org.cf
> Jun 11 06:39:13.419 [71425] dbg: config: read file
> /var/db/spamassassin/3.003001/updates_spamassassin_org.cf
> Jun 11 06:39:13.420 [71425] dbg: config: using
> "/usr/local/etc/mail/spamassassin" for site rules dir
>
>
> we could put it into out ../etc/mail/spamassassin dir?
That's what I suggested. Putting it into the local customizations dir
keeps it visible. If the symlink is buried down under /var/mumble then you
may forget about it and not remove it once we get the sa-update process
straightened out.
> right? scores are ok now?
This isn't to fix the scores, it's to ensure that you're using the latest
generated SOUGHT rules.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Our government should bear in mind the fact that the American
Revolution was touched off by the then-current government
attempting to confiscate firearms from the people.
-----------------------------------------------------------------------
184 days since the first successful private orbital launch (SpaceX)
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Sat, 2011-06-11 at 06:45 -0400, Michael Scheidell wrote:
> On 6/10/11 1:14 PM, Karsten Bräckelmann wrote:
> > This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
> > in your rule updates dir, pointing at the channel generated cf should
> > do. These channel cf files only hold include statements, to pull in the
> > actual cf files in the per-channel dir.
> ln -s /var/db/spamassassin/{ver}/sought_rules_yerp_org.cf
> /var/db/spamassassin/{ver}/updates_spamassassin_org.cf/zzz_sought.cf
^^^
That's a file, not a directory.
For each update channel, there is an appropriately named directory and a
cf file. The per-channel sub-directories are not parsed for cf files.
The cf files in the updatedir itself are used -- and they consist of a
bunch of include directives, pulling in their channels contents.
# cd /var/lib/spamassassin/$VERSION
# ln -s sought_rules_yerp_org.cf z-INCLUDE-LATE.cf
See 'man sa-update' for the actual updatedir used by your OS / distro.
As a result, you'll get something like this.
drwxr-xr-x sought_rules_yerp_org/
-rw-r--r-- sought_rules_yerp_org.cf
drwxr-xr-x updates_spamassassin_org/
-rw-r--r-- updates_spamassassin_org.cf
lrwxrwxrwx z-INCLUDE-LATE.cf -> sought_rules_yerp_org.cf
To verify the SOUGHT rules are indeed included a second time, after the
stock rule-set, have a look at the debug output.
$ spamassassin --lint -D config 2>&1 | grep 'read file /var/' | less
You'll see the per-channel cf files for sought, stock, and the symlink.
Followed by their respective channel contents, included from these
files. So you'll see sought actually being read twice, before and after
stock.
The symlink needs to be created exactly once, after each SA upgrade,
since the updatedir is versioned.
Also, it should be possible to symlink from your site config dir, or
even use an include statement in a site config cf file. Didn't test
that, though. However, this too needs to be updated after each SA
upgrade.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by Michael Scheidell <mi...@secnap.com>.
On 6/10/11 9:56 PM, Karsten Bräckelmann wrote:
> spamassassin -D config --lint 2>&1 | less
so, one MORE option, we don't need to add the symlink to crontab?
Jun 11 06:39:13.419 [71425] dbg: config: read file
/var/db/spamassassin/3.003001/sought_rules_yerp_org.cf
Jun 11 06:39:13.419 [71425] dbg: config: read file
/var/db/spamassassin/3.003001/updates_spamassassin_org.cf
Jun 11 06:39:13.420 [71425] dbg: config: using
"/usr/local/etc/mail/spamassassin" for site rules dir
we could put it into out ../etc/mail/spamassassin dir?
to summarize:
On 6/10/11 1:14 PM, Karsten Bräckelmann wrote:
> This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
> in your rule updates dir, pointing at the channel generated cf should
> do. These channel cf files only hold include statements, to pull in the
> actual cf files in the per-channel dir.
>
(or freebsd, your MMV)
cd ../updates/spamassassin_org.cf (or ../etc/mail/spamassassin)
ln -s /var/db/spamassassin/{ver}/sought_rules_yerp_org.cf
/var/db/spamassassin/{ver}/updates_spamassassin_org.cf/zzz_sought.cf
right? scores are ok now?
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
READ THIS Re: Sought rules
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 6/10/2011 11:13 PM, Warren Togami Jr. wrote:
> Wait a sec, I'm confused about this. "JM_SOUGHT_2 hitting on every legit
> Facebook message" on dev@ list February 17th 2011. If the SOUGHT channel
> was being overridden by the sa-update rules, how would this problem
> appear from the SOUGHT channel? Doesn't this suggest that spamassassin
> was successfully using the SOUGHT channel?
>
> (I still agree we should remove the static SOUGHT from the sa-update
> rules.)
>
> Warren
NOTE: I'm skeptical that this bug is effecting us on Linux. I am not
using a re-order hack and yet SOUGHT seems to be changing its behavior
on a daily basis.
Warren
Re: Sought rules
Posted by Justin Mason <jm...@jmason.org>.
On Sunday, June 12, 2011, Warren Togami Jr. <wt...@gmail.com> wrote:
> On 6/12/2011 12:32 AM, Warren Togami Jr. wrote:
>
> On 6/11/2011 10:03 AM, Justin Mason wrote:
>
> guys -- I'm going to make the whole question moot (in trunk at least)
> -- the only reason SOUGHT and SOUGHT_FRAUD were being checked in there
> was to make their accuracy visible in ruleqa. It's been months since
> I've looked at that, so it's needless. I'll remove them from svn
> asap.
>
> --j.
>
>
> WAIT!!! Wouldn't this remove our ability to check for false positives of
> your patterns against the much larger ham collection of nightly masscheck?
>
>
> The alternative is to filter SOUGHT from the sa-update rule updates with a script, but still allow it in the nightly masschecks. Testing sought in nightly masschecks has been useful to occasionally find obvious SOUGHT problems, or sometimes to locate spam that was misplaced in the ham folder.
Sure -- if that's preferable we can do that. We just need to be
diligent about tflags Nopublish.
> Warren
>
Re: Sought rules
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 6/12/2011 12:32 AM, Warren Togami Jr. wrote:
> On 6/11/2011 10:03 AM, Justin Mason wrote:
>> guys -- I'm going to make the whole question moot (in trunk at least)
>> -- the only reason SOUGHT and SOUGHT_FRAUD were being checked in there
>> was to make their accuracy visible in ruleqa. It's been months since
>> I've looked at that, so it's needless. I'll remove them from svn
>> asap.
>>
>> --j.
>
> WAIT!!! Wouldn't this remove our ability to check for false positives of
> your patterns against the much larger ham collection of nightly masscheck?
The alternative is to filter SOUGHT from the sa-update rule updates with
a script, but still allow it in the nightly masschecks. Testing sought
in nightly masschecks has been useful to occasionally find obvious
SOUGHT problems, or sometimes to locate spam that was misplaced in the
ham folder.
Warren
Re: Sought rules
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 6/11/2011 10:03 AM, Justin Mason wrote:
> guys -- I'm going to make the whole question moot (in trunk at least)
> -- the only reason SOUGHT and SOUGHT_FRAUD were being checked in there
> was to make their accuracy visible in ruleqa. It's been months since
> I've looked at that, so it's needless. I'll remove them from svn
> asap.
>
> --j.
WAIT!!! Wouldn't this remove our ability to check for false positives of
your patterns against the much larger ham collection of nightly masscheck?
I wouldn't be concerned about this if there were a way to collaborate on
feeding more ham into SOUGHT's safety check corpus, but when I asked
about this earlier you seemed hesitant.
Warren
Re: Sought rules
Posted by Justin Mason <jm...@jmason.org>.
guys -- I'm going to make the whole question moot (in trunk at least)
-- the only reason SOUGHT and SOUGHT_FRAUD were being checked in there
was to make their accuracy visible in ruleqa. It's been months since
I've looked at that, so it's needless. I'll remove them from svn
asap.
--j.
2011/6/11 Karsten Bräckelmann <gu...@rudersport.de>:
> On Fri, 2011-06-10 at 23:13 -1000, Warren Togami Jr. wrote:
>> Wait a sec, I'm confused about this. "JM_SOUGHT_2 hitting on every
>> legit Facebook message" on dev@ list February 17th 2011. If the SOUGHT
>> channel was being overridden by the sa-update rules, how would this
>> problem appear from the SOUGHT channel? Doesn't this suggest that
>> spamassassin was successfully using the SOUGHT channel?
>
> Yes, and no. grep for SOUGHT in the stock rules...
>
> The stock rule-set has a snapshot of the SOUGHT_FRAUD patterns, they do
> NOT have the SOUGHT patterns.
>
>
> And, well... READ THIS instead. ;)
>
> --
> char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
> main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
> (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
>
>
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2011-06-14 at 09:58 -0400, pseudonymous Alex wrote:
> I see that recently Justin made some changes to the svn rules for
> this, but I wasn't sure if that was going to be reflected in the
> channel?
Yes, it will eventually. See the other direct reply by Justin from Sat
to the post you replied to.
> I just noticed that the last update I have is 1083704, from Apr 18th,
> and wasn't sure if I was somehow missing something?
$ host -t TXT 1.3.3.updates.spamassassin.org
1.3.3.updates.spamassassin.org descriptive text "1083704"
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by Alex <my...@gmail.com>.
Hi,
>> Wait a sec, I'm confused about this. "JM_SOUGHT_2 hitting on every
>> legit Facebook message" on dev@ list February 17th 2011. If the SOUGHT
>> channel was being overridden by the sa-update rules, how would this
>> problem appear from the SOUGHT channel? Doesn't this suggest that
>> spamassassin was successfully using the SOUGHT channel?
>
> Yes, and no. grep for SOUGHT in the stock rules...
>
> The stock rule-set has a snapshot of the SOUGHT_FRAUD patterns, they do
> NOT have the SOUGHT patterns.
>
>
> And, well... READ THIS instead. ;)
I see that recently Justin made some changes to the svn rules for
this, but I wasn't sure if that was going to be reflected in the
channel?
I just noticed that the last update I have is 1083704, from Apr 18th,
and wasn't sure if I was somehow missing something?
Thanks,
Alex
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-06-10 at 23:13 -1000, Warren Togami Jr. wrote:
> Wait a sec, I'm confused about this. "JM_SOUGHT_2 hitting on every
> legit Facebook message" on dev@ list February 17th 2011. If the SOUGHT
> channel was being overridden by the sa-update rules, how would this
> problem appear from the SOUGHT channel? Doesn't this suggest that
> spamassassin was successfully using the SOUGHT channel?
Yes, and no. grep for SOUGHT in the stock rules...
The stock rule-set has a snapshot of the SOUGHT_FRAUD patterns, they do
NOT have the SOUGHT patterns.
And, well... READ THIS instead. ;)
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by "Warren Togami Jr." <wt...@gmail.com>.
Wait a sec, I'm confused about this. "JM_SOUGHT_2 hitting on every
legit Facebook message" on dev@ list February 17th 2011. If the SOUGHT
channel was being overridden by the sa-update rules, how would this
problem appear from the SOUGHT channel? Doesn't this suggest that
spamassassin was successfully using the SOUGHT channel?
(I still agree we should remove the static SOUGHT from the sa-update rules.)
Warren
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-06-10 at 14:54 -1000, Warren Togami Jr. wrote:
> On 6/10/2011 2:01 PM, Karsten Bräckelmann wrote:
> >
> > IFF you use the sought channel with SA 3.3.x, you will need the reorder
> > hack to bend the alphabet.
>
> It is not entirely clear to me, what exactly are you supposed to rename
> for the reorder hack? You have to do it every time you sa-update?
No, only once.
The problem is, that cf files are parsed in lexical order per directory.
That means that 10_default comes before 72_active. However, have a look
at your updates conf dir (see 'man spamassassin' or 'man sa-update').
There's one directory per channel. And one identically named cf file per
channel. The dirs are not parsed recursively (and it wouldn't matter),
but the cf files are. THEIR order matters. Alphabetically.
Now since "s" is before "u" in any alphabet I've ever used, the sought
channel's cf file is parsed before the stock updates cf file. Both
consist of a bunch of include directives, actually loading the rules' cf
files.
To ensure some channel to be parsed after the general updates channel,
because the channel's content is supposed to overwrite stock, it must
begin with a char that is after "u".
Thus, a single symlink should do. Once. Also see the output of
spamassassin -D config --lint 2>&1 | less
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 6/10/2011 2:01 PM, Karsten Bräckelmann wrote:
>
> IFF you use the sought channel with SA 3.3.x, you will need the reorder
> hack to bend the alphabet.
>
It is not entirely clear to me, what exactly are you supposed to rename
for the reorder hack? You have to do it every time you sa-update?
Warren
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-06-10 at 19:32 -0400, Michael Scheidell wrote:
> On 6/10/11 5:49 PM, Karsten Bräckelmann wrote:
> >
> > While I do agree this is an issue -- at the very least, all third-party
> > sought channel docs should include that note -- I do not agree that this
> > is worrisome. The negative impact basically boils down to "the channel
> > does not work".
>
> so, the 'best practice' is to what?
> symlink/reorder hack? or stop running sought channels, and add your own
> scores?
Adding or adjusting the scores might be a good idea in either case.
Other than that...
Until the frequent re-scoring and stock rules updates are working
properly, you will have to use the dedicated channel, if you want the
SOUGHT rules. And even after that, if you want the freshest patterns,
you still need the sought channel -- unless stock rules are updated once
a day.
IFF you use the sought channel with SA 3.3.x, you will need the reorder
hack to bend the alphabet.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by Michael Scheidell <mi...@secnap.com>.
On 6/10/11 5:49 PM, Karsten Bräckelmann wrote:
>
> While I do agree this is an issue -- at the very least, all third-party
> sought channel docs should include that note -- I do not agree that this
> is worrisome. The negative impact basically boils down to "the channel
> does not work".
>
so, the 'best practice' is to what?
symlink/reorder hack? or stop running sought channels, and add your own
scores?
> Similar to not running sa-update for the main channel. Simply no update.
> No harm otherwise.
>
>
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-06-10 at 11:19 -1000, Warren Togami Jr. wrote:
> On 6/10/2011 7:14 AM, Karsten Bräckelmann wrote:
> > Now, the bad thing about this is that updates_spamassassin_org.cf is
> > lexically *after* sought_rules_yerp_org.cf in your rule update dir.
> > Which means the more recent rules in the dedicated Sought channel are
> > overwritten by the stock rules...
> >
> > This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
> > in your rule updates dir, pointing at the channel generated cf should
> > do. These channel cf files only hold include statements, to pull in the
> > actual cf files in the per-channel dir.
>
> Without a re-ordering hack, does this mean mean that essentially
> EVERYONE is using SOUGHT wrong? This is a bit worrisome.
You'd be closer with a lower-cased "everyone".
This is true since 3.3.0, since the SOUGHT rules got included into the
stock rule-set. It does not affect 3.2.x users.
It does not affect those *not* using the third-party sought channel, but
using the version in the main channel. Granted, those got some rather
stale patterns currently, and are unlikely to hit much recent spam, but
that's a different topic.
It *does* affect anyone using the third-party sought channel explicitly
to get frequent updates of fresh spam-phrase patterns the sought process
extracts. Basically, they just don't get what they wanted, unless...
Unless they participated in the previous thread discussing this issue,
or at least where lurking one way or the other. This topic came up
before, I just summarized the whole thread in my previous answer. :)
While I do agree this is an issue -- at the very least, all third-party
sought channel docs should include that note -- I do not agree that this
is worrisome. The negative impact basically boils down to "the channel
does not work".
Similar to not running sa-update for the main channel. Simply no update.
No harm otherwise.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 6/10/2011 7:14 AM, Karsten Bräckelmann wrote:
>
> You are generally correct about the numerical (actually lexical) order,
> though it doesn't apply to the files you are talking about. The
> mentioned 72_active and 20_sought are in different sa-update channels.
>
> Now, the bad thing about this is that updates_spamassassin_org.cf is
> lexically *after* sought_rules_yerp_org.cf in your rule update dir.
> Which means the more recent rules in the dedicated Sought channel are
> overwritten by the stock rules...
>
> This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
> in your rule updates dir, pointing at the channel generated cf should
> do. These channel cf files only hold include statements, to pull in the
> actual cf files in the per-channel dir.
>
>
Without a re-ordering hack, does this mean mean that essentially
EVERYONE is using SOUGHT wrong? This is a bit worrisome.
Warren
Re: Sought rules
Posted by John Hardin <jh...@impsec.org>.
On Sat, 11 Jun 2011, Jezz wrote:
> So here's the thing: I'm actually running SA on Windows, via the MDaemon
> mail server. So I can't so easily create a symlink as you've described.
http://lifehacker.com/5496652/how-to-use-symlinks-in-windows
But then, this is just FYI as Justin is fixing the problem...
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The third basic rule of firearms safety:
Keep your booger hook off the bang switch!
-----------------------------------------------------------------------
184 days since the first successful private orbital launch (SpaceX)
Re: Sought rules
Posted by Jezz <je...@fairbrass.co.nz>.
"Karsten Bräckelmann" <gu...@rudersport.de> wrote in message
news:1307971023.4766.8.camel@monkey...
> On Mon, 2011-06-13 at 13:21 +0200, Jezz wrote:
>> > Yes, works. Regardless of using symlinks, a copy of the cf file, or
>> > include'ing it from another config file -- the target will be read
>> > again.
>> >
>> > But don't just take my word for it -- see for yourself!
>> >
>> > spamassassin --lint -D config
>
>> Looking at the output in the attached file, I can see the
>> 'sought-rules.cf'
>> file, which I created inside my 'rules' directory, there on line 20. It's
>> getting read *after* the files inside the default_rules directory (lines
>> 7-11), which is good.
>>
>> What's confusing me is that lines 21 onwards show the individual .cf
>> files
>> getting loaded from inside each of the channel subdirectories in
>> default_rules. And this is *after* my sought-rules.cf file gets loaded.
>> So
>> does that mean the stock rules are still overriding the sought rules? Or
>> am
>> I misinterpreting the output here?
>
> Have a close look at the output again. In particular search for all
> lines containing "sought".
>
> You'll see the actual sought channel's contents being loaded -- twice.
> Right before the stock rules, and again as the very last cf files being
> read.
>
> Does that answer the question? ;)
>
>
Oh yeah, look at that - silly me!
**slaps palm on forehead**
:)
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2011-06-13 at 13:21 +0200, Jezz wrote:
> > Yes, works. Regardless of using symlinks, a copy of the cf file, or
> > include'ing it from another config file -- the target will be read
> > again.
> >
> > But don't just take my word for it -- see for yourself!
> >
> > spamassassin --lint -D config
> Looking at the output in the attached file, I can see the 'sought-rules.cf'
> file, which I created inside my 'rules' directory, there on line 20. It's
> getting read *after* the files inside the default_rules directory (lines
> 7-11), which is good.
>
> What's confusing me is that lines 21 onwards show the individual .cf files
> getting loaded from inside each of the channel subdirectories in
> default_rules. And this is *after* my sought-rules.cf file gets loaded. So
> does that mean the stock rules are still overriding the sought rules? Or am
> I misinterpreting the output here?
Have a close look at the output again. In particular search for all
lines containing "sought".
You'll see the actual sought channel's contents being loaded -- twice.
Right before the stock rules, and again as the very last cf files being
read.
Does that answer the question? ;)
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by Jezz <je...@fairbrass.co.nz>.
"Karsten Bräckelmann" <gu...@rudersport.de> wrote in message
news:1307824487.4844.55.camel@monkey...
>
> Yes, works. Regardless of using symlinks, a copy of the cf file, or
> include'ing it from another config file -- the target will be read
> again.
>
> But don't just take my word for it -- see for yourself!
>
> spamassassin --lint -D config
>
Thanks Karsten, much appreciated! So I've just run the lint check, and I've
attached the results here in this message (lint.txt). I'm not entirely sure
if it's working as desired or not - can you help?
I specifically ran this command:
spamassassin.exe --lint -D
config -x --siteconfigpath=rules --configpath=default_rules
Looking at the output in the attached file, I can see the 'sought-rules.cf'
file, which I created inside my 'rules' directory, there on line 20. It's
getting read *after* the files inside the default_rules directory (lines
7-11), which is good.
What's confusing me is that lines 21 onwards show the individual .cf files
getting loaded from inside each of the channel subdirectories in
default_rules. And this is *after* my sought-rules.cf file gets loaded. So
does that mean the stock rules are still overriding the sought rules? Or am
I misinterpreting the output here?
- Jezz
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Sat, 2011-06-11 at 20:01 +0200, Jezz wrote:
> So currently I'm thinking about this plan: I could create a file called
> 'zz_sought.cf' and place it into my /rules directory where it's safe. AFAIK
> the files in here would be parsed *after* the files inside the
> /default_rules directory - at least that would seem logical to me.
>
> And inside this zz_sought.cf file I can include one line like this:
>
> include C:\PATH\TO\default_rules\sought_rules_yerp_org.cf
>
> ...which is pointing to the .cf file from the SOUGHT channel, which itself
> contains two 'include' lines pointing to 20_sought.cf and
> 20_sought_fraud.cf.
Yes, include directives may be nested. Just tested this. Hey, you could
have tested it, too. ;)
Also, since your site config is read after the default config, there is
no need for the z-prefix. Just have a file named sought.cf including the
Sought channel cf, which in turn includes the channel's contents...
> Secondly, I now essentially have two files containing 'include' lines which
> point to the two SOUGHT .cf files. First (lexically) is
> sought_rules_yerp_org.cf. Then comes updates_spamassassin_org.cf which
> overwrites the SOUGHT channel entries. Then after that we have my
> zz_sought.cf file being parsed last, pointing back to the SOUGHT channel
> again. So is there any issue or problem with having two files
> (sought_rules_yerp_org.cf and zz_sought.cf) pointing to the two actual
> SOUGHT .cf files? I'm guessing not, but I want to make sure.
Yes, works. Regardless of using symlinks, a copy of the cf file, or
include'ing it from another config file -- the target will be read
again.
But don't just take my word for it -- see for yourself!
spamassassin --lint -D config
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Sought rules
Posted by Jezz <je...@fairbrass.co.nz>.
"Karsten Bräckelmann" <gu...@rudersport.de> wrote in message
news:1307726044.7307.29.camel@monkey...
> On Fri, 2011-06-10 at 18:07 +0200, Jezz wrote:
>> I recently upgraded SpamAssassin from 3.2.5 to 3.3.1, and I discovered
>> that
>> the JM_SOUGHT_FRAUD_x rules are now included within the official ruleset,
>> within the 72_active.cf file.
>>
>> However, as far as I can tell, these rules seem to be different to the
>> same-named rules that are within the latest copy of 20_sought_fraud.cf
>> which
>> is downloaded from the sought.rules.yerp.org channel. Which is to say,
>> the
>> contents of the 'meta' rule is different between these two files. My
>> guess
>> is that the version of these rules contained inside 72_active.cf is
>> perhaps
>> an older version than the ones inside 20_sought_fraud.cf. Is that the
>> case?
>
> Yes. Well, currently at least.
>
> The Sought rule-set is re-generated multiple times a day, which is what
> you get from the dedicated sa-update channel. With 3.3.x the plan is, to
> frequently perform mass-checks and re-scoring, distributed via the
> regular channel. This includes a recent snapshot of the Sought rules, so
> the dedicated channel is almost obsolete. Alas, the re-scoring currently
> does not happen as we plan for.
>
>
>> What's more, I also see that these three FRAUD rules all have a score of
>> 0
>> inside 50_scores.cf. My first question then is why they are zeroed out?
>
> It's a safety default. If you want the FRAUD subset, assign them a score
> in your local config.
>
>> Secondly, I'm wondering how I can enable these rules again if I do want
>> to
>> use them. In other words, if I want to use the latest version contained
>> within 20_sought_fraud.cf - I don't see how this could be possible.
>> Certainly I can add 'score' values for those three rules into my local.cf
>> file, which will override the zeroed-out scores in 50_scores.cf file.
>> However, because 72_active.cf comes numerically after 20_sought_fraud.cf,
>> that means the (assumedly older) FRAUD rules inside 72_active.cf will
>> override the (assumedly newer) FRAUD rules inside 20_sought_fraud.cf -
>> right? If so, there's no way for me to use the rules from
>> 20_sought_fraud.cf
>> at all?
>
> The score in your local config will take precedence, thus enabling the
> rules.
>
> You are generally correct about the numerical (actually lexical) order,
> though it doesn't apply to the files you are talking about. The
> mentioned 72_active and 20_sought are in different sa-update channels.
>
> Now, the bad thing about this is that updates_spamassassin_org.cf is
> lexically *after* sought_rules_yerp_org.cf in your rule update dir.
> Which means the more recent rules in the dedicated Sought channel are
> overwritten by the stock rules...
>
> This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
> in your rule updates dir, pointing at the channel generated cf should
> do. These channel cf files only hold include statements, to pull in the
> actual cf files in the per-channel dir.
>
>
> --
> char
> *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
> main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8?
> c<<=1:
> (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){
> putchar(t[s]);h=m;s=0; }}}
>
>
Thanks Karsten, and everyone else - that's very helpful!
So here's the thing: I'm actually running SA on Windows, via the MDaemon
mail server. So I can't so easily create a symlink as you've described.
However I think I can do something similar - let me know if this sounds
right:
Currently I've got the default rules located in a directory called
/default_rules. Inside that directory is the 'sought_rules_yerp_org' and
'updates_spamassassin_org' sub-directories, each containing their respective
.cf files, and there is also the sought_rules_yerp_org.cf and
updates_spamassassin_org.cf files with their 'include' entries inside,
pointing to the .cf files inside their sub-directories.
Then I also have a separate directory called /rules which is parallel to
/default_rules. In the /rules directory goes my local.cf file and my other
personal .cf files that I've created for myself.
The problem with your idea of a symlink or similar is that I can't change
anything inside the /default_rules directory, because MDaemon wipes and
replaces all the files and folders inside that directory each time I install
a new version of MDaemon - much to my annoyance. However, MDaemon doesn't
touch the contents of the /rules directory, so I can do whatever I want in
there.
So currently I'm thinking about this plan: I could create a file called
'zz_sought.cf' and place it into my /rules directory where it's safe. AFAIK
the files in here would be parsed *after* the files inside the
/default_rules directory - at least that would seem logical to me.
And inside this zz_sought.cf file I can include one line like this:
include C:\PATH\TO\default_rules\sought_rules_yerp_org.cf
...which is pointing to the .cf file from the SOUGHT channel, which itself
contains two 'include' lines pointing to 20_sought.cf and
20_sought_fraud.cf.
Hopefully that would work, but this raises a couple of questions:
Firstly, is it okay to have one 'include' line pointing to a .cf file which
itself contains 'include' lines? So we have two layers of includes? If this
works I think it would be preferable, as it protects me from any future
additions or filename changes within the SOUGHT channel.
Secondly, I now essentially have two files containing 'include' lines which
point to the two SOUGHT .cf files. First (lexically) is
sought_rules_yerp_org.cf. Then comes updates_spamassassin_org.cf which
overwrites the SOUGHT channel entries. Then after that we have my
zz_sought.cf file being parsed last, pointing back to the SOUGHT channel
again. So is there any issue or problem with having two files
(sought_rules_yerp_org.cf and zz_sought.cf) pointing to the two actual
SOUGHT .cf files? I'm guessing not, but I want to make sure.
Sorry if any of that is confusing to follow! (It makes sense in my head at
least...)
By the way, you mentioned that the SOUGHT channel normally gets updated a
few times a day, but currently I'm not seeing that - mine hasn't been
updated in a couple of weeks actually (for the 20_sought_fraud.cf file
specifically). I assume that's correct and I'm not doing something wrong -
perhaps the author of SOUGHT is taking a break or something?
FWIW, my vote would also be to keep the SOUGHT rules completely out of the
official updates channel altogether, and leave it up to folks to download it
separately via its own channel if they want to use it - unless the updates
channel is going to be updated at least once a day and will always include
the latest SOUGHT rules with it - but if that's unlikely to happen, then I'd
rather keep SOUGHT separate.
Cheers,
Jezz
Re: Sought rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-06-10 at 18:07 +0200, Jezz wrote:
> I recently upgraded SpamAssassin from 3.2.5 to 3.3.1, and I discovered that
> the JM_SOUGHT_FRAUD_x rules are now included within the official ruleset,
> within the 72_active.cf file.
>
> However, as far as I can tell, these rules seem to be different to the
> same-named rules that are within the latest copy of 20_sought_fraud.cf which
> is downloaded from the sought.rules.yerp.org channel. Which is to say, the
> contents of the 'meta' rule is different between these two files. My guess
> is that the version of these rules contained inside 72_active.cf is perhaps
> an older version than the ones inside 20_sought_fraud.cf. Is that the case?
Yes. Well, currently at least.
The Sought rule-set is re-generated multiple times a day, which is what
you get from the dedicated sa-update channel. With 3.3.x the plan is, to
frequently perform mass-checks and re-scoring, distributed via the
regular channel. This includes a recent snapshot of the Sought rules, so
the dedicated channel is almost obsolete. Alas, the re-scoring currently
does not happen as we plan for.
> What's more, I also see that these three FRAUD rules all have a score of 0
> inside 50_scores.cf. My first question then is why they are zeroed out?
It's a safety default. If you want the FRAUD subset, assign them a score
in your local config.
> Secondly, I'm wondering how I can enable these rules again if I do want to
> use them. In other words, if I want to use the latest version contained
> within 20_sought_fraud.cf - I don't see how this could be possible.
> Certainly I can add 'score' values for those three rules into my local.cf
> file, which will override the zeroed-out scores in 50_scores.cf file.
> However, because 72_active.cf comes numerically after 20_sought_fraud.cf,
> that means the (assumedly older) FRAUD rules inside 72_active.cf will
> override the (assumedly newer) FRAUD rules inside 20_sought_fraud.cf -
> right? If so, there's no way for me to use the rules from 20_sought_fraud.cf
> at all?
The score in your local config will take precedence, thus enabling the
rules.
You are generally correct about the numerical (actually lexical) order,
though it doesn't apply to the files you are talking about. The
mentioned 72_active and 20_sought are in different sa-update channels.
Now, the bad thing about this is that updates_spamassassin_org.cf is
lexically *after* sought_rules_yerp_org.cf in your rule update dir.
Which means the more recent rules in the dedicated Sought channel are
overwritten by the stock rules...
This merely requires a re-ordering hack, though. A symlink zzz_sought.cf
in your rule updates dir, pointing at the channel generated cf should
do. These channel cf files only hold include statements, to pull in the
actual cf files in the per-channel dir.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}