You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Quanah Gibson-Mount <qu...@zimbra.com> on 2013/10/18 20:56:55 UTC
Best practice for packaging rules
For Zimbra, I need to ship a working SA, so I can't just have the end
admins install and then download the rules. Are there any best practices
on how to do this?
For example, right now I'm updating our SA version from 3.4.0 RC1 to 3.4.0
RC3.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2013-10-25 at 17:04 -0700, Quanah Gibson-Mount wrote:
> Well... Since we run SA via amavis, custom rules are stored in
> /opt/zimbra/conf/sa, and Amavis loads them with:
>
> $sa_siteconfigpath = '/opt/zimbra/conf/sa';
It would have been nice to mention this earlier. What you just said
feels like "Hey, I told you all our SA build config and stuff, but that
fucking doesn't matter because it's just where we dump things. It is not
what SA will use, because we'll override any of the arguments I told
you."
Sorry, but given that one statement, you could have set DEFRULESDIR,
LOCALRULESDIR and LOCALSTATEDIR just as well to /dev/null...
In production, you're using different values all over the place.
Quoting the amavisd release notes, introducing that variable:
"this makes it easier to run multiple instances of amavisd, each with a
different SpamAssassin configuration"
That's what it is intended for. Ad-hoc debugging or multiple instances.
Not to overrule SA general config.
Same with the sa-update --updatedir option.
That stuff is not intended to be a replacement of build time arguments.
Just get your build right, once. Then let the commands use their
suitable defaults. They are defaults for a reason.
> So custom rules are still supported. ;)
>
> However, clearly this entire thing needs a redesign. Hopefully I can get
> to it next week.
First rule of redesign: Any custom command-line option or configuration
MUST have a solid reason. This holds even stronger for paths.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Best practice for packaging rules
Posted by Benny Pedersen <me...@junc.eu>.
Quanah Gibson-Mount skrev den 2013-10-26 02:04:
> Well... Since we run SA via amavis, custom rules are stored in
> /opt/zimbra/conf/sa, and Amavis loads them with:
try spampd ? :=)
when it works you are ready for problems with amavis
as you test now there is hidded trees in the woods
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On May 5, 2014 at 2:11:02 PM -0700 Quanah Gibson-Mount
<qu...@zimbra.com> wrote:
> --On Friday, October 25, 2013 6:04 PM -0700 Quanah Gibson-Mount
> <qu...@zimbra.com> wrote:
> PREFIX=/opt/zimbra/zimbramon
> DATADIR=/opt/zimbra/conf/spamassassin
> CONFDIR=/opt/zimbra/conf/spamassassin
> LOCALSTATEDIR=/opt/zimbra/data/spamassassin;
Decided to go with:
PREFIX=/opt/zimbra/zimbramon
DATADIR=/opt/zimbra/data/spamassassin/rules
CONFDIR=/opt/zimbra/data/spamasssassin/localrules
LOCALSTATEDIR=/opt/zimbra/data/spamassassin/state
--Quanah
--
Quanah Gibson-Mount
Server Architect
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Friday, October 25, 2013 6:04 PM -0700 Quanah Gibson-Mount
<qu...@zimbra.com> wrote:
Ok, so I want to finally get Zimbra's rules packaging and SA layout correct.
To rehash: We run SpamAsassin from Amavis. We currently run with the
following values:
$DEF_RULES_DIR = '/opt/zimbra/conf/spamassassin';
$LOCAL_RULES_DIR = '/opt/zimbra/conf/spamassassin';
$LOCAL_STATE_DIR = '/opt/zimbra/data/spamassassin';
I need everything to be under /opt/zimbra, and I understand that having the
LOCAL_RULES_DIR match DEF_RULES_DIR is incorrect, and I need a solution for
LOCAL_STATE_DIR.
Right now, I build SA with:
PREFIX=/opt/zimbra/zimbramon
DATADIR=/opt/zimbra/conf/spamassassin
CONFDIR=/opt/zimbra/conf/spamassassin
LOCALSTATEDIR=/opt/zimbra/data/spamassassin;
Makefile.PL notes:
'DEFRULESDIR', # A synonym for 'DATADIR'.
'LOCALRULESDIR', # " " " 'CONFDIR'.
'LOCALSTATEDIR', # normally determined based on $*PREFIX.
So I believe what I want is:
DATADIR=/opt/zimbra/conf/spamassassin
CONFDIR=/opt/zimbra/data/spamassassin
However, I'm stuck exactly on what I should do for LOCALSTATEDIR. It
cannot be based off of /opt/zimbra/zimbramon, as that's only writeable by
root. Would:
LOCALSTATEDIR=/opt/zimbra/data/spamassassin/3.004.000 (to reflect 3.4.0 as
the release) be correct?
Thanks!
--Quanah
--
Quanah Gibson-Mount
Server Architect
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Saturday, October 26, 2013 2:01 AM +0200 Karsten Bräckelmann
<gu...@rudersport.de> wrote:
> OK, you have even more problems...
>
> On Fri, 2013-10-25 at 09:43 -0700, Quanah Gibson-Mount wrote:
>> --On Wednesday, October 23, 2013 3:01 AM +0200 Karsten Bräckelmann
>> wrote:
>
>> >> > Well, what did you set $LOCAL_STATE_DIR to?
>> >
>> >> There's no versioned data anywhere. ;)
>> >>
>> >> There is only /opt/zimbra/conf/spamassassin
>> >>
>> >> (and after sa-update)
>> >>
>> >> /opt/zimbra/conf/spamassassin/updates_spamassassin_org/
>> >
>> > That... is rather strange. Without actually checking the code, I
>> > believe sa-update (and SA in general) adds the version number. Is
>> > there any custom patch to change that behavior?
>>
>> No, we don't patch SA.
>>
>> The build runs:
>> $(PERL) -I$(DEST_LIB_DIR) Makefile.PL
>> PREFIX=$(ZIMBRA_HOME)/zimbramon LIB=$(ZIMBRA_HOME)/zimbramon/lib
>> DATADIR=$(ZIMBRA_HOME)/conf/spamassassin
>> CONFDIR=/opt/zimbra/conf/spamassassin
>
> DATADIR and CONFDIR are synonyms for DEFRULESDIR and LOCALRULESDIR
> respectively.
>
> Given previous statements, home too is /opt/zimbra, and both are using
> the same directory. This is an issue, because according to the docs, the
> files in DEFRULESDIR are removed when SA is installed. See the PACKAGING
> docs.
>
> No problem if used as intended, i.e. for stock rules (package). In your
> case that also will flame any custom configuration or rules.
Well... Since we run SA via amavis, custom rules are stored in
/opt/zimbra/conf/sa, and Amavis loads them with:
$sa_siteconfigpath = '/opt/zimbra/conf/sa';
So custom rules are still supported. ;)
However, clearly this entire thing needs a redesign. Hopefully I can get
to it next week.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
OK, you have even more problems...
On Fri, 2013-10-25 at 09:43 -0700, Quanah Gibson-Mount wrote:
> --On Wednesday, October 23, 2013 3:01 AM +0200 Karsten Bräckelmann wrote:
> >> > Well, what did you set $LOCAL_STATE_DIR to?
> >
> >> There's no versioned data anywhere. ;)
> >>
> >> There is only /opt/zimbra/conf/spamassassin
> >>
> >> (and after sa-update)
> >>
> >> /opt/zimbra/conf/spamassassin/updates_spamassassin_org/
> >
> > That... is rather strange. Without actually checking the code, I believe
> > sa-update (and SA in general) adds the version number. Is there any
> > custom patch to change that behavior?
>
> No, we don't patch SA.
>
> The build runs:
> $(PERL) -I$(DEST_LIB_DIR) Makefile.PL
> PREFIX=$(ZIMBRA_HOME)/zimbramon LIB=$(ZIMBRA_HOME)/zimbramon/lib
> DATADIR=$(ZIMBRA_HOME)/conf/spamassassin
> CONFDIR=/opt/zimbra/conf/spamassassin
DATADIR and CONFDIR are synonyms for DEFRULESDIR and LOCALRULESDIR
respectively.
Given previous statements, home too is /opt/zimbra, and both are using
the same directory. This is an issue, because according to the docs, the
files in DEFRULESDIR are removed when SA is installed. See the PACKAGING
docs.
No problem if used as intended, i.e. for stock rules (package). In your
case that also will flame any custom configuration or rules.
> LOCALSTATEDIR=$(ZIMBRA_HOME)/data/spamassassin; \
>
> Although interestingly I never create or use /opt/zimbra/data/spamassassin.
> :P
So you are running sa-update always with --updatedir specified on the
command line?
That would explain the last pieces of the puzzle -- this directory not
existing, and the weird un-versioned path as per above. Without patching
the code, --updatedir is the only way to create that mess. Otherwise
sa-update would create LOCALSTATEDIR/3.xxx.yyy/.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Wednesday, October 23, 2013 3:01 AM +0200 Karsten Bräckelmann
<gu...@rudersport.de> wrote:
> On Tue, 2013-10-22 at 17:03 -0700, Quanah Gibson-Mount wrote:
>> --On Wednesday, October 23, 2013 1:53 AM +0200 Karsten Bräckelmann
>> wrote:
>
>> > Well, what did you set $LOCAL_STATE_DIR to?
>
>> There's no versioned data anywhere. ;)
>>
>> There is only /opt/zimbra/conf/spamassassin
>>
>> (and after sa-update)
>>
>> /opt/zimbra/conf/spamassassin/updates_spamassassin_org/
>
> That... is rather strange. Without actually checking the code, I believe
> sa-update (and SA in general) adds the version number. Is there any
> custom patch to change that behavior?
No, we don't patch SA.
The build runs:
$(PERL) -I$(DEST_LIB_DIR) Makefile.PL
PREFIX=$(ZIMBRA_HOME)/zimbramon LIB=$(ZIMBRA_HOME)/zimbramon/lib
DATADIR=$(ZIMBRA_HOME)/conf/spamassassin
CONFDIR=/opt/zimbra/conf/spamassassin
LOCALSTATEDIR=$(ZIMBRA_HOME)/data/spamassassin; \
Although interestingly I never create or use /opt/zimbra/data/spamassassin.
:P
>> with /opt/zimbra/conf/updates_spamassassin_org.cf added for loading the
>> rules.
>
> (Missed a "spamassassin/" here. Did you really, repeatedly type in that
> string manually?)
No, but I did that one time. :P
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2013-10-22 at 17:03 -0700, Quanah Gibson-Mount wrote:
> --On Wednesday, October 23, 2013 1:53 AM +0200 Karsten Bräckelmann wrote:
> > Well, what did you set $LOCAL_STATE_DIR to?
> There's no versioned data anywhere. ;)
>
> There is only /opt/zimbra/conf/spamassassin
>
> (and after sa-update)
>
> /opt/zimbra/conf/spamassassin/updates_spamassassin_org/
That... is rather strange. Without actually checking the code, I believe
sa-update (and SA in general) adds the version number. Is there any
custom patch to change that behavior?
> with /opt/zimbra/conf/updates_spamassassin_org.cf added for loading the
> rules.
(Missed a "spamassassin/" here. Did you really, repeatedly type in that
string manually?)
With a layout like that, having an un-versioned SA update channel inside
your site-specific $LOCAL_RULES_DIR, you are indeed lucky enough to have
it just work. Because alphabet.
> They are definitely not equal values:
>
> ./Mail/SpamAssassin.pm: $self->{LOCAL_STATE_DIR} ||= '/opt/zimbra/data/spamassassin';
> ./Mail/SpamAssassin.pm: $self->{LOCAL_RULES_DIR} ||= '/opt/zimbra/conf/spamassassin';
>
> but /opt/zimbra/data/spamassassin doesn't even exist. :P
>
> I'll definitely spend some time on getting this mess sorted out.
Good. As should be clear by now, these really should be 3 distinct
directories with different precedence.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Wednesday, October 23, 2013 1:53 AM +0200 Karsten Bräckelmann
<gu...@rudersport.de> wrote:
> On Tue, 2013-10-22 at 16:25 -0700, Quanah Gibson-Mount wrote:
>> --On Tuesday, October 22, 2013 12:18 AM +0200 Karsten Bräckelmann wrote:
>>
>> > By setting $DEF_RULES_DIR to the same value as $LOCAL_RULES_DIR,
>> > placing stock rules in what's supposed to be strictly for
>> > site-specific conf, you override the fresh rules pulled by sa-update.
>>
>> I'm not positive that is correct -- When sa-update runs, it pulls down
>> the rules and adds a cf file to include them, with the name
>> updates_spamassassin_cf. Given load order seems to be alphabetically
>> dependent, it should still override, I'm hoping. ;)
>
> Well, what did you set $LOCAL_STATE_DIR to?
>
> Continuing with the default dirs I used in my previous post, local state
> dir would be /var/lib/spamassassin. SA generally, and sa-update in
> particular are using a *versioned* schema inside local state dir.
>
> Thus, if you set $LOCAL_STATE_DIR to the same as $LOCAL_RULES_DIR, you
> will end up with a 3.00x00y/ directory in your site-specific config.
> That versioned directory contains the update-channels in a dedicated dir
> each (and a .cf file to include the contents). The versioned directory
> will not be traversed recursively.
>
> I stand to what I said.
>
>
> But hey, don't just take my word for it. Please run 'spamassassin -D'.
> The debug output will tell you which configuration SA reads, in what
> order.
There's no versioned data anywhere. ;)
There is only /opt/zimbra/conf/spamassassin
(and after sa-update)
/opt/zimbra/conf/spamassassin/updates_spamassassin_org/
with /opt/zimbra/conf/updates_spamassassin_org.cf added for loading the
rules.
They are definitely not equal values:
./Mail/SpamAssassin.pm: $self->{LOCAL_STATE_DIR} ||=
'/opt/zimbra/data/spamassassin';
./Mail/SpamAssassin.pm: $self->{LOCAL_RULES_DIR} ||=
'/opt/zimbra/conf/spamassassin';
but /opt/zimbra/data/spamassassin doesn't even exist. :P
I'll definitely spend some time on getting this mess sorted out.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2013-10-22 at 16:25 -0700, Quanah Gibson-Mount wrote:
> --On Tuesday, October 22, 2013 12:18 AM +0200 Karsten Bräckelmann wrote:
>
> > By setting $DEF_RULES_DIR to the same value as $LOCAL_RULES_DIR, placing
> > stock rules in what's supposed to be strictly for site-specific conf,
> > you override the fresh rules pulled by sa-update.
>
> I'm not positive that is correct -- When sa-update runs, it pulls down the
> rules and adds a cf file to include them, with the name
> updates_spamassassin_cf. Given load order seems to be alphabetically
> dependent, it should still override, I'm hoping. ;)
Well, what did you set $LOCAL_STATE_DIR to?
Continuing with the default dirs I used in my previous post, local state
dir would be /var/lib/spamassassin. SA generally, and sa-update in
particular are using a *versioned* schema inside local state dir.
Thus, if you set $LOCAL_STATE_DIR to the same as $LOCAL_RULES_DIR, you
will end up with a 3.00x00y/ directory in your site-specific config.
That versioned directory contains the update-channels in a dedicated dir
each (and a .cf file to include the contents). The versioned directory
will not be traversed recursively.
I stand to what I said.
But hey, don't just take my word for it. Please run 'spamassassin -D'.
The debug output will tell you which configuration SA reads, in what
order.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Tuesday, October 22, 2013 12:18 AM +0200 Karsten Bräckelmann
<gu...@rudersport.de> wrote:
> By setting $DEF_RULES_DIR to the same value as $LOCAL_RULES_DIR, placing
> stock rules in what's supposed to be strictly for site-specific conf,
> you override the fresh rules pulled by sa-update.
I'm not positive that is correct -- When sa-update runs, it pulls down the
rules and adds a cf file to include them, with the name
updates_spamassassin_cf. Given load order seems to be alphabetically
dependent, it should still override, I'm hoping. ;)
>> I'm wondering if there's a better layout strategy than this. I.e., keep
>> the .pre files in a specific location (/opt/zimbra/conf/spamassassin)
>> and keep the rule files + updates in /opt/zimbra/data/spamassassin (our
>> generalized data location for app specific data).
>>
>> Any opinions on that?
>
> Yes, definitely.
Ok, added to my to-do list. ;)
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2013-10-21 at 13:59 -0700, Quanah Gibson-Mount wrote:
> --On Sunday, October 20, 2013 3:39 AM +0200 Karsten Bräckelmann wrote:
>
> > There is. Install SA, run sa-update. See the INSTALL file.
> >
> > On this topic it is worth pointing out, that the Mail-SpamAssassin
> > tarball contains everything necessary but the actual rules files. That
> > includes the most basic .pre files.
>
> Yes, but I was thinking if you didn't even have the .pre files, there's no
> way to bootstrap.
That... would be correct as stated.
There's a fundamental problem with that, though. sa-update requires
Mail::SpamAssassin. Which, at that stage, you did not install. If you
would have installed the core SA parts, you would also have the .pre
files.
You are getting that error message only, because some magic in the
sa-update BEGIN block picks up a relative located M::SA in your build
tree.
Again: First install SA. Then run sa-update.
> However, this has opened up a question to me as to whether or not my
> current configuration (which was designed by someone else long ago) is
> optimal.
>
> Currently, we store everything (.pre files and rules) in
> /opt/zimbra/conf/spamassassin:
>
> zimbra@zre-ldap001:~/conf/spamassassin$ ls
> 10_default_prefs.cf 20_freemail.cf 20_ratware.cf 25_pyzor.cf
[...]
> v340.pre
>
>
> And we have SA built with:
>
> my $DEF_RULES_DIR = '/opt/zimbra/conf/spamassassin';
> my $LOCAL_RULES_DIR = '/opt/zimbra/conf/spamassassin';
Ouch. Do NOT keep stock rules in the site-specific configuration dir.
Let me show what that configuration does. I'll stick to common defaults
for the dirs, trying to refer to synonyms and variables. Parts are
copied straight from the man page.
Default configuration data is loaded from the first existing directory
in:
/var/lib/spamassassin/3.00x00y # $LOCAL_STATE_DIR plus SA version
/usr/share/spamassassin # $DEF_RULES_DIR
The first one is populated by sa-update, the second is pre sa-update and
a suitable place for a stock rules package.
Site-specific configuration data is used to override any values which
had already been set. This is loaded from the first existing directory
in:
/etc/mail/spamassassin # $LOCAL_RULES_DIR
That is where the core .pre files are located, and any site-specific
configuration -- including custom rules, overriding stock scores, etc.
By setting $DEF_RULES_DIR to the same value as $LOCAL_RULES_DIR, placing
stock rules in what's supposed to be strictly for site-specific conf,
you override the fresh rules pulled by sa-update.
> I'm wondering if there's a better layout strategy than this. I.e., keep
> the .pre files in a specific location (/opt/zimbra/conf/spamassassin) and
> keep the rule files + updates in /opt/zimbra/data/spamassassin (our
> generalized data location for app specific data).
>
> Any opinions on that?
Yes, definitely.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Tuesday, October 22, 2013 8:56 AM -0400 "Kevin A. McGrail"
<KM...@PCCC.com> wrote:
> On 10/21/2013 4:59 PM, Quanah Gibson-Mount wrote:
>> I'm wondering if there's a better layout strategy than this. I.e.,
>> keep the .pre files in a specific location
>> (/opt/zimbra/conf/spamassassin) and keep the rule files + updates in
>> /opt/zimbra/data/spamassassin (our generalized data location for app
>> specific data).
>>
>> Any opinions on that?
> Definite opinions.
>
> You need to have one dir that is designed for rules/config files you
> create, modify and override. On my system this is:
> /etc/mail/spamassassin.
>
> Then you have your default rules location for sa-update which is
> /var/lib/spamassassin/<version> dir by default. You really shouldn't
> interact with the files in this dir so if you are intermingling them, it
> could be very confusing to administer the system.
>
> It sounds a bit like what you inherited predates the concept of sa-update
> but I off the cuff, even something like setting LOCALSTATEDIR to
> /opt/zimbra/conf/spamassassin/var/lib and DEFRULESDIR to /opt/zimbra/conf
> might be better.
>
> That should give you a couple of pre files and a local.cf in
> /opt/zimbra/conf/spamassassin and then allow sa-update to run more
> cleanly.
Thanks. I believe this configuration dates back to about 2005. ;)
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/21/2013 4:59 PM, Quanah Gibson-Mount wrote:
> I'm wondering if there's a better layout strategy than this. I.e.,
> keep the .pre files in a specific location
> (/opt/zimbra/conf/spamassassin) and keep the rule files + updates in
> /opt/zimbra/data/spamassassin (our generalized data location for app
> specific data).
>
> Any opinions on that?
Definite opinions.
You need to have one dir that is designed for rules/config files you
create, modify and override. On my system this is: /etc/mail/spamassassin.
Then you have your default rules location for sa-update which is
/var/lib/spamassassin/<version> dir by default. You really shouldn't
interact with the files in this dir so if you are intermingling them, it
could be very confusing to administer the system.
It sounds a bit like what you inherited predates the concept of
sa-update but I off the cuff, even something like setting LOCALSTATEDIR
to /opt/zimbra/conf/spamassassin/var/lib and DEFRULESDIR to
/opt/zimbra/conf might be better.
That should give you a couple of pre files and a local.cf in
/opt/zimbra/conf/spamassassin and then allow sa-update to run more cleanly.
Regards,
KAM
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Sunday, October 20, 2013 3:39 AM +0200 Karsten Bräckelmann
<gu...@rudersport.de> wrote:
> There is. Install SA, run sa-update. See the INSTALL file.
>
> On this topic it is worth pointing out, that the Mail-SpamAssassin
> tarball contains everything necessary but the actual rules files. That
> includes the most basic .pre files.
Yes, but I was thinking if you didn't even have the .pre files, there's no
way to bootstrap.
In any case, I have something put together now that uses the .pre files
that ship with the tarball.
However, this has opened up a question to me as to whether or not my
current configuration (which was designed by someone else long ago) is
optimal.
Currently, we store everything (.pre files and rules) in
/opt/zimbra/conf/spamassassin:
zimbra@zre-ldap001:~/conf/spamassassin$ ls
10_default_prefs.cf 20_freemail.cf 20_ratware.cf 25_pyzor.cf
30_text_pl.cf 72_active.cf
STATISTICS-set0-72_scores.cf.txt
10_hasbase.cf 20_freemail_domains.cf 20_uri_tests.cf
25_razor2.cf 30_text_pt_br.cf 72_scores.cf
STATISTICS-set1-72_scores.cf.txt
20_advance_fee.cf 20_head_tests.cf 20_vbounce.cf
25_replace.cf 50_scores.cf 73_sandbox_manual_scores.cf
STATISTICS-set2-72_scores.cf.txt
20_aux_tlds.cf 20_html_tests.cf 23_bayes.cf 25_spf.cf
60_adsp_override_dkim.cf active.list
STATISTICS-set3-72_scores.cf.txt
20_body_tests.cf 20_imageinfo.cf 25_accessdb.cf
25_textcat.cf 60_awl.cf init.pre
user_prefs.template
20_compensate.cf 20_mailspike.cf 25_antivirus.cf 25_uribl.cf
60_shortcircuit.cf languages v310.pre
20_dnsbl_tests.cf 20_meta_tests.cf 25_asn.cf
30_text_de.cf 60_whitelist.cf local.cf
v312.pre
20_drugs.cf 20_net_tests.cf 25_dcc.cf
30_text_fr.cf 60_whitelist_dkim.cf MIRRORED.BY
v320.pre
20_dynrdns.cf 20_phrases.cf 25_dkim.cf
30_text_it.cf 60_whitelist_spf.cf regression_tests.cf
v330.pre
20_fake_helo_tests.cf 20_porn.cf 25_hashcash.cf
30_text_nl.cf 60_whitelist_subject.cf sa-update-pubkey.txt
v340.pre
And we have SA built with:
my $DEF_RULES_DIR = '/opt/zimbra/conf/spamassassin';
my $LOCAL_RULES_DIR = '/opt/zimbra/conf/spamassassin';
I'm wondering if there's a better layout strategy than this. I.e., keep
the .pre files in a specific location (/opt/zimbra/conf/spamassassin) and
keep the rule files + updates in /opt/zimbra/data/spamassassin (our
generalized data location for app specific data).
Any opinions on that?
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
Quanah, I see you already found a solution. However, for completeness
and to clarify potential misunderstanding:
On Fri, 2013-10-18 at 16:07 -0700, Quanah Gibson-Mount wrote:
> --On Friday, October 18, 2013 7:04 PM -0400 "Kevin A. McGrail"
> <KM...@PCCC.com> wrote:
>
> > I don't believe that is correct. It appears you are missing the
> > /etc/mail/spamassassin/ pre files which includes v320.pre which includes
> > this include:
>
> We of course don't install into /etc/mail
That's just a default, actually depending on your $PREFIX.
The resulting "site-specific configuration" directory is shared by all
SA commands, including sa-update. The SA installation is responsible for
that dir and its content to exist. Your failing sa-update command
suggests you ran it from within your build tree, prior to installation.
> And while building, we build out into a temporary location, related to the
> OS of the system being built upon.
>
> It seems like a significant design flaw that there is no way to bootstrap
> SA.
There is. Install SA, run sa-update. See the INSTALL file.
On this topic it is worth pointing out, that the Mail-SpamAssassin
tarball contains everything necessary but the actual rules files. That
includes the most basic .pre files.
The rules snapshot tarball is equivalent to the "default configuration".
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Friday, October 18, 2013 7:15 PM -0400 "Kevin A. McGrail"
<KM...@PCCC.com> wrote:
> I'm sure there is but wherever your bootstrap location is configured for
> config files needs a pre file that at a minimum loads the check
> functionality. It's a missing configuration issue.
Yeah, I figured out a way to do it using the .pre files shipped with the
Mail-SpamAssassin-3.4.0/rules bits. So this is solved, thanks. :)
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/18/2013 7:07 PM, Quanah Gibson-Mount wrote:
> --On Friday, October 18, 2013 7:04 PM -0400 "Kevin A. McGrail"
> <KM...@PCCC.com> wrote:
>
>> I don't believe that is correct. It appears you are missing the
>> /etc/mail/spamassassin/ pre files which includes v320.pre which includes
>> this include:
>
> We of course don't install into /etc/mail
>
> And while building, we build out into a temporary location, related to
> the OS of the system being built upon.
>
> It seems like a significant design flaw that there is no way to
> bootstrap SA.
I'm sure there is but wherever your bootstrap location is configured for
config files needs a pre file that at a minimum loads the check
functionality. It's a missing configuration issue.
Regards,
KAM
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Friday, October 18, 2013 7:04 PM -0400 "Kevin A. McGrail"
<KM...@PCCC.com> wrote:
> I don't believe that is correct. It appears you are missing the
> /etc/mail/spamassassin/ pre files which includes v320.pre which includes
> this include:
We of course don't install into /etc/mail
And while building, we build out into a temporary location, related to the
OS of the system being built upon.
It seems like a significant design flaw that there is no way to bootstrap
SA.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/18/2013 6:58 PM, Quanah Gibson-Mount wrote:
> --On Friday, October 18, 2013 3:47 PM -0700 Quanah Gibson-Mount
> <qu...@zimbra.com> wrote:
>
>> --On Friday, October 18, 2013 10:56 PM +0200 Axb <ax...@gmail.com>
>> wrote:
>>
>>> As to rules, running sa-update as the last step in your setup routine
>>> will be your best bet to get uptodate rules.
>>> I think nobody would recommend delivering a static package.
>
> I would note sa-update depends on already having rules:
>
> build@zre-sles11-64:~/p4/zimbra/main/ThirdParty/Perl> perl -T -I
> zimbramon/lib zimbramon/bin/sa-update --updatedir=./spamassassin
> config: no configuration text or files found! do you need to run
> 'sa-update'?
> Timeout::_run: check: no loaded plugin implements 'check_main': cannot
> scan!
> Check the necessary '.pre' files are in the config directory.
>
> I also find it rather amusing that sa-update tells you to run
> sa-update...
>
> So if you have *nothing* but the sa-update command and SA perl
> modules, what is the best practice for getting the rules? At the
> moment, there appears to be a circular dependency.
I don't believe that is correct. It appears you are missing the
/etc/mail/spamassassin/ pre files which includes v320.pre which includes
this include:
# Check - Provides main check functionality
#
loadplugin Mail::SpamAssassin::Plugin::Check
At a minimum, you need those files in a pre.
regards,
KAM
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Friday, October 18, 2013 3:47 PM -0700 Quanah Gibson-Mount
<qu...@zimbra.com> wrote:
> --On Friday, October 18, 2013 10:56 PM +0200 Axb <ax...@gmail.com>
> wrote:
>
>> As to rules, running sa-update as the last step in your setup routine
>> will be your best bet to get uptodate rules.
>> I think nobody would recommend delivering a static package.
I would note sa-update depends on already having rules:
build@zre-sles11-64:~/p4/zimbra/main/ThirdParty/Perl> perl -T -I
zimbramon/lib zimbramon/bin/sa-update --updatedir=./spamassassin
config: no configuration text or files found! do you need to run
'sa-update'?
Timeout::_run: check: no loaded plugin implements 'check_main': cannot scan!
Check the necessary '.pre' files are in the config directory.
I also find it rather amusing that sa-update tells you to run sa-update...
So if you have *nothing* but the sa-update command and SA perl modules,
what is the best practice for getting the rules? At the moment, there
appears to be a circular dependency.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Friday, October 18, 2013 10:56 PM +0200 Axb <ax...@gmail.com>
wrote:
> As to rules, running sa-update as the last step in your setup routine
> will be your best bet to get uptodate rules.
> I think nobody would recommend delivering a static package.
We have a job to update rules once they are set up, but we still need to
ship a default set of rules.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Axb <ax...@gmail.com>.
On 10/18/2013 10:50 PM, Quanah Gibson-Mount wrote:
> --On Friday, October 18, 2013 10:29 PM +0200 Axb <ax...@gmail.com>
> wrote:
>
>> On 10/18/2013 09:01 PM, Quanah Gibson-Mount wrote:
>>> Only in spamassassin/: sandbox-felicity.pm
>>> Only in spamassassin/: sandbox-hstern.pm
>>> Only in spamassassin/: SIQ.pm
>>
>> KAM,
>>
>> iirc, these shouldn't be distributed
>
> Those aren't in his ruleset tarball. ;) If they aren't to be
> distributed, I can delete it from what I'm shipping.
Yep, these shouldn't be distributed.
> I likely got them from subversion at some point.
Suggest you use a clean/different path from svn/trunk, etc for packaging.
As to rules, running sa-update as the last step in your setup routine
will be your best bet to get uptodate rules.
I think nobody would recommend delivering a static package.
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Friday, October 18, 2013 10:29 PM +0200 Axb <ax...@gmail.com>
wrote:
> On 10/18/2013 09:01 PM, Quanah Gibson-Mount wrote:
>> Only in spamassassin/: sandbox-felicity.pm
>> Only in spamassassin/: sandbox-hstern.pm
>> Only in spamassassin/: SIQ.pm
>
> KAM,
>
> iirc, these shouldn't be distributed
Those aren't in his ruleset tarball. ;) If they aren't to be distributed,
I can delete it from what I'm shipping. I likely got them from subversion
at some point.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
Re: Best practice for packaging rules
Posted by Axb <ax...@gmail.com>.
On 10/18/2013 09:01 PM, Quanah Gibson-Mount wrote:
> Only in spamassassin/: sandbox-felicity.pm
> Only in spamassassin/: sandbox-hstern.pm
> Only in spamassassin/: SIQ.pm
KAM,
iirc, these shouldn't be distributed
any ideas?
Re: Best practice for packaging rules
Posted by Quanah Gibson-Mount <qu...@zimbra.com>.
--On Friday, October 18, 2013 11:56 AM -0700 Quanah Gibson-Mount
<qu...@zimbra.com> wrote:
> For Zimbra, I need to ship a working SA, so I can't just have the end
> admins install and then download the rules. Are there any best practices
> on how to do this?
>
> For example, right now I'm updating our SA version from 3.4.0 RC1 to
> 3.4.0 RC3.
And here's an example of why getting this done is always a question for me.
I downloaded KAM's 3.4.0 RC3 ruleset from his link, and compare it to what
I have now:
quanah@zre-ldap001:~/p4/zimbra/main/ZimbraServer/conf$ diff -ru
spamassassin/ 3.4.0-RC3/ | grep "Only in"
Only in spamassassin/: active.list
Only in spamassassin/: init.pre
Only in spamassassin/: sandbox-felicity.pm
Only in spamassassin/: sandbox-hstern.pm
Only in spamassassin/: SIQ.pm
Only in spamassassin/: STATISTICS-set0.txt
Only in spamassassin/: STATISTICS-set1.txt
Only in spamassassin/: STATISTICS-set2.txt
Only in spamassassin/: STATISTICS-set3.txt
Only in spamassassin/: v310.pre
Only in spamassassin/: v312.pre
Only in spamassassin/: v320.pre
Only in spamassassin/: v330.pre
Only in spamassassin/: v340.pre
And a number of files are missing, at least some of which (v*.pre) I'm
pretty sure are required for having a functional installation.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration