Re: config status; rewrite_header not taking

[Ray <rs...@misinformation.org>]

Karsten Bräckelmann <guenther <at> rudersport.de> writes:

> > just local.cf, IIUC, but potentially any of the 47 files in my hosting
> > provider's /usr/share/spamassassin and /etc/mail/spamassassin dirs (or any
>
> Aah... no. :)  The stuff in /usr/share/spamassassin (granted, plus the .pre
> files) is exactly the *base*. Stock SA. No user-servicable parts inside. This
> dir won't even be used, after an sa-update.

Ah, okay, that's good to learn, thanks.  That's not a detail I found out from
the docs (but I should have understood it from the directory hierarchy).

> Frankly, there are some important differences between SA and postfix
> "configuration". Just to start the list, not exhaustive:
>
> 'postconf' without the handy -n switch dumps about 500 lines. The
> equivalent dump for SA including the rules is about 6000 lines. And
> that's a plain dump, *without* following and unfolding meta rules or
> anything.
>
> Also, frankly, I don't think SA rules are really the same as settings.

This is maybe one of the communication difficulties here.  When I think "config"
I'm really thinking what you're calling "settings".  At this point I'm not as
interested in the rules.  (I haven't gotten past grokking the more fundamental
"config" of the system, which I feel I should understand before I move on to
rules.)  Perhaps the confusion here arises from the fact that rule definition is
a subset of "proper" config?  (I don't even know if this is the case for sure as
I haven't gotten to rules.)

> There are exactly two (sensible) possible places for custom configu-
> ration. /etc/mail/spamassassin and the user_prefs, if any.

I'm not assured a sensible installation when I am not the person who did the
installation.  (Perhaps not even then.)  But, again, good to learn this
information.  Thank you.  I note that so far this seems like orally-transmitted
folk knowledge more so than documented system nature.

> > others if they happen to have configured such), plus my user_prefs file
> > (_except_ any items which are prohibited from being overridden (except the
> > privileged settings which are actually allowed by allow_user_rules (except
> > those privileged settings which are actually "administrator" privileged
> > settings which cannot be allowed via allow_user_rules))), but minus
> > misspellings and possibly minus rules following misspellings in any of the
> > config files.
>
> Hell, no!  Assuming there are mis-spellings is inherently broken. Do lint
> check your configuration after *any* change. No complaints, no mis-spellings.

I'm not sure I understand you here.

I think that assuming there are _no_ misspellings in someone else's site-wide
config is leaving a door open to problems.  As you appear to indicate, lint
checking the config to validate it is very important.  No complaints and I can
then assume that the effective config is not modulated by errors, which is a
good (yet additional) step toward knowing the effective config.  I would be
sorely pressed to understand the implications of lint complaints that I couldn't
understand like:

[> check: no loaded plugin implements 'check_main': cannot scan! at
[>   /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 164.

The response here may be "that's not a lint message, and you can safely ignore
it".  But my point is that I'm required to understand this to know its effect on
the config if I am manually parsing the config and don't have a tool to show the
effective config.

I'm not sure if I'm clear with where I've been going with all this.  To know the
effective config I am having to search for more bodies.  And the process appears
to be unbounded.  Thankfully I have SA sherpas to help out, but I really didn't
want to bother you guys in the first place and I think that it would be nicer if
noobs like me didn't bother you guys.  Not that I want to encourage surliness
and "RTFM!" from the natives, but an in-effect config printer might help, as it
might also help a lot of debugging.

> > [...]  Meaning, if I want to know for sure exactly what results in the
> > effective config, do I consult the POD?  Or maybe the POD and the man pages,
> > and perhaps a particular wiki article and that's it, period.
>
> POD == man pages.
>
> Anyway, you're contradicting yourself. POD plus a single wiki page -- to grok
> the FULL configuration? That's what you requested to be dumped by a
> postconf-alike. Right, 6000 lines full of meta-rules and ghastly REs.
> Understood after the tiny bit of lecture you mentioned? No way.

I imagine the full config with rules would be awful, yeah.  But what about
config aside from rules?  That's really what I'm talking about.  (postconf
doesn't really output "rules".)  Isn't there something of a semantic distinction
between "bayes_auto_learn" and "redirector_pattern"?  I think the Conf POD page
gives one just about everything one needs to understand any non-rule config.

Don't folks ever want to list out non-rule configs?  The response here may be
"if I change a config I can see the effects".  What happens if you can't?  Maybe
one never runs into a situation where the config one places in user_prefs fails
to go into effect exactly as intended?

> > I imagine just about every SA admin here has played out the scenario
> > repeatedly and knows the locations of the bodies.  "Come on, there are only
> > four."  Heck, even though I didn't install the SA here and I'm coming at
> > this as a total noob I may already have all the critical details I need.
> > But how am I to know that?  This is what I mean by open-ended.
>
> As a "noob", you aren't interested in the full postconf dump, are you?  You
> want your custom settings, -n...

Right.  Either the non-rule settings, or the custom (non-default) settings.
Preferrably option for either.

> I guess you're approaching this from the wrong end. It's almost like you don't
> trust the default config. The first thing for a new user after installing any
> application (and ensuring it works) is to customize it.  Not to dissect and
> anatomize its deepest innards.

Not "default config", site-wide config.

The Vim installation at this hosting provider is twitted so it can't syntax
highlight, among other problems.  I take that and the above `spamassassin -D`
error message as signs that I ought not necessarily trust the SA config.

Together, the untrustworthiness of the site-wide config and the complexity and
folk knowledge required to parse out the effective config, combined with config
documentation syntax underspecification (I'll get to this in a moment) lead me
to wishing I had a simple way to check the effective config.

I'm not asking to dissect to deepest innards.  If I could just say something
like `spamassassin --print-config rewrite_header`, that would do it.

> Maybe you should try to ask specific questions, and explain what you want to
> accomplish.

I maintain that a config printer (at least without rules) is a good idea, but
I'll stop trying to persuade about it.

One specific problem I'm having is that my user_prefs config for undoing the
site-wide rewrite_header does not appear to be working.  How does a user stop SA
from rewriting the header?  (Note that this effort is a step towards the goal of
preserving spam for later manually-directed `sa-learn` training.)

RSK

Re: config status; rewrite_header not taking

[Ray <rs...@misinformation.org>]

Karsten Bräckelmann <guenther <at> rudersport.de> writes:

> You must not assume or allow for mis-spelled configuration keywords or
> otherwise illegal syntax. Just lint check. If it comes back clean, all
> is good. If it doesn't, you NEED to fix it anyway.

I don't have privs, and conceivably a misspelling isn't fatal.

The power is not mine to allow or deny misspellings in site-wide config.

> > I think that assuming there are _no_ misspellings in someone else's
> > site-wide config is leaving a door open to problems.  As you appear to
> > indicate, lint checking the config to validate it is very important.
>
> Yes, you must exactly assume that. There are no site-wide mis-spellings.  And
> you can verify it easily.

But it's not called "assuming" if you verify?  I must be missing you here,
sorry.  I would side with "you must verify" rather than "you must assume".

> > [> check: no loaded plugin implements 'check_main': cannot scan! at
> > [>   /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line
> > [>   164.
>
> Ahem. Your SA is crippled. It does nothing. It can't. (And no, this is
> not about a mis-spelling...)

I'm trying not to come across as unnecessarily contrary; please pardon me when I
say that you are wrong here.  The SA installation works well enough to score and
judge (add X-Spam-Score, e.g.), and it's doing a pretty good job so far.  It
even marks up subject lines in a way I don't like.  It's pretty clear it's
operating.

It seems likely then that this config check error is from my account running
jailed.  They've given me visibility to things like /usr/share/spamassasin, but
it's likely they haven't given me the entirety of what's driving the site-wide
SA.  Since I'm not the user that spamd runs as, it's probably not critical to
give me access to all of this hosts's SA config.

(Querying for the in-effect config becomes especially important if the site-wide
config is in part hidden from my account.  But it also becomes more complicated
-- IPC with spamd?  I don't know how spamd is architected.  Anyway, sorry, I
said I'd stop harping on this.)

I'll bypass the portion of your message predicated on SA not operating.  Let me
know if there's anything in that I shouldn't bypass.

> I don't know about general trustworthiness of site-wide config in your case.
> The above is a gross failure. Which might just be a broken install and simply
> needs to be fixed. Bad, but not necessarily affecting trust.  Trustworthiness
> is much more -- it involves not breaking or even changing without knowledge.
> If you can't trust your system admin, switch your system.

I feel I can trust the competence of the site admins to mostly run a pretty good
system.  They've locked things down pretty hard, though.  This is a difficulty,
but the validity of ... well, I wasn't going to talk about that.  Anyway the
following topic still stands:

> > One specific problem I'm having is that my user_prefs config for undoing the
> > site-wide rewrite_header does not appear to be working.  How does a user
> > stop SA from rewriting the header?  (Note that this effort is a step towards
> > the goal of preserving spam for later manually-directed `sa-learn`
> > training.)

I _can_ in fact adjust required_score.  Is there any reason I wouldn't be able
to adjust rewrite_header?

RSK

Re: config status; rewrite_header not taking

[Karsten Bräckelmann <gu...@rudersport.de>]

Skipping all the "print my config" stuff. Maybe tomorrow. For now,
there's more important things than custom config...


On Sun, 2009-03-01 at 01:24 +0000, Ray wrote:
> Karsten Bräckelmann <guenther <at> rudersport.de> writes:

> > > those privileged settings which are actually "administrator" privileged
> > > settings which cannot be allowed via allow_user_rules))), but minus
> > > misspellings and possibly minus rules following misspellings in any of the
> > > config files.
> >
> > Hell, no!  Assuming there are mis-spellings is inherently broken. Do lint
> > check your configuration after *any* change. No complaints, no mis-spellings.
> 
> I'm not sure I understand you here.

You must not assume or allow for mis-spelled configuration keywords or
otherwise illegal syntax. Just lint check. If it comes back clean, all
is good. If it doesn't, you NEED to fix it anyway.


> I think that assuming there are _no_ misspellings in someone else's site-wide
> config is leaving a door open to problems.  As you appear to indicate, lint
> checking the config to validate it is very important.  

Yes, you must exactly assume that. There are no site-wide mis-spellings.
And you can verify it easily.

>                                                        No complaints and I can
> then assume that the effective config is not modulated by errors, which is a
> good (yet additional) step toward knowing the effective config.  I would be
> sorely pressed to understand the implications of lint complaints that I couldn't
> understand like:
> 
> [> check: no loaded plugin implements 'check_main': cannot scan! at
> [>   /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 164.

Ahem. Your SA is crippled. It does nothing. It can't. (And no, this is
not about a mis-spelling...)

At this point, back down and take a deep breath. By chasing a non-
functional rewrite_header setting, you are seriously barking up the
wrong tree. Don't care about that setting. Don't care about any such
setting. SA doesn't work -- this is what you need to check.

The above is supposed to be enabled by /etc/mail/spamassassin/v320.pre:
  loadplugin Mail::SpamAssassin::Plugin::Check

A stock and most vital part. If it isn't, your install is seriously
broken. To stress this part again -- the stock, vanilla v320.pre ships
with that very line.

Unless you messed with these .pre files, removing loadplugin settings, I
suggest you go back to square one -- install SA properly. Which might
involve telling your hoster to do so.


FWIW, if that line exists, another explanation is a missing Perl module.
In either case it's about the equivalent of a missing .so in your
postfix analogy. postconf would be useless here...


> The response here may be "that's not a lint message, and you can safely ignore
> it".  But my point is that I'm required to understand this to know its effect on
> the config if I am manually parsing the config and don't have a tool to show the
> effective config.

Nope, this is by far *worse* than a failing lint check.


> > I guess you're approaching this from the wrong end. It's almost like you don't
> > trust the default config. The first thing for a new user after installing any
> > application (and ensuring it works) is to customize it.  Not to dissect and
> > anatomize its deepest innards.
> 
> Not "default config", site-wide config.
> 
> The Vim installation at this hosting provider is twitted so it can't syntax
> highlight, among other problems.  I take that and the above `spamassassin -D`
> error message as signs that I ought not necessarily trust the SA config.
> 
> Together, the untrustworthiness of the site-wide config and the complexity and
> folk knowledge required to parse out the effective config, combined with config
> documentation syntax underspecification (I'll get to this in a moment) lead me
> to wishing I had a simple way to check the effective config.

I don't know about general trustworthiness of site-wide config in your
case. The above is a gross failure. Which might just be a broken install
and simply needs to be fixed. Bad, but not necessarily affecting trust.
Trustworthiness is much more -- it involves not breaking or even
changing without knowledge. If you can't trust your system admin, switch
your system.


> One specific problem I'm having is that my user_prefs config for undoing the
> site-wide rewrite_header does not appear to be working.  How does a user stop SA
> from rewriting the header?  (Note that this effort is a step towards the goal of
> preserving spam for later manually-directed `sa-learn` training.)

It looks more like SA isn't doing anything, cause basically SA has never
been run on your system. See above.

First, install SA properly, and make it run. Then check back with the
custom settings again.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}