You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Don Levey <sp...@the-leveys.us> on 2005/04/04 15:35:52 UTC

Update on Autolearn, SA/SA-milter ID problem, etc

If the definition of insanity is doing the same thing multiple times and
expecting a different result, what is it when you're doing the same thing
multiple times, expecting the same result, and you get DIFFERENT results?
That's what seems to be happening to me.  I've been trying to search the
archives for any helpful information, but I'm having difficulty in
extracting anything that might be of use.

For some reason, I can't seem to get all the features on SpamAssassin to
work at the same time. Allow me to elaborate - here's what I've got:

	* OS: Fedora Core 2
	* MTA: Sendmail 8.12.11-4.6
	* Spamassassin: 3.0.2-1.1.fc2.rf
	* Spamass-milter: 0.3.0-1.1.fc2.rf

Here's what I want to do with them:

	* The spamd/spamass-milter processes should not run as root (user
'spamassassin').
	* I want a single set of user preferences/bayes DB.
	  While additional user preferences could in theory be OK,
 	  I want only one Bayes DB. * As the above may mention, I want to
	  use the Bayes DB for learning and auto-learning.
	* I want tagged spam to rewrite the subject.
	* I want to attch the original message to the report.
	* I want to use RBLs for things not covered otherwise in sendmail
	  (i.e. for URLs in the messages)
	* I want to use Razor/Pyzor
	* Eventually, I may drop egregious spam examples,
	  but I'm not sure I want to do that yet.

What seems to happen is that I can get some subset of these things, but not
all at once. Additionally, while I often think I've got things working
correctly, they appear to change randomly from working to non-working. The
last point, on dropping spam, seems to be happening anyway. From what I can
tell, anything with a score greater than 15 is being rejected automatically.
This is seriously reducing my spam load.

As I mentioned last week, I was getting "autolearn=failed" when BAYES_00 was
the only rule that hit. If I got ANY other rule that also hit, autolearn did
not fail. At least part of the problem there had to do with creating the
lock file for the Bayes DB; Even though I thought I was running as root, and
root owned the directory in question (/etc/mail/spamassassin) I needed to
open the permissions in order for things to work correctly.

>From what I see now, this is because if root is running it then the user
shifts to 'nobody'. This is damn inconvenient. So, I've tried to shift to
using user 'spamassassin' by using the "-u spamassassin" switch on both
spamd and spamass-milter. When I do this, though, I can't actually read the
user_prefs file for user root. But why am I even trying to open it for root,
when spamassassin is the UID?

The biggest problem right now is that for some reason message rewriting has
stopped for spam messages.  The header is tagged correctly, but the message
is never rewritten.  From my local.cf file (below), it looks like this
should be happening.  I don't know of any change I made which could account
for this, and indeed this seemed to happen overnight, when I didn't do
anything.

[local.cf]
required_score 		5
rewrite_header Subject  *** SPAM: _SCORE_ points ***
#subject_tag 		[SPAM?]
report_safe 		1
#use_terse_report 	0
use_bayes 		1
#bayes_path		/etc/mail/spamassassin/bayes_db
bayes_path		/SA-shared/bayes_db
bayes_file_mode		0666
bayes_auto_learn 	1
skip_rbl_checks 	0
use_razor2 		1
use_dcc			1
use_pyzor		1
trusted_networks	192.168/16 127/8
ok_languages		en he ru yi
ok_locales		en ru

[user_prefs for root]
# How many hits before a mail is considered spam.
 required_score		5

# Whitelist and blacklist addresses are now file-glob-style patterns, so
# "friend@somewhere.com", "*@isp.com", or "*.domain.net" will all work.
# whitelist_from	someone@somewhere.com
auto_whitelist_path	/etc/mail/spamassassin/auto-whitelist
auto_whitelist_file_mode	0666


bayes_auto_learn	1


score BIZ_TLD 			4.5
score RCVD_IN_SORBS_DUL         0.1
score RCVD_IN_SORBS_WEB		0.5
score SUBJECT_DRUG_GAP_C        3.5
score SUBJECT_DRUG_GAP_L        3.5
score SUBJECT_DRUG_GAP_VIA      3.5
score VIA_GAP_GRA               3.5
score FORGED_YAHOO_RCVD         1.5
score GAPPY_SUBJECT             2.5
score HTML_IMAGE_ONLY_04        3.5

score	BAYES_00 0 0 -4.901 -4.900
score	BAYES_05 0 0 -0.925 -2.599
score	BAYES_20 0 0 -0.730 -1.951
score	BAYES_40 0 0 -0.276 -1.096
score	BAYES_50 0 0  1.567  0.001
score	BAYES_60 0 0  3.515  1.592
score	BAYES_80 0 0  3.608  2.087
score	BAYES_95 0 0  3.514  3.514
score	BAYES_99 0 0  4.070  5.400

I don't want to clog up the bandwidth with too many files in-line that may
not be of use, so I've got:

http://www.eruditer.org:6080/spamassassin/local.cf
http://www.eruditer.org:6080/spamassassin/root-user_prefs
http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin
http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin-milter

Any help or information would be greatly appreciated.
Tnanks,
 -Don

Re: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Craig McLean <cr...@craig.dnsalias.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Don Levey wrote:
[snip]

| The latest in my quest to get SA to work properly...
|
| I've made sure that the whitelist and Bayes DB can be written to and
be read
| by 'spamassassin'.  I've set the '-u spamassassin' flag for both the
| /etc/sysconfig/spamassassin and /etc/sysconfig/spamass-milter startup
files.
| I've restarted spamd, spamass-milter, and sendmail.
|
| My ps list shows that 'spamassassin' is running spamd, and 'root' is
running
| spamass-milter.  In my maillog file, I am getting errors:
| * for 'named' accounts, spamd can't find the user_prefs file
| * for 'aliased' accounts, spamd can't find the username.
|
| I know that I can solve the latter by putting the '-x' flag on the
| spamass-milter startup line.  Do I need to worry about the former?
That is,
| am I causing any problems by running this way, or am I simply now set
up so
| that I can run user-specific rules in addition to the site-wide ones?

No need to worry, SA is looking for per-user user_prefs files, and can't
find them. Which is not a problem. It's a function of the -u option to
spamass-milter. If you don't want individual user_prefs you might turn
it off.
And as you said, -x might be useful as well....

Craig.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFCU8MSMDDagS2VwJ4RAnteAKCg71c8ufHkrHcWOHWBA55Ll28gogCfR9Yq
mcfugS5jgb9417bNibe+LcI=
=4YBD
-----END PGP SIGNATURE-----

RE: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Don Levey <sp...@the-leveys.us>.
Craig McLean wrote:
> Don, some thoughts inline..
>
> Don Levey wrote:

>>
>>> From what I see now, this is because if root is running it then the
>>> user
>> shifts to 'nobody'. This is damn inconvenient. So, I've tried to
>> shift to
>> using user 'spamassassin' by using the "-u spamassassin" switch on
>> both
>> spamd and spamass-milter. When I do this, though, I can't actually
>> read the
>> user_prefs file for user root. But why am I even trying to open it
>> for root,
>> when spamassassin is the UID?
>
> Why not combine the user_prefs and the local.cf, and move the
> whitelist somewhere where 'spamassassin' user can read/write to it?
>
The latest in my quest to get SA to work properly...

I've made sure that the whitelist and Bayes DB can be written to and be read
by 'spamassassin'.  I've set the '-u spamassassin' flag for both the
/etc/sysconfig/spamassassin and /etc/sysconfig/spamass-milter startup files.
I've restarted spamd, spamass-milter, and sendmail.

My ps list shows that 'spamassassin' is running spamd, and 'root' is running
spamass-milter.  In my maillog file, I am getting errors:
* for 'named' accounts, spamd can't find the user_prefs file
* for 'aliased' accounts, spamd can't find the username.

I know that I can solve the latter by putting the '-x' flag on the
spamass-milter startup line.  Do I need to worry about the former?  That is,
am I causing any problems by running this way, or am I simply now set up so
that I can run user-specific rules in addition to the site-wide ones?

 -Don

p.s. In case it wasn't clear, SpamAssassin really rocks!  I don't want my
current frustration to get in the way of the appreciation and adulation the
developers deserve.

Re: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Craig McLean <cr...@craig.dnsalias.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Don Levey wrote:
| Craig McLean wrote:
|
|
|>>	* The spamd/spamass-milter processes should not run as root (user
|>>'spamassassin').
|>
|>I gather from your previous mail that you already run this as
|>"spamassassin". Make sure it owns the bayes files defined by
|>bayes_path. I created a subdirectory owned by the user and let SA get
|>on with it.
|>
|
| I had tried running as 'spamassassin', but ran into difficulties.  In
| particular, it kept giving errors that it couldn't open
| /root/.spamassassin/user_prefs for writing, even when I made the file and
| the directory wide-open (777).  Since I seem to recall seeing
somewhere that
| I should make changed to the user_prefs and not the local.cf (as that
might
| be updated and overwritten with upgrades), I had been using the user_prefs
| instead.  I even went to the point of setting up a wide-open
user_prefs file
| in a wide open directory, and linking to that for all users, but that
didn't
| help (it still looked only for the one in the root home dir)

I didn't think local.cf is overwritten during upgrades. I hope it
doesn't, that would be counter-productive. It is true that the
/usr/[local]/share/spamassassin directory may well get overwritten,
which is why local rules should be in local.cf.
Also, I believe SA reads *all* files ending in .cf in
/etc/mail/spamassassin for configuration, so you could just call yours
localconfig.cf or some such.

|
| I'm getting header tags, but I'm not getting message rewriting/attachment,
| or a subject rewrite.
|

Spooky. I don't want to sound like a windows specialist, but have you
tried stopping and starting spamd?

|>>	* I want to use RBLs for things not covered otherwise in sendmail
|>>	  (i.e. for URLs in the messages)
|>
|>Make sure you have the perl Net::DNS stuff installed. Check with
|>'spamassassin -D --lint, look for:
|>debug: is Net::DNS::Resolver available? yes
|>
|
| I *think* this is set up correctly; I'm not currently getting any errors
| that I can see.  That line is indeed present.

It should be fairly obvious from the spam you get if you see rules like
RCVD_IN_SORBS, RCVD_IN_BL_SPAMCOP_NET or other hits on RBL: rules.


|
|>>	* Eventually, I may drop egregious spam examples,
|>>	  but I'm not sure I want to do that yet.
|>
|>Well, it can be done if you choose to.
|>
|
| Not only that, but it seems to be happening now!  I vaguely remember
seeing
| which config file would control this, but re-Googling for it doesn't turn
| anything up now.  Damn this memory!

Likewise..

|
|>>What seems to happen is that I can get some subset of these things,
|>>but not
|>>all at once. Additionally, while I often think I've got things
|>>working
|>>correctly, they appear to change randomly from working to
|>>non-working.
|>
|>Can you be more specific? What's not working? Any error messages in
|>messages/maillog/&c.
|>
|
| At this particular moment, the big problem is the subject/message
rewriting.
| But then I'm still running as root (or, apparently, 'nobody') and I'm not
| sure this is the best thing to do.

Probably not, get a dedicated user for spamd and use that, keeps things
tidy.

|
|>>The last point, on dropping spam, seems to be happening anyway. From
|>>what I can
|>>tell, anything with a score greater than 15 is being rejected
|>>automatically.
|>>This is seriously reducing my spam load.
|>
|>That may well be a function of how SA/sendmail are configured on
|>Fedora?
|>
|
| It could be - but that wasn't happening as of Friday.  I was seeing scores
| into the 20s come through - but tagged/rewritten.

Out of interest, you don't have a conflicting user_prefs laying about in
~ either root's or spamassassin's $HOME/.spamassassin do you? If so, get
them out of the way until you get the basic config up and running. You
never know...

[snip]

|
| I don't think I'm getting errors on the whitelist, just user_prefs.  But I
| *could* combine the user_prefs and local.cf files (I did that briefly,
but I
| thought that was a bad idea for some reason or another).

Seeing as you want site-wide config and bayes, a site-wide config file
would make more sense that using a basic config in
/etc/mail/spamassassin and added configuration elsewhere.

|
| I'm not seeing any error in maillog (yes, you've got the location correct)
| nor anything in 'spamassassin -D --lint'.  Running the latest message
itself
| through spamassassin -D shows that it is tagged correctly, and indeed
it is
| being rewritten properly (sbject and body).  I ran that test as root; this
| must have something to do with user IDs but I'm seeing no errors that
I can
| find.

Well, spamd is just spamassassin (sort of), so if it works from the
command-line as root but not as "nobody" or "spamassassin", it probably:
1) something in root's user_prefs that's not in nobody's or spamassassin's
2) the opposite.

I'd consolidate the stuff in root's user_prefs into
/etc/mail/spamassassin/local.cf and (re)move all user_prefs from
$HOME's, then re-start spamd (remember it only picks up config changes
at startup...) and see if it's fixed.

|
|
|>>http://www.eruditer.org:6080/spamassassin/local.cf
|>>http://www.eruditer.org:6080/spamassassin/root-user_prefs
|>>http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin
|>>http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin-milter
|>
|>Can't get to those URL's, timeout...
|>
|
| Hmm... It seems to be working from the outside; perhaps you're on a
firewall
| that blocks my "special" port?  Unfortunately, my ISP doesn't want me
to run
| on port 80.  I think I posted the first two anyway; the latter two are the
| startups from the /etc/sysconfig directory.  I'll post those if you think
| it's helpful, or anything else for that matter...

Yeah, it's working now. God knows what was going on.

| Thanks for your help!

My pleasure.
Good Luck!
Craig.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFCUWrmMDDagS2VwJ4RAouuAKDbrVT9un8+4ZRwszXXjhaLG0amdwCghSpK
cKgh3vpujbQKcwvJaHatROc=
=QFRw
-----END PGP SIGNATURE-----

RE: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Don Levey <sp...@the-leveys.us>.
Don Levey wrote:
> Craig McLean wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Don Levey wrote:
>>
>> [snip]
>> [Spamassassin rejecting mail above a certain score]
>>> Not only that, but it seems to be happening now!  I vaguely remember
>>> seeing which config file would control this, but re-Googling for it
>>> doesn't turn anything up now.  Damn this memory!
>>
>> AHA! It came to me, it's the spamass-milter. There is a startup
>> option (-r <n>) where n is the score to reject at.
>> Also, check that it's not running with -m/-M, that would screw thing
>> up. In fact, it's probably worth checking the whole milter config
>> against the man page.
>>
>> Cheers!
>> Craig.
>
> Craig,
> I think that's it!  This was being set in the /etc/init.d startup
> script, -r
> 15.
> Also, -m was set, which (according to the man page) would disable
> subject/body rewriting.
> Of all the things I played around with, THAT one was from the stock
> files.
>
> Thanks for all your help;  I'll make sure that this works correctly
> (awaiting the next spam message now), take a snapshot, and then start
> playing around with the dedicated user ID.
>
>  -Don

That was indeed the problem.  I was just able to check, and my spam is now
being tagged.
Thanks again!
 -Don


RE: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Don Levey <sp...@the-leveys.us>.
Craig McLean wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Don Levey wrote:
>
> [snip]
> [Spamassassin rejecting mail above a certain score]
>> Not only that, but it seems to be happening now!  I vaguely remember
>> seeing which config file would control this, but re-Googling for it
>> doesn't turn anything up now.  Damn this memory!
>
> AHA! It came to me, it's the spamass-milter. There is a startup option
> (-r <n>) where n is the score to reject at.
> Also, check that it's not running with -m/-M, that would screw thing
> up. In fact, it's probably worth checking the whole milter config
> against the man page.
>
> Cheers!
> Craig.

Craig,
I think that's it!  This was being set in the /etc/init.d startup script, -r
15.
Also, -m was set, which (according to the man page) would disable
subject/body rewriting.
Of all the things I played around with, THAT one was from the stock files.

Thanks for all your help;  I'll make sure that this works correctly
(awaiting the next spam message now), take a snapshot, and then start
playing around with the dedicated user ID.

 -Don

Re: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Craig McLean <cr...@craig.dnsalias.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Don Levey wrote:

[snip]
[Spamassassin rejecting mail above a certain score]
| Not only that, but it seems to be happening now!  I vaguely remember
seeing
| which config file would control this, but re-Googling for it doesn't turn
| anything up now.  Damn this memory!

AHA! It came to me, it's the spamass-milter. There is a startup option
(-r <n>) where n is the score to reject at.
Also, check that it's not running with -m/-M, that would screw thing up.
In fact, it's probably worth checking the whole milter config against
the man page.

Cheers!
Craig.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFCUW3oMDDagS2VwJ4RAmjTAJ0YEc5vkmDcfx+GHO2RQ4ocsqtZKACgoOA/
LwJjGjySxEJj7dYgC1RRN5Q=
=2ZAQ
-----END PGP SIGNATURE-----

RE: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Don Levey <sp...@the-leveys.us>.
Craig McLean wrote:

>>
>> 	* The spamd/spamass-milter processes should not run as root (user
>> 'spamassassin').
>
> I gather from your previous mail that you already run this as
> "spamassassin". Make sure it owns the bayes files defined by
> bayes_path. I created a subdirectory owned by the user and let SA get
> on with it.
>
I had tried running as 'spamassassin', but ran into difficulties.  In
particular, it kept giving errors that it couldn't open
/root/.spamassassin/user_prefs for writing, even when I made the file and
the directory wide-open (777).  Since I seem to recall seeing somewhere that
I should make changed to the user_prefs and not the local.cf (as that might
be updated and overwritten with upgrades), I had been using the user_prefs
instead.  I even went to the point of setting up a wide-open user_prefs file
in a wide open directory, and linking to that for all users, but that didn't
help (it still looked only for the one in the root home dir)

>> 	* I want a single set of user preferences/bayes DB.
>> 	  While additional user preferences could in theory be OK,
>>  	  I want only one Bayes DB.
>
> OK, the prefs in /etc/mail/spamassassin/*.cf and the bayes BD in
> bayes_path then.
>
I think I'm there now; when I tried to use the -u flag on the startup
command for spamassassin and spamass-milter, I got checks to each individual
user.

>> 	* As the above may mention, I want to use the Bayes DB for learning
>> and auto-learning.
>
> Should work fine as long as the user running spamd owns the
> directory/files used by bayes.
>
So far this seems to be working.

>> 	* I want tagged spam to rewrite the subject.
>> 	* I want to attch the original message to the report.
>
> looks like that's set up fine, judging by your local.cf
>
I'm getting header tags, but I'm not getting message rewriting/attachment,
or a subject rewrite.

>> 	* I want to use RBLs for things not covered otherwise in sendmail
>> 	  (i.e. for URLs in the messages)
>
> Make sure you have the perl Net::DNS stuff installed. Check with
> 'spamassassin -D --lint, look for:
> debug: is Net::DNS::Resolver available? yes
>
I *think* this is set up correctly; I'm not currently getting any errors
that I can see.  That line is indeed present.

>
>> 	* Eventually, I may drop egregious spam examples,
>> 	  but I'm not sure I want to do that yet.
>
> Well, it can be done if you choose to.
>
Not only that, but it seems to be happening now!  I vaguely remember seeing
which config file would control this, but re-Googling for it doesn't turn
anything up now.  Damn this memory!

>> What seems to happen is that I can get some subset of these things,
>> but not
>> all at once. Additionally, while I often think I've got things
>> working
>> correctly, they appear to change randomly from working to
>> non-working.
>
> Can you be more specific? What's not working? Any error messages in
> messages/maillog/&c.
>
At this particular moment, the big problem is the subject/message rewriting.
But then I'm still running as root (or, apparently, 'nobody') and I'm not
sure this is the best thing to do.

>> The last point, on dropping spam, seems to be happening anyway. From
>> what I can
>> tell, anything with a score greater than 15 is being rejected
>> automatically.
>> This is seriously reducing my spam load.
>
> That may well be a function of how SA/sendmail are configured on
> Fedora?
>
It could be - but that wasn't happening as of Friday.  I was seeing scores
into the 20s come through - but tagged/rewritten.

>> As I mentioned last week, I was getting "autolearn=failed" when
>> BAYES_00 was
>> the only rule that hit. If I got ANY other rule that also hit,
>> autolearn did
>> not fail. At least part of the problem there had to do with creating
>> the
>> lock file for the Bayes DB; Even though I thought I was running as
>> root, and
>> root owned the directory in question (/etc/mail/spamassassin) I
>> needed to
>> open the permissions in order for things to work correctly.
>
> I'd imagine that spamd runs as root only for long enough to create the
> priv'd socket it needs, and then drops privs. I have everything in
> /var/bayesdb/bayes_* and /var/bayesdb is 755 owned by 'spam' user
> (which runs the milter/spamd). /etc/mail/spamassassin is 755 owned by
> root. No problems..
>
I've tried to move things off to a new directory /SA-shared.  The Bayes DB
is there now.  but I'm still back to running as root, to avoid the
user_prefs errors mentioned above.

>>
>>> From what I see now, this is because if root is running it then the
>>> user
>> shifts to 'nobody'. This is damn inconvenient. So, I've tried to
>> shift to
>> using user 'spamassassin' by using the "-u spamassassin" switch on
>> both
>> spamd and spamass-milter. When I do this, though, I can't actually
>> read the
>> user_prefs file for user root. But why am I even trying to open it
>> for root,
>> when spamassassin is the UID?
>
> Why not combine the user_prefs and the local.cf, and move the
> whitelist somewhere where 'spamassassin' user can read/write to it?
>
I don't think I'm getting errors on the whitelist, just user_prefs.  But I
*could* combine the user_prefs and local.cf files (I did that briefly, but I
thought that was a bad idea for some reason or another).

>> The biggest problem right now is that for some reason message
>> rewriting has
>> stopped for spam messages.  The header is tagged correctly, but the
>> message
>> is never rewritten.  From my local.cf file (below), it looks like
>> this
>> should be happening.  I don't know of any change I made which could
>> account
>> for this, and indeed this seemed to happen overnight, when I didn't
>> do
>> anything.
>
> [snip]
> The config looks ok to me, but I'm no expert. Any error messages in
> /var/log/maillog (or wherever on Fedora), or in the output from
> spamassassin -D --lint?
>
I'm not seeing any error in maillog (yes, you've got the location correct)
nor anything in 'spamassassin -D --lint'.  Running the latest message itself
through spamassassin -D shows that it is tagged correctly, and indeed it is
being rewritten properly (sbject and body).  I ran that test as root; this
must have something to do with user IDs but I'm seeing no errors that I can
find.

>>
>> http://www.eruditer.org:6080/spamassassin/local.cf
>> http://www.eruditer.org:6080/spamassassin/root-user_prefs
>> http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin
>> http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin-milter
>
> Can't get to those URL's, timeout...
>
Hmm... It seems to be working from the outside; perhaps you're on a firewall
that blocks my "special" port?  Unfortunately, my ISP doesn't want me to run
on port 80.  I think I posted the first two anyway; the latter two are the
startups from the /etc/sysconfig directory.  I'll post those if you think
it's helpful, or anything else for that matter...

Thanks for your help!
 -Don

Re: Update on Autolearn, SA/SA-milter ID problem, etc

Posted by Craig McLean <cr...@craig.dnsalias.com>.
Don, some thoughts inline..

Don Levey wrote:
> If the definition of insanity is doing the same thing multiple times and
> expecting a different result, what is it when you're doing the same thing
> multiple times, expecting the same result, and you get DIFFERENT results?

Sounds like the definition of computer sciences to me ;-)

[snip]

> Here's what I want to do with them:

Sounds like the setup I have here (with the exception of pyzor) but I 
run it on BSD instead of Fedora. Some comments below...

> 
> 	* The spamd/spamass-milter processes should not run as root (user
> 'spamassassin').

I gather from your previous mail that you already run this as 
"spamassassin". Make sure it owns the bayes files defined by bayes_path. 
I created a subdirectory owned by the user and let SA get on with it.

> 	* I want a single set of user preferences/bayes DB.
> 	  While additional user preferences could in theory be OK,
>  	  I want only one Bayes DB.

OK, the prefs in /etc/mail/spamassassin/*.cf and the bayes BD in 
bayes_path then.

> 	* As the above may mention, I want to use the Bayes DB for learning and auto-learning.

Should work fine as long as the user running spamd owns the 
directory/files used by bayes.

> 	* I want tagged spam to rewrite the subject.
> 	* I want to attch the original message to the report.

looks like that's set up fine, judging by your local.cf

> 	* I want to use RBLs for things not covered otherwise in sendmail
> 	  (i.e. for URLs in the messages)

Make sure you have the perl Net::DNS stuff installed. Check with 
'spamassassin -D --lint, look for:
debug: is Net::DNS::Resolver available? yes

> 	* I want to use Razor/Pyzor

OK. Haven't bothered with them yet.

> 	* Eventually, I may drop egregious spam examples,
> 	  but I'm not sure I want to do that yet.

Well, it can be done if you choose to.

> What seems to happen is that I can get some subset of these things, but not
> all at once. Additionally, while I often think I've got things working
> correctly, they appear to change randomly from working to non-working.

Can you be more specific? What's not working? Any error messages in 
messages/maillog/&c.

> The last point, on dropping spam, seems to be happening anyway. From what I can
> tell, anything with a score greater than 15 is being rejected automatically.
> This is seriously reducing my spam load.

That may well be a function of how SA/sendmail are configured on Fedora?

> As I mentioned last week, I was getting "autolearn=failed" when BAYES_00 was
> the only rule that hit. If I got ANY other rule that also hit, autolearn did
> not fail. At least part of the problem there had to do with creating the
> lock file for the Bayes DB; Even though I thought I was running as root, and
> root owned the directory in question (/etc/mail/spamassassin) I needed to
> open the permissions in order for things to work correctly.

I'd imagine that spamd runs as root only for long enough to create the 
priv'd socket it needs, and then drops privs. I have everything in 
/var/bayesdb/bayes_* and /var/bayesdb is 755 owned by 'spam' user (which 
runs the milter/spamd). /etc/mail/spamassassin is 755 owned by root. No 
problems..

> 
>>>From what I see now, this is because if root is running it then the user
> shifts to 'nobody'. This is damn inconvenient. So, I've tried to shift to
> using user 'spamassassin' by using the "-u spamassassin" switch on both
> spamd and spamass-milter. When I do this, though, I can't actually read the
> user_prefs file for user root. But why am I even trying to open it for root,
> when spamassassin is the UID?

Why not combine the user_prefs and the local.cf, and move the whitelist 
somewhere where 'spamassassin' user can read/write to it?

> The biggest problem right now is that for some reason message rewriting has
> stopped for spam messages.  The header is tagged correctly, but the message
> is never rewritten.  From my local.cf file (below), it looks like this
> should be happening.  I don't know of any change I made which could account
> for this, and indeed this seemed to happen overnight, when I didn't do
> anything.

[snip]
The config looks ok to me, but I'm no expert. Any error messages in 
/var/log/maillog (or wherever on Fedora), or in the output from 
spamassassin -D --lint?

> 
> http://www.eruditer.org:6080/spamassassin/local.cf
> http://www.eruditer.org:6080/spamassassin/root-user_prefs
> http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin
> http://www.eruditer.org:6080/spamassassin/sysconfig-spamassassin-milter

Can't get to those URL's, timeout...

Cheers!
Craig.