You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Paul R. Ganci" <pr...@mric.coop> on 2005/05/14 18:12:59 UTC
Bayes Database RW Lock
I am at my wits end regarding this issue. I am getting very frequent:
May 14 09:58:05 citlatepetl spamd[5125]: Cannot open bayes databases
/home/spam-filter/etc/mail/spamassassin/bayes_* R/W: lock failed: File
exists
messages. From what I have googled I have done just about everything I
could find regarding this issue. As I have a site wide Bayes database I
have the following lines in my /etc/mail/spamassassin/local.cf:
bayes_auto_expire 0
bayes_learn_to_journal 1
bayes_file_mode 0777
I have limited the number of spamd instances via:
daemon /home/spam-filter/bin/spamd -d -c -m 5
--max-conn-per-child=10 -p 783 -H /home/spam-filter/razor --user-config
--virtual-config-dir=/home/sites/%d/users/%u
The host server is RaQ550 with 2GB SDRAM so memory shouldn't be an
issue. The directory protections for the database are:
[root etc]# ls -alt /home/spam-filter/etc/mail
total 4
drwxrwxrwx 2 root root 4096 May 14 10:04 spamassassin
The file protections are:
-rw-rw-rw- 1 prganci users 165988 May 14 10:05 bayes_journal
-rw------- 1 pangione users 34 May 14 10:00 bayes.lock
-rw-rw-rw- 1 hubbard users 136548 May 14 09:58 bayes_journal.old
-rw-rw-rw- 1 root root 5279744 May 14 09:38 bayes_toks
-rw-rw-rw- 1 root root 5226496 May 9 20:23 bayes_seen
Please note that I always start off with bayes.lock with -rw-rw-rw- so
it is spamd/spamc which is changing the owner/protection on the lock
file. The /etc/procmailrc file looks like:
DROPPRIVS=yes
SHELL=/bin/sh
#LOGFILE=/var/log/procmail.log
#VERBOSE=on
TIMEOUT=60
:0fw:
* < 200000
|/home/spam-filter/bin/spamc -p 783 -u $1 -t 120
#
# Drop high scoring spam into bit bucket.
:0:
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*
/home/sites/magnoliaroad.net/users/definitespam/mail/saved-messages
# Work around procmail bug: any output on stderr will cause the "F" in
"From"
# to be dropped. This will re-add it.
# NOTE: This is probably NOT needed in recent versions of procmail
:0
* ^^rom[ ]
{
LOG="*** Dropped F off From_ header! Fixing up. "
:0 fhw
| sed -e '1s/^/F/'
}
Does anybody have any idea how I cure this problem. About the only thing
I haven't attempted is moving from a global Bayes db to a local user
Bayes db.
Thank you for any suggestions you might have.
--
Paul (prganci@mric.coop)
Re: Bayes Database RW Lock
Posted by "Paul R. Ganci" <pr...@mric.coop>.
Paul R. Ganci wrote:
> I assume this message implies my Bayes database is corrupted and that
> is why I keep finding stale lock files. Namely every
> bayes_journal_max_size = 102400 Bytes (or whatever unit) the system
> attempts to sync the Bayes database which crashes leaving behind the
> lock file.
I fixed my database (or at least so far it seems) with this sequence:
[spamassassin]# mv bayes_seen bayes_seen-raq-3.0.2
[spamassassin]# mv bayes_toks bayes_toks-raq-3.0.2
[spamassassin]# db_dump bayes_seen-raq-3.0.2 >bayes_seen-raq-3.0.2.dump
[spamassassin]# db_dump bayes_toks-raq-3.0.2 >bayes_toks-raq-3.0.2.dump
[spamassassin]# db_load -f ./bayes_seen-raq-3.0.2.dump bayes_seen
[spamassassin]# db_load -f ./bayes_toks-raq-3.0.2.dump bayes_toks
[spamassassin]# sa-learn --import
Now instead of a core dump I get:
[spamassassin]# sa-learn --sync
synced Bayes databases from journal in 0 seconds: 1005 unique entries
(1331 total entries)
All seems better now.
--
Paul (prganci@mric.coop)
Re: Bayes Database RW Lock
Posted by "Paul R. Ganci" <pr...@mric.coop>.
Matt Kettler wrote:
>Paul R. Ganci wrote:
>
>
>>-rw-rw-rw- 1 prganci users 165988 May 14 10:05 bayes_journal
>>-rw------- 1 pangione users 34 May 14 10:00 bayes.lock
>>
>>
>That's a little troubling.. are you having problems with spamd instances
>crashing? The lock is at least 5 minutes old here.. that's a long time unless an
>expiry is being run.
>
This is a month old thread, but I think I may have found out what is
going on here. I am having trouble running an expiry:
[spamassassin]# /home/spam-filter/bin/sa-learn --force-expire
__db_assert: "TYPE(h) == P_INVALID" failed: file
"../dist/../db/db_meta.c", line 106
Aborted (core dumped)
I assume this message implies my Bayes database is corrupted and that is
why I keep finding stale lock files. Namely every bayes_journal_max_size
= 102400 Bytes (or whatever unit) the system attempts to sync the Bayes
database which crashes leaving behind the lock file. Does this scenario
seem plausible or am I all wet? Is there something I can do to fix this
problem?
--
Paul (prganci@mric.coop)
Re: Bayes Database RW Lock
Posted by "Paul R. Ganci" <pr...@mric.coop>.
Matt Kettler wrote:
>>bayes_auto_expire 0
>>
>>
>With this setting you've got a cronjob running sa-learn --force-expire. Right?
>
>
Yes.
>If not, fix that.
>
>>rw-rw-rw- 1 prganci users 165988 May 14 10:05 bayes_journal
>>-rw------- 1 pangione users 34 May 14 10:00 bayes.lock
>>
>>
>That's a little troubling.. are you having problems with spamd instances
>crashing?
>
Not to my knowledge, but I will look into it.
>Also note the lock should NEVER be anything but -rw------. EVER.
>
>
Yes, good point.
--
Paul (prganci@mric.coop)
Re: Bayes Database RW Lock
Posted by Matt Kettler <mk...@evi-inc.com>.
Paul R. Ganci wrote:
> I am at my wits end regarding this issue. I am getting very frequent:
>
> May 14 09:58:05 citlatepetl spamd[5125]: Cannot open bayes databases
> /home/spam-filter/etc/mail/spamassassin/bayes_* R/W: lock failed: File
> exists
>
This is very common, and is not a problem UNLESS your autolearner never kicks
in. (ie: autolearning always fails)
Basically all it means is two SA processes tried to update the bayes DB at the
same time. If two messages come in at the same time and both get autolearned,
the second one will fail to lock the bayes DB and generate the above message.
Now, if a SA instance crashes and leaves a lockfile laying about, this can be a
problem. From then on, all SA learning will fail until the lock gets removed.
> bayes_auto_expire 0
With this setting you've got a cronjob running sa-learn --force-expire. Right?
If not, fix that.
> -rw-rw-rw- 1 prganci users 165988 May 14 10:05 bayes_journal
> -rw------- 1 pangione users 34 May 14 10:00 bayes.lock
That's a little troubling.. are you having problems with spamd instances
crashing? The lock is at least 5 minutes old here.. that's a long time unless an
expiry is being run.
However, I'd not worry too much yet, but I would check on it. If the lockfile
gets to be really old (1hr) then I'd worry.
> Please note that I always start off with bayes.lock with -rw-rw-rw- so it is spamd/spamc which is changing the owner/protection on the lock file. The /etc/procmailrc file looks like:
Please note you shouldn't start off with a bayes.lock at all. That file should
only ever be created by spamassassin, and should only exist while bayes
manipulation is in progress.
Also note the lock should NEVER be anything but -rw------. EVER.
If it's -rw-rw-rw- the whole purpose of the file is defeated.