You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Chris <cp...@earthlink.net> on 2005/05/30 04:59:42 UTC
cannot open bayes databases
While running my sa-learn script I'm suddenly getting the below:
debug: lock: 26313
created /home/chris/.spamassassin/bayes.lock.cpollock.localdomain.26313
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
0 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
1 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
2 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
3 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
4 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
5 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
6 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
7 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
8 retries
debug: lock: 26313 trying to get lock on /home/chris/.spamassassin/bayes with
9 retries
Cannot open bayes databases /home/chris/.spamassassin/bayes_* R/W: lock
failed: File exists
Looking in my ~.SpamAssassin folder I see a bayes.lock file is still there
after running sa-learn. I've deleted it and the error persists. I'm sure
there is an easy fix for this.
Thanks
Chris
--
Chris
Registered Linux User 283774 http://counter.li.org
21:56:49 up 9:37, 2 users, load average: 0.74, 0.81, 0.68
Mandriva Linux 10.1 Official, kernel 2.6.8.1-12mdk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
30. Ooops. Save your work, everyone. FAST!
--Top 100 things you don't want the sysadmin to say
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: cannot open bayes databases
Posted by Anthony DeRobertis <an...@derobert.net>.
FYI, I just saw this message after upgrading to Sarge. The solution is to use
db4.2_upgrade on your bayes and AWL dbs.
Re: cannot open bayes databases
Posted by Chris <cp...@earthlink.net>.
On Monday 30 May 2005 05:58 pm, Bruno Delbono wrote:
> On Mon, 30 May 2005 15:47:01 -0700, Chris <cp...@earthlink.net> wrote:
> > The problem appears to be that database is corrupted. I renamed the old
> > ~/.spamassassin, ran my reporter script an all was well. Luckily I have
> > a
> > good 1000 or so spams saved up, hams are another matter. Is there any
> > way to save the database or is it just 'gone'?
>
> This is vital in Bayes database maintenance. Over time a lot of valuable
> information will be stored in the Bayes database. Should the database
> become corrupted for some reason you don't want to start all over with
> seeding it and then having to wait the time it takes to accumulate the
> number of tokens that make up a mature system again. Create a daily backup
> for ~/.spamassassin (this is where the database files live) and do daily
> full backups.
>
> I've found that recovering bayes berkeley db databases can be a very big
> chore and would suggest that you move to a proper SQL datbase (MySQL,
> PostgreSQL, SQLite) as soon as possible. BerkeleyDB is horrible! Both in
> performance and reliability.
>
> http://spamassassin.apache.org/full/3.0.x/dist/sql/README.bayes
>
> -Bruno
Thanks Bruno, I'll give that at look through this coming weekend when I've got
more time. For now I ran sa-learn --backup > backup.txt as you suggested.
Looking at the backup file I see there was one line, the last one, that
wasn't complete. I deleted that line, ran sa-learn --restore backup.txt and
the database appears to have been mostly recreated. The number of spam and
ham are the same, and, running the reporter script gave no errors this time.
Maybe I got lucky, maybe not. Only time will tell over the rest of this week.
I know where the corruption came from, I had some bad ram that was causing
lockups and reboots during the day when I was at work. That appears to have
been fixed now.
Chris
--
Chris
Registered Linux User 283774 http://counter.li.org
19:05:34 up 1 day, 6:46, 2 users, load average: 1.31, 1.37, 0.99
Mandriva Linux 10.1 Official, kernel 2.6.8.1-12mdk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Bite off, dirtball."
Richard Sexton, richard@gryphon.COM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: cannot open bayes databases
Posted by Kris Deugau <kd...@vianet.ca>.
> Bruno Delbono wrote:
> > I've found that recovering bayes berkeley db databases can be a
> > very big chore and would suggest that you move to a proper SQL
> > datbase (MySQL, PostgreSQL, SQLite) as soon as possible.
> > BerkeleyDB is horrible! Both in performance and reliability.
Michael Parker replied:
> 1) This is bordering on FUD. Berkeley DB has it's uses, and as such
> it is good to recognize those uses and use the right DB for the right
> job.
Indeed. For the record, in the time I've been using SA with Bayes
(almost since it was introduced - I've been using SA since ~v2.2 or
2.3), I've *NEVER* had Bayes corruption, and I'm still using the "same"
databases as I started with. I've had *other* problems related to how
(auto)learning was implemented, and how expiry works, but that didn't
corrupt the DB, just made the rest of the system bog down occasionally.
<g>
The closest thing to a BerkeleyDB-related issue that I've had was
transferring a Bayes DB from a RH7.3 system to a Debian Woody system,
where I had to do some magic to get the right version of the BDB library
linked into Perl's DB_File module. (It didn't help that I'd added db4
to the RH system, when db3 was the "real" version of BDB shipping with
the OS, and Debian shipped with (IIRC) db2 linked to DB_File.)
-kgd
--
Get your mouse off of there! You don't know where that email has been!
Re: cannot open bayes databases
Posted by Michael Parker <pa...@pobox.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bruno Delbono wrote:
> I've found that recovering bayes berkeley db databases can be a
> very big chore and would suggest that you move to a proper SQL
> datbase (MySQL, PostgreSQL, SQLite) as soon as possible.
> BerkeleyDB is horrible! Both in performance and reliability.
>
1) This is bordering on FUD. Berkeley DB has it's uses, and as such
it is good to recognize those uses and use the right DB for the right
job. For those interested, here is a benchmark of the various bayes
backend storage methods:
http://wiki.apache.org/spamassassin/BayesBenchmarkResults
You can judge for yourself how well each database performs.
2) I'm sorry, but I simply can not recommend SQLite as a backend store
for the bayes data. Unless someone can prove me wrong, it's simply
not up to the task.
Michael
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFCm/dfG4km+uS4gOIRAoVpAJsFBwzF3bhIkXrqsBG3BjAS0ZqX1QCfRS6i
nfJwLAYz7C9ILdw/mcP568s=
=1ZEA
-----END PGP SIGNATURE-----
Re: cannot open bayes databases
Posted by Bruno Delbono <br...@mail.ac>.
On Mon, 30 May 2005 15:47:01 -0700, Chris <cp...@earthlink.net> wrote:
> The problem appears to be that database is corrupted. I renamed the old
> ~/.spamassassin, ran my reporter script an all was well. Luckily I have
> a
> good 1000 or so spams saved up, hams are another matter. Is there any
> way to save the database or is it just 'gone'?
This is vital in Bayes database maintenance. Over time a lot of valuable
information will be stored in the Bayes database. Should the database
become corrupted for some reason you don't want to start all over with
seeding it and then having to wait the time it takes to accumulate the
number of tokens that make up a mature system again. Create a daily backup
for ~/.spamassassin (this is where the database files live) and do daily
full backups.
I've found that recovering bayes berkeley db databases can be a very big
chore and would suggest that you move to a proper SQL datbase (MySQL,
PostgreSQL, SQLite) as soon as possible. BerkeleyDB is horrible! Both in
performance and reliability.
http://spamassassin.apache.org/full/3.0.x/dist/sql/README.bayes
-Bruno
Re: cannot open bayes databases
Posted by Chris <cp...@earthlink.net>.
On Monday 30 May 2005 05:24 pm, Bruno Delbono wrote:
>
> Ah. It could be number of things - corrupt database, bad berkley db
> install, missing libdb_deadlock (usually libdb3/4_deadlock), permissions.
> I would suggest
>
> 1) Make sure that ~/.spamassassin is owned by you chris (both uid:gid)
> 2) If 1), then remove ~/.spamassassin (or mv it) and re-run sa-learn with
> the message and see if this problem is reproducable
> 3) Make sure that spamassassin is linked/compiled with the proper libdb
> (Berkley DB)
>
> DEBUG follows:
The problem appears to be that database is corrupted. I renamed the old
~/.spamassassin, ran my reporter script an all was well. Luckily I have a
good 1000 or so spams saved up, hams are another matter. Is there any way to
save the database or is it just 'gone'?
--
Chris
Registered Linux User 283774 http://counter.li.org
17:44:57 up 1 day, 5:25, 2 users, load average: 1.86, 1.03, 0.67
Mandriva Linux 10.1 Official, kernel 2.6.8.1-12mdk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I tripped over a hole that was sticking up out of the ground.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: cannot open bayes databases
Posted by Bruno Delbono <br...@mail.ac>.
CC'g the list as you sent the message only to me.
On Mon, 30 May 2005 15:10:16 -0700, Chris <cp...@earthlink.net> wrote:
> On Monday 30 May 2005 11:05 am, Bruno Delbono wrote:
>> Chris wrote:
>> > No joy Martin. Tried both --sync and --force-expire. Anyone else
>> with
>> > ideas?
>>
>> strace sa-learn --sync and see what exactly is it doing
>>
>> -Bruno
> Ok, here is what happens. If there is no lockfile present in the
> ~/.spamassassin folder strace sa-learn --sync seems to run ok. If I run
> sa-learn --spam [msg] it learns ok, but, leaves the lockfiles in the
> folder
> instead of removing them as I believe it is supposed to.
Ah. It could be number of things - corrupt database, bad berkley db
install, missing libdb_deadlock (usually libdb3/4_deadlock), permissions.
I would suggest
1) Make sure that ~/.spamassassin is owned by you chris (both uid:gid)
2) If 1), then remove ~/.spamassassin (or mv it) and re-run sa-learn with
the message and see if this problem is reproducable
3) Make sure that spamassassin is linked/compiled with the proper libdb
(Berkley DB)
DEBUG follows:
> debug: lock: 300
> created /home/chris/.spamassassin/bayes.lock.cpollock.localdomain.300
> debug: lock: 300 trying to get lock on /home/chris/.spamassassin/bayes
> with 0
> retries
>
> Here is the ending output of the strace. When on the "select(0,NULL....
> lines, it will continue until I stop it. Thats with a lockfile in the
> folder. I have no idea what I'm looking at so bear with me and if you
> need
> any more let me know. I tried to capture the whole strace to a file
> "strace
> sa-learn --sync > chrissync, however that created an empty file.
>
>
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=26, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {0, 652268}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=52, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {1, 160488}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=78, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {1, 39738}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=104, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {0, 848799}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=130, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {1, 345979}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=156, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {0, 570221}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=182, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {1, 307284}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=208, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {1, 5495}) = 0 (Timeout)
> write(5, "cpollock.localdomain.1163\n", 26) = 26
> link("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> "/home/chris/.spamassassin/bayes.lock") = -1 EEXIST (File exists)
> lstat64("/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.1163",
> {st_mode=S_IFREG|0600, st_size=234, ...}) = 0
> lstat64("/home/chris/.spamassassin/bayes.lock", {st_mode=S_IFREG|0600,
> st_size=25, ...}) = 0
> select(0, NULL, NULL, NULL, {1, 350244} <unfinished ...>
> [chris@cpollock chris]$
>
Re: cannot open bayes databases
Posted by Bruno Delbono <br...@mail.ac>.
Chris wrote:
> No joy Martin. Tried both --sync and --force-expire. Anyone else with ideas?
strace sa-learn --sync and see what exactly is it doing
-Bruno
Re: cannot open bayes databases
Posted by Chris <cp...@earthlink.net>.
On Monday 30 May 2005 05:02 am, martin smith wrote:
> M>-----Original Message-----
> M>From: Chris [mailto:cpollock@earthlink.net]
> M>Sent: 30 May 2005 04:00
> M>To: users@spamassassin.apache.org
> M>Subject: cannot open bayes databases
> M>
> M>While running my sa-learn script I'm suddenly getting the below:
> M>
> M>debug: lock: 26313
> M>created
> M>/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.26313
> M>debug: lock: 26313 trying to get lock on
> M>/home/chris/.spamassassin/bayes with 0 retries
> M>debug: lock: 26313 trying to get lock on
> M>/home/chris/.spamassassin/bayes with
> M>1 retries
>
> Hi Chris I had the same and deleted the .lock file then ran sa-learn --sync
> or you could use --force-expire, worth a go. If that don't work then post
> back and someone else may have more ideas.
>
> Martin
No joy Martin. Tried both --sync and --force-expire. Anyone else with ideas?
--
Chris
Registered Linux User 283774 http://counter.li.org
05:50:37 up 17:31, 2 users, load average: 0.47, 0.43, 0.39
Mandriva Linux 10.1 Official, kernel 2.6.8.1-12mdk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BOFH excuse #340:
Well fix that in the next (upgrade, update, patch release, service pack).
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RE: cannot open bayes databases
Posted by martin smith <ma...@ntlworld.com>.
M>-----Original Message-----
M>From: Chris [mailto:cpollock@earthlink.net]
M>Sent: 30 May 2005 04:00
M>To: users@spamassassin.apache.org
M>Subject: cannot open bayes databases
M>
M>While running my sa-learn script I'm suddenly getting the below:
M>
M>debug: lock: 26313
M>created
M>/home/chris/.spamassassin/bayes.lock.cpollock.localdomain.26313
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with 0 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>1 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>2 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>3 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>4 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>5 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>6 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>7 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>8 retries
M>debug: lock: 26313 trying to get lock on
M>/home/chris/.spamassassin/bayes with
M>9 retries
M>Cannot open bayes databases /home/chris/.spamassassin/bayes_*
M>R/W: lock
M>failed: File exists
M>
M>Looking in my ~.SpamAssassin folder I see a bayes.lock file
M>is still there after running sa-learn. I've deleted it and
M>the error persists. I'm sure there is an easy fix for this.
M>
M>Thanks
M>Chris
M>
M>--
M>Chris
Hi Chris I had the same and deleted the .lock file then ran sa-learn --sync
or you could use --force-expire, worth a go. If that don't work then post
back and someone else may have more ideas.
Martin