You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Greg Stein <gs...@lyra.org> on 2000/04/20 22:24:05 UTC
Re: [Python-Dev] [OT] [Q] corruption in DB files? (fwd)
FYI, here is the response that I got regarding DB corruption.
It would be interesting to track down the change log for 1.86 and see if
it says anything about the hash problems.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------- Forwarded message ----------
Date: Thu, 20 Apr 2000 08:29:37 -0500 (CDT)
From: Skip Montanaro <sk...@mojam.com>
To: Greg Stein <gs...@lyra.org>
Cc: python-dev@python.org
Subject: Re: [Python-Dev] [OT] [Q] corruption in DB files?
>...
Greg> A question just came up elsewhere about DB and I seemed to recall
Greg> somebody mentioning the occasional corruption. Oh, maybe it was
Greg> related to multiple threads.
Yes, Berkeley DB 1.85 (exposed through the bsddb module in Python) has bugs
in the hash implementation. They never fixed them (well maybe in 1.86?),
but moved on to version 2.x. Of course, they changed the function call
interface and the file format, so many people didn't follow. They do
provide a 1.85-compatible API but you have to #include db_185.h instead of
db.h.
As far as I know, if you stick to the btree interface with 1.85 you should
be okay. Unfortunately, both the anydbm and dbhash modules both use the
hash interface, so if you're trying to be more or less portable and not
modify your Python sources, you've also got buggy db files...
Someone did create a libdb 2.x-compatible module that exposes more of the
underlying functionality. Check the VoP for it.
libdb == Berkeley DB == Sleepycat...
Skip
Re: [Python-Dev] [OT] [Q] corruption in DB files? (fwd)
Posted by Tony Finch <do...@dotat.at>.
Greg Stein <gs...@lyra.org> wrote:
>FYI, here is the response that I got regarding DB corruption.
>
>It would be interesting to track down the change log for 1.86 and see if
>it says anything about the hash problems.
Hmm. http://www.sleepycat.com/historic.html says the following:
| Specifically, we recommend that you avoid the following operations
| when using versions 1.85 and 1.86:
|
| Btree cursor (seq and put using a cursor) operations.
|
| Large numbers of btree duplicates (specifically, avoid migrating
| duplicate keys to internal pages).
|
| Large numbers of btree deletes (you should periodically dump and
| rebuild the database if you delete large numbers of records).
|
| Overwriting or deleting overflow hash key/data pairs (pairs with
| items larger than the page size).
|
| Intermixing hash cursor operations with deletes.
|
| In addition:
|
| As there was no locking support in version 1.85, you cannot
| perform concurrent read/write operations in the database.
|
| As there was no logging or transaction support in version 1.85,
| you must re-create your database whenever abnormal application
| termination occurs (e.g., either the application or the system
| crashes) as the database may have been left in a corrupted state.
Sounds like btrees are more flaky than hashes to me...
Tony.
--
f.a.n.finch fanf@demon.net dot@dotat.at
331 face first into the furnace