Using Berkeley DB

[Jim Blandy <ji...@savonarola.red-bean.com>]

Karl Fogel <kf...@galois.collab.net> writes:

> Brian Behlendorf <br...@collab.net> writes:
> > > Yes.  You'll need to run it on a local disk.
> 
> Do we definitely need Berkeley DB's transaction guarantees?  I mean,
> those low-level transactions make things more convenient for the
> filesystem implementation, but they're not essential (if things work
> the way I think?).
> 
> What I don't know is how *much* more convenient... :-)  Jim?

I used to think it would be easy to squeak by without the usual
database-y kinds of facilities.  Now, though, that seems really
annoying.

Transactions are to multiple-reader, multiple-writer databases what
mutexes and condition variables are to multi-theaded programming.  You
can always get by with weaker promises from the underlying medium (for
databases, the filesystem; for multi-threaded programming, the
instruction set and your memory/cache system), but it's a pain.  And
in fact, the first thing you'd probably do is implement yourself some
mutexes and condition variables.  :)

If you want recoverability, you pretty much end up implementing
something like what Berkeley DB has.  If someone can suggest an
alternative to Berkeley DB that provides transactions and
recoverability, but also works over NFS, then that would be great.  At
this point, it wouldn't be much trouble to switch media.

I think Greg Stein mentioned using mySQL instead of Berkeley DB.  I
don't know a lot about mySQL, but my understanding was that it didn't
really care about recoverability much.

Recoverability and quick locking (i.e., no 30-second waits) were
things I was very much looking forward to getting right.  I don't care
how those things happen, but cutting those corners would be extremely
disappointing.  Fortunately, I don't think anyone's suggesting that.


Here's the manual's bit on Berkeley DB vs. NFS.  There are only
certain files that need to be on a local disk.  The bulk of the data
can live on NFS-mounted filesystems, although the system will run more
slowly.  Note especially the bit at the end about Linux.

(Some terminology:
  - "home directory" refers to the directory containing the various Unix files
    that make up the DB.
  - "regions" are the shared memory regions used as cache and, on
    certain architectures, for locking.
  - an "environment" is a group of database tables that can be protected
    with a single transactions.  The entire Subversion filesystem lives in
    a bunch of tables within a single environment. 
)


 Berkeley DB Reference Guide:                           [Prev] [Ref][Next]
      Environment

                             Remote filesystems

When regions are backed by the filesystem, it is a
common error to attempt to create Berkeley DB environments backed by
remote file systems such as the Network File System (NFS) or the
Andrew File System (AFS). Remote filesystems rarely support mapping
files into process memory, and even more rarely support correct
semantics for mutexes after the attempt succeeds. For this reason, we
strongly recommend that the database environment directory reside in a
local filesystem.

For remote file systems that do allow system files to be mapped into process
memory, home directories accessed via remote file systems cannot be used
simultaneously from multiple clients. None of the commercial remote file
systems available today implement coherent, distributed shared memory for
remote-mounted files. As a result, different machines will see different
versions of these shared regions and the system behavior is undefined.

Databases, log files and temporary files may be placed on remote
filesystems, *as long as the remote filesystem fully supports standard
POSIX filesystem semantics*, although the application may incur a
performance penalty for doing so. Obviously, NFS-mounted databases
cannot be accessed from more than one Berkeley DB environment (and
therefore from more than one system), at a time since no Berkeley DB
database may be accessed from more than one Berkeley DB environment at
a time.

Linux note:
     Some Linux releases are known to not support complete semantics for the
     POSIX fsync call on NFS-mounted filesystems. No Berkeley DB files
     should be placed on NFS-mounted filesystems on these systems.

                                                        [Prev] [Ref][Next]

Copyright Sleepycat Software

Re: Using Berkeley DB

[Jim Blandy <ji...@savonarola.red-bean.com>]

> Let's get this right. Jim questioned "recoverability". MySQL is *very*
> strong on the recoverability side. I'll refer people to:
> 
>     http://www.phpbuilder.com/columns/tim20000705.php3
> 
>     "... it makes up for it by apparently never corrupting its data files.
>     ... In over a year of running MySQL, I haven't ever seen a single case
>     of database or index corruption. In the same timeframe, I have done 2 or
>     3 recoveries of a couple different Postgres databases."
>     
>     [ this from one of the SourceForge engineers ]

That's a good user experience, which certainly counts for something.
But what's the architecture?  How is MySQL designed to ensure that the
database won't be corrupted?

Re: Using Berkeley DB

[Greg Stein <gs...@lyra.org>]

On Mon, Sep 11, 2000 at 11:53:20PM -0700, Bryan O'Sullivan wrote:
> j> I think Greg Stein mentioned using mySQL instead of Berkeley DB.  I
> j> don't know a lot about mySQL, but my understanding was that it
> j> didn't really care about recoverability much.
> 
> MySQL only acquired support for transactions recently, and its
> developers are still quite early in the development cycle for that set
> of features.  In addition, MySQL uses Berkeley DB to handle tables
> with transactional properties, so it will continue to be turtles most
> of the way down, even if you stack a rhino on top.

Woah, woah, woah.

Let's get this right. Jim questioned "recoverability". MySQL is *very*
strong on the recoverability side. I'll refer people to:

    http://www.phpbuilder.com/columns/tim20000705.php3

    "... it makes up for it by apparently never corrupting its data files.
    ... In over a year of running MySQL, I haven't ever seen a single case
    of database or index corruption. In the same timeframe, I have done 2 or
    3 recoveries of a couple different Postgres databases."
    
    [ this from one of the SourceForge engineers ]

Sometimes people use "transactional" to refer to how data is placed onto the
disk. Does it save it in a safe manner? Basically, "does it retain its
integrity".

The second form of "transactional" refers to transacting a group of changes.
The classic example is a credit/debit financial system. Make sure both parts
get committed, or make sure neither is committed.

The latest *development* version of MySQL now has transactions and rollbacks
of sets of changes. Only in this recent dev version does MySQL begin to use
Berkeley DB. Historically, MySQL has had its own database format. IOW, it
isn't necessarily "turtles all the way down".

So to address Jim's question: MySQL is one damn solid database, and very
fast. It does not have SQL transactions. When I brought up MySQL, it was in
reference to building the SVN FS such that SQL transactions would not be
required -- that we would be changing tables in a safe manner.

Personally, I would love to see SVN hosted on a MySQL database. That opens a
lot of doors for tools/utilities/apps, and provides a kickass SQL /
relational database to work with. But I also recognize that it will happen
post 1.0 :-)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Re: Using Berkeley DB

[Bryan O'Sullivan <bo...@serpentine.com>]

j> I think Greg Stein mentioned using mySQL instead of Berkeley DB.  I
j> don't know a lot about mySQL, but my understanding was that it
j> didn't really care about recoverability much.

MySQL only acquired support for transactions recently, and its
developers are still quite early in the development cycle for that set
of features.  In addition, MySQL uses Berkeley DB to handle tables
with transactional properties, so it will continue to be turtles most
of the way down, even if you stack a rhino on top.

j> Here's the manual's bit on Berkeley DB vs. NFS. [...]  Note
j> especially the bit at the end about Linux.

My understanding of the potential pitfalls in this area is a bit more
detailed.  There are three problems.

1.  Linux distributions prior to the 2.2 kernel series didn't use or
    support the NFS lock daemon, which is used to provide Unix-style
    locking semantics over NFS.  Recent kernels support lockd, and
    recent distributions ship with the necessary user-space support.

2.  The default behaviour for current Linux NFSv2 and NFSv3
    implementations is to use async writes.  This violates the NFS
    specs, which require that writes be synchronous (i.e. the server
    doesn't send an RPC return message to the client until the data is
    on disk), moutns can be forced to use synchronous writes if
    necessary.  The 2.4 kernel defaults to synchronous writes for
    NFSv3.  Async-mounted NFS filesystems will appear to honour
    fsync() calls without actually doing so.

    Since synchronous writes yield truly crappy performance, it's not
    a bad idea to leave async writes turned on if the client and
    server are running stable kernels with reliable electrical power.

3.  All stock (i.e. issued by Alan Cox) Linux 2.2 kernels have NFS
    implementations that are deficient in various respects.  There's a
    set of patches that bring them up to scratch, but Alan hasn't
    integrated them into the official tree yet.  All recent Linux
    distributions include these patches in the kernels they ship.

What the above amounts to is a set of *potential* problems for people
using SVN over NFS with Linux servers or clients, each of which can be
evaluated and avoided if necessary.

	<b