Re: [RFC] Conceptual locking procedure for database access [#11511]

[Keith Bostic <bo...@abyssinian.sleepycat.com>]

I've committed changes into the Berkeley DB sources to support
serializing DB_ENV handle creation, and automatically running
recovery whenever a thread of control dies with a DB_ENV handle
open.

This support is based on a new flag to the DB_ENV->open method,
DB_REGISTER.  The DB_REGISTER flag is documented at:

    http://www.bostic.com/docs/api_c/env_open.html#DB_REGISTER

    DB_REGISTER
	    Check to see if a process has failed while using the
	    database environment, that is, if a process has exited with
	    an open DB_ENV handle. (For this check to be accurate, all
	    processes using the environment must specify DB_REGISTER
	    when opening the environment.) If failure is detected and
	    the DB_RECOVER or DB_RECOVER_FATAL flags are also specified,
	    recovery will be performed and the open will proceed
	    normally. If failure is detected and no recovery flag is
	    specified, DB_RUNRECOVERY will be returned.

I have also re-written the "Architecting Transactional Data
Store applications" section of the Berkeley DB Reference Guide
to include DB_REGISTER:

    http://www.bostic.com/docs/ref/transapp/app.html

The DB_REGISTER discussion in that page is as follows:

	3. The third way to architect transactional applications is as
	a group of unrelated processes (the processes may or may not be
	multithreaded).

	If it is not practical to have a single parent for the processes
	sharing a Berkeley DB environment, each process sharing the
	database environment should log their connection to and exit
	from the environment in a way allowing a monitoring process to
	detect if a thread of control might have acquired Berkeley DB
	resources and never released them.

	Berkeley DB supports this architecture with the DB_REGISTER flag
	to the DB_ENV->open method. If the DB_REGISTER flag is set, each
	process opening the database environment first checks that no
	other process has failed while holding an open DB_ENV handle.
	If no failure is detected, the environment open will proceed
	normally. If failure is detected, and the DB_RECOVER flag was
	specified to the DB_ENV->open method, then recovery will be
	performed and the open will proceed normally. If failure is
	detected and DB_RECOVER was not specified, the open will return
	DB_RUNRECOVERY.

	For this architecture to work, all applications using the
	database environment must specify the DB_REGISTER flag when
	opening the environment. However, there is no additional
	requirement the application choose a single process to recover
	the environment, as the first process to open the database
	environment will know to perform recovery.

	There is a performance cost associated with this configuration
	of roughly one system call per open DB_ENV handle. As DB_ENV
	handles are relatively long-lived, this should not be a problem
	for most applications.

	Alternatively, applications can implement their own monitoring.
	For example, initial "watcher" process would open/create the
	Berkeley DB environment and run recovery, and then create a
	sentinel file. Any other process wanting to use the Berkeley DB
	environment checks for the sentinel file; if the sentinel file
	exists, the other process registers its process ID with the
	watcher and joins the database environment. When the other
	process finishes with the environment, it unregisters its
	process ID with the watcher. The watcher periodically checks to
	ensure that no process has failed while using the environment.
	If a process does fail while using the environment, the watcher
	removes the sentinel file, kills all processes currently using
	the environment, runs recovery, and re-creates the sentinel
	file.

A few people expressed interest in reviewing and testing this
work in Subversion when it became available.  I have contacted
them off-line.

If anyone else is interested, I would be happy to make test
snapshots of Berkeley DB available to anyone that would like to
try out this feature.

Regards,
--keith

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Keith Bostic			bostic@sleepycat.com
Sleepycat Software Inc.		keithbosticim (ymsgid)
118 Tower Rd.			+1-781-259-3139
Lincoln, MA 01773		http://www.sleepycat.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org