Aborting Subversion transactions.

["C. Michael Pilato" <cm...@collab.net>]

To abort a Subversion txn (transaction) involves the following steps:

   1.  Read the txn, making sure it hasn't been committed.
   2.  Delete the mutable nodes under that txn.
   3.  Delete any 'copies' table records created in the txn.
   4.  Delete any 'changes' table records created in the txn.
   5.  Remove the txn itself from the 'transactions' table.

Today, this is all done in a single Berkeley transaction (or "trail",
if you are familiar with the libsvn_fs code).

The problem with doing all this junk in one trail is that any locks
taken out for the purposes of removing records are held until the
until trail completes.  Given a big enough commit which happens to
fail, the cleanup of the failed commit could itself exhaust the
Berkeley DB lock allocation and wedge the repository.

Now, it would be simple to break up the steps of the cleanup into
various trails, but now we introduce timing problems.  If some part of
the cleanup fails, what do we want to be left with?  A txn that's
missing some or all of its 'copies' records?  A txn that's missing
some or all of its 'changes' records?  Or what about no txn at all,
but dangling 'copies', 'nodes', and 'changes' table records?  The
cosmetic danger is that we might lose the chance to fully cleanup the
txn.  But the far more dangerous aspect of breaking this up into
various trails is that, in theory, someone might come along and decide
to start working on this txn again, possible even committing it.

I need not speak of the bad things that could happen if someone was
able to successfully commit a txn that was missing some parts of its
record-keeping data whose absence wouldn't be noted in the commit_txn
process.

To handle this cleanup process correctly, it seems we need a new state
for transactions, to denote that they are "dead".  The first action of
svn_fs_abort_txn() is to set the transaction state to "dead", and then
call a new public function svn_fs_cleanup_txn (which takes a
transaction name, not an svn_fs_txn_t object).  You are *never*
allowed to open a dead transaction -- you may only call
svn_fs_cleanup_txn() on it.

The new svn_fs_cleanup_txn() would always resume cleanup from wherever
it left off, and always leave enough pointers around so that if
something goes wrong, the cleaup attempt can be re-tried/continued
later.  In other words, it leaves no dangling nodes, doesn't lose
references to copies before the copies themselves are gone, etc.

But then again, this could all just be overkill.  Maybe it's enough to make
svn_fs_abort_txn() just delete the transaction first, and then go
about cleaning up the other stuff.  If that fails, we get back an
error and there is just unreachable cruft in the filesystem that sits
around until the next dump/load cycle.

Thoughts (quickly, please)?

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Aborting Subversion transactions.

["C. Michael Pilato" <cm...@collab.net>]

mark benedetto king <mb...@lowlatency.com> writes:

> On Mon, Nov 10, 2003 at 08:57:55PM -0600, C. Michael Pilato wrote:
> > 
> > The new svn_fs_cleanup_txn() would always resume cleanup from wherever
> > it left off, and always leave enough pointers around so that if
> > something goes wrong, the cleaup attempt can be re-tried/continued
> > later.  In other words, it leaves no dangling nodes, doesn't lose
> > references to copies before the copies themselves are gone, etc.
> 
> This may be even harder than it sounds, if you want to have fixed
> upper bounds on your txn size (which is the whole point of the
> change).

No, actually, this shouldn't be that difficult.  The removal of
mutable nodes can be done bottom-up, one trail-per-path.  The removal
of changes is a single Berkeley call to delete all the stuff
associated with the transaction ID.  The removal of copies is a little
uglier, and involves, per-trail, doing:

   1.  Read the txn, and pick the first copy from its copies list.
   2.  Remove the copy.
   3.  Write the txn, with its copies list absent that first copy.

Fortunately, not many copies happen in a single transaction, usually.

> Or perhaps is optionally harvested as part of an "svntunefs" run.

I dunno.  Deltification (and other suggested operations that could be
part of 'svntunefs') isn't about correctness -- it's about us doing
our best to be nice to the user's hard disk space. :-) But cleaning up
(or at least marking as dead) transactions that a user asks to be
aborted is a correctness issue, because failing to do this is just
*asking* for something terrible to happen.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Aborting Subversion transactions.

[mark benedetto king <mb...@lowlatency.com>]

On Mon, Nov 10, 2003 at 08:57:55PM -0600, C. Michael Pilato wrote:
> 
> The new svn_fs_cleanup_txn() would always resume cleanup from wherever
> it left off, and always leave enough pointers around so that if
> something goes wrong, the cleaup attempt can be re-tried/continued
> later.  In other words, it leaves no dangling nodes, doesn't lose
> references to copies before the copies themselves are gone, etc.

This may be even harder than it sounds, if you want to have fixed
upper bounds on your txn size (which is the whole point of the change).

> 
> But then again, this could all just be overkill.  Maybe it's enough to make
> svn_fs_abort_txn() just delete the transaction first, and then go
> about cleaning up the other stuff.  If that fails, we get back an
> error and there is just unreachable cruft in the filesystem that sits
> around until the next dump/load cycle.
> 

Or perhaps is optionally harvested as part of an "svntunefs" run.

--ben


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org