Hot vs. Live Backups

[Drew Leske <dl...@uvic.ca>]

Hi everybody,

I'm trying to understand the need for hot copies before backing up data. 
  I believe I've read through the relevant documentation, but as I 
understand it transaction commits are atomic, so it's impossible to back 
up half a revision.  On unversioned properties, I don't see confirmation 
but I can't see any reason these wouldn't occur as part of a transaction 
as well.

However, I see statements like this in the documentation:

"In the case of Berkeley DB, Sleepycat documents describe a certain 
order in which database files can be copied that will guarantee a valid 
backup copy. And a similar ordering exists for FSFS data."

If hot copies need to be done, fine, this is more what I expect (and 
consistent with other services we back up, of course ;).  I'd just like 
to have a better understanding.

Apologies if this has been asked and answered, but I looked and none of 
the questions about backups seem to answer this.


Thanks,
Drew.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Hot vs. Live Backups

[Drew Leske <dl...@uvic.ca>]

Thanks for the responses folks... it clarifies things for me.

Drew.

Dimitri Papadopoulos-Orfanos wrote:
> Hi,
> 
>> For things to be recoverable, the syscalls have to be done in the
>> right order, so that a crash at any time leaves the on-disk data in a
>> state where things can be recovered.  It's easy to see how that can be
>> true for bdb; it's not so obvious with fsfs given that you're
>> depending on file system ordering and caching then.
> 
> 
> I guess Subversion calls sync() at strategic places.
> 
> Actually I think it's not much different whether you're using bdb or 
> fsfs. In both cases many files are modified sequentially.
> 
> Dimitri Papadopoulos
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Hot vs. Live Backups

[Dimitri Papadopoulos-Orfanos <pa...@shfj.cea.fr>]

Hi,

> For things to be recoverable, the syscalls have to be done in the
> right order, so that a crash at any time leaves the on-disk data in a
> state where things can be recovered.  It's easy to see how that can be
> true for bdb; it's not so obvious with fsfs given that you're
> depending on file system ordering and caching then.

I guess Subversion calls sync() at strategic places.

Actually I think it's not much different whether you're using bdb or 
fsfs. In both cases many files are modified sequentially.

Dimitri Papadopoulos

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Hot vs. Live Backups

[Paul Koning <pk...@equallogic.com>]

>>>>> "Dimitri" == Dimitri Papadopoulos-Orfanos <pa...@shfj.cea.fr> writes:

 Dimitri> Hi,
 >> I'm trying to understand the need for hot copies before backing up
 >> data. I believe I've read through the relevant documentation, but
 >> as I understand it transaction commits are atomic, so it's
 >> impossible to back up half a revision.

 Dimitri> The atomicity of a transaction is achieved through lock
 Dimitri> files and such. As a result the transaction seems atomic to
 Dimitri> users of Subversion, but that doesn't mean the files in the
 Dimitri> Subversion repository are always in a consistent state. For
 Dimitri> example what happens if the database is backed up while a
 Dimitri> file is locked and being written?

 Dimitri> A Subversion transaction is not atomic as far as the OS is
 Dimitri> concerned. Only the Subversion user sees the transaction as
 Dimitri> an atomic operation through the use of an additional layer
 Dimitri> over the OS (Berkeley DB).

That makes sense, but there's an additional angle.

For things to be recoverable, the syscalls have to be done in the
right order, so that a crash at any time leaves the on-disk data in a
state where things can be recovered.  It's easy to see how that can be
true for bdb; it's not so obvious with fsfs given that you're
depending on file system ordering and caching then.

In any case, that's how it needs to work.  If it does, then a backup
system that uses snapshots (common on storage area networks) would
work, because such a system has "crash consistent semantics".

      paul


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Hot vs. Live Backups

[Dimitri Papadopoulos-Orfanos <pa...@shfj.cea.fr>]

Hi,

> I'm trying to understand the need for hot copies before backing up data. 
>  I believe I've read through the relevant documentation, but as I 
> understand it transaction commits are atomic, so it's impossible to back 
> up half a revision.

The atomicity of a transaction is achieved through lock files and such. 
As a result the transaction seems atomic to users of Subversion, but 
that doesn't mean the files in the Subversion repository are always in a 
consistent state. For example what happens if the database is backed up 
while a file is locked and being written?

A Subversion transaction is not atomic as far as the OS is concerned. 
Only the Subversion user sees the transaction as an atomic operation 
through the use of an additional layer over the OS (Berkeley DB).

Dimitri Papadopoulos

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org