You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Harry Bullen <hb...@gmail.com> on 2012/02/14 20:11:29 UTC
SVN backup with lvm snapshots and rsync
I'm looking at using lvm snapshots and rsync to back up a bunch of svn
repositories every night. The repositories total about 200GB (some
people might have committed movies to svn but I may not fix this)
which is why I want to do this instead of svn hotcopy, which will have
to copy all 200GB every day. I wanted to know if their is any danger
from doing this, and if anyone might have some advice for me.
Thank you.
--------------------------------------
Harry William Bullen IV
Re: SVN backup with lvm snapshots and rsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
If the snapshots are atomic that's fine, though you'll potentially need
to run 'svnadmin recover' before they're usable.
Running dump / verify or otherwise verifying your snapshots remains
a good idea.
Harry Bullen wrote on Tue, Feb 14, 2012 at 14:11:29 -0500:
> I'm looking at using lvm snapshots and rsync to back up a bunch of svn
> repositories every night. The repositories total about 200GB (some
> people might have committed movies to svn but I may not fix this)
> which is why I want to do this instead of svn hotcopy, which will have
> to copy all 200GB every day. I wanted to know if their is any danger
> from doing this, and if anyone might have some advice for me.
>
> Thank you.
>
> --------------------------------------
> Harry William Bullen IV
Re: SVN backup with lvm snapshots and rsync
Posted by Blair Zajac <bl...@orcaware.com>.
On 02/15/2012 11:05 AM, Philip Martin wrote:
> Stefan Sperling<st...@elego.de> writes:
>
>> But rather than going through that effort, I would recommend using
>> svnadmin dump/load, or svnsync with file:// URLs, until Subversion 1.8
>> is released. At which point you can switch over to using
>> "svnadmin hotcopy --incremental", which will copy rep-cache.db via
>> the appropriate sqlite APIs.
>
> Are you suggesting this because you know the LVM snapshot method doesn't
> work or just because it's a method that avoids non-standard Subversion
> tools?
>
> Running rsync on an LVM snapshot looks like it should work to me. An
> LVM snapshot freezes the filesystem by diverting all writes into the
> snapshot area. A Subversion repository in the frozen filesystem may
> have transactions in progress, Subversion transactions or SQLite
> transactions, but they won't change while the copy is made. If the
> copied repository is ever used then these partial transactions will
> simply be ignored. Provided Subversion gets its fsyncs and writes in
> the correct order, and I believe it does, this looks like it should
> work. I'd go as far as saying if the copied repository doesn't work
> then this is a Subversion bug, or perhaps an LVM/OS bug.
One needs to use at least 1.6.5, there is one fsync I added to ensure
that fsfs works without causing empty revprop files. This was seen on a
GPFS filesystem [1].
Blair
[1] http://subversion.tigris.org/issues/show_bug.cgi?id=3442
Re: SVN backup with lvm snapshots and rsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Sperling wrote on Thu, Feb 16, 2012 at 13:53:04 +0100:
> On Thu, Feb 16, 2012 at 03:56:29AM +0200, Daniel Shahaf wrote:
> > Nico Kadel-Garcia wrote on Wed, Feb 15, 2012 at 19:41:57 -0500:
> > > On Wed, Feb 15, 2012 at 6:19 PM, Philip Martin
> > > <ph...@wandisco.com> wrote:
> > > > Nico Kadel-Garcia <nk...@gmail.com> writes:
> > > >
> > > >> Unless you do a "sync"
> > > >> command, or various low level flush commands, you don't know that you
> > > >> write to disk has actually made it to the platter.
> > > >
> > > > Subversion does that. It uses fsync (plus fsync on directories on
> > > > Linux) before assuming data is on disk.
> > >
> > > So the risk is reduced, which is good. It lowers the window of
> > > vulnerability (between when a commit is published, and when the fsync
> > > occurs). This is actually good progamming. Not everyone who deals with
> > > real data is this thoughtful.
> >
> > There is no "window of vulnerability".
>
> There is in the case where the operating system or disk lies to the
> application about data having been stored on platters.
>
Stefan, you're correct, it would have been more accurate for me to say
"There is no window of vulnerability if specific system calls that
Subversion uses fulfill their contract".
It's just that getting to this level of detail seemed to be more
hair-splitting than was necessary for me to rebut Nico's point.
Thanks for clarifying.
>
> I am not saying Subversion is careless. There is nothing Subversion
> can do about any of this. But these are things to consider when
> building a system for high reliability purposes. The systems are
> complex and the less complexity you've got the more reliability you
> tend to get.
Re: SVN backup with lvm snapshots and rsync
Posted by Les Mikesell <le...@gmail.com>.
On Thu, Feb 16, 2012 at 6:53 AM, Stefan Sperling <st...@elego.de> wrote:
> >
> > There is no "window of vulnerability".
>
>
> What if the disk controller lies and sends the acknowledgement before
> the actual work has completed (e.g. it cached the write request and is
> going to get to it soon)? How can the OS know what the disk controller
> is really doing?
>
That is a risk in the case of a power failure with a disk/raid controller
that has a cache without battery backup, but in the case of an LVM snapshot
and the power staying on, wouldn't the controller return the cached block
regardless of whether it had made the round trip to disk or not?
--
Les Mikesell
lesmikesell@gmail.com
Re: SVN backup with lvm snapshots and rsync
Posted by Stefan Sperling <st...@elego.de>.
On Thu, Feb 16, 2012 at 03:56:29AM +0200, Daniel Shahaf wrote:
> Nico Kadel-Garcia wrote on Wed, Feb 15, 2012 at 19:41:57 -0500:
> > On Wed, Feb 15, 2012 at 6:19 PM, Philip Martin
> > <ph...@wandisco.com> wrote:
> > > Nico Kadel-Garcia <nk...@gmail.com> writes:
> > >
> > >> Unless you do a "sync"
> > >> command, or various low level flush commands, you don't know that you
> > >> write to disk has actually made it to the platter.
> > >
> > > Subversion does that. It uses fsync (plus fsync on directories on
> > > Linux) before assuming data is on disk.
> >
> > So the risk is reduced, which is good. It lowers the window of
> > vulnerability (between when a commit is published, and when the fsync
> > occurs). This is actually good progamming. Not everyone who deals with
> > real data is this thoughtful.
>
> There is no "window of vulnerability".
There is in the case where the operating system or disk lies to the
application about data having been stored on platters.
E.g. it depends on how long the OS blocks the application during fsync.
Will it make the app wait all the way until the disk controller sent
an acknowledgement interrupt for the write? Or will it allow the
application to continue as soon as the buffer no cache no longer
considers the buffer dirty because it issued a write? This descision
is entirely up to the OS.
What if the disk controller lies and sends the acknowledgement before
the actual work has completed (e.g. it cached the write request and is
going to get to it soon)? How can the OS know what the disk controller
is really doing?
Now, insert some network storage layer that runs on top of IP into
the picture and you've got some more opportunities for things to go
wrong. Now the request is passing through network stacks and possibly
firewalls, and the network may treat a lost packet as collateral
damage.
I am not saying Subversion is careless. There is nothing Subversion
can do about any of this. But these are things to consider when
building a system for high reliability purposes. The systems are
complex and the less complexity you've got the more reliability you
tend to get.
Re: SVN backup with lvm snapshots and rsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nico Kadel-Garcia wrote on Wed, Feb 15, 2012 at 19:41:57 -0500:
> On Wed, Feb 15, 2012 at 6:19 PM, Philip Martin
> <ph...@wandisco.com> wrote:
> > Nico Kadel-Garcia <nk...@gmail.com> writes:
> >
> >> Unless you do a "sync"
> >> command, or various low level flush commands, you don't know that you
> >> write to disk has actually made it to the platter.
> >
> > Subversion does that. It uses fsync (plus fsync on directories on
> > Linux) before assuming data is on disk.
>
> So the risk is reduced, which is good. It lowers the window of
> vulnerability (between when a commit is published, and when the fsync
> occurs). This is actually good progamming. Not everyone who deals with
> real data is this thoughtful.
There is no "window of vulnerability".
Re: SVN backup with lvm snapshots and rsync
Posted by Nico Kadel-Garcia <nk...@gmail.com>.
On Wed, Feb 15, 2012 at 6:19 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Nico Kadel-Garcia <nk...@gmail.com> writes:
>
>> Unless you do a "sync"
>> command, or various low level flush commands, you don't know that you
>> write to disk has actually made it to the platter.
>
> Subversion does that. It uses fsync (plus fsync on directories on
> Linux) before assuming data is on disk.
So the risk is reduced, which is good. It lowers the window of
vulnerability (between when a commit is published, and when the fsync
occurs). This is actually good progamming. Not everyone who deals with
real data is this thoughtful.
Re: SVN backup with lvm snapshots and rsync
Posted by Philip Martin <ph...@wandisco.com>.
Nico Kadel-Garcia <nk...@gmail.com> writes:
> Unless you do a "sync"
> command, or various low level flush commands, you don't know that you
> write to disk has actually made it to the platter.
Subversion does that. It uses fsync (plus fsync on directories on
Linux) before assuming data is on disk.
--
Philip
Re: SVN backup with lvm snapshots and rsync
Posted by Les Mikesell <le...@gmail.com>.
On Wed, Feb 15, 2012 at 3:37 PM, Nico Kadel-Garcia <nk...@gmail.com> wrote:
> If the repository is busy in a commit or partial commit at the moment
> of the snapshot, you're in potentially very deep trouble. You've
> entered not only "split-brain" territory, where the repositories are
> out of sync, but partial commit territory. Subversion is careful that
> its database writes are atomic operations, but you can't assure that
> if the filesystem is in the midst of being written when snapshotted.
>
Wouldn't this be the same state of the disk if the system crashed at that
point? So if you can't expect to recover from that copy you should also
be worried about recovering after a power failure.
--
Les Mikesell
lesmikesell@gmail.com
Re: SVN backup with lvm snapshots and rsync
Posted by Nico Kadel-Garcia <nk...@gmail.com>.
On Wed, Feb 15, 2012 at 2:05 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Stefan Sperling <st...@elego.de> writes:
>
>> But rather than going through that effort, I would recommend using
>> svnadmin dump/load, or svnsync with file:// URLs, until Subversion 1.8
>> is released. At which point you can switch over to using
>> "svnadmin hotcopy --incremental", which will copy rep-cache.db via
>> the appropriate sqlite APIs.
>
> Are you suggesting this because you know the LVM snapshot method doesn't
> work or just because it's a method that avoids non-standard Subversion
> tools?
>
> Running rsync on an LVM snapshot looks like it should work to me. An
It'll work great until it doesn't.
If the repository is busy in a commit or partial commit at the moment
of the snapshot, you're in potentially very deep trouble. You've
entered not only "split-brain" territory, where the repositories are
out of sync, but partial commit territory. Subversion is careful that
its database writes are atomic operations, but you can't assure that
if the filesystem is in the midst of being written when snapshotted.
You've casually stated "An LVM snapshot freezes the filesystem". No,
it freezes the *partition*, and metadata about the blocks used on the
raw device, and excludes them from further writing (until the buffer
space allocated for LVM overflows. To the casual eye, this looks like
freezing the filesystem, but if you look more closely, I think you'll
find that anything paged out and not yet written to disk will not be
in the snapshot. And modern filesystems do this kind of paging as a
matter of course for performance reasons. Unless you do a "sync"
command, or various low level flush commands, you don't know that you
write to disk has actually made it to the platter.
In a lightly loaded environment, middle of the night, no commits are
going on? No problem. But in that kind of environment, you should have
space to run an rsync based snapshot several times in a row to verify
operational consistency and be done with it, and avoid LVM altogether.
Robust backup takes thought.
In a heavily loaded system that nightly commits regression testing
tags or branches? You have a potential adventure.
Re: SVN backup with lvm snapshots and rsync
Posted by Andreas Krey <a....@gmx.de>.
On Wed, 15 Feb 2012 20:41:48 +0000, Stefan Sperling wrote:
....
> I don't know enough about SQlite to judge whether the DB will
> keep working at any frozen state a snapshot might create.
If it doesn't then it wouldn't be resilient to system crashes either,
and *that* wouldn't exactly be a recommendation to use it at all.
Andreas
--
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800
Re: SVN backup with lvm snapshots and rsync
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Feb 15, 2012 at 07:05:08PM +0000, Philip Martin wrote:
> Stefan Sperling <st...@elego.de> writes:
>
> > But rather than going through that effort, I would recommend using
> > svnadmin dump/load, or svnsync with file:// URLs, until Subversion 1.8
> > is released. At which point you can switch over to using
> > "svnadmin hotcopy --incremental", which will copy rep-cache.db via
> > the appropriate sqlite APIs.
>
> Are you suggesting this because you know the LVM snapshot method doesn't
> work or just because it's a method that avoids non-standard Subversion
> tools?
I don't know enough about SQlite to judge whether the DB will
keep working at any frozen state a snapshot might create.
And, yes, in general, using offical Subversion tools for backup
purposes has the advantage that they will keep working after an
upgrade. No matter what kinds of changes we make to repository
fomats, we'll keep the official tools working.
That said, I agree that filesystem snapshots should work fine,
provided that sqlite databases will cope, too.
Re: SVN backup with lvm snapshots and rsync
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Sperling <st...@elego.de> writes:
> But rather than going through that effort, I would recommend using
> svnadmin dump/load, or svnsync with file:// URLs, until Subversion 1.8
> is released. At which point you can switch over to using
> "svnadmin hotcopy --incremental", which will copy rep-cache.db via
> the appropriate sqlite APIs.
Are you suggesting this because you know the LVM snapshot method doesn't
work or just because it's a method that avoids non-standard Subversion
tools?
Running rsync on an LVM snapshot looks like it should work to me. An
LVM snapshot freezes the filesystem by diverting all writes into the
snapshot area. A Subversion repository in the frozen filesystem may
have transactions in progress, Subversion transactions or SQLite
transactions, but they won't change while the copy is made. If the
copied repository is ever used then these partial transactions will
simply be ignored. Provided Subversion gets its fsyncs and writes in
the correct order, and I believe it does, this looks like it should
work. I'd go as far as saying if the copied repository doesn't work
then this is a Subversion bug, or perhaps an LVM/OS bug.
--
Philip
Re: SVN backup with lvm snapshots and rsync
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Feb 15, 2012 at 12:03:15PM -0500, Harry Bullen wrote:
> From what I gather rep-cache.db, can be
> regenerated by svn. If I used rsync and excluded the rep-cache.db
> would I then want to run 'svnadmin recover' on these backup or is
> rep-cache.db regenerated automatically when the repository is used?
It will be re-created automatically. But you're losing the benefits
of rep-sharing because a fresh rep-cache.db will be created.
So you'll start with an empty cache all over again.
The cache exists to prevent unnecessary growth of your repository.
It does not affect correctness. It prevents duplicate content from
being redundantly stored in the repository.
E.g. say two people make two independent commits, which each add one file.
The file has a different name in either commit but the exact same content
in both commits.
rep-cache.db would store a checksum of the file's content when the file
is first added. Keyed on this checksum is the exact location of the
content in the repository. Locating the content without this cache would
involve parsing all existing revisions, which is prohibitively expensive.
During the second commit which adds the same content the content's
checksum will be computed and generate a cache hit. The file added
during the second commit will then be made to refer to the content
already stored during the first commit.
If the rep-cache is cleared between the two commits, there will be
a cache miss so the redundancy cannot be detected. The content will be
stored redundantly in the newer revision. But it will also cause a new
rep-cache entry. So now you're good again until the cache is cleared
once more.
To work around this limitation you could write a small tool that uses the
sqlite API to perform a hotcopy of rep-cache.db and run this tool in
addition to rsync (see http://sqlite.org/backup.html).
But rather than going through that effort, I would recommend using
svnadmin dump/load, or svnsync with file:// URLs, until Subversion 1.8
is released. At which point you can switch over to using
"svnadmin hotcopy --incremental", which will copy rep-cache.db via
the appropriate sqlite APIs.
Re: SVN backup with lvm snapshots and rsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Harry Bullen wrote on Wed, Feb 15, 2012 at 12:03:15 -0500:
> On Wed, Feb 15, 2012 at 6:24 AM, Stefan Sperling <st...@elego.de> wrote:
> > On Wed, Feb 15, 2012 at 03:02:04AM +0200, Daniel Shahaf wrote:
> >> Stefan Sperling wrote on Tue, Feb 14, 2012 at 23:19:58 +0100:
> >> > Until then, svnsync or svnadmin dump/load are the only officially
> >> > supported incremental backup solutions. But, as Daniel explained,
> >> > 'rsync' followed by 'svnadmin recover' produces valid copies of
> >> > FSFS repositories, too.
> >>
> >> I didn't say that, Stefan, since it's not true. rsync is not safe if
> >> the SQLite db is being read or written whilst rsync runs.
> >
> > Ah, true. I keep forgetting about the rep cache...
> > I'm sorry if you felt I was mis-attributing this to you.
> >
> > All I can say in my defense is that the whole point of adding incremental
> > hotcopy support (which of course handles the rep-cache properly) was to
> > finally stop worrying and forget about such details :)
>
> Thank you for this advice. From what I gather rep-cache.db, can be
> regenerated by svn. If I used rsync and excluded the rep-cache.db
> would I then want to run 'svnadmin recover' on these backup or is
> rep-cache.db regenerated automatically when the repository is used?
Yes and no. The data for new revisions will be written if fsfs.conf is
configured to enable the cache. The data for old revisions won't be
backfilled, but it's certainly possible to do so (by walking all the
node-revs and adding their reps to the table), but I don't know that
anyone has actually written the code to do so.
Re: SVN backup with lvm snapshots and rsync
Posted by Harry Bullen <hb...@gmail.com>.
On Wed, Feb 15, 2012 at 6:24 AM, Stefan Sperling <st...@elego.de> wrote:
> On Wed, Feb 15, 2012 at 03:02:04AM +0200, Daniel Shahaf wrote:
>> Stefan Sperling wrote on Tue, Feb 14, 2012 at 23:19:58 +0100:
>> > Until then, svnsync or svnadmin dump/load are the only officially
>> > supported incremental backup solutions. But, as Daniel explained,
>> > 'rsync' followed by 'svnadmin recover' produces valid copies of
>> > FSFS repositories, too.
>>
>> I didn't say that, Stefan, since it's not true. rsync is not safe if
>> the SQLite db is being read or written whilst rsync runs.
>
> Ah, true. I keep forgetting about the rep cache...
> I'm sorry if you felt I was mis-attributing this to you.
>
> All I can say in my defense is that the whole point of adding incremental
> hotcopy support (which of course handles the rep-cache properly) was to
> finally stop worrying and forget about such details :)
Thank you for this advice. From what I gather rep-cache.db, can be
regenerated by svn. If I used rsync and excluded the rep-cache.db
would I then want to run 'svnadmin recover' on these backup or is
rep-cache.db regenerated automatically when the repository is used?
Re: SVN backup with lvm snapshots and rsync
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Feb 15, 2012 at 03:02:04AM +0200, Daniel Shahaf wrote:
> Stefan Sperling wrote on Tue, Feb 14, 2012 at 23:19:58 +0100:
> > Until then, svnsync or svnadmin dump/load are the only officially
> > supported incremental backup solutions. But, as Daniel explained,
> > 'rsync' followed by 'svnadmin recover' produces valid copies of
> > FSFS repositories, too.
>
> I didn't say that, Stefan, since it's not true. rsync is not safe if
> the SQLite db is being read or written whilst rsync runs.
Ah, true. I keep forgetting about the rep cache...
I'm sorry if you felt I was mis-attributing this to you.
All I can say in my defense is that the whole point of adding incremental
hotcopy support (which of course handles the rep-cache properly) was to
finally stop worrying and forget about such details :)
Re: SVN backup with lvm snapshots and rsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Les Mikesell wrote on Tue, Feb 14, 2012 at 19:34:10 -0600:
> On Tue, Feb 14, 2012 at 7:02 PM, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > Stefan Sperling wrote on Tue, Feb 14, 2012 at 23:19:58 +0100:
> >> Until then, svnsync or svnadmin dump/load are the only officially
> >> supported incremental backup solutions. But, as Daniel explained,
> >> 'rsync' followed by 'svnadmin recover' produces valid copies of
> >> FSFS repositories, too.
> >
> > I didn't say that, Stefan, since it's not true. rsync is not safe if
> > the SQLite db is being read or written whilst rsync runs.
>
> You could just repeat the rsync command until nothing changes as a
> quick hack. Unless the filesystem is extremely busy, that should get
> a good snapshot.
Just exclude rep-cache.db from the rsync. _For now_ I think that
suffices, but there is no guarantee that that will remain true into the
future. 'svnadmin hotcopy' (and its incremental variant in 1.8) is the
supported, forward-compatible means.
Re: SVN backup with lvm snapshots and rsync
Posted by Les Mikesell <le...@gmail.com>.
On Tue, Feb 14, 2012 at 7:02 PM, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> Stefan Sperling wrote on Tue, Feb 14, 2012 at 23:19:58 +0100:
>> Until then, svnsync or svnadmin dump/load are the only officially
>> supported incremental backup solutions. But, as Daniel explained,
>> 'rsync' followed by 'svnadmin recover' produces valid copies of
>> FSFS repositories, too.
>
> I didn't say that, Stefan, since it's not true. rsync is not safe if
> the SQLite db is being read or written whilst rsync runs.
You could just repeat the rsync command until nothing changes as a
quick hack. Unless the filesystem is extremely busy, that should get
a good snapshot.
--
Les Mikesell
lesmikesell@gmail.com
Re: SVN backup with lvm snapshots and rsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Sperling wrote on Tue, Feb 14, 2012 at 23:19:58 +0100:
> Until then, svnsync or svnadmin dump/load are the only officially
> supported incremental backup solutions. But, as Daniel explained,
> 'rsync' followed by 'svnadmin recover' produces valid copies of
> FSFS repositories, too.
I didn't say that, Stefan, since it's not true. rsync is not safe if
the SQLite db is being read or written whilst rsync runs.
Re: SVN backup with lvm snapshots and rsync
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Feb 14, 2012 at 02:11:29PM -0500, Harry Bullen wrote:
> The repositories total about 200GB (some
> people might have committed movies to svn but I may not fix this)
> which is why I want to do this instead of svn hotcopy, which will have
> to copy all 200GB every day.
Subversion 1.8 will ship with incremental hotcopy support (for FSFS
only) which should address this problem.
There's no release date for Subversion 1.8 yet. I'm just noting this
in case this makes a difference to your long-term planning.
Until then, svnsync or svnadmin dump/load are the only officially
supported incremental backup solutions. But, as Daniel explained,
'rsync' followed by 'svnadmin recover' produces valid copies of
FSFS repositories, too.
Re: SVN backup with lvm snapshots and rsync
Posted by Les Mikesell <le...@gmail.com>.
On Tue, Feb 14, 2012 at 1:11 PM, Harry Bullen <hb...@gmail.com> wrote:
> I'm looking at using lvm snapshots and rsync to back up a bunch of svn
> repositories every night. The repositories total about 200GB (some
> people might have committed movies to svn but I may not fix this)
> which is why I want to do this instead of svn hotcopy, which will have
> to copy all 200GB every day. I wanted to know if their is any danger
> from doing this, and if anyone might have some advice for me.
>
Svnsync would do this relatively gracefully after the initial copy - but at
the expense of having to set up each repository's mirror separately.
--
Les Mikesell
lesmikesell@gmail.com
Re: SVN backup with lvm snapshots and rsync
Posted by Nico Kadel-Garcia <nk...@gmail.com>.
On Tue, Feb 14, 2012 at 2:11 PM, Harry Bullen <hb...@gmail.com> wrote:
> I'm looking at using lvm snapshots and rsync to back up a bunch of svn
> repositories every night. The repositories total about 200GB (some
> people might have committed movies to svn but I may not fix this)
> which is why I want to do this instead of svn hotcopy, which will have
> to copy all 200GB every day. I wanted to know if their is any danger
> from doing this, and if anyone might have some advice for me.
>
> Thank you.
LVM snapshots do not guarantee completion of commits that are in
progress when the snapshot has occurred, even if it's complete the
filesystem is concerned, because it may not yet have been written to
*dsk*. ve effectively used the stunt you describe for MySQL backups,
but always made sure to *shut down* the database and "sync" the
filesystem
before LVM snapshotting, re-activating the backup, and doing this backup.
For a big envornmnent, rather than trying to outsmart things yourself,
may I suggest considering Wandisco's multi-homed commercial system to
avoid any skew between the primary and locally available secondary
servers? And if that's not feasible, visit the idea of svnsync mirrors
with configuration files transferred via a different channel for
potential merging and activation?
> --------------------------------------
> Harry William Bullen IV