You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Dhiraj Soni <dh...@oracle.com> on 2007/04/27 16:16:45 UTC
Strategy for backup and recovering svn repository
Hi,
I'm trying to setup a backup/reco strategy for our svn repositories. My
question is if i take a full dump via 'svnadmin hotcopy' command then
taking incrementail daily via 'svnadmin dump', can i apply the daily
incremental dumps to the full dump taken by hotcopy command??
Thanks,
Dhiraj.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Strategy for backup and recovering svn repository
Posted by Dhiraj Soni <dh...@oracle.com>.
(SVN version 1.4.3 on Sun Hardware running RH Linux Enterprise.
Compiled natively from source. )
Hi all,
I implemented the strategy, it doesn't work. Somehow, when you apply
incremental dump after a full hotcopy, it results in an error on the
lines of:
> ERROR: *svnadmin* failed with the following output while loading
the dumpfile:
> *svnadmin*: *File* *already* *exists*: *filesystem* '.../db',
> transaction '462-1', path 'trunk'
This only happens for first incremental commit, the rest goes through
ok. Don't know why it doesn't work. Here's what i did:
1) Made a hotcopy backup of full repository
2) Grab the youngest revision of that hotcopy backup. Lets say its at rev. 3
3) Do daily or post-commit incremental backups from then on, eg. 3-4,
4-7 etc.
4) Refresh this mechanism monthly.
To recover:
1) Place the hotcopy in original's place
2) Apply the incremental dumps. The 3-4 dump always results in error.
3) Rest of incremental dumps go on fine.
Can anyone help me figure out what's going wrong??
Thanks,
Dhiraj.
Dhiraj Soni wrote:
> I tried this, seems to work. Anyone has experience with this method?
> The repositories are going to be huge 1GB to 50GB, that's why i'm
> thinking of taking this approach. Take a hotcopy dump, then
> incremental dumps. Refresh this monthly.
> My strategy is that after taking hotcopy ,grab its youngest version
> and then get dumps after that. After that just zip the data and get
> rid of directory. So, that way i ensure that i grab all the revisions
> after hotcopy.
> What do you guys think?? Any issues? Anyone taking this approach, or
> i'm better off just using svnadmin dump instead of hotcopy??
> The FAQ needs to address this, as searching this list brought me some
> similar questions but no concrete answer.
> -Thanks.
>
> Dhiraj Soni wrote:
>> Hi,
>> I'm trying to setup a backup/reco strategy for our svn repositories.
>> My question is if i take a full dump via 'svnadmin hotcopy' command
>> then taking incrementail daily via 'svnadmin dump', can i apply the
>> daily incremental dumps to the full dump taken by hotcopy command??
>> Thanks,
>> Dhiraj.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: users-help@subversion.tigris.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
Re: Strategy for backup and recovering svn repository
Posted by Dhiraj Soni <dh...@oracle.com>.
I tried this, seems to work. Anyone has experience with this method? The
repositories are going to be huge 1GB to 50GB, that's why i'm thinking
of taking this approach. Take a hotcopy dump, then incremental dumps.
Refresh this monthly.
My strategy is that after taking hotcopy ,grab its youngest version and
then get dumps after that. After that just zip the data and get rid of
directory. So, that way i ensure that i grab all the revisions after
hotcopy.
What do you guys think?? Any issues? Anyone taking this approach, or i'm
better off just using svnadmin dump instead of hotcopy??
The FAQ needs to address this, as searching this list brought me some
similar questions but no concrete answer.
-Thanks.
Dhiraj Soni wrote:
> Hi,
> I'm trying to setup a backup/reco strategy for our svn repositories.
> My question is if i take a full dump via 'svnadmin hotcopy' command
> then taking incrementail daily via 'svnadmin dump', can i apply the
> daily incremental dumps to the full dump taken by hotcopy command??
> Thanks,
> Dhiraj.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Strategy for backup and recovering svn repository
Posted by Tom Kielty <to...@calabrio.com>.
Sorry for the late follow-up to this.
The machine our subversion server is running on has a 900GB RAID 5 array
which is backed up incrementally by our IT department every night to
tape. On the weekend they do a full backup. We are using FSFS as the
backend for our SVN server.
Is this not a good plan and we should do a hot back up before the daily
backup?
Thanks
Tom
-----Original Message-----
From: Johnathan Gifford [mailto:jgifford@wernervas.com]
Sent: Friday, April 27, 2007 3:03 PM
To: Dhiraj Soni; users@subversion.tigris.org
Subject: Re: Strategy for backup and recovering svn repository
Dhiraj,
Yes, the plan you propose will work. But, will be a lot more work in
the event of a recovery situation.
Our backup strategy plan is this.
Requisites:
1. A hot back up of the repository created the svn-fast-backup script
every 12 hours. We only keep four days worth of backups as anything
above that is just taking up space for no real reason in our opinion,
but feel free keep them longer. Some folks even do this after every
commit, but that may really be to much over head on your server and
commit process.
2. When a user commits a change to the repository, the post-commit hook
does an incremental dump of the new revision immediately. We keep every
single one of these as a 'last ditch' source, but realistically you only
need the last 12 to 24 hours, but for good measure you should keep the
last four days worth.
3. Everything is kept on RAID 5 storage.
Recovery:
Using the last 'good' hot backup copy, we load each incremental dump
manually to bring the repository backup to up to the point of the known
last revision committed.
Additional Insurance:
A) Run 'svnadmin verify' daily. This will let you know if your
repository has become corrupt much sooner than when you upgrade to the
next version of Subversion. Nothing like the experience of doing a dump
and reload to take advantage of new features to find out that your
repository has a problem.
B) If your not getting and keeping the e-mail diffs for all commits,
you may want to consider that. This is excellent reference to what is
missing from the last hot-backup. In the event your repository becomes
corrupt at a revision that is older than your backups, these may help a
third party support provider like CollabNet or Polarion in fixing your
repository.
C) Archive your hot-backups and incremental dumps and move them to
another server at another location at least once a day.
D) Remember, tape backup will not do you much good on a live
repository. Tape is only good for restoring the configuration of your
system, not the repository. The only guaranteed way to backup your
repository is to use the hot-backup tools.
E) If you are running 1.4.x and have a disaster recovery site and
access to a server at the site, consider mirroring your
repository/repositories to that location using the the 'svnsync' tools.
We've only had to put the above recovery plan into effect once due to
hard configuration snafu. In another situation, the e-mail diffs saved
our butt when we upgraded our Subversion installation and discovered
seven 'bad' revisions. We don't want to go there again, but we're sure
glad we took this approach when they did occur.
Johnathan
>>> On Fri, Apr 27, 2007 at 11:16 AM, in message
<46...@oracle.com>,
Dhiraj Soni <dh...@oracle.com> wrote:
> Hi,
> I'm trying to setup a backup/reco strategy for our svn repositories.
My
> question is if i take a full dump via 'svnadmin hotcopy' command then
> taking incrementail daily via 'svnadmin dump', can i apply the daily
> incremental dumps to the full dump taken by hotcopy command??
> Thanks,
> Dhiraj.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Using svnsync for mirroring
Posted by Shawn Talbert <st...@exploreconsulting.com>.
Is anyone here using the new svnsync for real-time incremental backups?
I currently use "hotcopy" backups of the entire repo, created daily.
The documentation for svnsync is rather sparse it seems.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Strategy for backup and recovering svn repository
Posted by Brian Brophy <br...@email.com>.
Johnathan,
What does this post-commit script look like to do the incremental backup
on commit? It sounds like a great idea. I'd be interested to see it.
Thanks,
Brian
Johnathan Gifford wrote:
> Dhiraj,
>
> Yes, the plan you propose will work. But, will be a lot more work in the event of a recovery situation.
>
> Our backup strategy plan is this.
>
> Requisites:
> 1. A hot back up of the repository created the svn-fast-backup script every 12 hours. We only keep four days worth of backups as anything above that is just taking up space for no real reason in our opinion, but feel free keep them longer. Some folks even do this after every commit, but that may really be to much over head on your server and commit process.
> 2. When a user commits a change to the repository, the post-commit hook does an incremental dump of the new revision immediately. We keep every single one of these as a 'last ditch' source, but realistically you only need the last 12 to 24 hours, but for good measure you should keep the last four days worth.
> 3. Everything is kept on RAID 5 storage.
>
> Recovery:
> Using the last 'good' hot backup copy, we load each incremental dump manually to bring the repository backup to up to the point of the known last revision committed.
>
> Additional Insurance:
> A) Run 'svnadmin verify' daily. This will let you know if your repository has become corrupt much sooner than when you upgrade to the next version of Subversion. Nothing like the experience of doing a dump and reload to take advantage of new features to find out that your repository has a problem.
> B) If your not getting and keeping the e-mail diffs for all commits, you may want to consider that. This is excellent reference to what is missing from the last hot-backup. In the event your repository becomes corrupt at a revision that is older than your backups, these may help a third party support provider like CollabNet or Polarion in fixing your repository.
> C) Archive your hot-backups and incremental dumps and move them to another server at another location at least once a day.
> D) Remember, tape backup will not do you much good on a live repository. Tape is only good for restoring the configuration of your system, not the repository. The only guaranteed way to backup your repository is to use the hot-backup tools.
> E) If you are running 1.4.x and have a disaster recovery site and access to a server at the site, consider mirroring your repository/repositories to that location using the the 'svnsync' tools.
>
> We've only had to put the above recovery plan into effect once due to hard configuration snafu. In another situation, the e-mail diffs saved our butt when we upgraded our Subversion installation and discovered seven 'bad' revisions. We don't want to go there again, but we're sure glad we took this approach when they did occur.
>
> Johnathan
>
>
>>>> On Fri, Apr 27, 2007 at 11:16 AM, in message <46...@oracle.com>,
>>>>
> Dhiraj Soni <dh...@oracle.com> wrote:
>
>> Hi,
>> I'm trying to setup a backup/reco strategy for our svn repositories. My
>> question is if i take a full dump via 'svnadmin hotcopy' command then
>> taking incrementail daily via 'svnadmin dump', can i apply the daily
>> incremental dumps to the full dump taken by hotcopy command??
>> Thanks,
>> Dhiraj.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: users-help@subversion.tigris.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Strategy for backup and recovering svn repository
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Apr 27, 2007, at 15:03, Johnathan Gifford wrote:
> Yes, the plan you propose will work. But, will be a lot more work
> in the event of a recovery situation.
>
> Our backup strategy plan is this.
>
> Requisites:
> 1. A hot back up of the repository created the svn-fast-backup
> script every 12 hours. We only keep four days worth of backups as
> anything above that is just taking up space for no real reason in
> our opinion, but feel free keep them longer. Some folks even do
> this after every commit, but that may really be to much over head
> on your server and commit process.
Shouldn't there be close to no overhead? From svn-fast-backup itself:
# Multiple FSFS backups share data via hardlinks, meaning old
backups are
# almost free, since a newer revision of a repository is almost a
complete
# superset of an older revision.
--
To reply to the mailing list, please use your mailer's Reply To All
function
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Strategy for backup and recovering svn repository
Posted by Johnathan Gifford <jg...@wernervas.com>.
Dhiraj,
Yes, the plan you propose will work. But, will be a lot more work in the event of a recovery situation.
Our backup strategy plan is this.
Requisites:
1. A hot back up of the repository created the svn-fast-backup script every 12 hours. We only keep four days worth of backups as anything above that is just taking up space for no real reason in our opinion, but feel free keep them longer. Some folks even do this after every commit, but that may really be to much over head on your server and commit process.
2. When a user commits a change to the repository, the post-commit hook does an incremental dump of the new revision immediately. We keep every single one of these as a 'last ditch' source, but realistically you only need the last 12 to 24 hours, but for good measure you should keep the last four days worth.
3. Everything is kept on RAID 5 storage.
Recovery:
Using the last 'good' hot backup copy, we load each incremental dump manually to bring the repository backup to up to the point of the known last revision committed.
Additional Insurance:
A) Run 'svnadmin verify' daily. This will let you know if your repository has become corrupt much sooner than when you upgrade to the next version of Subversion. Nothing like the experience of doing a dump and reload to take advantage of new features to find out that your repository has a problem.
B) If your not getting and keeping the e-mail diffs for all commits, you may want to consider that. This is excellent reference to what is missing from the last hot-backup. In the event your repository becomes corrupt at a revision that is older than your backups, these may help a third party support provider like CollabNet or Polarion in fixing your repository.
C) Archive your hot-backups and incremental dumps and move them to another server at another location at least once a day.
D) Remember, tape backup will not do you much good on a live repository. Tape is only good for restoring the configuration of your system, not the repository. The only guaranteed way to backup your repository is to use the hot-backup tools.
E) If you are running 1.4.x and have a disaster recovery site and access to a server at the site, consider mirroring your repository/repositories to that location using the the 'svnsync' tools.
We've only had to put the above recovery plan into effect once due to hard configuration snafu. In another situation, the e-mail diffs saved our butt when we upgraded our Subversion installation and discovered seven 'bad' revisions. We don't want to go there again, but we're sure glad we took this approach when they did occur.
Johnathan
>>> On Fri, Apr 27, 2007 at 11:16 AM, in message <46...@oracle.com>,
Dhiraj Soni <dh...@oracle.com> wrote:
> Hi,
> I'm trying to setup a backup/reco strategy for our svn repositories. My
> question is if i take a full dump via 'svnadmin hotcopy' command then
> taking incrementail daily via 'svnadmin dump', can i apply the daily
> incremental dumps to the full dump taken by hotcopy command??
> Thanks,
> Dhiraj.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org