You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Eiren Smith <ei...@gmail.com> on 2010/07/30 17:57:12 UTC
Re: Recovering repository with multiple missing rev/ files
On Jun 15, 2010, at 11:31 AM, Stefan Sperling wrote:
> On Tue, Jun 15, 2010 at 11:09:39AM -0400, Eiren Smith wrote:
>> Dear SVNers,
>>
>> I'm trying to recover an SVN repository after a hard drive failure.
>> From 7,797 revisions, we lost the following seven (7) files from the
>> .../repo_dir/db/revs/7/ directory:
>>
>> 7437 (7-9 Dec 2009) (file also missing from revprops/ dir)
>>
>> 7461 (16 Dec 2009)
>>
>> 7519 (8 Jan 2010)
>> 7520 (8 Jan 2010)
>> 7521 (8 Jan 2010)
>>
>> 7679 (19 Mar 2010)
>>
>> 7683 (19 Mar 2010)
>>
>> This single SVN repository contains many different software
>> projects/products. So I would prefer to be able to rebuild my
>> repository and only lose the revisions that explicitly depend on
>> those seven missing revs/ files, rather than having everything stop
>> at rev. 7436. Being able to recover all the way to 7797 might mean
>> only five products are affected but all our other products would
>> have their history fully restored, which would be excellent.
>>
>> Is there a way to do this?
>
> Can you still create dumpfiles containing the revisions that
> did not get lost? If so, you could stitch together a new repository
> and fill in the missing revisions manually (if you still know what
> happened in those revisions, or can guess what happened).
>
> See the svnbook section on rewriting history:
> http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.filtering
>
> And see the section called "REVISIONIST HISTORY" of this file:
> http://svn.apache.org/repos/asf/subversion/README
>
>> P.S. Other files may also be missing from the revprops/ dir, not
>> just 7437 -- If I'm willing to lose commit messages, can I live
>> without some revprops/ files?
>
> You should recreate the revprops for all revisions.
>
> Stefan
Dear Stefan,
I'm picking up this oldish topic, which I'm back to after a month of distractions.
> Can you still create dumpfiles containing the revisions that
> did not get lost? If so, you could stitch together a new repository
> and fill in the missing revisions manually (if you still know what
> happened in those revisions, or can guess what happened).
Unfortunately, it appears I cannot create dump files from revisions after a missing revs/ file gap. All the dumpfiles I created which began with a revision past a point of continuity were unusable. They collapsed upon re-import (svnadmin load …) into a single revision instead of hundreds of revisions.
Any idea why that might be or how I might get around it?
Thanks for your continued help,
/eiren
Re: Recovering repository with multiple missing rev/ files
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Eiren Smith wrote on Fri, Jul 30, 2010 at 17:22:43 -0400:
> On Jul 30, 2010, at 3:41 PM, Daniel Shahaf wrote:
>
> > Stefan Sperling wrote on Fri, Jul 30, 2010 at 20:05:16 +0200:
> >> On Fri, Jul 30, 2010 at 01:57:12PM -0400, Eiren Smith wrote:
> >>> Unfortunately, it appears I cannot create dump files from revisions
> >>> after a missing revs/ file gap. All the dumpfiles I created which
> >>> began with a revision past a point of continuity were unusable.
> >
> > Did you pass --incremental and/or --deltas flags to 'svnadmin dump'?
>
>
> Not in the case I described in my earlier post. But I did experiment with
> --incremental and the result there was a tiny little dumpfile, which perhaps
> just had one revision in it.
>
cat $dumpfile | grep '^'Revision-number:
> I didn't try --deltas. The page
>
> http://svnbook.red-bean.com/en/1.2/svn.ref.svnadmin.c.dump.html
>
This page is a couple of years old, please try the 1.5 or 1.6 book.
> says, "... deltified dumpfiles are more CPU intensive to create ..." which
> makes me think the deltas get recalculated by --deltas, not just grabbed from
> the revs/ files directly. So it might not help, huh?
>
You can sit here and speculate whether or not it would work, or you can try the
command for yourself and see for yourself whether it works.
> /eiren
Daniel
(echo "Try it and see" >> ~/docs/quotes-I-want-on-T-shirts.txt)
Re: Recovering repository with multiple missing rev/ files
Posted by Eiren Smith <ei...@gmail.com>.
On Jul 30, 2010, at 3:41 PM, Daniel Shahaf wrote:
> Stefan Sperling wrote on Fri, Jul 30, 2010 at 20:05:16 +0200:
>> On Fri, Jul 30, 2010 at 01:57:12PM -0400, Eiren Smith wrote:
>>> Unfortunately, it appears I cannot create dump files from revisions
>>> after a missing revs/ file gap. All the dumpfiles I created which
>>> began with a revision past a point of continuity were unusable.
>
> Did you pass --incremental and/or --deltas flags to 'svnadmin dump'?
Not in the case I described in my earlier post. But I did experiment with --incremental and the result there was a tiny little dumpfile, which perhaps just had one revision in it.
I didn't try --deltas. The page
http://svnbook.red-bean.com/en/1.2/svn.ref.svnadmin.c.dump.html
says, "... deltified dumpfiles are more CPU intensive to create ..." which makes me think the deltas get recalculated by --deltas, not just grabbed from the revs/ files directly. So it might not help, huh?
/eiren
Re: Recovering repository with multiple missing rev/ files
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Sperling wrote on Fri, Jul 30, 2010 at 20:05:16 +0200:
> On Fri, Jul 30, 2010 at 01:57:12PM -0400, Eiren Smith wrote:
> > Unfortunately, it appears I cannot create dump files from revisions
> > after a missing revs/ file gap. All the dumpfiles I created which
> > began with a revision past a point of continuity were unusable.
Did you pass --incremental and/or --deltas flags to 'svnadmin dump'?
Re: Recovering repository with multiple missing rev/ files
Posted by Eiren Smith <ei...@gmail.com>.
On Jul 30, 2010, at 6:41 PM, Stefan Sperling wrote:
> On Fri, Jul 30, 2010 at 03:43:12PM -0400, Eiren Smith wrote:
>> 2. Copied revision revs/7/7263 file from that dumpfile-backup-based repo into my production repo (the one missing some revs/ files -- I'll call it my corrupt repo).
>
> Copying rev files like this most likely won't work.
>
>> 3. Tried to dump that corrupt repo to ensure that my transplanted 7263 file was well received. Result:
>>
>> ...
>> * Dumped revision 7262.
>> svnadmin: Corrupt node-revision '0-5919.0.r7252/272624'
>> svnadmin: Missing id field in node-rev
>
> To replay changes you've lost, you need to pretend to be a developer
> making the changes again.
> You need to use svn checkout to get a working copy, make the changes
> which happened (and have been lost) in that working copy, and use
> svn commit to commit the changes, thereby re-creating the missing
> revision. Then load additional revisions on top using svnadmin load.
>
> If you don't know exactly what changes were made in the lost revisions,
> maybe you can interpolate the changes by looking at past and
> future revisions around the time of the missing changes?
>
> You were talking about changes made to binary files in
> http://svn.haxx.se/users/archive-2010-06/0218.shtml so it can be
> quite hard to recreate the changes.
> Maybe you can get away with just omitting the missing revisions
> entirely? Dump all revisions you are able to dump. For the first dump,
> use svnadmin dump -r0:X (where X is the last good revision before the
> first one that got broken by the disk failure). For any dump
> of later revision use svnadmind dump -rA:B with the --incremental option
> of svnadmin dump. Then try to load all the dumps in succession.
>
> I'm not sure what you mean when you describe that loading a dump
> file into a repository causes a single large revision to be created.
> That doesn't make any sense to me. I've never seen this happen.
> When this happens, what's the output of the svnadmin load command,
> and what's the output of the svn log command? Maybe I'm misunderstanding
> what you are trying to say? Please show the exact command you are using
> for loading. In fact, it would really help if you could simply post a
> transcript of all the commands you have tried running so far, and their
> output, without explaining your interpretation of things. This should
> help avoid misunderstandings. Based on such a transcript, we could ask
> further questions and provide hints, and hopefully help you fix the problem.
>
> Stefan
Thank you very much for all your help, Stefan. My window of time in which to restore this repository has passed now. So I'm going to restore my old backup and just lose the revisions between then and now. I don't expect to lose any final file versions, so the version history should be the only casualty.
/eiren
Re: Recovering repository with multiple missing rev/ files
Posted by Stefan Sperling <st...@elego.de>.
On Fri, Jul 30, 2010 at 03:43:12PM -0400, Eiren Smith wrote:
> 2. Copied revision revs/7/7263 file from that dumpfile-backup-based repo into my production repo (the one missing some revs/ files -- I'll call it my corrupt repo).
Copying rev files like this most likely won't work.
> 3. Tried to dump that corrupt repo to ensure that my transplanted 7263 file was well received. Result:
>
> ...
> * Dumped revision 7262.
> svnadmin: Corrupt node-revision '0-5919.0.r7252/272624'
> svnadmin: Missing id field in node-rev
To replay changes you've lost, you need to pretend to be a developer
making the changes again.
You need to use svn checkout to get a working copy, make the changes
which happened (and have been lost) in that working copy, and use
svn commit to commit the changes, thereby re-creating the missing
revision. Then load additional revisions on top using svnadmin load.
If you don't know exactly what changes were made in the lost revisions,
maybe you can interpolate the changes by looking at past and
future revisions around the time of the missing changes?
You were talking about changes made to binary files in
http://svn.haxx.se/users/archive-2010-06/0218.shtml so it can be
quite hard to recreate the changes.
Maybe you can get away with just omitting the missing revisions
entirely? Dump all revisions you are able to dump. For the first dump,
use svnadmin dump -r0:X (where X is the last good revision before the
first one that got broken by the disk failure). For any dump
of later revision use svnadmind dump -rA:B with the --incremental option
of svnadmin dump. Then try to load all the dumps in succession.
I'm not sure what you mean when you describe that loading a dump
file into a repository causes a single large revision to be created.
That doesn't make any sense to me. I've never seen this happen.
When this happens, what's the output of the svnadmin load command,
and what's the output of the svn log command? Maybe I'm misunderstanding
what you are trying to say? Please show the exact command you are using
for loading. In fact, it would really help if you could simply post a
transcript of all the commands you have tried running so far, and their
output, without explaining your interpretation of things. This should
help avoid misunderstandings. Based on such a transcript, we could ask
further questions and provide hints, and hopefully help you fix the problem.
Stefan
Re: Recovering repository with multiple missing rev/ files
Posted by Eiren Smith <ei...@gmail.com>.
On Jul 30, 2010, at 2:05 PM, Stefan Sperling wrote:
> On Fri, Jul 30, 2010 at 01:57:12PM -0400, Eiren Smith wrote:
>> Unfortunately, it appears I cannot create dump files from revisions
>> after a missing revs/ file gap. All the dumpfiles I created which
>> began with a revision past a point of continuity were unusable. They
>> collapsed upon re-import (svnadmin load …) into a single revision
>> instead of hundreds of revisions.
>>
>> Any idea why that might be or how I might get around it?
>
> Hard to tell from your description, it's a bit vague.
>
> Can you show us exactly what commands you typed while attempting to fix
> the problem? And any error message in the output? The more, the better.
> Extra noise does not hurt. Feel free to anonymize usernames, path names etc.
>
> Stefan
Stefan,
Thanks for your instantaneous response.
When I encountered this dumpfile problem a month ago, I realize now, I was under the (very likely incorrect) impresion that I might be able to do a dump of revs from the middle of the repo before restoring (even with a dummy revision) the missing revisions before the range I was trying to dump. Not sure what I thought it'd do about those gaps, but I hoped it'd find a way to ignore them for the most part. And e.g. "svnadmin dump repo_dir -r100:200 > dumpfile" didn't produce errors, but when I tried to load that dumpfile into a new repo (just to test it), it produced one revision. No errors, just one big revision. I don't know exactly why that was but my working theory is that it was because of the missing revs/ files before those revisions (lower rev numbers than the range being dumped).
Today I see that this "flat" dumpfile problem could perhaps be remedied by creating dummy revisions for any revisions before the revision range I want to dump. Worth a try.
So I tried that, sort of. To do this, I'd need to restore some revisions from my dumpfile backup (the one that's very old) as I can't manually make dummy revisions for those because I don't have modified file lists for those particular missing-from-repo-but-present-in-my-backup-dumpfile revisions. Plus, if I can replace them with real revision files instead of making dummy revisions/commits, awesome, right? Well, I got an error when I tried this. Here's what I did:
1. Loaded my dumpfile backup (the only one I have, the one which is far too old) into an empty repo.
2. Copied revision revs/7/7263 file from that dumpfile-backup-based repo into my production repo (the one missing some revs/ files -- I'll call it my corrupt repo).
3. Tried to dump that corrupt repo to ensure that my transplanted 7263 file was well received. Result:
...
* Dumped revision 7262.
svnadmin: Corrupt node-revision '0-5919.0.r7252/272624'
svnadmin: Missing id field in node-rev
So that's my current barrier. It might seem at this point that I should just dump from 1:7262 and then load that into a new repo because then 7263 will probably drop nicely on top of that without the error above. But if I did that and it worked, I'd get through a few more revisions until I came to the point where my backup dumpfile leaves off (rev 7307 is the last rev in my backup dumpfile) and I'd have to suture my corrupt repo onto that. I must suspect I'd run into the same problem "merging" this dumpfile-based repo with the later revisions from my corrupt repo as I countered after (3) above, which would leave me at an impasse.
Any ideas?
Thanks for your help.
/eiren
Re: Recovering repository with multiple missing rev/ files
Posted by Stefan Sperling <st...@elego.de>.
On Fri, Jul 30, 2010 at 01:57:12PM -0400, Eiren Smith wrote:
> Unfortunately, it appears I cannot create dump files from revisions
> after a missing revs/ file gap. All the dumpfiles I created which
> began with a revision past a point of continuity were unusable. They
> collapsed upon re-import (svnadmin load …) into a single revision
> instead of hundreds of revisions.
>
> Any idea why that might be or how I might get around it?
Hard to tell from your description, it's a bit vague.
Can you show us exactly what commands you typed while attempting to fix
the problem? And any error message in the output? The more, the better.
Extra noise does not hurt. Feel free to anonymize usernames, path names etc.
Stefan