You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Edmund Horner <ed...@chrysophylax.cjb.net> on 2003/05/13 09:10:50 UTC
Some parts of repository corrupted - how to recover?
Disaster strikes!
Some preliminaries:
SVN version = 0.20
No. revisions = 255
Repository size = 157,955,540
OS = Windows XP
Disk format = NTFS
Dumping my repository fails on rev 81:
$ svnadmin dump e:\svn -r 81 > svn-dump-81
svn: Filesystem is corrupt
svn: txn_body_read_rep: checksum mismatch on rep "2bc":
expected: 55b7db82c802ce6418b983e6ac27516c
actual: f6cfdc3a53e178d783f614a820a2bf33
The last part of the partially dumped svn-dump-81 looks like:
Node-path: compendium1/trunk/work/dump.pcx
Node-kind: file
Node-action: add
Prop-content-length: 59
Text-content-length: 275595
Text-content-md5: 55b7db82c802ce6418b983e6ac27516c
Content-length: 275654
K 13
svn:mime-type
V 24
application/octet-stream
PROPS-END
some binary data...
Revisions 1-80 can be dumped. Revisions after 81 CANNOT be dumped.
"svnadmin recover" just goes ahead and runs as if nothing's wrong.
From experimentation, paths that don't contain this (or other) problem
files can be checked out. There are some other problem files where the
same problem occurs, in different revisions and paths.
My guess is that the data for certain paths, e.g.
compendium1/trunk/work/dump.pcx is screwed up. Or maybe it's just the
checksums.
Although I don't have a definitive list of problem paths, I'm prepared
to lose those files (the example I gave above is easily replacable).
How can I most easily recover those paths which aren't corrupted?
Is this another use case for "svn obliterate" ?
BTW, it's probably the unreliability of my machine that causes this (I
seem to get more than my share of cosmic rays here ;-), so I don't want
anyone to panic about bugs yet.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Some parts of repository corrupted - how to recover?
Posted by Edmund Horner <ed...@chrysophylax.cjb.net>.
> I was just suggesting db_dump/db_load as an alternative to hex editing
> the database files. As to how SVN uses the database files exactly and
> what to do when something gets corrupted, I don't have much of a clue.
>
> I suspect what you did above was what Karl was suggesting, though.
Very well... I'm still confidenct that I haven't lost everything, I'm
just worried about how much work it's going to be to extract all the
versions of all the other paths!
Time to start writing a script I guess. I had considered modifying
"svnadmin dump" to ignore the two problem paths, but I haven't compiled
Subversion on Windows yet so that idea's on the backburner.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Some parts of repository corrupted - how to recover?
Posted by Michael Wood <mw...@its.uct.ac.za>.
On Wed, May 14, 2003 at 11:19:51PM +1200, Edmund Horner wrote:
> >You could try db_dump and db_load. Not sure if you would have any more
> >luck with that, though :)
>
> Thanks for the tip Michael, that was the first time I've actually those
> enticing db_* programs.
>
> I dumped representations, and got a text file in which the target
> checksums appear twice (as opposed to four time in the binary). I
> edited the checksums to 0s, and loaded them into a new representations
> file (which was half the size of the original). However, Subversion
> appears to work exactly the same as before, complete with problem paths.
>
> Is that what you were suggesting, or was I supposed to dump/load
> something else?
I was just suggesting db_dump/db_load as an alternative to hex editing
the database files. As to how SVN uses the database files exactly and
what to do when something gets corrupted, I don't have much of a clue.
I suspect what you did above was what Karl was suggesting, though.
--
Michael Wood <mw...@its.uct.ac.za>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Some parts of repository corrupted - how to recover?
Posted by Edmund Horner <ed...@chrysophylax.cjb.net>.
> You could try db_dump and db_load. Not sure if you would have any more
> luck with that, though :)
Thanks for the tip Michael, that was the first time I've actually those
enticing db_* programs.
I dumped representations, and got a text file in which the target
checksums appear twice (as opposed to four time in the binary). I
edited the checksums to 0s, and loaded them into a new representations
file (which was half the size of the original). However, Subversion
appears to work exactly the same as before, complete with problem paths.
Is that what you were suggesting, or was I supposed to dump/load
something else?
Edmund.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Some parts of repository corrupted - how to recover?
Posted by Michael Wood <mw...@its.uct.ac.za>.
On Wed, May 14, 2003 at 02:22:58PM +1200, Edmund Horner wrote:
[snip]
> Ok, not sure if I need a special BDB tool for this... I loaded the
> file in a hex editor, and changed all four places where the string
> occurred to zeros.
[snip]
You could try db_dump and db_load. Not sure if you would have any more
luck with that, though :)
--
Michael Wood <mw...@its.uct.ac.za>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Some parts of repository corrupted - how to recover?
Posted by Edmund Horner <ed...@chrysophylax.cjb.net>.
> Do you know what the contents of that file should be? (And do those
> contents match either of the checksums?)
There are two problem paths, and I had a working copy containing the
other one, so I got the checksum of that file and it matches the
recorded checksum in the repository. I.e. I think the checksums are
fine, it's the data which is wrong.
> If you can't get the contents, then make a copy of your repository,
> find the checksum in the db/representations representations table and
> edit it from
>
> 55b7db82c802ce6418b983e6ac27516c
>
> to
>
> 00000000000000000000000000000000
>
> Then try again (using the copy, of course). The all-zeros checksum
> always matches in Subversion.
Ok, not sure if I need a special BDB tool for this... I loaded the file
in a hex editor, and changed all four places where the string occurred
to zeros.
$ svnadmin dump e:\svn > svn-dump-test
* Dumped revision 0.
...etc...
* Dumped revision 80.
svn: Malformed skeleton data
svn: Malformed representation skeleton
The last part of svn-dump test looks like:
Node-path: compendium1/trunk/work/dump.pcx
Node-kind: file
Node-action: add
Prop-content-length: 59
which is the name of the path that's not working. It looks like it
stops before it gets to "Text-content-length", "Text-content-md5", and
"Content-length".
So I'm assuming one of:
1. I should only have changed one or two of the places, or using a hex
editor isn't the right way.
2. I need to change something else, too.
3. Something else is corrupt.
>>Is this another use case for "svn obliterate" ?
>
> Nah, it's a case for finding & fixing the cause of the problem :-).
Yeah... the problem being the dodgy machine and the dodgy practices of
its owner, e.g. no backups!
> Oh, well, thanks, that's reassuring in a way... How can you work on a
> machine that randomly changes bits, man? :-)
Student = no money.
Thanks for help so far,
Edmund.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Some parts of repository corrupted - how to recover?
Posted by kf...@collab.net.
Edmund Horner <ed...@chrysophylax.cjb.net> writes:
> Dumping my repository fails on rev 81:
>
> $ svnadmin dump e:\svn -r 81 > svn-dump-81
> svn: Filesystem is corrupt
> svn: txn_body_read_rep: checksum mismatch on rep "2bc":
> expected: 55b7db82c802ce6418b983e6ac27516c
> actual: f6cfdc3a53e178d783f614a820a2bf33
Do you know what the contents of that file should be? (And do those
contents match either of the checksums?)
If you can't get the contents, then make a copy of your repository,
find the checksum in the db/representations representations table and
edit it from
55b7db82c802ce6418b983e6ac27516c
to
00000000000000000000000000000000
Then try again (using the copy, of course). The all-zeros checksum
always matches in Subversion.
> The last part of the partially dumped svn-dump-81 looks like:
>
> Node-path: compendium1/trunk/work/dump.pcx
> Node-kind: file
> Node-action: add
> Prop-content-length: 59
> Text-content-length: 275595
> Text-content-md5: 55b7db82c802ce6418b983e6ac27516c
> Content-length: 275654
>
> K 13
> svn:mime-type
> V 24
> application/octet-stream
> PROPS-END
> some binary data...
>
> Revisions 1-80 can be dumped. Revisions after 81 CANNOT be dumped.
>
> "svnadmin recover" just goes ahead and runs as if nothing's wrong.
>
> From experimentation, paths that don't contain this (or other)
> problem files can be checked out. There are some other problem files
> where the same problem occurs, in different revisions and paths.
>
> My guess is that the data for certain paths,
> e.g. compendium1/trunk/work/dump.pcx is screwed up. Or maybe it's
> just the checksums.
>
> Although I don't have a definitive list of problem paths, I'm prepared
> to lose those files (the example I gave above is easily
> replacable). How can I most easily recover those paths which aren't
> corrupted?
>
> Is this another use case for "svn obliterate" ?
Nah, it's a case for finding & fixing the cause of the problem :-).
> BTW, it's probably the unreliability of my machine that causes this (I
> seem to get more than my share of cosmic rays here ;-), so I don't
> want anyone to panic about bugs yet.
Oh, well, thanks, that's reassuring in a way... How can you work on a
machine that randomly changes bits, man? :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org