You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by David Hopkins <DH...@serck-controls.com.au> on 2011/09/15 04:30:46 UTC
How to fix corrupt revision in repo?
Greetings,
I have an SVN repo that is failing svnadmin verify on revision 192.
For some reason the verify output says:
[...various successful revisions...]
* Verified revision 191.
svnadmin: Can't read file 'E:\Repositories\Client_Name\db\revs\0\16
2': End of file found
I'm not sure why the verification command for revision 192 would throw
an error description for revision 162. Revision 192 affected a
completely different part of the repository to revision 162, so there is
no obvious relationship between them.
All the revisions from 193 to 332 (HEAD) are ok. It might be a one-off.
This looks like a FSFS-backed repository (I am very new to SVN and
inherited the server from someone else!). The server is VisualSVN 2.1.4,
which is based on SVN 1.6.13.
The clients are mostly TortoiseSVN 1.6.16, which uses SVN 1.6.17.
What steps should I take to fix the corrupted revision? Is there more
information that I should provide? (eg a copy of the rev 192 file?) This
problem is causing checkouts and updates to fail for files that were
last modified in that revision.
Regards,
David Hopkins
Serck Controls
===== PRIVACY AND CONFIDENTIALITY NOTICE =====
The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
NOTE - You should carry out your own virus checks before opening any attachment.
Re: How to fix corrupt revision in repo?
Posted by Neil Bird <ne...@jibbyjobby.co.uk>.
Around about 16/09/11 11:22, Tony Sweeney typed ...
> Unlike 'svnadmin dump', hotcopy will happily back up a corrupt revision
> and not tell you. It's really just a clever filesystem backup with a
> very careful time ordering of certain key files in case there is a
> transaction in progress when it runs. Having been bitten by this
> myself[*], we now run svnadmin out of cron every night before we run our
> hotcopy. We also keep a week's worth of hotcopies, and I check my cron
> emails every morning.
> [*] fsfsverify.py is your friend
OK, thanks: I'll update our backup script!
--
[neil@fnx ~]# rm -f .signature
[neil@fnx ~]# ls -l .signature
ls: .signature: No such file or directory
[neil@fnx ~]# exit
Re: How to fix corrupt revision in repo?
Posted by Daniel Shahaf <da...@elego.de>.
edg wrote on Fri, Sep 16, 2011 at 13:08:40 -0700:
> On 9/16/2011 3:22 AM, Tony Sweeney wrote:
> >Unlike 'svnadmin dump', hotcopy will happily back up a corrupt revision
> >and not tell you. It's really just a clever filesystem backup with a
> >very careful time ordering of certain key files in case there is a
> >transaction in progress when it runs. Having been bitten by this
> >myself[*], we now run svnadmin out of cron every night before we run our
> >hotcopy. We also keep a week's worth of hotcopies, and I check my cron
> >emails every morning.
> >
>
>
> I have been working under the assumption that the best way to
> "verify" is by:
>
> svnadmin dump REPO > NUL ( or to ' > /dev/null' for 'nix)
>
> rather than
>
> svnadmin verify REPO
>
> The reason being that the former also checks revision property files.
>
+0.5
> See:
>
> http://svn.haxx.se/users/archive-2009-10/0512.shtml
>
See http://subversion.tigris.org/issues/show_bug.cgi?id=3441
>
>
Re: How to fix corrupt revision in repo?
Posted by edg <eg...@gmail.com>.
On 9/16/2011 3:22 AM, Tony Sweeney wrote:
> Unlike 'svnadmin dump', hotcopy will happily back up a corrupt revision
> and not tell you. It's really just a clever filesystem backup with a
> very careful time ordering of certain key files in case there is a
> transaction in progress when it runs. Having been bitten by this
> myself[*], we now run svnadmin out of cron every night before we run our
> hotcopy. We also keep a week's worth of hotcopies, and I check my cron
> emails every morning.
>
I have been working under the assumption that the best way to "verify"
is by:
svnadmin dump REPO > NUL ( or to ' > /dev/null' for 'nix)
rather than
svnadmin verify REPO
The reason being that the former also checks revision property files.
See:
http://svn.haxx.se/users/archive-2009-10/0512.shtml
RE: How to fix corrupt revision in repo?
Posted by Tony Sweeney <ts...@omnifone.com>.
Unlike 'svnadmin dump', hotcopy will happily back up a corrupt revision
and not tell you. It's really just a clever filesystem backup with a
very careful time ordering of certain key files in case there is a
transaction in progress when it runs. Having been bitten by this
myself[*], we now run svnadmin out of cron every night before we run our
hotcopy. We also keep a week's worth of hotcopies, and I check my cron
emails every morning.
Tony.
[*] fsfsverify.py is your friend
-----Original Message-----
From: Neil Bird [mailto:neil@jibbyjobby.co.uk]
Sent: 16 September 2011 09:54
To: users@subversion.apache.org
Subject: Re: How to fix corrupt revision in repo?
Around about 15/09/11 03:30, David Hopkins typed ...
> I have an SVN repo that is failing svnadmin verify on revision 192.
Slightly OT, if I may: would a 'svnadmin hotcopy' also show up
errors that 'verify' would? We use hotcopy to pull our repos off the
SVN server onto a backup-up network drive, but we don't use verify
regularly.
The network drive has incremental backups so if we see an error
during the copy we can still get older, valid, copies of revs. back.
Do we need to be running verify as well?
--
[neil@fnx ~]# rm -f .signature
[neil@fnx ~]# ls -l .signature
ls: .signature: No such file or directory [neil@fnx ~]# exit
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 10.0.1410 / Virus Database: 1520/3899 - Release Date: 09/15/11
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Re: How to fix corrupt revision in repo?
Posted by Neil Bird <ne...@jibbyjobby.co.uk>.
Around about 15/09/11 03:30, David Hopkins typed ...
> I have an SVN repo that is failing svnadmin verify on revision 192.
Slightly OT, if I may: would a 'svnadmin hotcopy' also show up errors
that 'verify' would? We use hotcopy to pull our repos off the SVN server
onto a backup-up network drive, but we don't use verify regularly.
The network drive has incremental backups so if we see an error during
the copy we can still get older, valid, copies of revs. back.
Do we need to be running verify as well?
--
[neil@fnx ~]# rm -f .signature
[neil@fnx ~]# ls -l .signature
ls: .signature: No such file or directory
[neil@fnx ~]# exit
Re: How to fix corrupt revision in repo?
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag David Hopkins,
am Donnerstag, 15. September 2011 um 04:30 schrieben Sie:
> What steps should I take to fix the corrupted revision?
If you have a backup, replace the file wih the backed up one. If you
don't, really really bad and it's the next you should take care of. In
the worst case you have no other chance but to hope that one of you're
working copies has the most current version of every data, create a
new repository with the old, working revisions dumped into and make a
very large commit to get the most current source as a new version.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning
AM-SoFT IT-Systeme - Hameln | Potsdam | Leipzig
Telefon: Potsdam: 0331-743881-0
E-Mail: tschoening@am-soft.de
Web: http://www.am-soft.de
AM-SoFT GmbH IT-Systeme, Konsumhof 1-5, 14482 Potsdam
Amtsgericht Potsdam HRB 21278 P, Geschäftsführer: Andreas Muchow
RE: How to fix corrupt revision in repo?
Posted by David Hopkins <DH...@serck-controls.com.au>.
> Daniel Shahaf wrote on Monday, 19 September 2011 9:27 PM:
> You ought to be able to keep the rest of the history even without
fixing
> the brokenness in r192. (as the file is deleted in HEAD, a checkout
> should work; and you also have the option of dumping the history while
> excluding the problematic file from it (via authz+svnsync/svnrdump or
> svndumpfilter).)
I'll look into the authz+svnsync/svnrdump option. Svndumpfilter doesn't
work for me because the 'svnadmin dump' operation fails when it tries to
process 192 (before I get a chance to use svndumpfilter to eliminate the
bogus file). As far as I can tell svndumpfilter operates on dumpfiles
that already exist, and can't actually stop svnadmin from trying to
resolve the bogus node-rev header during the dump process. The
authz+svnsync solution will hopefully allow me to effectively do that
filtering at an earlier stage in the pipeline.
Thank you very much for all your help,
David Hopkins
Serck Controls
===== PRIVACY AND CONFIDENTIALITY NOTICE =====
The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
NOTE - You should carry out your own virus checks before opening any attachment.
RE: How to fix corrupt revision in repo?
Posted by David Hopkins <DH...@serck-controls.com.au>.
> > Daniel Shahaf wrote on Monday, 19 September 2011 9:27 PM:
> > You ought to be able to keep the rest of the history even without
> fixing
> > the brokenness in r192. (as the file is deleted in HEAD, a checkout
> > should work; and you also have the option of dumping the history
while
> > excluding the problematic file from it (via authz+svnsync/svnrdump
or
> > svndumpfilter).)
>
> I'll look into the authz+svnsync/svnrdump option. Svndumpfilter
doesn't
> work for me because the 'svnadmin dump' operation fails when it tries
to
> process 192 (before I get a chance to use svndumpfilter to eliminate
the
> bogus file). As far as I can tell svndumpfilter operates on dumpfiles
> that already exist, and can't actually stop svnadmin from trying to
> resolve the bogus node-rev header during the dump process. The
> authz+svnsync solution will hopefully allow me to effectively do that
> filtering at an earlier stage in the pipeline.
For the benefit of anyone else who comes across this message thread in
the future, I thought I'd post a final follow-up message with my
results.
The authz+svnrdump solution *did* work for creating a dumpfile without
references to the corrupted file revision. I ended up setting up a
temporary server where I could set custom authz permissions, and
downloaded a beta SVN 1.7 client so that I could use svnrdump rather
than svnsync (which was much simpler to set up). I've successfully
loaded the purged dumpfile into a new repository which now works with
svnadmin verify, svnadmin dump, svnadmin hotcopy etc.
Thanks once again for all your help (especially the authz+svnrdump
suggestion).
Regards,
David Hopkins
Serck Controls
===== PRIVACY AND CONFIDENTIALITY NOTICE =====
The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
NOTE - You should carry out your own virus checks before opening any attachment.
Re: How to fix corrupt revision in repo?
Posted by Daniel Shahaf <da...@elego.de>.
David Hopkins wrote on Mon, Sep 19, 2011 at 10:06:17 +0800:
> > Daniel Shahaf wrote:
> > One more thing. The fact that in r162 one file was deleted *and no
> > files were added or changed* implies that the only new representations
> > in r162 would be directory representations --- it wouldn't add any
> > *file* representations --- so the reference to r162 in the node-rev
> > header (the sequence of ASCII lines of which the "text:" line is part)
> > is almost certainly bogus.
> >
> > I'm curious to hear whether the problem was indeed that the noderev
> > referred to r162 instead of r192.
>
> Sadly, it wasn't. I've now experimented with that. The offset supposedly
> within r162 is listed as 670867 bytes, which is well outside the total
> length of r162 as we've already discussed. But it isn't a valid pointer
> within r192 either; offset 670867 points to the middle of one of the
> other rep blocks within the r192 file. I've had a look at the other
> node-rev headers and it appears that all the rep blocks in the r192 file
> are fully accounted for by the node-revs which have text: 192. (That is,
> there are no representations in the r192 file which don't already have a
> valid node-rev header).
>
So the question is whether the rep is lost or just misplaced.
> I've had a look through all the revs between 162 and 192 which are at
> least 600 KiB in size. But I can't find *any* rev files in the whole
> repository history leading up to 192 where an offset of 670867 points to
> the beginning of a DELTA or PLAIN representation.
>
> So, I'm now assuming that both the reference to r162, and the offset of
> 670867 bytes, are bogus. But there aren't any obvious candidates for a
> non-bogus representation of that particular file update.
>
Okay. FWIW, there is no easy way to find an orphaned/unreferenced rep
in the revision files. The rightest way would be to parse all node-ids
(which _is_ possible, think 'svn ls -R'), collect all the rep pointers
they contain, then see if there are any additional reps. And even then,
identifying their length would have to be done manually... (reps do not
contain their own length)
> Given that the file with the bogus node-rev is unimportant, and has
> since been deleted from the repo, is there any way to patch the r192
> rev-file so that the repository has enough internal consistency to
> produce a valid dump file?
>
I suppose you could do it in a number of ways, either by changing what
rep the node-rev refers to or by unlinking the node-rev from its parent.
Be careful about successors of the file --- whether new revision of the
file are expressed as deltas to the node-rev which is now corrupted.
You must know whether that's the case before deciding how to alter that
node-rev.
Also, you must cause no offsets in existing rev files to change (so, add
padding as needed). And probably a few other things that I forget right
now...
> At the moment it looks like the "nuclear option" is to check out the
> current version of everything and start a new repository with it. This
> *should* work because the corrupted file isn't included in recent
> revisions, so SVN won't need to de-reference the invalid reference in
> r192 when performing the check out. But if I can purge the broken-ness
> from the repo and keep the rest of the history, that would obviously be
> better. I certainly don't want to keep using a repo that doesn't
> validate and can't be dumped, though.
>
You ought to be able to keep the rest of the history even without fixing
the brokenness in r192. (as the file is deleted in HEAD, a checkout
should work; and you also have the option of dumping the history while
excluding the problematic file from it (via authz+svnsync/svnrdump or svndumpfilter).)
> >
> > Daniel Shahaf wrote on Fri, Sep 16, 2011 at 15:37:11 +0300:
> > > Quick reply, more verbose one might follow up later.
> > >
> > > Your reply breaks the nested quoting levels, please try to avoid it,
> > are
> > > you sending mail as text/plain?
> > >
>
> Sorry about breaking the nested quoting. I'm using Outlook which is
> pretty mediocre as a plain-text email client. I was already using
> text/plain, but Outlook's quoting style wasn't right, so I was trying to
> manually fix the text-wrapping and quote marks. Clearly I wasn't getting
> it right. I've now found a couple more Outlook settings which will
> hopefully address the problem.
>
Thanks, much better now.
> Unfortunately, it doesn't look like I'll be able to send you the actual
> rev file(s), at least not without a lot of inconvenience that I don't
> want to subject you to (ie an NDA, since we don't actually own the IP to
> any of the code which may be included in the rev file). Sorry about
> that.
>
No problems, and good luck.
If you need docs about the FS that 'structure' doesn't have, they would
be under ../libsvn_fs_base/.
> Regards,
>
> David Hopkins
>
Re: How to fix corrupt revision in repo?
Posted by Nico Kadel-Garcia <nk...@gmail.com>.
2011/9/19 Thorsten Schöning <ts...@am-soft.de>:
> Guten Tag David Hopkins,
> am Montag, 19. September 2011 um 04:06 schrieben Sie:
>
>> At the moment it looks like the "nuclear option" is to check out the
>> current version of everything and start a new repository with it.
>
> You can dump the old repository until the last working rev and start
> from that with your current working copies, so not loosing all of your
> history.
You *CAN*. You can also do that to a system without broken revisions.
You have to be very careful not to insert a more subtlely corrupted
revision, and you *should* change your uuid and check out new working
copies lest someone with the old revision in an old working copy wind
up severely horking themselves. Simply leaving out the broken
revision, renumbering, and keeping the old uuid will break things
very, very badly. (Did that once editing out a DVD image that someone
added, fortunately it was early in the project before the repository
was considered production grade.
>> But if I can purge the broken-ness
>> from the repo and keep the rest of the history, that would obviously be
>> better.
>
> My impression on problems like yours, which I had too, is, that in
> most cases trying to repair broken rev files was simply not
> successful. Learn from it and take the time to design a proper
> backup/sync mechanism.
This is not as easy as it sounds. Any revision in a core repository
that is corrupted online and for which rsync overwrites the backup
server's copy is a real problem, and keeping numerous old copies lying
around on tape or separate disks is a support issue. And "rsync" based
backups have real problems with revisions in the process of being
committed on the primary server. The bulky and lengthy svnadmin "dump"
and "load" processes are resource intensive and awkward to manage, and
they take a *long* time on a big repository, leaving a significant
window of opportunity for corruption.
The revisions are also vulnerable to filesystem operations by the SVN
administrator account, or shared account, on the SVN server. This is
one of the reasons I *loathe* shared write permissions for file://
access, it's vulnerable to mistakes. . A backup system that merely
replicates those without reliable off-line or distinct disk space
storage is vulnerable to all sorts of surprises. My personal favorite
solution is a NetApp storage server with filesystem ".snapshot"
directories, and an off-site backup with svnsync running in case the
NetApp goes toes-up.
Re: How to fix corrupt revision in repo?
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag David Hopkins,
am Montag, 19. September 2011 um 04:06 schrieben Sie:
> At the moment it looks like the "nuclear option" is to check out the
> current version of everything and start a new repository with it.
You can dump the old repository until the last working rev and start
from that with your current working copies, so not loosing all of your
history.
> But if I can purge the broken-ness
> from the repo and keep the rest of the history, that would obviously be
> better.
My impression on problems like yours, which I had too, is, that in
most cases trying to repair broken rev files was simply not
successful. Learn from it and take the time to design a proper
backup/sync mechanism.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning
AM-SoFT IT-Systeme - Hameln | Potsdam | Leipzig
Telefon: Potsdam: 0331-743881-0
E-Mail: tschoening@am-soft.de
Web: http://www.am-soft.de
AM-SoFT GmbH IT-Systeme, Konsumhof 1-5, 14482 Potsdam
Amtsgericht Potsdam HRB 21278 P, Geschäftsführer: Andreas Muchow
RE: How to fix corrupt revision in repo?
Posted by David Hopkins <DH...@serck-controls.com.au>.
> Daniel Shahaf wrote:
> One more thing. The fact that in r162 one file was deleted *and no
> files were added or changed* implies that the only new representations
> in r162 would be directory representations --- it wouldn't add any
> *file* representations --- so the reference to r162 in the node-rev
> header (the sequence of ASCII lines of which the "text:" line is part)
> is almost certainly bogus.
>
> I'm curious to hear whether the problem was indeed that the noderev
> referred to r162 instead of r192.
Sadly, it wasn't. I've now experimented with that. The offset supposedly
within r162 is listed as 670867 bytes, which is well outside the total
length of r162 as we've already discussed. But it isn't a valid pointer
within r192 either; offset 670867 points to the middle of one of the
other rep blocks within the r192 file. I've had a look at the other
node-rev headers and it appears that all the rep blocks in the r192 file
are fully accounted for by the node-revs which have text: 192. (That is,
there are no representations in the r192 file which don't already have a
valid node-rev header).
I've had a look through all the revs between 162 and 192 which are at
least 600 KiB in size. But I can't find *any* rev files in the whole
repository history leading up to 192 where an offset of 670867 points to
the beginning of a DELTA or PLAIN representation.
So, I'm now assuming that both the reference to r162, and the offset of
670867 bytes, are bogus. But there aren't any obvious candidates for a
non-bogus representation of that particular file update.
Given that the file with the bogus node-rev is unimportant, and has
since been deleted from the repo, is there any way to patch the r192
rev-file so that the repository has enough internal consistency to
produce a valid dump file?
At the moment it looks like the "nuclear option" is to check out the
current version of everything and start a new repository with it. This
*should* work because the corrupted file isn't included in recent
revisions, so SVN won't need to de-reference the invalid reference in
r192 when performing the check out. But if I can purge the broken-ness
from the repo and keep the rest of the history, that would obviously be
better. I certainly don't want to keep using a repo that doesn't
validate and can't be dumped, though.
>
> Daniel Shahaf wrote on Fri, Sep 16, 2011 at 15:37:11 +0300:
> > Quick reply, more verbose one might follow up later.
> >
> > Your reply breaks the nested quoting levels, please try to avoid it,
> are
> > you sending mail as text/plain?
> >
Sorry about breaking the nested quoting. I'm using Outlook which is
pretty mediocre as a plain-text email client. I was already using
text/plain, but Outlook's quoting style wasn't right, so I was trying to
manually fix the text-wrapping and quote marks. Clearly I wasn't getting
it right. I've now found a couple more Outlook settings which will
hopefully address the problem.
Unfortunately, it doesn't look like I'll be able to send you the actual
rev file(s), at least not without a lot of inconvenience that I don't
want to subject you to (ie an NDA, since we don't actually own the IP to
any of the code which may be included in the rev file). Sorry about
that.
Regards,
David Hopkins
===== PRIVACY AND CONFIDENTIALITY NOTICE =====
The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
NOTE - You should carry out your own virus checks before opening any attachment.
Re: How to fix corrupt revision in repo?
Posted by Daniel Shahaf <da...@elego.de>.
One more thing. The fact that in r162 one file was deleted *and no
files were added or changed* implies that the only new representations
in r162 would be directory representations --- it wouldn't add any
*file* representations --- so the reference to r162 in the node-rev
header (the sequence of ASCII lines of which the "text:" line is part)
is almost certainly bogus.
I'm curious to hear whether the problem was indeed that the noderev
referred to r162 instead of r192.
Daniel Shahaf wrote on Fri, Sep 16, 2011 at 15:37:11 +0300:
> Quick reply, more verbose one might follow up later.
>
> Your reply breaks the nested quoting levels, please try to avoid it, are
> you sending mail as text/plain?
>
> (more below)
>
> David Hopkins wrote on Fri, Sep 16, 2011 at 13:05:52 +0800:
> > David Hopkins wrote on Fri, Sep 16, 2011 at 08:30:14 +0800:
> > > It's normal for r192 to contain "text: 162" if rep-sharing is enabled
> > > or
> > > if you did a copy-without-textmods from r162.
> > >
> >
> > Ok. I think rep-sharing is probably enabled because this server was
> > installed
> > using SVN 1.6, and we haven't altered the setting. (It's on by default,
> > yes?)
> >
>
> Yes. See $REPOS/db/rep-cache.db
>
> > But, I can see from the CPATH which file in r192 is referencing r162
> > (EDGE.CSV), and that reference doesn't make sense.
> >
>
> 'cpath' is created path. It doesn't change in copies.
>
> > > > > To answer your question: yes, most definitely a copy of the r192
> > > > > (and/or r162) rev file would allow to pinpoint the problem,
> > > > > however you might not want to share those files on a public list
> > > > > as they may contain sensitive data (versioned file contents).
> > > >
> > > > I'll find out if I can release the broken revisions in their
> > entirety.
> > > >
> > >
> > > Perhaps someone would be willing to have a look at those two revision
> > > files privately.
> > >
> > > (In fact, I might be able to do this too. But I'm reluctant to make
> > > a promise or commitment about this.)
> > >
> > > > The corrupted revision doesn't actually contain anything
> > > > particularly important (almost all the modified files in it have
> > > > since been replaced by newer versions anyway). Can I fix the
> > > > repository by dumping every revision except 192, and then
> > > > reloading the good revisions into a new repo? Or will cause
> > > > problems for the revisions after 192 since one of the revisions no
> > > > longer exists?
> > > >
> > >
> > > That won't work if files after r192 are stored as deltas against the
> > > fulltext of r192.
> > >
> >
> > Hmm, ok.
> >
> > I'm thinking about making a copy of the repository folder, and seeing
> > what happens if I replace "text: 162" with "text: 192" in revs\0\192,
> > since the offsets appear to pass the "smell test" for file size. Is
> > there _any_ chance that that will work?
>
> I don't think I've heard of precedents of this bug, but sure, try it.
> If you want to test before doing it, use 'xxd -l 5 -s 670867 r162-rev-file',
> it should say either "DELTA" or "PLAIN" (with no trailing end of line).
>
> Also, standard practice, check your disk for hardware errors. (should
> have said that earlier, sorry)
>
> > Or are there other references
> > I would also need to patch inside the revs\0\192 file?
> >
>
> If there are they would all appear as ASCII '162'. Revision numbers are
> referred to in the text: node-revision header and sometimes in the header
> of DELTA-style reps. See
> https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure
Re: How to fix corrupt revision in repo?
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Quick reply, more verbose one might follow up later.
Your reply breaks the nested quoting levels, please try to avoid it, are
you sending mail as text/plain?
(more below)
David Hopkins wrote on Fri, Sep 16, 2011 at 13:05:52 +0800:
> David Hopkins wrote on Fri, Sep 16, 2011 at 08:30:14 +0800:
> > It's normal for r192 to contain "text: 162" if rep-sharing is enabled
> > or
> > if you did a copy-without-textmods from r162.
> >
>
> Ok. I think rep-sharing is probably enabled because this server was
> installed
> using SVN 1.6, and we haven't altered the setting. (It's on by default,
> yes?)
>
Yes. See $REPOS/db/rep-cache.db
> But, I can see from the CPATH which file in r192 is referencing r162
> (EDGE.CSV), and that reference doesn't make sense.
>
'cpath' is created path. It doesn't change in copies.
> > > > To answer your question: yes, most definitely a copy of the r192
> > > > (and/or r162) rev file would allow to pinpoint the problem,
> > > > however you might not want to share those files on a public list
> > > > as they may contain sensitive data (versioned file contents).
> > >
> > > I'll find out if I can release the broken revisions in their
> entirety.
> > >
> >
> > Perhaps someone would be willing to have a look at those two revision
> > files privately.
> >
> > (In fact, I might be able to do this too. But I'm reluctant to make
> > a promise or commitment about this.)
> >
> > > The corrupted revision doesn't actually contain anything
> > > particularly important (almost all the modified files in it have
> > > since been replaced by newer versions anyway). Can I fix the
> > > repository by dumping every revision except 192, and then
> > > reloading the good revisions into a new repo? Or will cause
> > > problems for the revisions after 192 since one of the revisions no
> > > longer exists?
> > >
> >
> > That won't work if files after r192 are stored as deltas against the
> > fulltext of r192.
> >
>
> Hmm, ok.
>
> I'm thinking about making a copy of the repository folder, and seeing
> what happens if I replace "text: 162" with "text: 192" in revs\0\192,
> since the offsets appear to pass the "smell test" for file size. Is
> there _any_ chance that that will work?
I don't think I've heard of precedents of this bug, but sure, try it.
If you want to test before doing it, use 'xxd -l 5 -s 670867 r162-rev-file',
it should say either "DELTA" or "PLAIN" (with no trailing end of line).
Also, standard practice, check your disk for hardware errors. (should
have said that earlier, sorry)
> Or are there other references
> I would also need to patch inside the revs\0\192 file?
>
If there are they would all appear as ASCII '162'. Revision numbers are
referred to in the text: node-revision header and sometimes in the header
of DELTA-style reps. See
https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure
RE: How to fix corrupt revision in repo?
Posted by David Hopkins <DH...@serck-controls.com.au>.
David Hopkins wrote on Fri, Sep 16, 2011 at 08:30:14 +0800:
> > Here's the context in the rev file:
> > id: y-31.0-32.r192/673830
> > type: file
> > pred: y-31.0.r31/264323
> > count: 1
> > text: 162 670867 6111 52486 5117fb0964ca1a78dd97447d23452e73
> > 609f4745460d6e14860daff0803ee7024c54898c 191-5r/_m
>
> That tells you that the 6111 bytes starting at offset 670867(bytes)
into
> the r162 rev file are a representation generating a file whose
checksums
> and uniquifier are given later. See subversion/libsvn_fs_fs/structure
> for details --- basically, it's DELTA\n or PLAIN\n up through
ENDREP\n.
That's interesting. It certainly explains the "end of file" error
message
that is getting thrown, because rev 162 is only 1,506 bytes long.
Rev 162 was a deletion of a single file from a different folder in the
repo so I'd be surprised if it contained any file representations at
all.
Rev 192 is 683,471 bytes long, so it *is* long enough for a 670867 byte
offset to make sense.
> >
> > Looking at the other nearby entries, they have "text: 192 [...]"
> > instead of "text: 162 [...]". Is that likely to be the problem?
> >
>
> It's normal for r192 to contain "text: 162" if rep-sharing is enabled
or
> if you did a copy-without-textmods from r162.
>
Ok. I think rep-sharing is probably enabled because this server was
installed
using SVN 1.6, and we haven't altered the setting. (It's on by default,
yes?)
But, I can see from the CPATH which file in r192 is referencing r162
(EDGE.CSV), and that reference doesn't make sense.
The history of EDGE.CSV is as follows:
R31: EDGE.CSV added to repo
R32: one of the directories in EDGE.CSV's parent path was renamed
(R162: a single file in a completely different part of the repo was
deleted.
Literally the only part of their file path in common was the repo root
folder. EDGE.CSV and the deleted file have no shared history,
relationships,
or even data in common - one is a CSV file and the deleted file was a
binary
archive!)
R192: EDGE.CSV was modified, along with several other files in the same
folder.
I've now checked, and every single other text: field in R192 references
R192.
There are no other revisions referenced.
R335: EDGE.CSV was deleted. This is because that file wasn't very
important,
and all the other files which changed in r192 were updated in later
revisions
and apparently can be successfully checked out/updated.
> > > To answer your question: yes, most definitely a copy of the r192
> > (and/or
> > > r162) rev file would allow to pinpoint the problem, however you
might
> > > not want to share those files on a public list as they may contain
> > > sensitive data (versioned file contents).
> >
> > I'll find out if I can release the broken revisions in their
entirety.
> >
>
> Perhaps someone would be willing to have a look at those two revision
> files privately.
>
> (In fact, I might be able to do this too. But I'm reluctant to make
> a promise or commitment about this.)
>
> > The corrupted revision doesn't actually contain anything
particularly
> > important (almost all the modified files in it have since been
replaced
> > by newer versions anyway). Can I fix the repository by dumping every
> > revision except 192, and then reloading the good revisions into a
new
> > repo? Or will cause problems for the revisions after 192 since one
of
> > the revisions no longer exists?
> >
>
> That won't work if files after r192 are stored as deltas against the
> fulltext of r192.
>
Hmm, ok.
I'm thinking about making a copy of the repository folder, and seeing
what
happens if I replace "text: 162" with "text: 192" in revs\0\192, since
the
offsets appear to pass the "smell test" for file size. Is there _any_
chance
that that will work? Or are there other references I would also need to
patch
inside the revs\0\192 file?
I thought I'd try doing an svndump and then use svndumpfilter to exclude
EDGE.CSV, since it seems to be the only thing with an invalid rev
reference,
but the svnadmin dump operation fails when it gets to r192, since it
can't
process the reference to r162 either.
Regards,
David Hopkins
Serck Controls
===== PRIVACY AND CONFIDENTIALITY NOTICE =====
The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
NOTE - You should carry out your own virus checks before opening any attachment.
Re: How to fix corrupt revision in repo?
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
David Hopkins wrote on Fri, Sep 16, 2011 at 08:30:14 +0800:
> Thanks Daniel. Responses inline.
>
> >> David Hopkins wrote on Thu, Sep 15, 2011 at 10:30:46 +0800: I'm not
> sure
> >> why the verification command for revision 192 would throw an error
> >> description for revision 162. Revision 192 affected a completely
> >> different part of the repository to revision 162, so there is no
> obvious
> >> relationship between them.
> >
> > Possibly due to rep-sharing? Does db/revs/0/192 contain the number
> "162"
> > in ASCII decimal delimited by whitespace? You can check that with the
> > following command: grep -a "^text:" db/revs/0/192
>
> Yes it does.
>
> Here's the context in the rev file:
> id: y-31.0-32.r192/673830
> type: file
> pred: y-31.0.r31/264323
> count: 1
> text: 162 670867 6111 52486 5117fb0964ca1a78dd97447d23452e73
> 609f4745460d6e14860daff0803ee7024c54898c 191-5r/_m
That tells you that the 6111 bytes starting at offset 670867(bytes) into
the r162 rev file are a representation generating a file whose checksums
and uniquifier are given later. See subversion/libsvn_fs_fs/structure
for details --- basically, it's DELTA\n or PLAIN\n up through ENDREP\n.
> cpath: [redacted]
> copyroot: 32 [redacted]
>
> Looking at the other nearby entries, they have "text: 192 [...]"
> instead of "text: 162 [...]". Is that likely to be the problem?
>
It's normal for r192 to contain "text: 162" if rep-sharing is enabled or
if you did a copy-without-textmods from r162.
> > Does 'svnadmin dump -r 162 >NUL' work ?
>
> Yes it does.
>
Hmmm.
> > To answer your question: yes, most definitely a copy of the r192
> (and/or
> > r162) rev file would allow to pinpoint the problem, however you might
> > not want to share those files on a public list as they may contain
> > sensitive data (versioned file contents).
>
> I'll find out if I can release the broken revisions in their entirety.
>
Perhaps someone would be willing to have a look at those two revision
files privately.
(In fact, I might be able to do this too. But I'm reluctant to make
a promise or commitment about this.)
> The corrupted revision doesn't actually contain anything particularly
> important (almost all the modified files in it have since been replaced
> by newer versions anyway). Can I fix the repository by dumping every
> revision except 192, and then reloading the good revisions into a new
> repo? Or will cause problems for the revisions after 192 since one of
> the revisions no longer exists?
>
That won't work if files after r192 are stored as deltas against the
fulltext of r192.
> Regards,
>
> David Hopkins
> Serck Controls
>
>
> ===== PRIVACY AND CONFIDENTIALITY NOTICE =====
> The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
> The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
> NOTE - You should carry out your own virus checks before opening any attachment.
>
RE: How to fix corrupt revision in repo?
Posted by David Hopkins <DH...@serck-controls.com.au>.
Thanks Daniel. Responses inline.
>> David Hopkins wrote on Thu, Sep 15, 2011 at 10:30:46 +0800: I'm not
sure
>> why the verification command for revision 192 would throw an error
>> description for revision 162. Revision 192 affected a completely
>> different part of the repository to revision 162, so there is no
obvious
>> relationship between them.
>
> Possibly due to rep-sharing? Does db/revs/0/192 contain the number
"162"
> in ASCII decimal delimited by whitespace? You can check that with the
> following command: grep -a "^text:" db/revs/0/192
Yes it does.
Here's the context in the rev file:
id: y-31.0-32.r192/673830
type: file
pred: y-31.0.r31/264323
count: 1
text: 162 670867 6111 52486 5117fb0964ca1a78dd97447d23452e73
609f4745460d6e14860daff0803ee7024c54898c 191-5r/_m
cpath: [redacted]
copyroot: 32 [redacted]
Looking at the other nearby entries, they have "text: 192 [...]"
instead of "text: 162 [...]". Is that likely to be the problem?
> Does 'svnadmin dump -r 162 >NUL' work ?
Yes it does.
> To answer your question: yes, most definitely a copy of the r192
(and/or
> r162) rev file would allow to pinpoint the problem, however you might
> not want to share those files on a public list as they may contain
> sensitive data (versioned file contents).
I'll find out if I can release the broken revisions in their entirety.
The corrupted revision doesn't actually contain anything particularly
important (almost all the modified files in it have since been replaced
by newer versions anyway). Can I fix the repository by dumping every
revision except 192, and then reloading the good revisions into a new
repo? Or will cause problems for the revisions after 192 since one of
the revisions no longer exists?
Regards,
David Hopkins
Serck Controls
===== PRIVACY AND CONFIDENTIALITY NOTICE =====
The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
NOTE - You should carry out your own virus checks before opening any attachment.
Re: How to fix corrupt revision in repo?
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
David Hopkins wrote on Thu, Sep 15, 2011 at 10:30:46 +0800:
> Greetings,
>
> I have an SVN repo that is failing svnadmin verify on revision 192.
>
> For some reason the verify output says:
>
> [...various successful revisions...]
> * Verified revision 191.
> svnadmin: Can't read file 'E:\Repositories\Client_Name\db\revs\0\16
> 2': End of file found
>
> I'm not sure why the verification command for revision 192 would throw
> an error description for revision 162. Revision 192 affected a
> completely different part of the repository to revision 162, so there is
>
> no obvious relationship between them.
>
Possibly due to rep-sharing? Does db/revs/0/192 contain the number
"162" in ASCII decimal delimited by whitespace? You can check that with
the following command: grep -a "^text:" db/revs/0/192
Does 'svnadmin dump -r 162 >NUL' work ?
To answer your question: yes, most definitely a copy of the r192 (and/or
r162) rev file would allow to pinpoint the problem, however you might
not want to share those files on a public list as they may contain
sensitive data (versioned file contents).
> All the revisions from 193 to 332 (HEAD) are ok. It might be a one-off.
>
> This looks like a FSFS-backed repository (I am very new to SVN and
> inherited the server from someone else!). The server is VisualSVN 2.1.4,
>
> which is based on SVN 1.6.13.
>
> The clients are mostly TortoiseSVN 1.6.16, which uses SVN 1.6.17.
>
> What steps should I take to fix the corrupted revision? Is there more
> information that I should provide? (eg a copy of the rev 192 file?) This
>
> problem is causing checkouts and updates to fail for files that were
> last modified in that revision.
>
> Regards,
>
> David Hopkins
> Serck Controls
>
>
> ===== PRIVACY AND CONFIDENTIALITY NOTICE =====
> The information contained in this message is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, you must not copy, distribute, take any action in reliance on it, or disclose any details of the message to any person, firm or corporation. If you have received this message in error, please notify the sender immediately by reply e-mail and delete all copies of this transmission together with any attachments.
> The views or opinions expressed in this e-mail or any attachment are not necessarily those of Serck Controls Pty Ltd.
> NOTE - You should carry out your own virus checks before opening any attachment.
>