You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by "Brian J. Creasy" <bc...@collaborativefusion.com> on 2006/10/04 20:00:28 UTC
Re: svnadmin: Malformed file
Johnathan Gifford wrote:
> I'm in the middle of planning an upgrade of our Subversion server from
> 1.3.0 to 1.4.0. While attempting to dump the current repository so I
> can load into a '1.4.0' repository I came across this.
>
> svnadmin: Malformed file
>
> Okay, got my attention, so did some checking with the book and tried
> svnadmin verify. Same error on the same revision number. Not much on
> this has been posted here on the mail list about this issue and I found
> only one link on Google with any hope. That one link did say to check
> revision file from the revprops directory in the db directory of the
> repository. The file was garbage and not clear text. The same holds
> true for the revs directory.
>
> I would roll back each revision, but this revision happened months ago,
> so that's about 5000 revisions, so this is not very practical and
> beside, all my incremental dumps with the exception of the last week are
> gone. The repository is fine and fully functional in production. But
> since svnadmin verify failed, this has me wondering how many more
> revisions are bad after the first one encountered. I do have the commit
> notification e-mail indicating what the user did.
>
> So what is an admin to do? Any suggestions on how to get my repository
> dumped using svnadmin dump? Any suggestions how to fix the garbage
> revprops and revs file?
>
> Thanks,
>
> Johnathan
I actually came upon this same exact problem with our main repository
this morning while doing the same thing you were.
I think I've found a way to get around our malformed file, but the
workaround takes advantage of the fact that the revision that is corrupt
is r200 and from about a year ago. HEAD is now at r6700. What we're
planning on doing is running:
$ svnadmin dump -r201:HEAD repositoryName > dump.svn
$ svnadmin create repositoryName2
$ svnadmin load repositoryName2 < dump.svn
$ mv repositoryName repositoryName_20061004
$ mv repositoryName2 repositoryName
Basically, we're just going to throw out those first 200 revisions
since, at this stage of development, they don't really matter much at
all. The only problem I can see with doing it this way is that all of
the revisions will now be skewed by 200 from what they used to be; r5000
will be r4800. I haven't tested doing the load into a new 1.4
repository. Only into one of the same version that it is already as we
have not upgraded yet.
Hope this helps if only a little.
Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: svnadmin: Malformed file
Posted by "Reedick, Andrew" <An...@BellSouth.com>.
> -----Original Message-----
> From: John Szakmeister [mailto:john@szakmeister.net]
> Sent: Wednesday, October 04, 2006 8:53 PM
> To: Reedick, Andrew
> Cc: Johnathan Gifford; users@subversion.tigris.org
> Subject: Re: svnadmin: Malformed file
>
>
> ----- Andrew Reedick <An...@BellSouth.com> wrote:
> > > -----Original Message-----
> > > From: Johnathan Gifford [mailto:jgifford@wernervas.com]
> > > Sent: Wednesday, October 04, 2006 4:43 PM
> > > To: users@subversion.tigris.org
> > > Subject: Re: svnadmin: Malformed file
> >
> >
> > > Unless you dump your repository or run svnadmin verify by
> > > chance, you'll never know you have a problem. So the moral of the
> > > story, schedule svnadmin verify to run at least weekly, if not
> > daily.
> >
> >
> > Does 'svnadmin verify' return a meaningful exit code or have a
> > particular error message that a script can parse for?
>
> Yes. 0 for success (everything is okay), non-zero for an error.
>
I have empirically confirmed your statement. *grumble*
./verify.sh
ERROR! verify failed for the_third_biggest_repository_we_have
#!/usr/bin/bash
find * -type d -prune |
while read i
do
svnadmin verify $i > verify.$i.log 2>&1
if [ $? -ne 0 ]
then
echo "ERROR! verify failed for $i"
fi
done
*****
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. GA624
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svnadmin: Malformed file
Posted by John Szakmeister <jo...@szakmeister.net>.
----- Andrew Reedick <An...@BellSouth.com> wrote:
> > -----Original Message-----
> > From: Johnathan Gifford [mailto:jgifford@wernervas.com]
> > Sent: Wednesday, October 04, 2006 4:43 PM
> > To: users@subversion.tigris.org
> > Subject: Re: svnadmin: Malformed file
>
>
> > Unless you dump your repository or run svnadmin verify by
> > chance, you'll never know you have a problem. So the moral of the
> > story, schedule svnadmin verify to run at least weekly, if not
> daily.
>
>
> Does 'svnadmin verify' return a meaningful exit code or have a
> particular error message that a script can parse for?
Yes. 0 for success (everything is okay), non-zero for an error.
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: svnadmin: Malformed file
Posted by "Reedick, Andrew" <An...@BellSouth.com>.
> -----Original Message-----
> From: Johnathan Gifford [mailto:jgifford@wernervas.com]
> Sent: Wednesday, October 04, 2006 4:43 PM
> To: users@subversion.tigris.org
> Subject: Re: svnadmin: Malformed file
> Unless you dump your repository or run svnadmin verify by
> chance, you'll never know you have a problem. So the moral of the
> story, schedule svnadmin verify to run at least weekly, if not daily.
Does 'svnadmin verify' return a meaningful exit code or have a
particular error message that a script can parse for?
*****
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. GA622
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svnadmin: Malformed file
Posted by John Szakmeister <jo...@szakmeister.net>.
----- Johnathan Gifford <jg...@wernervas.com> wrote:
[snip]
> Our 'Malformed file' error was due to the repository_name/db/rev/xxxx
> file and repository_name/db/revprops/xxxx file were completely garbled.
> It looked like binary when displayed unlike understandable text for
> other revisions. Double check your rev and revprops file for revision
> 200. If it is somewhat legible, look for the fsfsverify.py tool that
> John Szakmeister has written. It might fix your problem. If it
> doesn't, I'll let you (and everyone else) if CollabNet fixes our
> revisions our repository.
>
> I think what is interesting about this problem, the repository still
> functions. Unless you dump your repository or run svnadmin verify by
> chance, you'll never know you have a problem. So the moral of the
> story, schedule svnadmin verify to run at least weekly, if not daily.
Or run it on every commit (you can put it in the post-commit and just check the revision that was committed).
> But just as important make sure you do an incremental dump of each
> revision committed and keep them for a certain amount of time so you can
> rebuild the repository if the svnadmin verify fails.
This is where running 'svnadmin verify' on every commit would be beneficial. The problem is that the data is being corrupted as part of the commit process. So the commit succeeds without error, despite the fact that the data just became corrupted. Having it in the post-commit would allow you to throw an error back to the user and do something like send an email, so that an administrator can fix it ASAP.
Oh the other hand, Malcome just recently committed some code that implements some locking around the transactions, with the idea that another process won't be able to come and take over the transaction. This should prevent the corruption from occurring. I believe it's already been nominated for backport, so hopefully it won't be too much longer before that's released.
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svnadmin: Malformed file
Posted by Johnathan Gifford <jg...@wernervas.com>.
>>> On Wed, Oct 4, 2006 at 3:00 PM, in message
<45...@collaborativefusion.com>, "Brian J. Creasy"
<bc...@collaborativefusion.com> wrote:
> Johnathan Gifford wrote:
>> I'm in the middle of planning an upgrade of our Subversion server
from
>> 1.3.0 to 1.4.0. While attempting to dump the current repository so
I
>> can load into a '1.4.0' repository I came across this.
>>
>> svnadmin: Malformed file
>>
>> Okay, got my attention, so did some checking with the book and
tried
>> svnadmin verify. Same error on the same revision number. Not much
on
>> this has been posted here on the mail list about this issue and I
found
>> only one link on Google with any hope. That one link did say to
check
>> revision file from the revprops directory in the db directory of
the
>> repository. The file was garbage and not clear text. The same holds
>> true for the revs directory.
>>
>> I would roll back each revision, but this revision happened months
ago,
>> so that's about 5000 revisions, so this is not very practical and
>> beside, all my incremental dumps with the exception of the last week
are
>> gone. The repository is fine and fully functional in production.
But
>> since svnadmin verify failed, this has me wondering how many more
>> revisions are bad after the first one encountered. I do have the
commit
>> notification e- mail indicating what the user did.
>>
>> So what is an admin to do? Any suggestions on how to get my
repository
>> dumped using svnadmin dump? Any suggestions how to fix the garbage
>> revprops and revs file?
>>
>> Thanks,
>>
>> Johnathan
>
> I actually came upon this same exact problem with our main repository
> this morning while doing the same thing you were.
>
> I think I've found a way to get around our malformed file, but the
> workaround takes advantage of the fact that the revision that is
corrupt
> is r200 and from about a year ago. HEAD is now at r6700. What we're
> planning on doing is running:
>
> $ svnadmin dump - r201:HEAD repositoryName > dump.svn
> $ svnadmin create repositoryName2
> $ svnadmin load repositoryName2 < dump.svn
> $ mv repositoryName repositoryName_20061004
> $ mv repositoryName2 repositoryName
>
> Basically, we're just going to throw out those first 200 revisions
> since, at this stage of development, they don't really matter much at
> all. The only problem I can see with doing it this way is that all
of
> the revisions will now be skewed by 200 from what they used to be;
r5000
> will be r4800. I haven't tested doing the load into a new 1.4
> repository. Only into one of the same version that it is already as
we
> have not upgraded yet.
>
> Hope this helps if only a little.
>
> Brian
Well, after more digging into our repository, we have one revision with
the 'Malformed file' error and six more within twenty reivsions having a
'Found malformed header in revision file' error. So a copy of our
repository is off to CollabNet for 'hopefull' repair along with copies
of all e-mail notification of those commits describing what transpired
during those revisions. I think my employer is about to find out if the
support contract with CollabNet is worth the money or not.
I was in communication with John Szakmeister about this issue offline.
He and another developer have been trying to track these issues down.
He did suggest that if the offending revision is not referred to in a
later revision and you don't know what happened on that revision, there
is a way to create an 'empty' commit so revision numbers stay in sync.
With that you could dump revision 0:199 and 201:HEAD incrementally.
Import 0:199 into the new repository. Either recreate what transpired
on 200 or load the empty commit, then proceed with loading 201:HEAD.
I'm not sure you're going to be able to dump from revision 201, as it
may need to look into revision 200 for content and still cause an error.
I ran into this earlier, I think. It's been a couple of months since I
tried that, but I didn't know, comprehend, or understand what the
problem was at that time.
Our 'Malformed file' error was due to the repository_name/db/rev/xxxx
file and repository_name/db/revprops/xxxx file were completely garbled.
It looked like binary when displayed unlike understandable text for
other revisions. Double check your rev and revprops file for revision
200. If it is somewhat legible, look for the fsfsverify.py tool that
John Szakmeister has written. It might fix your problem. If it
doesn't, I'll let you (and everyone else) if CollabNet fixes our
revisions our repository.
I think what is interesting about this problem, the repository still
functions. Unless you dump your repository or run svnadmin verify by
chance, you'll never know you have a problem. So the moral of the
story, schedule svnadmin verify to run at least weekly, if not daily.
But just as important make sure you do an incremental dump of each
revision committed and keep them for a certain amount of time so you can
rebuild the repository if the svnadmin verify fails.
Johnathan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org