You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Nitin Bhide <ni...@gmail.com> on 2013/09/01 08:32:18 UTC
corrupted svn repository with “serialized hash missing terminator” error
I have a local subversion repository. I am trying to convert it to
Mercurial. In the process I discovered that repository is corrupted. I have
tried the following
1. svnadmin verify - confirmed that repository is corrupted at revision
1265. The verify stops at this revision.
2. svnadmin recover - was unable to recover the repository.
3. tried to dump the repository content. svnadmin dump fails at revision
1265. So dumped till 1264. Took another dump from revision 1322. (all
revision from 1265 to 1321 are giving the serialized hash terminator error).
4. I tried to combine the dumps and load it into a new repository. However,
some files were added in the 'missing' revisions (1265 to 1321) and while
loading dump I am getting errors.
I googled for error and advice seems to be dump the repository and reload
the contents in new repository. However, that does not work for me.
I tried to find what exactly is the meaning of the error. However, I could
not get any information. I tried to read the documentation of FSFS
structure (
http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure).
I tried to manually check revision file for 1265 and see what is wrong. But
could not find anything visibly wrong.
Is there any way to correct this error in the repository
(manually/programmatically) ?
If someone can give me a hint on what is wrong in the file and underwhat
conditions this error occurs, I will try to write a script to correct it.
Regards,
Nitin
Re: corrupted svn repository with “serialized hash missing terminator” error
Posted by Andreas Mohr <an...@lisas.de>.
Hi,
On Thu, Sep 05, 2013 at 07:54:51PM +0530, Nitin Bhide wrote:
> Hi Stefan,
>
> Thanks for the hint. I tried the fsfsverify and fsfixer also. Both did not
> work. Looking back I think the problem started when I upgraded to from
> 1.7.x to 1.8.1.
>
> What exactly is 'serialized hash terminator' error ?
"Use The Source, Luke!"?
HTH (perhaps not),
Andreas Mohr
Re: corrupted svn repository with “serialized hash missing terminator” error
Posted by Nitin Bhide <ni...@gmail.com>.
Hi Andreas,
Thanks.
I think such recovery tools should be there. Probably not as part of
'svnadmin' but at least in contrib. The FSFS verify helped me. I will post
the script I used to my website. I am not sure if the attachments are
allowed in this list. If allowed, I will post here as well.
I am a reasonably good python developer. However I don't know much about
svn internals. So if someone is interested in writing such tool, I am
willing to contribute.
Regards,
Nitin
On Sat, Sep 7, 2013 at 6:49 PM, Andreas Mohr <an...@lisas.de> wrote:
> Hi,
>
> On Sat, Sep 07, 2013 at 05:50:29PM +0530, Nitin Bhide wrote:
> > Philip/Stefan/Andreas,
> >
> > Thanks for the help. I was able to write a small python script
> > specific to
> > my needs and recover the repository. It was corrupted revision
> > property
> > file. I have written the details in blog post.
> > [1]
> http://nitinbhide.blogspot.in/2013/09/recovering-from-corrupted-subversion.html
>
> Thanks for openly documenting/sharing your experience!
> (and of course congrats for your smashing success)
>
> Now a question that would spring from this would be whether this is
> something that could be more automated in Subversion project circles
> (for this one user who was nicely able to draw the necessary conclusions
> and then even write his own tool, there's probably a dozen more users
> who gave up in despair).
> That's probably not a feature that svnadmin ought to provide ("form
> intermediate revisions out of thin air when encountering corruption in
> certain revisions"), but rather an external recovery tool that's made
> for that purpose.
> But those are just random thoughts of someone who's quite external to all
> this... ;)
>
> Andreas Mohr
>
> --
> GNU/Linux. It's not the software that's free, it's you.
>
>
--
============================================
In theory, there is NO difference between theory and practice.
But, in practice, there IS
Yogi Berra
============================================
My company's new product : BootstrapToday - Simple and Sensible Application
Lifecycle Management
http://www.bootstraptoday.com
Re: corrupted svn repository with “serialized hash missing terminator” error
Posted by Andreas Mohr <an...@lisas.de>.
Hi,
On Sat, Sep 07, 2013 at 05:50:29PM +0530, Nitin Bhide wrote:
> Philip/Stefan/Andreas,
>
> Thanks for the help. I was able to write a small python script
> specific to
> my needs and recover the repository. It was corrupted revision
> property
> file. I have written the details in blog post.
> [1]http://nitinbhide.blogspot.in/2013/09/recovering-from-corrupted-subversion.html
Thanks for openly documenting/sharing your experience!
(and of course congrats for your smashing success)
Now a question that would spring from this would be whether this is
something that could be more automated in Subversion project circles
(for this one user who was nicely able to draw the necessary conclusions
and then even write his own tool, there's probably a dozen more users
who gave up in despair).
That's probably not a feature that svnadmin ought to provide ("form
intermediate revisions out of thin air when encountering corruption in
certain revisions"), but rather an external recovery tool that's made
for that purpose.
But those are just random thoughts of someone who's quite external to all
this... ;)
Andreas Mohr
--
GNU/Linux. It's not the software that's free, it's you.
Re: corrupted svn repository with “serialized hash missing terminator” error
Posted by Nitin Bhide <ni...@gmail.com>.
Philip/Stefan/Andreas,
Thanks for the help. I was able to write a small python script specific to
my needs and recover the repository. It was corrupted revision property
file. I have written the details in blog post.
http://nitinbhide.blogspot.in/2013/09/recovering-from-corrupted-subversion.html
Thanks again (especially Philip),
Regards,
Nitin
On Fri, Sep 6, 2013 at 2:36 PM, Philip Martin <ph...@wandisco.com>wrote:
> Nitin Bhide <ni...@gmail.com> writes:
>
> > Thanks for the clarification. Does the message mean either in db/revs or
> in
> > db/revprops 'END' tag is missing ?
>
> It means the file ended before the END tag was seen, but it's not
> possible to say exactly what is missing.
>
> --
> Philip Martin | Subversion Committer
> WANdisco // *Non-Stop Data*
>
--
============================================
In theory, there is NO difference between theory and practice.
But, in practice, there IS
Yogi Berra
============================================
My company's new product : BootstrapToday - Simple and Sensible Application
Lifecycle Management
http://www.bootstraptoday.com
Re: corrupted svn repository with “serializedhash missing terminator” error
Posted by Philip Martin <ph...@wandisco.com>.
Nitin Bhide <ni...@gmail.com> writes:
> Thanks for the clarification. Does the message mean either in db/revs or in
> db/revprops 'END' tag is missing ?
It means the file ended before the END tag was seen, but it's not
possible to say exactly what is missing.
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Re: corrupted svn repository with “serialized hash missing terminator” error
Posted by Nitin Bhide <ni...@gmail.com>.
Hi Philip,
Thanks for the clarification. Does the message mean either in db/revs or in
db/revprops 'END' tag is missing ?
Regards,
Nitin
On Thu, Sep 5, 2013 at 11:50 PM, Philip Martin
<ph...@wandisco.com>wrote:
> Nitin Bhide <ni...@gmail.com> writes:
>
> > Hi Stefan,
> >
> > Thanks for the hint. I tried the fsfsverify and fsfixer also. Both did
> not
> > work. Looking back I think the problem started when I upgraded to from
> > 1.7.x to 1.8.1.
> >
> > What exactly is 'serialized hash terminator' error ?
>
> It means one of the repository files is corrupt. It could be a revision
> files in db/revs or it could be a revprop file in db/revprops. A
> serialized hash is a series of K/V pairs followed by END:
>
> K 10
> svn:author
> V 2
> pm
> K 8
> svn:date
> V 27
> 2013-09-05T18:00:22.881511Z
> K 7
> svn:log
> V 1
> m
> END
>
> --
> Philip Martin | Subversion Committer
> WANdisco // *Non-Stop Data*
>
--
============================================
In theory, there is NO difference between theory and practice.
But, in practice, there IS
Yogi Berra
============================================
My company's new product : BootstrapToday - Simple and Sensible Application
Lifecycle Management
http://www.bootstraptoday.com
Re: corrupted svn repository with “serializedhash missing terminator” error
Posted by Philip Martin <ph...@wandisco.com>.
Nitin Bhide <ni...@gmail.com> writes:
> Hi Stefan,
>
> Thanks for the hint. I tried the fsfsverify and fsfixer also. Both did not
> work. Looking back I think the problem started when I upgraded to from
> 1.7.x to 1.8.1.
>
> What exactly is 'serialized hash terminator' error ?
It means one of the repository files is corrupt. It could be a revision
files in db/revs or it could be a revprop file in db/revprops. A
serialized hash is a series of K/V pairs followed by END:
K 10
svn:author
V 2
pm
K 8
svn:date
V 27
2013-09-05T18:00:22.881511Z
K 7
svn:log
V 1
m
END
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Re: corrupted svn repository with “serialized hash missing terminator” error
Posted by Nitin Bhide <ni...@gmail.com>.
Hi Stefan,
Thanks for the hint. I tried the fsfsverify and fsfixer also. Both did not
work. Looking back I think the problem started when I upgraded to from
1.7.x to 1.8.1.
What exactly is 'serialized hash terminator' error ?
Regards,
Nitin
On Sun, Sep 1, 2013 at 4:33 PM, Stefan Sperling <st...@elego.de> wrote:
> On Sun, Sep 01, 2013 at 12:02:18PM +0530, Nitin Bhide wrote:
> > I have a local subversion repository. I am trying to convert it to
> > Mercurial. In the process I discovered that repository is corrupted. I
> have
> > tried the following
> >
> > 1. svnadmin verify - confirmed that repository is corrupted at revision
> > 1265. The verify stops at this revision.
> > 2. svnadmin recover - was unable to recover the repository.
> > 3. tried to dump the repository content. svnadmin dump fails at revision
> > 1265. So dumped till 1264. Took another dump from revision 1322. (all
> > revision from 1265 to 1321 are giving the serialized hash terminator
> error).
> > 4. I tried to combine the dumps and load it into a new repository.
> However,
> > some files were added in the 'missing' revisions (1265 to 1321) and while
> > loading dump I am getting errors.
> >
> > I googled for error and advice seems to be dump the repository and reload
> > the contents in new repository. However, that does not work for me.
> >
> > I tried to find what exactly is the meaning of the error. However, I
> could
> > not get any information. I tried to read the documentation of FSFS
> > structure (
> >
> http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure
> ).
> > I tried to manually check revision file for 1265 and see what is wrong.
> But
> > could not find anything visibly wrong.
> >
> > Is there any way to correct this error in the repository
> > (manually/programmatically) ?
> >
> > If someone can give me a hint on what is wrong in the file and underwhat
> > conditions this error occurs, I will try to write a script to correct it.
> >
> > Regards,
> > Nitin
>
> Give this script a try:
> https://svn.apache.org/repos/asf/subversion/trunk/contrib/server-side/fsfsverify.py
> It can fix some instances of corruption which was probably caused by
> older Subversion/APR versions.
>
--
============================================
In theory, there is NO difference between theory and practice.
But, in practice, there IS
Yogi Berra
============================================
My company's new product : BootstrapToday - Simple and Sensible Application
Lifecycle Management
http://www.bootstraptoday.com
Re: corrupted svn repository with “serialized hash missing terminator” error
Posted by Stefan Sperling <st...@elego.de>.
On Sun, Sep 01, 2013 at 12:02:18PM +0530, Nitin Bhide wrote:
> I have a local subversion repository. I am trying to convert it to
> Mercurial. In the process I discovered that repository is corrupted. I have
> tried the following
>
> 1. svnadmin verify - confirmed that repository is corrupted at revision
> 1265. The verify stops at this revision.
> 2. svnadmin recover - was unable to recover the repository.
> 3. tried to dump the repository content. svnadmin dump fails at revision
> 1265. So dumped till 1264. Took another dump from revision 1322. (all
> revision from 1265 to 1321 are giving the serialized hash terminator error).
> 4. I tried to combine the dumps and load it into a new repository. However,
> some files were added in the 'missing' revisions (1265 to 1321) and while
> loading dump I am getting errors.
>
> I googled for error and advice seems to be dump the repository and reload
> the contents in new repository. However, that does not work for me.
>
> I tried to find what exactly is the meaning of the error. However, I could
> not get any information. I tried to read the documentation of FSFS
> structure (
> http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure).
> I tried to manually check revision file for 1265 and see what is wrong. But
> could not find anything visibly wrong.
>
> Is there any way to correct this error in the repository
> (manually/programmatically) ?
>
> If someone can give me a hint on what is wrong in the file and underwhat
> conditions this error occurs, I will try to write a script to correct it.
>
> Regards,
> Nitin
Give this script a try: https://svn.apache.org/repos/asf/subversion/trunk/contrib/server-side/fsfsverify.py
It can fix some instances of corruption which was probably caused by
older Subversion/APR versions.