missing db/revs, db/revprops and db/current files?

["Ritenour, Russell C. (Contr)" <Ru...@ngc.com>]

I'm normally a user of SVN and inherited a working system.  I made the horrible mistake of assuming that backups were being done correctly.  
 
Our backup software failed AND we lost our SVN server.  I was able to use KNOPPIX to pull the svn repository sans some files.  We are missing db/current and the db/revs directory is missing some numbers (e.g. has versions 1-17, missing 18, has versions 19-32, missing 33-47, etc...).
 
 
I ran svnadmin recover on the repository (after creating a db/current containing "3556 1 1")
 
I then tried to run "svnadmin verify" and continually bump into the missing versions.
 
What is the best way to recover the repository? Or am I truly lost?

Re: missing db/revs, db/revprops and db/current files?

[Daniel Shahaf <d....@daniel.shahaf.name>]

Ryan Schmidt wrote on Sat, 20 Sep 2008 at 15:01 -0500:
> On Sep 20, 2008, at 3:59 AM, Daniel Shahaf wrote:
> > Ryan, I think you are confusing two different meanings of the term
> > 'revision'.
> 
> Wow. I think you're right, and I'm not. I don't like it when I'm not right...
> :-)
> 
> 
> > In skip-deltas, the term 'revision' means:
...
> > In the Subversion UI and API, the term 'revision' means[1]:
...
> > 
> > I'm not an FS expert, but as far as I know these two meanings are
> > entirely unrelated.
> 
> Yeah. Well this is entirely confusing.
> 
> It's been hammered into me so many times that a revision is of the repository
> as a whole. Files don't have revisions; the repository does.
> 

Every version control system allows one to revise a file (meaning: to
edit it), thus files have revisions (meaning: editions, versions).  Only
normally we don't call this concept "revisions".

That we call something "revision" doesn't imply that repositories have
it and files don't.  More generally, what we call something has zero
effect on its semantics.

> Could the skip-deltas document perhaps be persuaded not to use the term
> "revision" when it isn't used the way the term "revision" is used everywhere
> else in the Subversion universe? :-)
> 

"Delta" is the difference between two "editions" (to use a neutral term)
of a file.  That entire document discusses the history of one file.  So
I don't see a problem.

(Also, I'm not about to go in and change a document about a core FS
algorithm that was written around 1.0.)

> 
> 
> I guess what this means is that you can't know at all what impact a missing
> rev file will have on the repository based on its number.
> 

Agreed.

Daniel

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: missing db/revs, db/revprops and db/current files?

[Ryan Schmidt <su...@ryandesign.com>]

On Sep 20, 2008, at 3:59 AM, Daniel Shahaf wrote:

> Ryan, I think you are confusing two different meanings of the term
> 'revision'.

Wow. I think you're right, and I'm not. I don't like it when I'm not  
right... :-)


> In skip-deltas, the term 'revision' means:
>
>     For the purposes of this document, we will pretend that  
> revisions of a
>     file are numbered starting from 0.  In reality, this number
>     corresponds to the "change count" field of a node-revision in each
>     filesystem back end.

I guess I never read that bit, or never got out of it what it was  
trying to tell me.


> In the Subversion UI and API, the term 'revision' means[1]:
>
>     The number of commits to a repository.
>
>
> I'm not an FS expert, but as far as I know these two meanings are
> entirely unrelated.
>
> Daniel
>
> [1] Not entirely accurate, but close enough for the purpose of this
> discussion.

Yeah. Well this is entirely confusing.

It's been hammered into me so many times that a revision is of the  
repository as a whole. Files don't have revisions; the repository does.

Could the skip-deltas document perhaps be persuaded not to use the  
term "revision" when it isn't used the way the term "revision" is  
used everywhere else in the Subversion universe? :-)



I guess what this means is that you can't know at all what impact a  
missing rev file will have on the repository based on its number.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: missing db/revs, db/revprops and db/current files?

[Daniel Shahaf <d....@daniel.shahaf.name>]

Ryan, I think you are confusing two different meanings of the term
'revision'.


In skip-deltas, the term 'revision' means:

    For the purposes of this document, we will pretend that revisions of a
    file are numbered starting from 0.  In reality, this number
    corresponds to the "change count" field of a node-revision in each
    filesystem back end.

In the Subversion UI and API, the term 'revision' means[1]:

    The number of commits to a repository.


I'm not an FS expert, but as far as I know these two meanings are
entirely unrelated.

Daniel

[1] Not entirely accurate, but close enough for the purpose of this 
discussion.


Ryan Schmidt wrote on Fri, 19 Sep 2008 at 13:33 -0500:
> 
> On Sep 18, 2008, at 12:58 PM, Ritenour, Russell C. (Contr) wrote:
> 
> The files in db/revs are the revision data. These are stored as a difference
> against a previous revision (though not necessarily the immediately previous
> revision; see the skip-deltas algorithm [1]). According to skip-deltas,
> revision 19 is based on 18, so a missing revision 18 can cause problems for
> revision 19. Revision 48 is based on revision 32, so missing revisions 33-47
> won't impact that. But if other higher revisions are missing, there may be
> extant revisions which will be broken because of this. You'll have to try
> and see.
> 
> The files in db/revprops are the properties for the corresponding revision.
> These aren't stored as diffs but as complete files. If you're missing the
> revprop file corresponding to a rev file that you do still have, you may
> need to reconstruct the revprop file before you can recover the remaining
> revisions. It shouldn't be too hard to create a valid revprop file, since
> it's a fairly readable text format. You'll just have to make up the date,
> author and log message. You could use an author and log message like
> "recovered" for example, and either try to pick a date that makes
> chronological sense given the remaining revisions or just set it to some
> random date in the past, or even today's date. Using non-chronological
> revisions will mean that you cannot use the '-r{date}' or
> '-r{fromdate}:{todate}' revision syntax to find revisions, so consider
> whether that's important to you.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: missing db/revs, db/revprops and db/current files?

[Ryan Schmidt <su...@ryandesign.com>]

On Sep 18, 2008, at 12:58 PM, Ritenour, Russell C. (Contr) wrote:

> I'm normally a user of SVN and inherited a working system.  I made  
> the horrible mistake of assuming that backups were being done  
> correctly.
>
> Our backup software failed AND we lost our SVN server.  I was able  
> to use KNOPPIX to pull the svn repository sans some files.  We are  
> missing db/current and the db/revs directory is missing some  
> numbers (e.g. has versions 1-17, missing 18, has versions 19-32,  
> missing 33-47, etc...).
>
>
> I ran svnadmin recover on the repository (after creating a db/ 
> current containing "3556 1 1")
>
> I then tried to run "svnadmin verify" and continually bump into the  
> missing versions.
>
> What is the best way to recover the repository? Or am I truly lost?

Well, it's not good.

The files in db/revs are the revision data. These are stored as a  
difference against a previous revision (though not necessarily the  
immediately previous revision; see the skip-deltas algorithm [1]).  
According to skip-deltas, revision 19 is based on 18, so a missing  
revision 18 can cause problems for revision 19. Revision 48 is based  
on revision 32, so missing revisions 33-47 won't impact that. But if  
other higher revisions are missing, there may be extant revisions  
which will be broken because of this. You'll have to try and see.

The files in db/revprops are the properties for the corresponding  
revision. These aren't stored as diffs but as complete files. If  
you're missing the revprop file corresponding to a rev file that you  
do still have, you may need to reconstruct the revprop file before  
you can recover the remaining revisions. It shouldn't be too hard to  
create a valid revprop file, since it's a fairly readable text  
format. You'll just have to make up the date, author and log message.  
You could use an author and log message like "recovered" for example,  
and either try to pick a date that makes chronological sense given  
the remaining revisions or just set it to some random date in the  
past, or even today's date. Using non-chronological revisions will  
mean that you cannot use the '-r{date}' or '-r{fromdate}:{todate}'  
revision syntax to find revisions, so consider whether that's  
important to you.

To recover what's left of the repository, you would have to dump the  
revisions that remain, in chunks. For example, since you're missing  
revisions 18 and 33-47:

svnadmin dump -r1:17 /path/to/brokenrepo > dump1-17
svnadmin dump -r19:32 --incremental /path/to/brokenrepo > dump19-32
svnadmin dump -r48:xx --incremental /path/to/brokenrepo > dump48-xx

If you're lucky and this works, then you can create a new repo and  
try loading these dumps into it:

svnadmin create /path/to/newrepo
svnadmin load /path/to/newrepo --ignore-uuid < dump1-17
svnadmin load /path/to/newrepo < dump19-32
svnadmin load /path/to/newrepo < dump48-xx

If you're lucky and that works too, then you can move over any hook  
scripts or configuration files from the broken repo to the new one  
and start using the new repo. Anybody with existing working copies  
will have to move them aside and check out new ones, since we've  
assigned the new repo a new UUID since it is not the same as your old  
repo. You should keep everybody's old working copies around, in case  
they had uncommitted changes (they can check out new working copies  
and manually move over their changes and commit them). Also it's  
possible that somebody's working copy contains some data that was  
lost from your original repo; if so, they can manually identify those  
changes and commit them again.


[1] http://svn.collab.net/repos/svn/trunk/notes/skip-deltas




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org