changing DIR structure of SVN dump

[Shaaa <sh...@gmail.com>]

Hello everyone

So I am trying to move my repos to a new server. Problem is the old
guy did a really strange setup. Basically, in order to access the
repos I would have to use the following uri
file:///var/svn/repos/mysite/trunk/project1/trunk On the new server I
want to change it to file:///var/svn/repos/project1/trunk but I dont
want to show the changes in the revs. I have tried the following:

svnadmin dump /var/svn/repos > mydump
svnadminfilter include trunk < mydump > newdump
svnadmin mkdir file:///var/svn/repos/myproject1
svnadmin load /var/svn/repos --parent-dir myproject1 < newdump

but it gives me the following URI:
file:///var/svn/repos/myproject1/trunk/myproject1

Re: changing DIR structure of SVN dump

[Les Mikesell <le...@gmail.com>]

On Wed, Jan 11, 2012 at 10:00 AM, Ryan Schmidt
<su...@ryandesign.com> wrote:
>
>> The repos is /var/svn/repos but it has a hb folder, which holds all
>> the obsolete files and that dir holds another dir with the new
>> project. Which two of the below would be ideal method of managing
>> several projects:
>>
>> method 1:
>> /var/svn/repos/project1
>> /var/svn/repos/project2
>> /var/svn/repos/project3
>>
>> method 2:
>> /var/svn/project1
>> /var/svn/project2
>> /var/svn/project3
>>
>> The former are directories/projects within the main repository and in
>> the case of the latter, each project is its own repository
>
> Both organizational methods have advantages and disadvantages. There's lots of discussion on the topic in the mailing list archives over the years.

A couple of practical points:

You probably want to configure your service (whether svnserve or http)
to access a directory where you have the option to create more
repositories transparently whether or not you use them initially.

Eventually you may need to do maintenance like removing a DVD iso
image someone accidentally committed or something that for legal
reasons should not be there.   The only way to actually remove
anything from a repository is to use svnadmin dump on the whole thing.
 If you have combined many large projects in one repository, this can
become very cumbersome due to the time and space requirements.

-- 
  Les Mikesell
    lesmikesell@gmail.com

Re: changing DIR structure of SVN dump

[Ryan Schmidt <su...@ryandesign.com>]

On Jan 11, 2012, at 09:07, Shaaa wrote:

> The repos is /var/svn/repos but it has a hb folder, which holds all
> the obsolete files and that dir holds another dir with the new
> project. Which two of the below would be ideal method of managing
> several projects:
> 
> method 1:
> /var/svn/repos/project1
> /var/svn/repos/project2
> /var/svn/repos/project3
> 
> method 2:
> /var/svn/project1
> /var/svn/project2
> /var/svn/project3
> 
> The former are directories/projects within the main repository and in
> the case of the latter, each project is its own repository

Both organizational methods have advantages and disadvantages. There's lots of discussion on the topic in the mailing list archives over the years.

Re: changing DIR structure of SVN dump

[Shaaa <sh...@gmail.com>]

> I think where you're saying "repos" above you mean "project" or "directory". It still seems tht your repository is just /var/svn/repos.

The repos is /var/svn/repos but it has a hb folder, which holds all
the obsolete files and that dir holds another dir with the new
project. Which two of the below would be ideal method of managing
several projects:

method 1:
/var/svn/repos/project1
/var/svn/repos/project2
/var/svn/repos/project3

method 2:
/var/svn/project1
/var/svn/project2
/var/svn/project3

The former are directories/projects within the main repository and in
the case of the latter, each project is its own repository

On 11/01/2012, Ryan Schmidt <su...@ryandesign.com> wrote:
>
> On Jan 11, 2012, at 07:11, Shaaa wrote:
>
>> Thank you for your informative reply. I have managed to overcome the
>> initial problem, however I do have a new problem.
>>
>> Basically, this is the current path to my repos:
>> svn list file:///var/svn/repos/hb/trunk/chi
>>
>> I want to change it to conform to the 'standardised' structure. At the
>> moment it is a complete mess. /var/svn/repos/hb has a tonne of
>> obselete files, then you have a trunk within a trunk which is
>> basically a different project and should be in its own directory
>> instead of being a sub directory of an obselete project.
>>
>> So to clean up the structure I have done the following:
>>
>> Ive created a new directory:
>> svn mkdir file:///var/svn/repos/chi
>>
>> and moved the other repos to the new directory:
>> svn move file:///var/svn/repos/hb/trunk/chi
>> file:///var/svn/repos/chi/trunk
>
> I think where you're saying "repos" above you mean "project" or "directory".
> It still seems that your repository is just /var/svn/repos.
>
>
>> now I take a dump of the repos which will be taken to the new server
>> svnadmin dump /var/svn/repos/ > svn.dump
>>
>> Now I try to filter and extract *only* the chi project
>> (file:///var/svn/repos/chi).
>> svndumpfilter include chi < dump > dump.filtered
>>
>> Doing this however, gives me the following error:
>> svndumpfilter: Invalid copy source path '/trunk/chi'
>
>
> You will need to "include" every path "chi" was ever known as, not just the
> path it's at in the HEAD revision.
>
>
>

Re: changing DIR structure of SVN dump

[Ryan Schmidt <su...@ryandesign.com>]

On Jan 11, 2012, at 07:11, Shaaa wrote:

> Thank you for your informative reply. I have managed to overcome the
> initial problem, however I do have a new problem.
> 
> Basically, this is the current path to my repos:
> svn list file:///var/svn/repos/hb/trunk/chi
> 
> I want to change it to conform to the 'standardised' structure. At the
> moment it is a complete mess. /var/svn/repos/hb has a tonne of
> obselete files, then you have a trunk within a trunk which is
> basically a different project and should be in its own directory
> instead of being a sub directory of an obselete project.
> 
> So to clean up the structure I have done the following:
> 
> Ive created a new directory:
> svn mkdir file:///var/svn/repos/chi
> 
> and moved the other repos to the new directory:
> svn move file:///var/svn/repos/hb/trunk/chi file:///var/svn/repos/chi/trunk

I think where you're saying "repos" above you mean "project" or "directory". It still seems that your repository is just /var/svn/repos.


> now I take a dump of the repos which will be taken to the new server
> svnadmin dump /var/svn/repos/ > svn.dump
> 
> Now I try to filter and extract *only* the chi project
> (file:///var/svn/repos/chi).
> svndumpfilter include chi < dump > dump.filtered
> 
> Doing this however, gives me the following error:
> svndumpfilter: Invalid copy source path '/trunk/chi'


You will need to "include" every path "chi" was ever known as, not just the path it's at in the HEAD revision.

Re: changing DIR structure of SVN dump

[Shaaa <sh...@gmail.com>]

Hello Ryan

Thank you for your informative reply. I have managed to overcome the
initial problem, however I do have a new problem.

Basically, this is the current path to my repos:
svn list file:///var/svn/repos/hb/trunk/chi

I want to change it to conform to the 'standardised' structure. At the
moment it is a complete mess. /var/svn/repos/hb has a tonne of
obselete files, then you have a trunk within a trunk which is
basically a different project and should be in its own directory
instead of being a sub directory of an obselete project.

So to clean up the structure I have done the following:

Ive created a new directory:
svn mkdir file:///var/svn/repos/chi

and moved the other repos to the new directory:
svn move file:///var/svn/repos/hb/trunk/chi file:///var/svn/repos/chi/trunk

now I take a dump of the repos which will be taken to the new server
svnadmin dump /var/svn/repos/ > svn.dump

Now I try to filter and extract *only* the chi project
(file:///var/svn/repos/chi).
svndumpfilter include chi < dump > dump.filtered

Doing this however, gives me the following error:
svndumpfilter: Invalid copy source path '/trunk/chi'


Help on resolving this issue will be greatly appreciated.


Shaaa


On 10/01/2012, Ryan Schmidt <su...@ryandesign.com> wrote:
> On Jan 10, 2012, at 08:39, Shaaa wrote:
>
>> So I am trying to move my repos to a new server. Problem is the old
>> guy did a really strange setup. Basically, in order to access the
>> repos I would have to use the following uri
>> file:///var/svn/repos/mysite/trunk/project1/trunk On the new server I
>> want to change it to file:///var/svn/repos/project1/trunk but I dont
>> want to show the changes in the revs. I have tried the following:
>>
>> svnadmin dump /var/svn/repos > mydump
>> svnadminfilter include trunk < mydump > newdump
>> svnadmin mkdir file:///var/svn/repos/myproject1
>> svnadmin load /var/svn/repos --parent-dir myproject1 < newdump
>>
>> but it gives me the following URI:
>> file:///var/svn/repos/myproject1/trunk/myproject1
>
> You need to distinguish between:
>
> 1. the path of the repository on the old server's disk
> 2. the path of the repository on the new server's disk
> 3. the url through which the repository is accessed on the old server
> 4. the url through which the repository is accessed on the new server
> 5. the path of things inside the repository
>
> Hopefully you are not actually accessing repositories via the file:///
> protocol over for example a file share, and are instead running an apache
> server and accessing it over the http:// or https:// protocol, or an
> svnserve server and accessing it over the svn:// protocol, or have ssh
> access and are accessing it over the svn+ssh:// protocol.
>
> When making any change to your setup, you should change as few things at a
> time as possible. For example, change the paths within the repository (by
> doing "svn mv", or a dump/filter/load cycle), without changing where the
> repository is on disk or the URL used to access it. Or, move the repository
> to a new server, without changing its contents. Though I understand that the
> new server might have a different disk layout than the old server.
>
> So, let's take a look at your situation:
>
>> svnadmin dump /var/svn/repos > mydump
>
> This says the repository was on disk at /var/svn/repos.
>
>> svnadminfilter include trunk < mydump > newdump
>
> There is no command "svnadminfilter"; perhaps you meant "svndumpfilter"?
>
> Based on the URLs you showed above, the repository doesn't contain a
> directory "trunk"; it contains a directory "mysite" which contains a
> directory "trunk". So the newdump shouldn't contain anything.
>
>> svnadmin mkdir file:///var/svn/repos/myproject1
>
> There is no such command "svnadmin mkdir"; did you mean "svnadmin create"?
> That would be odd though since you showed earlier that /var/svn/repos is a
> repository; you don't create repositories inside other repositories. So
> perhaps you meant "svn mkdir" instead?
>
>> svnadmin load /var/svn/repos --parent-dir myproject1 < newdump
>
> You are loading a new dumpfile into the same old /var/svn/repos repository?
>
> There appears to be significant confusion. Some of what I wrote above may be
> wrong, since it is based on what you said, and we already know two lines of
> your transcript cannot be correct. Perhaps we should start with:
>
> 1. Do you have one repository or multiple repositories? Do you want to
> change that?
> 2. What is the path of the repository or repositories on disk? Do you want
> to change that?
> 3. What is the path of the items inside the repository? I presume you at
> least want to change that, to remove the doubled "trunk" directories.
>
> Finally, note that if you really want to dump/filter/load and thus not
> change the revision numbers, what that means is that you are rewriting the
> repository's history (i.e. changing the contents of (at least some of) those
> revisions). As a consequence, you must ensure that you assign the new
> repository a new UUID, and thus everybody who has a working copy will have
> to throw it away and check out a new one. Of course while you are doing your
> work to rewrite the repository history (while you are running dump / filter
> / create / load), you'll have to disable write access to the repository.
> It's often more convenient to just "svn mv" things and accept a slightly
> "dirtier" history than to go through these steps; some would even argue the
> history is not dirty in this case; instead, it's accurate, in that it shows
> what was moved, when, and by whom (and if you write a good commit message,
> why).
>
>

Re: changing DIR structure of SVN dump

[Ryan Schmidt <su...@ryandesign.com>]

On Jan 10, 2012, at 08:39, Shaaa wrote:

> So I am trying to move my repos to a new server. Problem is the old
> guy did a really strange setup. Basically, in order to access the
> repos I would have to use the following uri
> file:///var/svn/repos/mysite/trunk/project1/trunk On the new server I
> want to change it to file:///var/svn/repos/project1/trunk but I dont
> want to show the changes in the revs. I have tried the following:
> 
> svnadmin dump /var/svn/repos > mydump
> svnadminfilter include trunk < mydump > newdump
> svnadmin mkdir file:///var/svn/repos/myproject1
> svnadmin load /var/svn/repos --parent-dir myproject1 < newdump
> 
> but it gives me the following URI:
> file:///var/svn/repos/myproject1/trunk/myproject1

You need to distinguish between:

1. the path of the repository on the old server's disk
2. the path of the repository on the new server's disk
3. the url through which the repository is accessed on the old server
4. the url through which the repository is accessed on the new server
5. the path of things inside the repository

Hopefully you are not actually accessing repositories via the file:/// protocol over for example a file share, and are instead running an apache server and accessing it over the http:// or https:// protocol, or an svnserve server and accessing it over the svn:// protocol, or have ssh access and are accessing it over the svn+ssh:// protocol.

When making any change to your setup, you should change as few things at a time as possible. For example, change the paths within the repository (by doing "svn mv", or a dump/filter/load cycle), without changing where the repository is on disk or the URL used to access it. Or, move the repository to a new server, without changing its contents. Though I understand that the new server might have a different disk layout than the old server.

So, let's take a look at your situation:

> svnadmin dump /var/svn/repos > mydump

This says the repository was on disk at /var/svn/repos.

> svnadminfilter include trunk < mydump > newdump

There is no command "svnadminfilter"; perhaps you meant "svndumpfilter"?

Based on the URLs you showed above, the repository doesn't contain a directory "trunk"; it contains a directory "mysite" which contains a directory "trunk". So the newdump shouldn't contain anything.

> svnadmin mkdir file:///var/svn/repos/myproject1

There is no such command "svnadmin mkdir"; did you mean "svnadmin create"? That would be odd though since you showed earlier that /var/svn/repos is a repository; you don't create repositories inside other repositories. So perhaps you meant "svn mkdir" instead?

> svnadmin load /var/svn/repos --parent-dir myproject1 < newdump

You are loading a new dumpfile into the same old /var/svn/repos repository?

There appears to be significant confusion. Some of what I wrote above may be wrong, since it is based on what you said, and we already know two lines of your transcript cannot be correct. Perhaps we should start with:

1. Do you have one repository or multiple repositories? Do you want to change that?
2. What is the path of the repository or repositories on disk? Do you want to change that?
3. What is the path of the items inside the repository? I presume you at least want to change that, to remove the doubled "trunk" directories.

Finally, note that if you really want to dump/filter/load and thus not change the revision numbers, what that means is that you are rewriting the repository's history (i.e. changing the contents of (at least some of) those revisions). As a consequence, you must ensure that you assign the new repository a new UUID, and thus everybody who has a working copy will have to throw it away and check out a new one. Of course while you are doing your work to rewrite the repository history (while you are running dump / filter / create / load), you'll have to disable write access to the repository. It's often more convenient to just "svn mv" things and accept a slightly "dirtier" history than to go through these steps; some would even argue the history is not dirty in this case; instead, it's accurate, in that it shows what was moved, when, and by whom (and if you write a good commit message, why).