You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by San Martino <sa...@gmail.com> on 2010/11/10 20:09:59 UTC
Help or suggestions on porting to subversion
Hello,
we are porting hundreds of projects from an old versioning system to
subversion. We would like to make use of the trunk, tag and branch
concepts.
For the convertion we used an automatic tool which preserved the
original layout under trunk/ . By looking at the layout, one problem
is that the files of these projects are not semantically grouped into
different directories (one directory for project), but are grouped
according to their extensions (.txt, .dll, etc..) or software in which
they are installed. For example, under trunk we have automatically
obtained:
Database/
Scripts/
Packages/
Application Server
libs/
servlets/
while /tag and branch/ are empty
There are hundreds of files under each directory. We want to preserve
this layout, since it's basically impossible to reorganize all the
files in projects/directories.
Furthermore, we cannot checkout/update these huge directories whenever
we want to change a single file, for example a single package.
While it will no longer be the same with the new projects (some of
which might depend on old stuff), what we would like to do with the
old stuff is to be able to checkout a small number of single files
semantically representing a project and tag the release or snapshot
under /tag when the changes are done.
How could we do in your opinion?
Re: Help or suggestions on porting to subversion
Posted by Les Mikesell <le...@gmail.com>.
On 11/10/2010 3:05 PM, San Martino wrote:
> Hello Ryan,
>
> Of course the cleanest way is to checkout the whole trunk/ and tag the
> trunk each time.
> A single checkout of the whole / directory is tens of GB over a
> network (non-LAN), multiply this for all the deveopers.
But this should only have to happen once per workspace needed - and you
could even copy the initial workspace if you need multiple copies at a
remote location. Or ship them a workspace on a USB drive. From then
on, only differences should be transferred.
> The solution does not scale well even if checkout with --depth is
> disabled. Furthermore tagging the whole trunk each time would be make
> the concept it is for pointless.
Tagging really just gets you a human-friendly name, at least in the
conventional use where the tag is a copy from a branch or trunk
revision. There's no difference in what you can do with a tag url than
the url plus revision that was the source of the copy. And there should
also not be much if any difference in efficiency if you 'svn switch'
your working copy there. It's just easier to document your processes
when you can make up the tag names you should use for certain steps.
--
Les Mikesell
lesmikesell@gmail.com
Re: Help or suggestions on porting to subversion
Posted by Les Mikesell <le...@gmail.com>.
On 11/11/2010 5:18 AM, San Martino wrote:
> 2010/11/11 Cooke, Mark<ma...@siemens.com>:
>>>>> Database/
>>>>> Scripts/
>>>>> Packages/
>>>>> Application Server
>>>>> libs/
>>>>> servlets/
>>>>>
>>>>> while /tag and branch/ are empty
>>>>>
>>>>> There are hundreds of files under each directory. We want to
>>>>> preserve this layout, since it's basically impossible to
>>>>> reorganize all the files in projects/directories.
>>>>> Furthermore, we cannot checkout/update these huge directories
>>>>> whenever we want to change a single file, for example a single
>>>>> package.
>
>> Did you go through an evaluation process when you selected subversion as your new tool? You must have decided that it has features you want to use? The best course of action, even if painful in the short term, is probably to use svn as it is designed rather than trying to make it fit your old working practices...
>>
>> What development tools are you using that you ended up with this repositor layout? I think a little understanding of the process that got you to where you are (and why they are now different?) might help us to help you.
>
> Thanks for your answer. In substance we have no choice, we must use
> Subversion for our new development since our new IDE is integrated
> with Subversion. We cannot change our new IDE for many reasons that
> have higher priority than Subversion. We are trying to do the best. Of
> course new projects will profit by Subversion features in a more
> evident way than before. The problem is the old stuff in the old (and
> now bad) layout controlled by PVCS previously. Since we are still in
> transition, we want to preserve as much information as possible and we
> want to control all the sources with a single tool. The new projects
> might still depend semantically on the old files which not organized
> on a "per-project" criteria. The cost of reorganizing the old layout
> is huge and not affordable.
Perhaps you can break this step into manageable chunks - like initially
importing to the old layout so old tools can find the old copies - and
make a branch copy so you can find that old layout and maintain some
backwards compatibility there for as long as you need it while the trunk
gets organized into better subdirectory groupings as you make changes.
But really 'hundreds of files' in a directory is not a big number these
days. If you aren't deleting and checking out new working copies all the
time I'm not sure you should expect a problem.
> The theoric solution we thought at the beginning to integrate the old
> stuff with the new projects is to consider the old layout and files as
> part of one big virtual project maintained under /trunk. Then we could
> starting by checking-out the root /trunk/ with "--depth empty" to a
> working local copy which is a basis when accessing the old stuff. This
> way, when needed, we just sparse-checkout single files belonging to
> the same real project (fortunately, this is automatically possible (as
> workaround) in tortoisesvn with a loop of "checkout dir --depth empty"
> until a final update of the single file). Once the changes are done we
> create a tag of the current working copy.
Theoretically you can copy a working directory to a tag. It's not
necessarily a good idea because you normally think of tags as a 'human
friendly' name for snapshot of a specific revision on the trunk or a
branch, and the stuff in your workspace may not all be committed and may
not match anything else.
> But here is the problem: the
> tag will still represent everything under /trunk and not only the
> group of files (the modified project). How are tags really "cheaps"
> (time and space on disk) when the whole /trunk is measured in Gbs of
> data? Is there really a point to tag the whole /trunk when we just
> intended to make a change to a real project?
You can basically think of anything that is copied from somewhere else
in the repository as just another name for the thing at the
location/revision that is the source of the copy. That works even if
you make the copy from your workspace - for the things that came from
the repository at least. And, if you have a checked workspace
containing the HEAD or some other revision of the whole trunk and 'svn
switch some_tag_name' only the differences should be transferred from
the repository, just the same as if you had done an 'update -r
rev_number' to the revision where the tag was made. Other than the
initial checkout time and the space on the client (2x the actual size),
having extra stuff in the wrong place may not bother you that much at all.
--
Les Mikesell
lesmikesell@gmail.com
Re: Help or suggestions on porting to subversion
Posted by Mike Dixon <mi...@denovosoftware.com>.
On 11/11/2010 3:18 AM, San Martino wrote:
> How are tags really "cheaps"
> (time and space on disk) when the whole /trunk is measured in Gbs of
> data? Is there really a point to tag the whole /trunk when we just
> intended to make a change to a real project?
>
> thanks
http://svnbook.red-bean.com/nightly/en/svn.branchmerge.using.html#svn.branchmerge.using.create
At the bottom of that section there's an orange box with the title
"Cheap Copies". I recommend reading it. (And actually, I'd recommend
reading the book in its entirety if you're going to be administering a
svn deployment.)
-Mike
Re: Help or suggestions on porting to subversion
Posted by San Martino <sa...@gmail.com>.
2010/11/12 Ryan Schmidt <su...@ryandesign.com>:
> On Nov 11, 2010, at 18:33, San Martino wrote:
>
>> $ svn mkdir file:///home/x/repo/tag/change001/
>> $ svn copy --parents file:///home/x/repo/trunk/test/java_imports
>> file:///home/x/repo/tag/change001/
>> $ svn ls file:///home/x/repo/tag/change001/
>> java_imports
>>
>> From the help "svn copy --parents" should create "some" missing
>> directories..doesn't it?
>> Do you have any idea why I don't see any directories in the repository
>> under change001?
>
>
> You do see a directory under change001: the java_imports directory you told it to copy there. What did you expect to see instead?
Sorry, my fault. I misunderstood the semantic behind --parents.
Re: Help or suggestions on porting to subversion
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Nov 11, 2010, at 18:33, San Martino wrote:
> $ svn mkdir file:///home/x/repo/tag/change001/
> $ svn copy --parents file:///home/x/repo/trunk/test/java_imports
> file:///home/x/repo/tag/change001/
> $ svn ls file:///home/x/repo/tag/change001/
> java_imports
>
> From the help "svn copy --parents" should create "some" missing
> directories..doesn't it?
> Do you have any idea why I don't see any directories in the repository
> under change001?
You do see a directory under change001: the java_imports directory you told it to copy there. What did you expect to see instead?
If you wanted it to be in a "test" directory then you should presumably instead type:
$ svn copy --parents file:///home/x/repo/trunk/test/java_imports file:///home/x/repo/tag/change001/test/
or maybe:
$ svn copy --parents file:///home/x/repo/trunk/test/java_imports file:///home/x/repo/tag/change001/test/java_imports
You probably don't need to (and in fact shouldn't) "svn mkdir" the change001 directory first.
Usually, when you want to tag something, you just copy the entire trunk, i.e.:
$ svn copy file:///home/x/repo/trunk file:///home/x/repo/tag/change001
Re: Help or suggestions on porting to subversion
Posted by San Martino <sa...@gmail.com>.
Hello Johan
2010/11/11 Johan Corveleyn <jc...@gmail.com>:
> I think you can also make a tag or branch of a subset of files
> completely on the server (without the need to create a sparse working
> copy first). This can be done with the --parents option of "svn copy",
> and multiple SRC arguments (which can all be url's):
>
> svn copy --parents
> https://svn.example.com/trunk/dirwithmanyfiles/file1
> https://svn.example.com/trunk/dirwithmanyfiles/file2
> https://svn.example.com/branches/mybranch/dirwithmanyfiles
$ svn --version
svn, versione 1.6.5 (r38866)
Aug 31 2009, 18:42:02
$ svn mkdir file:///home/x/repo/tag/change001/
$ svn copy --parents file:///home/x/repo/trunk/test/java_imports
file:///home/x/repo/tag/change001/
$ svn ls file:///home/x/repo/tag/change001/
java_imports
Re: Help or suggestions on porting to subversion
Posted by San Martino <sa...@gmail.com>.
Hello Johan
2010/11/11 Johan Corveleyn <jc...@gmail.com>:
> I think you can also make a tag or branch of a subset of files
> completely on the server (without the need to create a sparse working
> copy first). This can be done with the --parents option of "svn copy",
> and multiple SRC arguments (which can all be url's):
>
> svn copy --parents
> https://svn.example.com/trunk/dirwithmanyfiles/file1
> https://svn.example.com/trunk/dirwithmanyfiles/file2
> https://svn.example.com/branches/mybranch/dirwithmanyfiles
$ svn --version
svn, versione 1.6.5 (r38866)
Aug 31 2009, 18:42:02
$ svn mkdir file:///home/x/repo/tag/change001/
$ svn copy --parents file:///home/x/repo/trunk/test/java_imports
file:///home/x/repo/tag/change001/
$ svn ls file:///home/x/repo/tag/change001/
java_imports
>From the help "svn copy --parents" should create "some" missing
directories..doesn't it?
Do you have any idea why I don't see any directories in the repository
under change001?
thanks
Re: Help or suggestions on porting to subversion
Posted by Johan Corveleyn <jc...@gmail.com>.
On Thu, Nov 11, 2010 at 12:18 PM, San Martino <sa...@gmail.com> wrote:
> 2010/11/11 Cooke, Mark <ma...@siemens.com>:
>>> > > Database/
>>> > > Scripts/
>>> > > Packages/
>>> > > Application Server
>>> > > libs/
>>> > > servlets/
>>> > >
>>> > > while /tag and branch/ are empty
>>> > >
>>> > > There are hundreds of files under each directory. We want to
>>> > > preserve this layout, since it's basically impossible to
>>> > > reorganize all the files in projects/directories.
>>> > > Furthermore, we cannot checkout/update these huge directories
>>> > > whenever we want to change a single file, for example a single
>>> > > package.
>
>> Did you go through an evaluation process when you selected subversion as your new tool? You must have decided that it has features you want to use? The best course of action, even if painful in the short term, is probably to use svn as it is designed rather than trying to make it fit your old working practices...
>>
>> What development tools are you using that you ended up with this repositor layout? I think a little understanding of the process that got you to where you are (and why they are now different?) might help us to help you.
>
> Thanks for your answer. In substance we have no choice, we must use
> Subversion for our new development since our new IDE is integrated
> with Subversion. We cannot change our new IDE for many reasons that
> have higher priority than Subversion. We are trying to do the best. Of
> course new projects will profit by Subversion features in a more
> evident way than before. The problem is the old stuff in the old (and
> now bad) layout controlled by PVCS previously. Since we are still in
> transition, we want to preserve as much information as possible and we
> want to control all the sources with a single tool. The new projects
> might still depend semantically on the old files which not organized
> on a "per-project" criteria. The cost of reorganizing the old layout
> is huge and not affordable.
> The theoric solution we thought at the beginning to integrate the old
> stuff with the new projects is to consider the old layout and files as
> part of one big virtual project maintained under /trunk. Then we could
> starting by checking-out the root /trunk/ with "--depth empty" to a
> working local copy which is a basis when accessing the old stuff. This
> way, when needed, we just sparse-checkout single files belonging to
> the same real project (fortunately, this is automatically possible (as
> workaround) in tortoisesvn with a loop of "checkout dir --depth empty"
> until a final update of the single file). Once the changes are done we
> create a tag of the current working copy. But here is the problem: the
> tag will still represent everything under /trunk and not only the
> group of files (the modified project). How are tags really "cheaps"
> (time and space on disk) when the whole /trunk is measured in Gbs of
> data? Is there really a point to tag the whole /trunk when we just
> intended to make a change to a real project?
I think you can also make a tag or branch of a subset of files
completely on the server (without the need to create a sparse working
copy first). This can be done with the --parents option of "svn copy",
and multiple SRC arguments (which can all be url's):
svn copy --parents
https://svn.example.com/trunk/dirwithmanyfiles/file1
https://svn.example.com/trunk/dirwithmanyfiles/file2
https://svn.example.com/branches/mybranch/dirwithmanyfiles
(all on one line, i.e. one command). Or same with tags instead of branches.
I'm not sure how that works out with merging, but maybe you don't need
merge (tracking) anyway ... YMMV.
Cheers,
--
Johan
Re: Help or suggestions on porting to subversion
Posted by San Martino <sa...@gmail.com>.
2010/11/11 Cooke, Mark <ma...@siemens.com>:
>> > > Database/
>> > > Scripts/
>> > > Packages/
>> > > Application Server
>> > > libs/
>> > > servlets/
>> > >
>> > > while /tag and branch/ are empty
>> > >
>> > > There are hundreds of files under each directory. We want to
>> > > preserve this layout, since it's basically impossible to
>> > > reorganize all the files in projects/directories.
>> > > Furthermore, we cannot checkout/update these huge directories
>> > > whenever we want to change a single file, for example a single
>> > > package.
> Did you go through an evaluation process when you selected subversion as your new tool? You must have decided that it has features you want to use? The best course of action, even if painful in the short term, is probably to use svn as it is designed rather than trying to make it fit your old working practices...
>
> What development tools are you using that you ended up with this repositor layout? I think a little understanding of the process that got you to where you are (and why they are now different?) might help us to help you.
Thanks for your answer. In substance we have no choice, we must use
Subversion for our new development since our new IDE is integrated
with Subversion. We cannot change our new IDE for many reasons that
have higher priority than Subversion. We are trying to do the best. Of
course new projects will profit by Subversion features in a more
evident way than before. The problem is the old stuff in the old (and
now bad) layout controlled by PVCS previously. Since we are still in
transition, we want to preserve as much information as possible and we
want to control all the sources with a single tool. The new projects
might still depend semantically on the old files which not organized
on a "per-project" criteria. The cost of reorganizing the old layout
is huge and not affordable.
The theoric solution we thought at the beginning to integrate the old
stuff with the new projects is to consider the old layout and files as
part of one big virtual project maintained under /trunk. Then we could
starting by checking-out the root /trunk/ with "--depth empty" to a
working local copy which is a basis when accessing the old stuff. This
way, when needed, we just sparse-checkout single files belonging to
the same real project (fortunately, this is automatically possible (as
workaround) in tortoisesvn with a loop of "checkout dir --depth empty"
until a final update of the single file). Once the changes are done we
create a tag of the current working copy. But here is the problem: the
tag will still represent everything under /trunk and not only the
group of files (the modified project). How are tags really "cheaps"
(time and space on disk) when the whole /trunk is measured in Gbs of
data? Is there really a point to tag the whole /trunk when we just
intended to make a change to a real project?
thanks
RE: Help or suggestions on porting to subversion
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
>
Linedata Limited
Registered Office: 85 Gracechurch St., London, EC3V 0AA
Registered in England and Wales No 3475006 VAT Reg No 710 3140 03
-----Original Message-----
> From: Cooke, Mark [mailto:mark.cooke@siemens.com]
> Sent: 11 November 2010 08:50
> To: Giulio Troccoli; users@subversion.apache.org
> Cc: San Martino
> Subject: RE: Help or suggestions on porting to subversion
>
> > -----Original Message-----
> > From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
> > Sent: 11 November 2010 08:45
> > Subject: RE: Help or suggestions on porting to subversion
> >
> > [CUT]
> >
> > >
> > > > The thing we want is:
> > > > - checkout single/scattered files from different directories
> > > > - modify/commit them
> > > > - group the files and tag this group (only the files in
> > > > this group).
> > > >
> > > This is not AFAIK directly possible in subversion (or any other
> > > similar tool that I have used). You could try using file
> externals
> > > in a new project directory but I have not used them and believe
> > > there are "issues".
> >
> > According to the book it is possible (at least the last bit)
> >
> http://svnbook.red-bean.com/nightly/en/svn.branchmerge.tags.ht
> ml#svn.bra
> nchmerge.tags.mkcomplex
> >
> > Giulio
> >
> Ooo, thanks for that, learn a new thing every day.
Just learned that myself yesterday, while looking for something else :-)
> However, I still think that the first step will be at least
> as "expensive" as doing the work to reorganise into a more
> relevant project tree which would then simplify the remainder
> of the steps. Without knowing why the OP is where they are
> it is hard to understand why they have to stay there.
There is a Python script to automate sparse checkout
https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/svn-viewspec.py
Maybe that would help.
RE: Help or suggestions on porting to subversion
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
>
Linedata Limited
Registered Office: 85 Gracechurch St., London, EC3V 0AA
Registered in England and Wales No 3475006 VAT Reg No 710 3140 03
-----Original Message-----
> From: Cooke, Mark [mailto:mark.cooke@siemens.com]
> Sent: 11 November 2010 08:50
> To: Giulio Troccoli; users@subversion.apache.org
> Cc: San Martino
> Subject: RE: Help or suggestions on porting to subversion
>
> > -----Original Message-----
> > From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
> > Sent: 11 November 2010 08:45
> > Subject: RE: Help or suggestions on porting to subversion
> >
> > [CUT]
> >
> > >
> > > > The thing we want is:
> > > > - checkout single/scattered files from different directories
> > > > - modify/commit them
> > > > - group the files and tag this group (only the files in
> > > > this group).
> > > >
> > > This is not AFAIK directly possible in subversion (or any other
> > > similar tool that I have used). You could try using file
> externals
> > > in a new project directory but I have not used them and believe
> > > there are "issues".
> >
> > According to the book it is possible (at least the last bit)
> >
> http://svnbook.red-bean.com/nightly/en/svn.branchmerge.tags.ht
> ml#svn.bra
> nchmerge.tags.mkcomplex
> >
> > Giulio
> >
> Ooo, thanks for that, learn a new thing every day.
Just learned that myself yesterday, while looking for something else :-)
> However, I still think that the first step will be at least
> as "expensive" as doing the work to reorganise into a more
> relevant project tree which would then simplify the remainder
> of the steps. Without knowing why the OP is where they are
> it is hard to understand why they have to stay there.
There is a Python script to automate sparse checkout
https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/svn-viewspec.py
Maybe that would help.
RE: Help or suggestions on porting to subversion
Posted by "Cooke, Mark" <ma...@siemens.com>.
> -----Original Message-----
> From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
> Sent: 11 November 2010 08:45
> Subject: RE: Help or suggestions on porting to subversion
>
> [CUT]
>
> >
> > > The thing we want is:
> > > - checkout single/scattered files from different directories
> > > - modify/commit them
> > > - group the files and tag this group (only the files in
> > > this group).
> > >
> > This is not AFAIK directly possible in subversion (or any
> > other similar tool that I have used). You could try using
> > file externals in a new project directory but I have not used
> > them and believe there are "issues".
>
> According to the book it is possible (at least the last bit)
>
http://svnbook.red-bean.com/nightly/en/svn.branchmerge.tags.html#svn.bra
nchmerge.tags.mkcomplex
>
> Giulio
>
Ooo, thanks for that, learn a new thing every day.
However, I still think that the first step will be at least as
"expensive" as doing the work to reorganise into a more relevant project
tree which would then simplify the remainder of the steps. Without
knowing why the OP is where they are it is hard to understand why they
have to stay there.
~ mark c
RE: Help or suggestions on porting to subversion
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
[CUT]
>
> > The thing we want is:
> > - checkout single/scattered files from different directories
> > - modify/commit them
> > - group the files and tag this group (only the files in this group).
> >
> This is not AFAIK directly possible in subversion (or any
> other similar tool that I have used). You could try using
> file externals in a new project directory but I have not used
> them and believe there are "issues".
According to the book it is possible (at least the last bit)
http://svnbook.red-bean.com/nightly/en/svn.branchmerge.tags.html#svn.branchmerge.tags.mkcomplex
Giulio
Linedata Limited
Registered Office: 85 Gracechurch St., London, EC3V 0AA
Registered in England and Wales No 3475006 VAT Reg No 710 3140 03
RE: Help or suggestions on porting to subversion
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
[CUT]
>
> > The thing we want is:
> > - checkout single/scattered files from different directories
> > - modify/commit them
> > - group the files and tag this group (only the files in this group).
> >
> This is not AFAIK directly possible in subversion (or any
> other similar tool that I have used). You could try using
> file externals in a new project directory but I have not used
> them and believe there are "issues".
According to the book it is possible (at least the last bit)
http://svnbook.red-bean.com/nightly/en/svn.branchmerge.tags.html#svn.branchmerge.tags.mkcomplex
Giulio
Linedata Limited
Registered Office: 85 Gracechurch St., London, EC3V 0AA
Registered in England and Wales No 3475006 VAT Reg No 710 3140 03
RE: Help or suggestions on porting to subversion
Posted by "Cooke, Mark" <ma...@siemens.com>.
(please do not top-post on this list)
> 2010/11/10 Ryan Schmidt <su...@ryandesign.com>:
> >
> > On Nov 10, 2010, at 14:09, San Martino wrote:
> >
> > > we are porting hundreds of projects from an old versioning
> > > system to subversion. We would like to make use of the trunk,
> > > tag and branch concepts. For the convertion we used an
> > > automatic tool which preserved the original layout under
> > > trunk/ . By looking at the layout, one problem is that the
> > > files of these projects are not semantically grouped into
> > > different directories (one directory for project), but are
> > > grouped according to their extensions (.txt, .dll, etc..) or
> > > software in which they are installed. For example, under
> > > trunk we have automatically obtained:
> > >
> > > Database/
> > > Scripts/
> > > Packages/
> > > Application Server
> > > libs/
> > > servlets/
> > >
> > > while /tag and branch/ are empty
> > >
> > > There are hundreds of files under each directory. We want to
> > > preserve this layout, since it's basically impossible to
> > > reorganize all the files in projects/directories.
> > > Furthermore, we cannot checkout/update these huge directories
> > > whenever we want to change a single file, for example a single
> > > package.
> >
> > Why can't you?
> >
> > > While it will no longer be the same with the new projects (some
> > > of which might depend on old stuff), what we would like to do
> > > with the old stuff is to be able to checkout a small number of
> > > single files semantically representing a project and tag the
> > > release or snapshot under /tag when the changes are done.
> > >
> > > How could we do in your opinion?
> >
> > You could check out the large directory.
> >
> > Or you could check out a sparse directory of only what you
> > need to work on.
> >
> > Or you could reorganize the files by project so that you
> > can check out each project. If there are hundreds of projects
> > and this is daunting, you could just convert projects as you
> > need to work on them.
> >
> >
> -----Original Message-----
> From: San Martino [mailto:sanmrtn96@gmail.com]
> Sent: 10 November 2010 21:05
> Subject: Re: Help or suggestions on porting to subversion
>
> Hello Ryan,
>
> Of course the cleanest way is to checkout the whole trunk/ and tag the
> trunk each time.
> A single checkout of the whole / directory is tens of GB over a
> network (non-LAN), multiply this for all the deveopers.
> The solution does not scale well even if checkout with --depth is
> disabled. Furthermore tagging the whole trunk each time would be make
> the concept it is for pointless.
>
...however, the full checkout is only done once (per developer) and then only updates are transferred. This is probably not as big a problem as you may think. Also, if the developers are co-located you can probably copy a single fresh WC between developers within your own network (before anyone messes with the virgin WC).
> The thing we want is:
> - checkout single/scattered files from different directories
> - modify/commit them
> - group the files and tag this group (only the files in this group).
>
This is not AFAIK directly possible in subversion (or any other similar tool that I have used). You could try using file externals in a new project directory but I have not used them and believe there are "issues".
> The slow transition of reorganizing by moving the files in projects as
> we modify them is basically not possible and might be confusing. Also
> not every developer has an idea of all the dependencies involved when
> he/she just wants to modify a single file. It's even possible that
> base/common libraries will never be touched at all. Just Tto be short
> there are other reasons I will not mention ..
>
I do not know your project but when we converted from another SCCS product, we decided to import the complete tree as-was into one repo (so the history was available) but then made that read-only and started new, fresh repos for related projects with sensible layouts.
Did you go through an evaluation process when you selected subversion as your new tool? You must have decided that it has features you want to use? The best course of action, even if painful in the short term, is probably to use svn as it is designed rather than trying to make it fit your old working practices...
What development tools are you using that you ended up with this repositor layout? I think a little understanding of the process that got you to where you are (and why they are now different?) might help us to help you.
Good luck,
~ mark c
Re: Help or suggestions on porting to subversion
Posted by San Martino <sa...@gmail.com>.
Hello Ryan,
Of course the cleanest way is to checkout the whole trunk/ and tag the
trunk each time.
A single checkout of the whole / directory is tens of GB over a
network (non-LAN), multiply this for all the deveopers.
The solution does not scale well even if checkout with --depth is
disabled. Furthermore tagging the whole trunk each time would be make
the concept it is for pointless.
The thing we want is:
- checkout single/scattered files from different directories
- modify/commit them
- group the files and tag this group (only the files in this group).
The slow transition of reorganizing by moving the files in projects as
we modify them is basically not possible and might be confusing. Also
not every developer has an idea of all the dependencies involved when
he/she just wants to modify a single file. It's even possible that
base/common libraries will never be touched at all. Just Tto be short
there are other reasons I will not mention ..
2010/11/10 Ryan Schmidt <su...@ryandesign.com>:
>
> On Nov 10, 2010, at 14:09, San Martino wrote:
>
>> we are porting hundreds of projects from an old versioning system to
>> subversion. We would like to make use of the trunk, tag and branch
>> concepts.
>> For the convertion we used an automatic tool which preserved the
>> original layout under trunk/ . By looking at the layout, one problem
>> is that the files of these projects are not semantically grouped into
>> different directories (one directory for project), but are grouped
>> according to their extensions (.txt, .dll, etc..) or software in which
>> they are installed. For example, under trunk we have automatically
>> obtained:
>>
>> Database/
>> Scripts/
>> Packages/
>> Application Server
>> libs/
>> servlets/
>>
>> while /tag and branch/ are empty
>>
>> There are hundreds of files under each directory. We want to preserve
>> this layout, since it's basically impossible to reorganize all the
>> files in projects/directories.
>> Furthermore, we cannot checkout/update these huge directories whenever
>> we want to change a single file, for example a single package.
>
> Why can't you?
>
>> While it will no longer be the same with the new projects (some of
>> which might depend on old stuff), what we would like to do with the
>> old stuff is to be able to checkout a small number of single files
>> semantically representing a project and tag the release or snapshot
>> under /tag when the changes are done.
>>
>> How could we do in your opinion?
>
> You could check out the large directory.
>
> Or you could check out a sparse directory of only what you need to work on.
>
> Or you could reorganize the files by project so that you can check out each project. If there are hundreds of projects and this is daunting, you could just convert projects as you need to work on them.
>
>
Re: Help or suggestions on porting to subversion
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Nov 10, 2010, at 14:09, San Martino wrote:
> we are porting hundreds of projects from an old versioning system to
> subversion. We would like to make use of the trunk, tag and branch
> concepts.
> For the convertion we used an automatic tool which preserved the
> original layout under trunk/ . By looking at the layout, one problem
> is that the files of these projects are not semantically grouped into
> different directories (one directory for project), but are grouped
> according to their extensions (.txt, .dll, etc..) or software in which
> they are installed. For example, under trunk we have automatically
> obtained:
>
> Database/
> Scripts/
> Packages/
> Application Server
> libs/
> servlets/
>
> while /tag and branch/ are empty
>
> There are hundreds of files under each directory. We want to preserve
> this layout, since it's basically impossible to reorganize all the
> files in projects/directories.
> Furthermore, we cannot checkout/update these huge directories whenever
> we want to change a single file, for example a single package.
Why can't you?
> While it will no longer be the same with the new projects (some of
> which might depend on old stuff), what we would like to do with the
> old stuff is to be able to checkout a small number of single files
> semantically representing a project and tag the release or snapshot
> under /tag when the changes are done.
>
> How could we do in your opinion?
You could check out the large directory.
Or you could check out a sparse directory of only what you need to work on.
Or you could reorganize the files by project so that you can check out each project. If there are hundreds of projects and this is daunting, you could just convert projects as you need to work on them.
Re: Help or suggestions on porting to subversion
Posted by David Weintraub <qa...@gmail.com>.
It's a little late for you, but I've found that sometimes it is best
to start "clean" and give up your old history. Just keep your old
repository around if you need this information.
There are many times I've moved a group to Subversion, and we simply
copied over the end points of branches, and relevant tags. Wholesale
conversion simply doesn't work out much of the time.
The problem with wholesale conversion is that you've changed your
tools around and the way you do your work. For example, your old build
scripts probably no longer work because they were made to work with
your old system. Like you have, I've seen sites where old build tools
required strange repository layouts that no longer make sense under
Subversion.
The first time I realized this was a conversion from StarTeam to
ClearCase. There were no automated tools for the conversion, and
StarTeam was practically impossible to write a script for. Commands
were limited, and each command needed the entire login information in
order to work. In the end, we decided our StarTeam repository would
still be available, and we would simply convert over just a few
branches and tags that we needed.
This suddenly freed me to rearrange the repository. We got rid of the
cruft that had built up over 7 years of development, and arranged
things in a more logical layout. At first, the developers were unhappy
because they would have to go into StarTeam to look up history, but
then they realized they rarely did that anyway. After a while, the
developers were happier with this new arrangement.
You haven't mentioned your IDE tool, and why your repository is in
this strange layout, or why is it setup with so many files in one
directory structure and why files you need are scattered all over the
place. So, it's a bit difficult to understand. It could be that you
have some sort of proprietary software project that requires this
structure -- maybe some sort of document repository front end.
If your repository REALLY needs to be setup this way, Subversion isn't
the tool for you. Subversion is highly optimized for directory
checkout structure. Its revisioning system is based upon it. Its tools
assume this. You'll be better off with something that allows each and
every file to be independently versioned like CVS did. Is your only
choice with your IDE Subversion?
Right now, you have three choices:
1). Restructure your repository to make using Subversion efficient.
2). Switching to another version control system that better matches
what you want.
3). Keep what you have and try to use Subversion in a way that's not
very efficient, and will be hard for new employees to support.
The last option will take more work in the long run, and in the end,
you'll end up with a bunch of unhappy users.
--
David Weintraub
qazwart@gmail.com