You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@subversion.apache.org by Mike Cepek <mi...@yahoo.com> on 2011/08/11 16:27:42 UTC

Parallel branches/tags/trunk directories

(I did some searching for an answer here and elsewhere, and didn't find it; sorry if I missed it.)
 
The project I'm dealing with has the SVN repository arranged like so:
 
proj/top1/branches/
proj/top1/tags/
proj/top1/trunk/
proj/top2/common/branches/
proj/top2/common/tags/
proj/top2/common/trunk/
proj/top2/thingA/branches/
proj/top2/thingA/tags/
proj/top2/thingA/trunk/
proj/top2/thingB/branches/
proj/top2/thingB/tags/
proj/top2/thingB/trunk/
proj/top2/deploy/common/branches/
proj/top2/deploy/common/tags/
proj/top2/deploy/common/trunk/
proj/top2/deploy/thingA/env1/branches/
proj/top2/deploy/thingA/env1/tags/
proj/top2/deploy/thingA/env1/trunk/
proj/top2/deploy/thingA/env2/branches/
proj/top2/deploy/thingA/env2/tags/
proj/top2/deploy/thingA/env2/trunk/
proj/top2/deploy/thingB/env1/branches/
proj/top2/deploy/thingB/env1/tags/
proj/top2/deploy/thingB/env1/trunk/
(etc)
 
First, I completely agree that this organization has issues.  Changing is isn't an option.  I'll spare you the details.  :-/
 
My question is:  how to checkout and update at the proj/ level without getting the content of all the tags/ and branches/ directories?
 
The "Sparse Directories" features of SVN (--depth and --set-depth) sounded good:
http://svnbook.red-bean.com/en/1.5/svn.advanced.sparsedirs.html
 
But those features are not sufficient.  In order to do a fresh checkout of the above and avoid the massive on-disk redundancy that would occur due to the content of tags/ and branches/, each trunk/ directory must be checked out individually.  Other than creating a brittle script that lists each trunk/ directory, I don't see a solution.  Even then, each branches/ and tags/ directory needs to be marked with --set-depth as well.
 
I also discovered that "svn co URL/proj/top1 --depth immediates" can take a very long time for a tree with tons of content.  Seems like a bug (using SlikSvn 1.6.15).  Similarly, something like "svn update URL/proj/top1/tags -depth immediates" would be handy at times to get just the empty subdirectories, named after each branch or tag.  Updating just one of those subdirectories would allow mass-diffs between the trunk or another branch/tag.  I don't see why a depth-limited operation like that should take so much time.
 
I though about using 'svn list http://.../proj/' and grep'ing the result for "/trunk", but that one command takes over 10 minutes - not feasible.  A checkout of all the trunk/ directory contents only takes 2.5 minutes.
 
So: has anyone found a clean way to handle checkouts and updates when branches/tags/trunk directories are strewn all over a tree?  Thanks.

Re: Parallel branches/tags/trunk directories

Posted by Mark Phippard <ma...@gmail.com>.

On Thu, Mar 29, 2012 at 5:33 AM, coolie <pc...@senscient.com> wrote:
>
> I have exactly the same issue - I want to inhibit checkout of all tags (and
> branches) folders, but still be able to do an update from the root. I've
> read the replies on this post and none seem to solve the problem. Did you
> find a simple solution? Ideally this should be a property set on the root
> folder a bit like svn:ignore so that it is registered at the server. Looks
> like I'll just have to bite the bullet and write a co script that  everyone
> has to run to do the initial checkout (don't have Python or Perl). I don't
> know where the depth information is held for each folder - maybe there is
> some way of  modifying this file?
>
> As I understand it you would do the following:
> - checkout root (immediate files and folders only, which includes the
> script)
> - run the script, which would do the remaining checkout (but only the
> immediate children of tags or branches folders)
> - updating the root hereafter will not update sub-folders of tags or
> branches folders
>
> Also need a script to modify the tree for existing checkouts. This would
> delete the tags sub-folder contents from the local disk and set the co depth

Take a look at the svn-viewspec.py script for an example of such a script:

http://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: Parallel branches/tags/trunk directories

Posted by coolie <pc...@senscient.com>.

I have exactly the same issue - I want to inhibit checkout of all tags (and 
branches) folders, but still be able to do an update from the root. I've 
read the replies on this post and none seem to solve the problem. Did you 
find a simple solution? Ideally this should be a property set on the root 
folder a bit like svn:ignore so that it is registered at the server. Looks 
like I'll just have to bite the bullet and write a co script that  everyone 
has to run to do the initial checkout (don't have Python or Perl). I don't 
know where the depth information is held for each folder - maybe there is 
some way of  modifying this file?

As I understand it you would do the following:
- checkout root (immediate files and folders only, which includes the 
script)
- run the script, which would do the remaining checkout (but only the 
immediate children of tags or branches folders)
- updating the root hereafter will not update sub-folders of tags or 
branches folders

Also need a script to modify the tree for existing checkouts. This would 
delete the tags sub-folder contents from the local disk and set the co depth


On Thursday, 11 August 2011 15:27:42 UTC+1, Mike Cepek wrote:
>
> (I did some searching for an answer here and elsewhere, and didn't find 
> it; sorry if I missed it.)
>  
> The project I'm dealing with has the SVN repository arranged like so:
>  
> proj/top1/branches/
> proj/top1/tags/
> proj/top1/trunk/
> proj/top2/common/branches/
> proj/top2/common/tags/
> proj/top2/common/trunk/
> proj/top2/thingA/branches/
> proj/top2/thingA/tags/
> proj/top2/thingA/trunk/
> proj/top2/thingB/branches/
> proj/top2/thingB/tags/
> proj/top2/thingB/trunk/
> proj/top2/deploy/common/branches/
> proj/top2/deploy/common/tags/
> proj/top2/deploy/common/trunk/
> proj/top2/deploy/thingA/env1/branches/
> proj/top2/deploy/thingA/env1/tags/
> proj/top2/deploy/thingA/env1/trunk/
> proj/top2/deploy/thingA/env2/branches/
> proj/top2/deploy/thingA/env2/tags/
> proj/top2/deploy/thingA/env2/trunk/
> proj/top2/deploy/thingB/env1/branches/
> proj/top2/deploy/thingB/env1/tags/
> proj/top2/deploy/thingB/env1/trunk/
> (etc)
>  
> First, I completely agree that this organization has issues.  Changing is 
> isn't an option.  I'll spare you the details.  :-/
>  
> My question is:  how to checkout and update at the proj/ level without 
> getting the content of all the tags/ and branches/ directories?
>  
> The "Sparse Directories" features of SVN (--depth and --set-depth) sounded 
> good:
> http://svnbook.red-bean.com/en/1.5/svn.advanced.sparsedirs.html
>  
> But those features are not sufficient.  In order to do a fresh checkout of 
> the above and avoid the massive on-disk redundancy that would occur due to 
> the content of tags/ and branches/, each trunk/ directory must be checked 
> out individually.  Other than creating a brittle script that lists each 
> trunk/ directory, I don't see a solution.  Even then, each branches/ and 
> tags/ directory needs to be marked with --set-depth as well.
>  
> I also discovered that "svn co URL/proj/top1 --depth immediates" can take 
> a very long time for a tree with tons of content.  Seems like a bug (using 
> SlikSvn 1.6.15).  Similarly, something like "svn update URL/proj/top1/tags 
> -depth immediates" would be handy at times to get just the empty 
> subdirectories, named after each branch or tag.  Updating just one of those 
> subdirectories would allow mass-diffs between the trunk or another 
> branch/tag.  I don't see why a depth-limited operation like that should 
> take so much time.
>  
> I though about using 'svn list http://.../proj/' and grep'ing the result 
> for "/trunk", but that one command takes over 10 minutes - not feasible.  A 
> checkout of all the trunk/ directory contents only takes 2.5 minutes.
>  
> So: has anyone found a clean way to handle checkouts and updates when 
> branches/tags/trunk directories are strewn all over a tree?  Thanks.
>

On Thursday, 11 August 2011 15:27:42 UTC+1, Mike Cepek wrote:
>
> (I did some searching for an answer here and elsewhere, and didn't find 
> it; sorry if I missed it.)
>  
> The project I'm dealing with has the SVN repository arranged like so:
>  
> proj/top1/branches/
> proj/top1/tags/
> proj/top1/trunk/
> proj/top2/common/branches/
> proj/top2/common/tags/
> proj/top2/common/trunk/
> proj/top2/thingA/branches/
> proj/top2/thingA/tags/
> proj/top2/thingA/trunk/
> proj/top2/thingB/branches/
> proj/top2/thingB/tags/
> proj/top2/thingB/trunk/
> proj/top2/deploy/common/branches/
> proj/top2/deploy/common/tags/
> proj/top2/deploy/common/trunk/
> proj/top2/deploy/thingA/env1/branches/
> proj/top2/deploy/thingA/env1/tags/
> proj/top2/deploy/thingA/env1/trunk/
> proj/top2/deploy/thingA/env2/branches/
> proj/top2/deploy/thingA/env2/tags/
> proj/top2/deploy/thingA/env2/trunk/
> proj/top2/deploy/thingB/env1/branches/
> proj/top2/deploy/thingB/env1/tags/
> proj/top2/deploy/thingB/env1/trunk/
> (etc)
>  
> First, I completely agree that this organization has issues.  Changing is 
> isn't an option.  I'll spare you the details.  :-/
>  
> My question is:  how to checkout and update at the proj/ level without 
> getting the content of all the tags/ and branches/ directories?
>  
> The "Sparse Directories" features of SVN (--depth and --set-depth) sounded 
> good:
> http://svnbook.red-bean.com/en/1.5/svn.advanced.sparsedirs.html
>  
> But those features are not sufficient.  In order to do a fresh checkout of 
> the above and avoid the massive on-disk redundancy that would occur due to 
> the content of tags/ and branches/, each trunk/ directory must be checked 
> out individually.  Other than creating a brittle script that lists each 
> trunk/ directory, I don't see a solution.  Even then, each branches/ and 
> tags/ directory needs to be marked with --set-depth as well.
>  
> I also discovered that "svn co URL/proj/top1 --depth immediates" can take 
> a very long time for a tree with tons of content.  Seems like a bug (using 
> SlikSvn 1.6.15).  Similarly, something like "svn update URL/proj/top1/tags 
> -depth immediates" would be handy at times to get just the empty 
> subdirectories, named after each branch or tag.  Updating just one of those 
> subdirectories would allow mass-diffs between the trunk or another 
> branch/tag.  I don't see why a depth-limited operation like that should 
> take so much time.
>  
> I though about using 'svn list http://.../proj/' and grep'ing the result 
> for "/trunk", but that one command takes over 10 minutes - not feasible.  A 
> checkout of all the trunk/ directory contents only takes 2.5 minutes.
>  
> So: has anyone found a clean way to handle checkouts and updates when 
> branches/tags/trunk directories are strewn all over a tree?  Thanks.
>

RE: Parallel branches/tags/trunk directories

Posted by Bob Archer <Bo...@amsi.com>.

> On Aug 11, 2011, at 12:21, Andy Levy wrote:
> 
> > I'm trying to find the documentation (release notes, CHANGES file) to
> > corroborate Bob's description of how Sparse Directories work with
> > pre-1.6 servers, but I can't. I have the same recollection he does -
> > that you're still downloading everything and just discarding the items
> > that don't match the depth you've set. I see the same behavior with my
> > 1.5.2 server and 1.6 clients.
> 
> http://subversion.apache.org/docs/release-notes/1.5.html#sc-compat
> 

Thanks for that. I was close. It is actually pre 1.5 servers that will still send all the data. And the OP is running 1.4... perhaps if he upgrades to 1.6 and the perf is improved he won't struggle as much with sparse directories.

BOb

Re: Parallel branches/tags/trunk directories

Posted by Ryan Schmidt <su...@ryandesign.com>.

On Aug 11, 2011, at 12:21, Andy Levy wrote:

> I'm trying to find the documentation (release notes, CHANGES file) to
> corroborate Bob's description of how Sparse Directories work with
> pre-1.6 servers, but I can't. I have the same recollection he does -
> that you're still downloading everything and just discarding the items
> that don't match the depth you've set. I see the same behavior with my
> 1.5.2 server and 1.6 clients.

http://subversion.apache.org/docs/release-notes/1.5.html#sc-compat

Re: Parallel branches/tags/trunk directories

Posted by Andy Levy <an...@gmail.com>.

On Thu, Aug 11, 2011 at 13:07, Mike Cepek <mi...@yahoo.com> wrote:

>> What version of the server are you running? I think if you use Sparse directories with a pre 1.6 server
>> the server sends all the files and the client just throws away what doesn't fit into your requested depth.
>
> Ah.  Our server reports itself as 1.4.2.  That probably explains why it took so long.  I have no idea what the odds are of having our server upgraded (used by many other projects).

You're on a very old, unsupported release. That should be a very
strong influence in getting the system upgraded to a newer version (or
stand up a new server & migrate projects as schedules allow). The
farther you fall behind, the more painful your upgrade path will
become (this is true for almost all software).

I'm trying to find the documentation (release notes, CHANGES file) to
corroborate Bob's description of how Sparse Directories work with
pre-1.6 servers, but I can't. I have the same recollection he does -
that you're still downloading everything and just discarding the items
that don't match the depth you've set. I see the same behavior with my
1.5.2 server and 1.6 clients.

Re: Parallel branches/tags/trunk directories

Posted by Les Mikesell <le...@gmail.com>.

On 8/11/2011 12:07 PM, Mike Cepek wrote:
>
>> What version of the server are you running? I think if you use Sparse directories with a pre 1.6 server
>> the server sends all the files and the client just throws away what doesn't fit into your requested depth.
>
> Ah.  Our server reports itself as 1.4.2.  That probably explains why it took so long.  I have no idea what the odds are of having our server upgraded (used by many other projects).

The likely reason for having one that old is that it was what was 
included in a Red Hat or CentOS distribution and you haven't done a 
system update recently (or it is older than the 5.x version).  If it is 
a 5.x, there are plenty of good reasons to update it, including bringing 
subversion up to a usable 1.6.x.

>> That all said, I don't really see the benefit of having one working copy that you can update at once.
>
> As opposed to having 14 working copies that every developer has to manually maintain?  Really?  The layout in my OP is only part of this project.  We really do have 14 separate branches/tags/trunks trees.  This is a new project, so that number will probably go up over time.

Why can't you build your own unversioned top level tree and check out 
the parts you want in the places they should be underneath it?  A script 
to do that shouldn't be any harder than loading the right pieces under a 
sparse tree and you can jump around in the branches with svn switch.  Or 
build a versioned tree that uses externals to pull what you want. All 
you lose is the ability to do a single top-level commit when you have 
changes in multiple components.

-- 
   Les Mikesell
    lesmikesell@gmail.com

RE: Parallel branches/tags/trunk directories

Posted by Bob Archer <Bo...@amsi.com>.

> > Not "sufficient" because it takes time and manual work to do? Or, because
> it doesn't do what you need?
> 
> Taking 97 minutes to pull 25 GB from all the tags/ and branches/ is
> unacceptable when it takes only 3 minutes to pull the 660 MB we actually
> need from just the trunk/ directories.  Getting everything and then manually
> pruning isn't a viable option.  Getting just the trunk/ directories without a
> brittle script is the whole point of my post.
> 
> > Sparse directories is exactly what you need to use to get what you want.
> > 1.7 added a -parents (I think it is) argument to the update/checkout
> > [...]
> I'll check out that option.  Unfortunately...
> 
> > What version of the server are you running? I think if you use Sparse
> > directories with a pre 1.6 server the server sends all the files and the client
> just throws away what doesn't fit into your requested depth.
> 
> Ah.  Our server reports itself as 1.4.2.  That probably explains why it took so
> long.  I have no idea what the odds are of having our server upgraded (used
> by many other projects).
> 
> > That all said, I don't really see the benefit of having one working copy that
> you can update at once.
> 
> As opposed to having 14 working copies that every developer has
> to manually maintain?  Really?  The layout in my OP is only part of this
> project.  We really do have 14 separate branches/tags/trunks trees.  This is a
> new project, so that number will probably go up over time.

Yes really. I have perhaps 20 or more working copies checked out. Many of those are various versions (branches) of one project. I update the ones I am working on as needed. I have no need to update WC's that I'm not working on every time I update the stuff I am working on. Once again, I don't see the benefit. 

BOb


> 
> > I prefer to check out the branch and/or trunks that I need to work on
> > as needed and update each one as needed. If you want to update all
> > your working copies with one command you can create a local batch file
> which goes through and calls update on each folder.
> 
> With --max-depth it should be possible to avoid needing a script for
> updates.  I've been working on a script for the initial checkout. It's non-trivial
> and brittle.  I will postpone the gory details unless there is interest.

Re: Parallel branches/tags/trunk directories

Posted by km...@rockwellcollins.com.

Mike Cepek <mi...@yahoo.com> wrote on 08/11/2011 12:07:44 PM:
> > Not "sufficient" because it takes time and manual work to do? Or, 
> because it doesn't do what you need?
> 
> Taking 97 minutes to pull 25 GB from all the tags/ and branches/ is 
> unacceptable when it takes only 3 minutes to pull the 660 MB we 
> actually need from just the trunk/ directories.  Getting everything 
> and then manually pruning isn't a viable option.  Getting just the 
> trunk/ directories without a brittle script is the whole point of my 
post.
> 
> > Sparse directories is exactly what you need to use to get what you 
want.
> > 1.7 added a -parents (I think it is) argument to the update/checkout 
[...]
> I'll check out that option.  Unfortunately...
> 
> > What version of the server are you running? I think if you use 
> Sparse directories with a pre 1.6 server
> > the server sends all the files and the client just throws away 
> what doesn't fit into your requested depth.
> 
> Ah.  Our server reports itself as 1.4.2.  That probably explains why
> it took so long.  I have no idea what the odds are of having our 
> server upgraded (used by many other projects).
> 
> > That all said, I don't really see the benefit of having one 
> working copy that you can update at once.
> 
> As opposed to having 14 working copies that every developer has 
> to manually maintain?  Really?  The layout in my OP is only part of 
> this project.  We really do have 14 separate branches/tags/trunks 
> trees.  This is a new project, so that number will probably go up over 
time.
> 
> > I prefer to check out the branch and/or trunks that I need to work
> on as needed and update each
> > one as needed. If you want to update all your working copies with 
> one command you can create
> > a local batch file which goes through and calls update on each folder.
> 
> With --max-depth it should be possible to avoid needing a script for
> updates.  I've been working on a script for the initial checkout. 
> It's non-trivial and brittle.  I will postpone the gory details 
> unless there is interest.

How about creating a separate directory that only contains a bunch of
svn:externals values to each trunk dir you need?  Then the user
just checks out this one directory which will automatically
pull only the directories you want.

I don't normally recommend externals since each external causes
a separate connection to the server.  This can be a big performance
problem for high latency connections...  Especially when it
is abused by having thousands of externals...

Kevin R.

Re: Parallel branches/tags/trunk directories

Posted by Mike Cepek <mi...@yahoo.com>.

> Not "sufficient" because it takes time and manual work to do? Or, because it doesn't do what you need?

Taking 97 minutes to pull 25 GB from all the tags/ and branches/ is unacceptable when it takes only 3 minutes to pull the 660 MB we actually need from just the trunk/ directories.  Getting everything and then manually pruning isn't a viable option.  Getting just the trunk/ directories without a brittle script is the whole point of my post.

> Sparse directories is exactly what you need to use to get what you want.
> 1.7 added a -parents (I think it is) argument to the update/checkout [...]
I'll check out that option.  Unfortunately...

> What version of the server are you running? I think if you use Sparse directories with a pre 1.6 server
> the server sends all the files and the client just throws away what doesn't fit into your requested depth.

Ah.  Our server reports itself as 1.4.2.  That probably explains why it took so long.  I have no idea what the odds are of having our server upgraded (used by many other projects).

> That all said, I don't really see the benefit of having one working copy that you can update at once.

As opposed to having 14 working copies that every developer has to manually maintain?  Really?  The layout in my OP is only part of this project.  We really do have 14 separate branches/tags/trunks trees.  This is a new project, so that number will probably go up over time.

> I prefer to check out the branch and/or trunks that I need to work on as needed and update each
> one as needed. If you want to update all your working copies with one command you can create
> a local batch file which goes through and calls update on each folder.

With --max-depth it should be possible to avoid needing a script for updates.  I've been working on a script for the initial checkout. It's non-trivial and brittle.  I will postpone the gory details unless there is interest.

RE: Parallel branches/tags/trunk directories

Posted by Bob Archer <Bo...@amsi.com>.

> (resend as plaintext to fix formatting)
> 
> (I did some searching for an answer here and elsewhere, and didn't find it;
> sorry if I missed it.)
> 
> The project I'm dealing with has the SVN repository arranged like so:
> 
> proj/top1/branches/
> proj/top1/tags/
> proj/top1/trunk/
> proj/top2/common/branches/
> proj/top2/common/tags/
> proj/top2/common/trunk/
> proj/top2/thingA/branches/
> proj/top2/thingA/tags/
> proj/top2/thingA/trunk/
> proj/top2/thingB/branches/
> proj/top2/thingB/tags/
> proj/top2/thingB/trunk/
> proj/top2/deploy/common/branches/
> proj/top2/deploy/common/tags/
> proj/top2/deploy/common/trunk/
> proj/top2/deploy/thingA/env1/branches/
> proj/top2/deploy/thingA/env1/tags/
> proj/top2/deploy/thingA/env1/trunk/
> proj/top2/deploy/thingA/env2/branches/
> proj/top2/deploy/thingA/env2/tags/
> proj/top2/deploy/thingA/env2/trunk/
> proj/top2/deploy/thingB/env1/branches/
> proj/top2/deploy/thingB/env1/tags/
> proj/top2/deploy/thingB/env1/trunk/
> (etc)
> 
> First, I completely agree that this organization has issues.  Changing is isn't an
> option.  I'll spare you the details.  :-/
> 
> My question is:  how to checkout and update at the proj/ level without
> getting the content of all the tags/ and branches/ directories?
> 
> The "Sparse Directories" features of SVN (--depth and --set-depth) sounded
> good:
> http://svnbook.red-bean.com/en/1.5/svn.advanced.sparsedirs.html
> 
> But those features are not sufficient.  In order to do a fresh checkout of the

Not "sufficient" because it takes time and manual work to do? Or, because it doesn't do what you need? Sparse directories is exactly what you need to use to get what you want.

1.7 added a -parents (I think it is) argument to the update/checkout so that you didn't have to do

Svn checkout ^/proj --depth empty
Cd proj
Svn checkout ^/proj/top1 --depth empty
Cd top1
Svn checkout ^/proj/top1/trunk 
Etc...

Also, if you are on Windows tortoiseSVN makes the above easier without the need to walking the path... iirc.

That all said, I don't really see the benefit of having one working copy that you can update at once. I prefer to check out the branch and/or trunks that I need to work on as needed and update each one as needed. If you want to update all your working copies with one command you can create a local batch file which goes through and calls update on each folder.

> above and avoid the massive on-disk redundancy that would occur due to
> the content of tags/ and branches/, each trunk/ directory must be checked
> out individually.  Other than creating a brittle script that lists each trunk/
> directory, I don't see a solution.  Even then, each branches/ and tags/
> directory needs to be marked with --set-depth as well.
> 
> I also discovered that "svn co URL/proj/top1 --depth immediates" can take a
> very long time for a tree with tons of content.  Seems like a bug (using SlikSvn
> 1.6.15).  Similarly, something like "svn update URL/proj/top1/tags -depth
> immediates" would be handy at times to get just the empty subdirectories,
> named after each branch or tag.  Updating just one of those subdirectories
> would allow mass-diffs between the trunk or another branch/tag.  I don't see
> why a depth-limited operation like that should take so much time.

What version of the server are you running? I think if you use Sparse directories with a pre 1.6 server the server sends all the files and the client just throws away what doesn't fit into your requested depth.

BOb


> 
> I though about using 'svn list http://.../proj/'; and grep'ing the result for
> "/trunk", but that one command takes over 10 minutes - not feasible.  A
> checkout of all the trunk/ directory contents only takes 2.5 minutes.
> 
> So: has anyone found a clean way to handle checkouts and updates when
> branches/tags/trunk directories are strewn all over a tree?  Thanks.

Re: Parallel branches/tags/trunk directories

Posted by Mike Cepek <mi...@yahoo.com>.

> Maybe something like this might work out better for you. Use "svn ls -R"
> and find the directories you have to checkout:

> svn ls -R http://myserver/svn/proj | grep "/trunk/$"

Thanks, David.  I did bring up using 'svn list' in my original post and grep'ing for trunk, but that took 4 times as long as the actual checkouts that need to be performed.  And even a 1.6 server won't improve that.

I've started the ball rolling on an SVN server upgrade.  Without that, I don't see any feasible non-brittle solution.  With that, I'm thinking that iterating N levels deep with "--depth immediates" will be performant.  I'm not sure how Ant does at emulating 'find . -type d' command...

@Bob Archer:  Yes, I've had dozens of SVN sandboxes on-disk at once both inside and outside of Eclipse.  They too were various versions of projects.  I strongly suspect our situation is different.  This one project has different branches and tags at different levels.  In fact, our production support team will be mainly responsible for the proj/top2/deploy/** subtree, including tagging, whereas the other trees will be managed by us product developers.  I suspect there will be at least 5 different "sets" of trunk/tag/branch trees, updated at different times, some perhaps with different naming conventions (for branches and tags).

Re: Parallel branches/tags/trunk directories

Posted by David Weintraub <qa...@gmail.com>.

Okay solutions:

Since you're dealing with an older version of Subversion, you might
find the -N (no-recursive) flag to work better than --depth. Do a "svn
co -N" and then go into each of those directories and do a "svn update
-N" unless those directories are branches or tags (you skip) or is
trunk (you update without the -N flag).

Maybe something like this might work out better for you. Use "svn ls
-R" and find the directories you have to checkout:

    svn ls -R http://myserver/svn/proj | grep "/trunk/$"

That's probably faster than using "svn co --depths" a bunch of times.
You can then use that list to checkout each of those directories in
their own working directory. You'll have a dozen working directories.
to get a listing of all the trunks you have to checkout, and then
checking out each trunk in its own directory independently of the
others. It might work better than checking out proj in a sparse
checkout and working your way down to all of the trunks.

Completely untested, but you get the idea:

svn ls -R $URL | grep "/trunk/$" | while read directory
do
    parent=$(dirname $directory)
    mkdir -p $parent
    svn co $URL/$directory $directory
done

-- 
David Weintraub
qazwart@gmail.com

Re: Parallel branches/tags/trunk directories

Posted by Les Mikesell <le...@gmail.com>.

On 8/11/2011 1:25 PM, Mike Cepek wrote:
> I appreciate the creative solutions, but I feel that we may be wandering a bit off track.

They mostly aren't 'creative' - just standard practices.

> Otherwise, it sounds like there is no consensus solution for dealing with this kind of repository layout.  Thanks, all.

No, it is pretty common for people to pull in an assortment of 
components from different locations in a repository and to choose from a 
large number of branches.  Your only odd part is arranging things so 
that you have to do it even for the main portions of the project.

-- 
   Les Mikesell
    lesmikesell@gmail.com

Re: Parallel branches/tags/trunk directories

Posted by Mike Cepek <mi...@yahoo.com>.

I appreciate the creative solutions, but I feel that we may be wandering a bit off track.
 
This SVN repository layout will NOT be changing.  Mapping externals to every trunk/ directory seems like overkill, especially when features like --max-depth exist.
 
What little I've been able to find out about --parents suggests that it isn't going to be much of a help.  It sound like it just simplifies having to deal with intermediate directories.  That might simplify my checkout script a bit, but not much.  And since SVN 1.7 appears to be bleeding-edge, I expect any SVN server upgrade I can initiate will be to 1.6 anyway.
 
If there's interest, I can post the bash shell script I'm working on that does most of the job of a full trunk-only checkout.  Since it's mostly just SVN commands, it's likely that we'll migrate this into Ant eventually.  I'm not happy about how brittle this script is, and I can't fully optimize it due to my ancient SVN server version.  But I'll share my half-baked script if anyone cares.
 
Otherwise, it sounds like there is no consensus solution for dealing with this kind of repository layout.  Thanks, all.

Re: Parallel branches/tags/trunk directories

Posted by Stefan Sperling <st...@elego.de>.

On Thu, Aug 11, 2011 at 07:18:46PM +0200, Andreas Krey wrote:
> There already were comments on the usefulness of a complete checkout. I
> mainly find (resp. found, in the days of CVS, without the path-emulated
> tags/branches) it helpful to be able to search the whole repo for usages
> of my APIs.

For this purpose there are solutions which perform data-mining directly
on the repository and provide full-text search of all files in all
revisions via a web interface.  Examples are svnsearch (open source)
and fisheye (proprietary).

Re: Parallel branches/tags/trunk directories

Posted by Andreas Krey <a....@gmx.de>.

On Thu, 11 Aug 2011 07:52:46 +0000, Mike Cepek wrote:
...
> But those features are not sufficient.  In order to do a fresh checkout of the above and avoid the massive on-disk redundancy that would occur due to the content of tags/ and branches/, each trunk/ directory must be checked out individually. 

There already were comments on the usefulness of a complete checkout. I
mainly find (resp. found, in the days of CVS, without the path-emulated
tags/branches) it helpful to be able to search the whole repo for usages
of my APIs.

Anyway, one way to not have to script this would be to have a separate
tree that pulls in all the interesting parts as svn:externals. That way,
you get easy updates; in exchange for needing some care when committing.
(I don't suppose you actually *want* the ../trunk/.. in the paths?)

Andreas

-- 
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800