You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Stas Cherkassky <sc...@gmail.com> on 2010/01/08 00:31:09 UTC

Re: suspected bug report or feature request - tagging a sparse directory

Julian,

I don't want to remove parts of the tree from the repository.
What I want is, effectively, a way to independently mark separate
parts of the tree with the same tag.
(I tried to explain it in my example). I don't see how your method
(using svn rm) does it.

If a tag is a property of a file (like in CVS), it's natural operation :
cvs tag -R dirA -t relX    # tags whatever files and versions I have in WC
and then
cvs tag -R dirB -t relX
In the end I have both dirA and dirB tagged with same tag relX.

If a tag is an object (like in Perforce), and it's property is a list
of files/versions it applies to, it's also easily achievable in very
similar manner - 2nd p4 tag call would just add more files to that
property of the tag object.

With SVN's implementation of tags (logical directory in repo), it
would also be possible, had 'svn copy' behave like I suggested.
IMHO, this way is more intuitive - that's what unix copy command would do.

regards,
Stas

On Thu, Jan 7, 2010 at 2:42 PM, Julian Foad <ju...@wandisco.com> wrote:
> Stas Cherkassky wrote:
> [...]
>> % svn co $SVNROOT/myproj -–depth empty myproj
>> % svn update -–depth infinity myproj/dirA
>>
>>  .. modifies it in some way and tries to tag:
>>
>> % svn mkdir $SVNROOT/tags/rel_X
>> % svn copy myproj $SVNROOT/tags/rel_X/ -m “tagging part A of myproj
>> for the next integration”
>>
>> The problem that we see is SVN for some reason also copies dirB under
>> $SVNROOT/branches/rel_X !!
>> % svn ls $SVNROOT/tags/rel_X/myproj
>> dirA
>> dirB <- this should not be there !
>>
>> my SVN version is 1.6.2 (r37639)
>>
>> I was under impression that svn copy WC -> URL should take whatever I
>> have in my WC,
>> and ONLY that, similar to unix copy.
>> The whole point of this mode (WC -> URL) it to remember (tag) any
>> weird combination of versions/components
>> that I've built in my WC. So why copy the rest of the world ?
>
> The idea of the sparse WC is "I know that other stuff exists as part of
> this project tree, but I don't need to see that other stuff right now."
>
> If you want to create a WC that represents your project tree pruned down
> (with parts removed), you have to use the "svn rm" command to do that:
> "svn rm dirB". Otherwise the other stuff is still conceptually "there"
> in terms of the versioned tree.
>
> I don't know if you can currently use "svn rm" on a node that is
> excluded from the WC because of a sparse WC. If not, that would be a
> good feature enhancement to make that possible.
>
>>  If
>> that's not a bug but feature, would it be possible
>> to have a switch to copy command that would behave as I described ?
>
> I think the method above is better. There are several ways a node can be
> missing from the WC, and adding a switch (or switches) to the copy
> command to control this behaviour is probably not the best way to design
> the solution. Also "copy" is not the only operation you might want to do
> [1]. It's better to get the WC into the state you want (which as I say
> is not achieved just by making it sparse) and then issue a simple "copy"
> command, rather than use a more complex copy command that has options.
>
> [1] Hmm, but what does "export" from the WC do? Does it go to the repo
> to get the excluded parts?
>
> - Julian
>
>
>

Re: suspected bug report or feature request - tagging a sparse directory

Posted by Johan Corveleyn <jc...@gmail.com>.
On Fri, Jan 8, 2010 at 1:31 AM, Stas Cherkassky <sc...@gmail.com> wrote:
> Julian,
>
> I don't want to remove parts of the tree from the repository.
> What I want is, effectively, a way to independently mark separate
> parts of the tree with the same tag.
> (I tried to explain it in my example). I don't see how your method
> (using svn rm) does it.
>
> If a tag is a property of a file (like in CVS), it's natural operation :
> cvs tag -R dirA -t relX    # tags whatever files and versions I have in WC
> and then
> cvs tag -R dirB -t relX
> In the end I have both dirA and dirB tagged with same tag relX.
>
> If a tag is an object (like in Perforce), and it's property is a list
> of files/versions it applies to, it's also easily achievable in very
> similar manner - 2nd p4 tag call would just add more files to that
> property of the tag object.
>
> With SVN's implementation of tags (logical directory in repo), it
> would also be possible, had 'svn copy' behave like I suggested.
> IMHO, this way is more intuitive - that's what unix copy command would do.

You could also try to accomplish such a "sparse tag" by doing a pure
server-side copy, using multiple source targets (and with the
--parents option to automatically create the tag directory itself and
any other subdirs needed). Like:

% svn copy --parents $SVNROOT/myproj/dirA $SVNROOT/tags/rel_X/myproj
-m “tagging part A of myproj for the next integration”

Or if you want to tag only dirA, dirC and dirF, ignoring other dirXXX subdirs:

% svn copy --parents $SVNROOT/myproj/dirA $SVNROOT/myproj/dirC
$SVNROOT/myproj/dirF $SVNROOT/tags/rel_X/myproj -m “tagging part A, C
and F of myproj for the next integration”
(all the above on one line)

This makes use of the feature of svn copy that it can take multiple
source arguments:
    usage: copy SRC[@REV]... DST

and the --parents option:
    --parents                : make intermediate directories


Ok, it might not be as convenient for you as creating the tag directly
out of your WC that you've already set up the correct way, but still
... it works.

HTH
Regards,
Johan

Re: suspected bug report or feature request - tagging a sparse directory

Posted by Stas Cherkassky <sc...@gmail.com>.
- Great! Make it 'svn cp --sparse --force WC URL' - and we are all set !

[ --force means that we can override file that already exists in the URL,
e.g. by creating a new revision of it, and not issue error message like it
does now ]

On Thu, Jan 14, 2010 at 8:59 PM, Philip Martin
<ph...@wandisco.com>wrote:

> "C. Michael Pilato" <cm...@collab.net> writes:
>
> > Stas, you are quite right that the messaging here is important, if only
> to
> > avoid hurt feelings.  Your use-case is a valid one, and we'd all do well
> not
> > to discount it.  But Subversion simply wasn't designed to accommodate it.
> > And this isn't even a small "oops we overlooked it" thing.  This is a
> > fundamental difference between the tagging and branching paradigms in CVS
> > and Subversion -- a difference introduced intentionally when Subversion
> was
> > conceived.
>
> I think Stas has a point here.  Subversion already does something
> similar for mixed revision working copies and switched working copies.
> If one copies such a working copy to the repository the revision will
> include appropriate deletes and replaces to "match" the working copy.
> Now if one checks out such an URL it produces a simple, single
> revision working copy, not the original mixed or switched one, but
> from a content point of view it looks like the original.  I don't
> think it would be unreasonable for Subversion to do treat sparse
> working copies in the same way.
>
> --
> Philip
>

Re: suspected bug report or feature request - tagging a sparse directory

Posted by Philip Martin <ph...@wandisco.com>.
"C. Michael Pilato" <cm...@collab.net> writes:

> Stas, you are quite right that the messaging here is important, if only to
> avoid hurt feelings.  Your use-case is a valid one, and we'd all do well not
> to discount it.  But Subversion simply wasn't designed to accommodate it.
> And this isn't even a small "oops we overlooked it" thing.  This is a
> fundamental difference between the tagging and branching paradigms in CVS
> and Subversion -- a difference introduced intentionally when Subversion was
> conceived.

I think Stas has a point here.  Subversion already does something
similar for mixed revision working copies and switched working copies.
If one copies such a working copy to the repository the revision will
include appropriate deletes and replaces to "match" the working copy.
Now if one checks out such an URL it produces a simple, single
revision working copy, not the original mixed or switched one, but
from a content point of view it looks like the original.  I don't
think it would be unreasonable for Subversion to do treat sparse
working copies in the same way.

-- 
Philip

Re: suspected bug report or feature request - tagging a sparse directory

Posted by Stas Cherkassky <sc...@gmail.com>.
- yeah, that's the capability I miss compared to any other SCM system I know
(which is CVS, Perforce, and now - a little of git).
Add it to other smaller but also annoying things, and I am half the way into
git manual... Git has handy submodules which can be brought together into
any tree configuration ...

On Wed, Jan 20, 2010 at 5:31 AM, Roy Franz <ro...@gmail.com> wrote:

> On Thu, Jan 14, 2010 at 1:36 PM, Stas Cherkassky <sc...@gmail.com>
> wrote:
> > Michael,
> >
> > I think you are right - Subversion just wasn't designed for this
> > methodology.
> > But I can't believe SVN users don't have the same problem. I wonder how
> they
> > solve it ...
>
> We have the same problem, but our 'solution' is to just live with it.
> This comes up periodically,
> and is a capability that users miss (compared to CVS.)
>
>

Re: suspected bug report or feature request - tagging a sparse directory

Posted by Roy Franz <ro...@gmail.com>.
On Thu, Jan 14, 2010 at 1:36 PM, Stas Cherkassky <sc...@gmail.com> wrote:
> Michael,
>
> I think you are right - Subversion just wasn't designed for this
> methodology.
> But I can't believe SVN users don't have the same problem. I wonder how they
> solve it ...

We have the same problem, but our 'solution' is to just live with it.
This comes up periodically,
and is a capability that users miss (compared to CVS.)

> In other words, the sparse tag is just a mean to achieve what I need, which
> is a communication
> between sub-module designers and integrator. How is this supposed to be done
> in SVN ?
> How does integrator knows which version of each sub-module to take as a
> release-candidate
> for next integration ?
>
> The "view attached to tag" that you suggested is interesting (it resembles a
> bit a tag implementation
> in Perforce, where tag is an object which knows all files/versions it's put
> on - which, IMHO is the best),
> ... but only resembles. In my case there is no special goal to reproduce
> that partial tree. I need to collect a few sub-modules to build a module
> (this is hierarchical, the module may in turn be only part of the project).
> And, as far as I understand, such "view" attached to a tag would not help me
> to achieve this goal.
>
> thanks,
> Stas
>
> On Thu, Jan 14, 2010 at 6:39 PM, C. Michael Pilato <cm...@collab.net>wrote:
>
>> Stas, you are quite right that the messaging here is important, if only to
>> avoid hurt feelings.  Your use-case is a valid one, and we'd all do well
>> not
>> to discount it.  But Subversion simply wasn't designed to accommodate it.
>> And this isn't even a small "oops we overlooked it" thing.  This is a
>> fundamental difference between the tagging and branching paradigms in CVS
>> and Subversion -- a difference introduced intentionally when Subversion was
>> conceived.
>>
>> I wonder:  if you could attach to your tag a view specification that, upon
>> checkout of the tag, would "kick in" and cause a working copy to be
>> constructed sparsely as defined in that specification, would your concerns
>> be alleviated?  My line of thinking is this:  some work was done to build
>> out the working copy that serves as the source of your copy.  That's work
>> that might need to be done again someday.  It would be nice to preserve
>> that
>> work a non-destructive fashion:  to remember the sparse configuration of
>> your tree so that it can be "replayed" later, or perhaps by someone else.
>> Well, what if you could marshal that sparse configuration into the
>> repository in some fashion so that it could be applied to tags created from
>> your working copy?  What complications or problems does this introduce in
>> your situation?  What problems does it alleviate?
>>
>>
>> Stas Cherkassky wrote:
>> > Julian,
>> >
>> > thanks you for your reply.
>> > Your approach may technically work, but, I am afraid, it's not realistic
>> in
>> > real large project.
>> >
>> > The [everything except dirA and dirB] part may be quite complicated for
>> > large sparse tree.
>> >
>> > The second approach (multiple svn cp WC URL) is also cumbersome and
>> requires
>> > multiple commits.
>> > It also won't work if some files were already tagged before (you can't
>> svn
>> > cp over existing file).
>> >
>> > Both approaches are not easy to automate (too many cases to consider).
>> > Just consider, what will happen 2nd time, when user B wants to update the
>> > tag in his WC. Some files were tagged before, some  - not. They may be in
>> > the same dir, deep inside the tree.
>> >
>> > Just compare it to 'cvs tag -R -F tagX myproj'. The -F switch would tag
>> > untagged files, and move tag on tagged files. Whatever files I have in my
>> > WC. In one simple command.
>> >
>> > You may point our (correctly) that this feature was not requested or was
>> not
>> > considered important enough - after all it's open source community, and
>> > developers do what they think is important and interesting to them.  And
>> > this is all correct. Incorrect is to claim that SVN does have the
>> ability.
>> > Even if it technically does, its absolutely unusable (all IMHO, of
>> course).
>> > The only real solution that I can see, it to have some switch for 'svn
>> cp'
>> > that will do exactly what's needed - including override files already
>> under
>> > the ^/tags/ dir..
>> >
>> > thanks,
>> > Stas
>> >
>> >
>> >
>> > On Wed, Jan 13, 2010 at 8:00 PM, Julian Foad <julianfoad@btopenworld.com
>> >wrote:
>> >
>> >> Hi Stas.
>> >>
>> >> Your question raises an important issue: "What are the semantics of
>> >> sparse directories?" You expected "svn copy" to copy just the parts that
>> >> are present in the WC, whereas in fact it copies the whole logical tree.
>> >> There is a reason for that, but I think it is not clear how a user
>> >> should expect each Subversion command to behave.
>> >>
>> >> In this email I will just address how it works at the moment and how to
>> >> achieve the tagging that you wanted. I will start a separate email
>> >> thread to discuss the semantics of sparse directories.
>> >>
>> >>
>> >> Stas Cherkassky wrote:
>> >>> I don't want to remove parts of the tree from the repository.
>> >>> What I want is, effectively, a way to independently mark separate
>> >>> parts of the tree with the same tag.
>> >>> (I tried to explain it in my example). I don't see how your method
>> >>> (using svn rm) does it.
>> >> I'll explain my method using "svn rm" first, and then the method using
>> >> "svn copy" with multiple source arguments.
>> >>
>> >> You don't exactly want to remove parts from the repository, but one way
>> >> of thinking about what you want to create in the repository (the tagged
>> >> data) is a copy of your normal working tree except with some parts
>> >> removed. A common and flexible way to create a tree in a Subversion
>> >> repository is to create it in a WC and then commit (or in this case
>> >> copy) it. That's what my suggestion was for.
>> >>
>> >> More precisely,
>> >>
>> >>  svn rm myproj/[everything except dirA and dirB]
>> >>  svn copy myproj ^/tags/rel_X/
>> >>
>> >> And then if you want to restore your WC to how it was, you can revert
>> >> the parts that are scheduled for deletion:
>> >>
>> >>  svn revert -R myproj/[everything except dirA and dirB]
>> >>
>> >>
>> >> Another way of thinking about the task is to ask Subversion to copy only
>> >> the parts you want:
>> >>
>> >>  $ svn copy --parents myproj/dirA myproj/dirB ^/tags/rel_X/myproj/
>> >>  Committed revision 2.
>> >>
>> >>  $ svn log -vq -c2
>> >>  ------------------------------------------------------------------
>> >>  r2 | julianfoad | 2010-01-08 10:37:34 +0000 (Fri, 08 Jan 2010)
>> >>  Changed paths:
>> >>     A /tags/rel_X/myproj
>> >>     A /tags/rel_X/myproj/dirA (from /myproj/dirA:1)
>> >>     A /tags/rel_X/myproj/dirB (from /myproj/dirB:1)
>> >>  ------------------------------------------------------------------
>> >>
>> >> There is a limitation with this form of the copy command. With a single
>> >> "svn copy" command, all the specified source objects (dirA and dirB in
>> >> this case) are created as direct children of the target directory. There
>> >> is no way to extend that example to make myproj/dirC/foo go to
>> >> ^/tags/rel_X/myproj/dirC/foo at the same time as copying dirA and dirB.
>> >> However, you can extend a tag by making a further commit to it with a
>> >> second copy command, unless committing to a tag is disabled in your
>> >> repository. So you could do:
>> >>
>> >>  $ svn mkdir ^/tags/rel_X/myproj      -m "New tag. Incomplete."
>> >>  $ svn copy dirA ^/tags/rel_X/myproj/ -m "Incomplete."
>> >>  $ svn copy dirB ^/tags/rel_X/myproj/ -m "Incomplete."
>> >>  $ svn copy --parents dirC/foo ^/tags/rel_X/myproj/dirC/foo
>> >>             -m "Completed tag for rel_X of myproj."
>> >>
>> >>
>> >>> If a tag is a property of a file (like in CVS), it's natural operation
>> :
>> >>> cvs tag -R dirA -t relX    # tags whatever files and versions I have in
>> >> WC
>> >>> and then
>> >>> cvs tag -R dirB -t relX
>> >>> In the end I have both dirA and dirB tagged with same tag relX.
>> >> You can certainly do the equivalent thing in Subversion: it is the "svn
>> >> copy" method that I described above.
>> >>
>> >>> If a tag is an object (like in Perforce), and it's property is a list
>> >>> of files/versions it applies to, it's also easily achievable in very
>> >>> similar manner - 2nd p4 tag call would just add more files to that
>> >>> property of the tag object.
>> >> That is also like the "svn copy" method that I described above.
>> >>
>> >>> With SVN's implementation of tags (logical directory in repo), it
>> >>> would also be possible, had 'svn copy' behave like I suggested.
>> >> That is true.
>> >>
>> >>> IMHO, this way is more intuitive - that's what unix copy command would
>> >> do.
>> >>
>> >> The Unix copy command is not a fair comparison, because a normal Unix
>> >> directory is not a temporary working representation of the real data
>> >> stored on the server, and so does not have a way to mark certain files
>> >> and directories as "excluded from the local working copy". Or, looking
>> >> at it another way, if you mark some of the directory's children as
>> >> hidden (by renaming them to start with a dot in Unix), then a copy of
>> >> the whole directory will still include those hidden items.
>> >>
>> >>
>> >> - Julian
>> >>
>> >>
>> >>
>> >
>> >
>>
>>
>> --
>> C. Michael Pilato <cm...@collab.net>
>> CollabNet   <>   www.collab.net   <>   Distributed Development On Demand
>>
>>
>
>
> --
> Stas Cherkassky
> email:   scherkas@gmail.com
> phone:  +972-54-4261959
>

RE: suspected bug report or feature request - tagging a sparse directory

Posted by "Strahovsky, Ido" <id...@intel.com>.
No doubt that you can achieve the same result running several commands.
But the usability is much less friendly, and require the developer to run several commands, while in CVS he could do it in one command.
Also, using wrappers and scripts will solve it, but again we would like to have this functionality as a core of Subversion - achieved by running single command.

The sparse checkout do enhance the capabilities, now it is easier to create your "own" working area, containing subset of folders, and it make sense to enable also tagging this structure.
More than that, imagine a case when developer get a working copy by a wrapper which use sparse checkout, after he finished he tag his working copy, and then... oops he got sources which were not part of his work.

This could be achieved with:
svn cp --sparse WC -> URL
while --sparse: cp only the files in my WC

-- Ido

-----Original Message-----
From: C. Michael Pilato [mailto:cmpilato@collab.net] 
Sent: Thursday, January 14, 2010 11:57 PM
To: Stas Cherkassky
Cc: Julian Foad; dev@subversion.apache.org; Strahovsky, Ido
Subject: Re: suspected bug report or feature request - tagging a sparse directory

I suspect that most folks that have this problem had it long before sparse checkouts was introduced as a feature.  I mean, there's nothing about sparse checkouts that's specific to the goal you have in mind, right?  So I suspect those that have come before you have taken an additive approach to the problem (versus the somewhat subtractive nature of sparse checkouts).  Maybe they've used skeletal trees with svn:externals that pull in particular sub-modules.  Or maybe they've hand-constructed their modules with appropriate copies and deletes and such.  Or maybe they do this kind of configuration in the integration/build scripts.  As I'm sure you can imagine, people can get awfully creative with their development processes. :-)

Something that I don't *think* I've mention before as an option to you is to consider extending your sparse checkouts by a single level, scheduling the deletion (pruning) of those extensions, and then copying that.  Allow me to explain.

Say today you do this:

   svn checkout --depth empty REPO-URL module-to-be
   cd module-to-be
   svn update --set-depth infinity submodule1
   svn update --set-depth infinity submodule3
   svn update --set-depth infinity submodule8
   cd ..
   svn cp module-to-be NEW-REPO-URL
   # Oops!  Got the whole tree instead of just the three modules I wanted

Consider this route:

   svn checkout --depth immediates REPO-URL module-to-be
   cd module-to-be
   svn update --set-depth infinity submodule1
   svn update --set-depth infinity submodule3
   svn update --set-depth infinity submodule8
   svn delete [everything else in this directory)
   cd ..
   svn cp module-to-be NEW-REPO-URL
   # Now I get a copy of everything minus the stuff I don't want!
   svn revert -R .  # so you don't accidentally commit those deletions

I've presented a simplified use-case, of course, but perhaps it can scale to the degree that you need it.

Just trying to add more goodies to the bag of tricks you might consider here.


Stas Cherkassky wrote:
> Michael,
> 
> I think you are right - Subversion just wasn't designed for this 
> methodology.
> But I can't believe SVN users don't have the same problem. I wonder 
> how they solve it ...
> In other words, the sparse tag is just a mean to achieve what I need, 
> which is a communication between sub-module designers and integrator. 
> How is this supposed to be done in SVN ?
> How does integrator knows which version of each sub-module to take as 
> a release-candidate for next integration ?
> 
> The "view attached to tag" that you suggested is interesting (it 
> resembles a bit a tag implementation in Perforce, where tag is an 
> object which knows all files/versions it's put on - which, IMHO is the 
> best), ... but only resembles. In my case there is no special goal to 
> reproduce that partial tree. I need to collect a few sub-modules to 
> build a module (this is hierarchical, the module may in turn be only 
> part of the project).
> And, as far as I understand, such "view" attached to a tag would not 
> help me to achieve this goal.
> 
> thanks,
> Stas
> 
> On Thu, Jan 14, 2010 at 6:39 PM, C. Michael Pilato 
> <cmpilato@collab.net <ma...@collab.net>> wrote:
> 
>     Stas, you are quite right that the messaging here is important, if
>     only to
>     avoid hurt feelings.  Your use-case is a valid one, and we'd all do
>     well not
>     to discount it.  But Subversion simply wasn't designed to
>     accommodate it.
>     And this isn't even a small "oops we overlooked it" thing.  This is a
>     fundamental difference between the tagging and branching paradigms
>     in CVS
>     and Subversion -- a difference introduced intentionally when
>     Subversion was
>     conceived.
> 
>     I wonder:  if you could attach to your tag a view specification
>     that, upon
>     checkout of the tag, would "kick in" and cause a working copy to be
>     constructed sparsely as defined in that specification, would your
>     concerns
>     be alleviated?  My line of thinking is this:  some work was done to
>     build
>     out the working copy that serves as the source of your copy.  That's
>     work
>     that might need to be done again someday.  It would be nice to
>     preserve that
>     work a non-destructive fashion:  to remember the sparse configuration of
>     your tree so that it can be "replayed" later, or perhaps by someone
>     else.
>     Well, what if you could marshal that sparse configuration into the
>     repository in some fashion so that it could be applied to tags
>     created from
>     your working copy?  What complications or problems does this
>     introduce in
>     your situation?  What problems does it alleviate?
> 
> 
>     Stas Cherkassky wrote:
>     > Julian,
>     >
>     > thanks you for your reply.
>     > Your approach may technically work, but, I am afraid, it's not
>     realistic in
>     > real large project.
>     >
>     > The [everything except dirA and dirB] part may be quite
>     complicated for
>     > large sparse tree.
>     >
>     > The second approach (multiple svn cp WC URL) is also cumbersome
>     and requires
>     > multiple commits.
>     > It also won't work if some files were already tagged before (you
>     can't svn
>     > cp over existing file).
>     >
>     > Both approaches are not easy to automate (too many cases to consider).
>     > Just consider, what will happen 2nd time, when user B wants to
>     update the
>     > tag in his WC. Some files were tagged before, some  - not. They
>     may be in
>     > the same dir, deep inside the tree.
>     >
>     > Just compare it to 'cvs tag -R -F tagX myproj'. The -F switch
>     would tag
>     > untagged files, and move tag on tagged files. Whatever files I
>     have in my
>     > WC. In one simple command.
>     >
>     > You may point our (correctly) that this feature was not requested
>     or was not
>     > considered important enough - after all it's open source
>     community, and
>     > developers do what they think is important and interesting to
>     them.  And
>     > this is all correct. Incorrect is to claim that SVN does have the
>     ability.
>     > Even if it technically does, its absolutely unusable (all IMHO, of
>     course).
>     > The only real solution that I can see, it to have some switch for
>     'svn cp'
>     > that will do exactly what's needed - including override files
>     already under
>     > the ^/tags/ dir..
>     >
>     > thanks,
>     > Stas
>     >
>     >
>     >
>     > On Wed, Jan 13, 2010 at 8:00 PM, Julian Foad
>     <julianfoad@btopenworld.com <ma...@btopenworld.com>>wrote:
>     >
>     >> Hi Stas.
>     >>
>     >> Your question raises an important issue: "What are the semantics of
>     >> sparse directories?" You expected "svn copy" to copy just the
>     parts that
>     >> are present in the WC, whereas in fact it copies the whole
>     logical tree.
>     >> There is a reason for that, but I think it is not clear how a user
>     >> should expect each Subversion command to behave.
>     >>
>     >> In this email I will just address how it works at the moment and
>     how to
>     >> achieve the tagging that you wanted. I will start a separate email
>     >> thread to discuss the semantics of sparse directories.
>     >>
>     >>
>     >> Stas Cherkassky wrote:
>     >>> I don't want to remove parts of the tree from the repository.
>     >>> What I want is, effectively, a way to independently mark separate
>     >>> parts of the tree with the same tag.
>     >>> (I tried to explain it in my example). I don't see how your method
>     >>> (using svn rm) does it.
>     >> I'll explain my method using "svn rm" first, and then the method
>     using
>     >> "svn copy" with multiple source arguments.
>     >>
>     >> You don't exactly want to remove parts from the repository, but
>     one way
>     >> of thinking about what you want to create in the repository (the
>     tagged
>     >> data) is a copy of your normal working tree except with some parts
>     >> removed. A common and flexible way to create a tree in a Subversion
>     >> repository is to create it in a WC and then commit (or in this case
>     >> copy) it. That's what my suggestion was for.
>     >>
>     >> More precisely,
>     >>
>     >>  svn rm myproj/[everything except dirA and dirB]
>     >>  svn copy myproj ^/tags/rel_X/
>     >>
>     >> And then if you want to restore your WC to how it was, you can revert
>     >> the parts that are scheduled for deletion:
>     >>
>     >>  svn revert -R myproj/[everything except dirA and dirB]
>     >>
>     >>
>     >> Another way of thinking about the task is to ask Subversion to
>     copy only
>     >> the parts you want:
>     >>
>     >>  $ svn copy --parents myproj/dirA myproj/dirB ^/tags/rel_X/myproj/
>     >>  Committed revision 2.
>     >>
>     >>  $ svn log -vq -c2
>     >>  ------------------------------------------------------------------
>     >>  r2 | julianfoad | 2010-01-08 10:37:34 +0000 (Fri, 08 Jan 2010)
>     >>  Changed paths:
>     >>     A /tags/rel_X/myproj
>     >>     A /tags/rel_X/myproj/dirA (from /myproj/dirA:1)
>     >>     A /tags/rel_X/myproj/dirB (from /myproj/dirB:1)
>     >>  ------------------------------------------------------------------
>     >>
>     >> There is a limitation with this form of the copy command. With a
>     single
>     >> "svn copy" command, all the specified source objects (dirA and
>     dirB in
>     >> this case) are created as direct children of the target
>     directory. There
>     >> is no way to extend that example to make myproj/dirC/foo go to
>     >> ^/tags/rel_X/myproj/dirC/foo at the same time as copying dirA and
>     dirB.
>     >> However, you can extend a tag by making a further commit to it with a
>     >> second copy command, unless committing to a tag is disabled in your
>     >> repository. So you could do:
>     >>
>     >>  $ svn mkdir ^/tags/rel_X/myproj      -m "New tag. Incomplete."
>     >>  $ svn copy dirA ^/tags/rel_X/myproj/ -m "Incomplete."
>     >>  $ svn copy dirB ^/tags/rel_X/myproj/ -m "Incomplete."
>     >>  $ svn copy --parents dirC/foo ^/tags/rel_X/myproj/dirC/foo
>     >>             -m "Completed tag for rel_X of myproj."
>     >>
>     >>
>     >>> If a tag is a property of a file (like in CVS), it's natural
>     operation :
>     >>> cvs tag -R dirA -t relX    # tags whatever files and versions I
>     have in
>     >> WC
>     >>> and then
>     >>> cvs tag -R dirB -t relX
>     >>> In the end I have both dirA and dirB tagged with same tag relX.
>     >> You can certainly do the equivalent thing in Subversion: it is
>     the "svn
>     >> copy" method that I described above.
>     >>
>     >>> If a tag is an object (like in Perforce), and it's property is a
>     list
>     >>> of files/versions it applies to, it's also easily achievable in very
>     >>> similar manner - 2nd p4 tag call would just add more files to that
>     >>> property of the tag object.
>     >> That is also like the "svn copy" method that I described above.
>     >>
>     >>> With SVN's implementation of tags (logical directory in repo), it
>     >>> would also be possible, had 'svn copy' behave like I suggested.
>     >> That is true.
>     >>
>     >>> IMHO, this way is more intuitive - that's what unix copy command
>     would
>     >> do.
>     >>
>     >> The Unix copy command is not a fair comparison, because a normal Unix
>     >> directory is not a temporary working representation of the real data
>     >> stored on the server, and so does not have a way to mark certain
>     files
>     >> and directories as "excluded from the local working copy". Or,
>     looking
>     >> at it another way, if you mark some of the directory's children as
>     >> hidden (by renaming them to start with a dot in Unix), then a copy of
>     >> the whole directory will still include those hidden items.
>     >>
>     >>
>     >> - Julian
>     >>
>     >>
>     >>
>     >
>     >
> 
> 
>     --
>     C. Michael Pilato <cmpilato@collab.net <ma...@collab.net>>
>     CollabNet   <>   www.collab.net <http://www.collab.net>   <>  
>     Distributed Development On Demand
> 
> 
> 
> 
> --
> Stas Cherkassky
> email:   scherkas@gmail.com <ma...@gmail.com>
> phone:  +972-54-4261959
> 


-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

---------------------------------------------------------------------
A member of the Intel Corporation group of companies

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: suspected bug report or feature request - tagging a sparse directory

Posted by "C. Michael Pilato" <cm...@collab.net>.
I suspect that most folks that have this problem had it long before sparse
checkouts was introduced as a feature.  I mean, there's nothing about sparse
checkouts that's specific to the goal you have in mind, right?  So I suspect
those that have come before you have taken an additive approach to the
problem (versus the somewhat subtractive nature of sparse checkouts).  Maybe
they've used skeletal trees with svn:externals that pull in particular
sub-modules.  Or maybe they've hand-constructed their modules with
appropriate copies and deletes and such.  Or maybe they do this kind of
configuration in the integration/build scripts.  As I'm sure you can
imagine, people can get awfully creative with their development processes. :-)

Something that I don't *think* I've mention before as an option to you is to
consider extending your sparse checkouts by a single level, scheduling the
deletion (pruning) of those extensions, and then copying that.  Allow me to
explain.

Say today you do this:

   svn checkout --depth empty REPO-URL module-to-be
   cd module-to-be
   svn update --set-depth infinity submodule1
   svn update --set-depth infinity submodule3
   svn update --set-depth infinity submodule8
   cd ..
   svn cp module-to-be NEW-REPO-URL
   # Oops!  Got the whole tree instead of just the three modules I wanted

Consider this route:

   svn checkout --depth immediates REPO-URL module-to-be
   cd module-to-be
   svn update --set-depth infinity submodule1
   svn update --set-depth infinity submodule3
   svn update --set-depth infinity submodule8
   svn delete [everything else in this directory)
   cd ..
   svn cp module-to-be NEW-REPO-URL
   # Now I get a copy of everything minus the stuff I don't want!
   svn revert -R .  # so you don't accidentally commit those deletions

I've presented a simplified use-case, of course, but perhaps it can scale to
the degree that you need it.

Just trying to add more goodies to the bag of tricks you might consider here.


Stas Cherkassky wrote:
> Michael,
> 
> I think you are right - Subversion just wasn't designed for this
> methodology.
> But I can't believe SVN users don't have the same problem. I wonder how
> they solve it ...
> In other words, the sparse tag is just a mean to achieve what I need,
> which is a communication
> between sub-module designers and integrator. How is this supposed to be
> done in SVN ?
> How does integrator knows which version of each sub-module to take as a
> release-candidate
> for next integration ?
> 
> The "view attached to tag" that you suggested is interesting (it
> resembles a bit a tag implementation
> in Perforce, where tag is an object which knows all files/versions it's
> put on - which, IMHO is the best),
> ... but only resembles. In my case there is no special goal to reproduce
> that partial tree. I need to collect a few sub-modules to build a module
> (this is hierarchical, the module may in turn be only part of the project).
> And, as far as I understand, such "view" attached to a tag would not
> help me to achieve this goal.
> 
> thanks,
> Stas
> 
> On Thu, Jan 14, 2010 at 6:39 PM, C. Michael Pilato <cmpilato@collab.net
> <ma...@collab.net>> wrote:
> 
>     Stas, you are quite right that the messaging here is important, if
>     only to
>     avoid hurt feelings.  Your use-case is a valid one, and we'd all do
>     well not
>     to discount it.  But Subversion simply wasn't designed to
>     accommodate it.
>     And this isn't even a small "oops we overlooked it" thing.  This is a
>     fundamental difference between the tagging and branching paradigms
>     in CVS
>     and Subversion -- a difference introduced intentionally when
>     Subversion was
>     conceived.
> 
>     I wonder:  if you could attach to your tag a view specification
>     that, upon
>     checkout of the tag, would "kick in" and cause a working copy to be
>     constructed sparsely as defined in that specification, would your
>     concerns
>     be alleviated?  My line of thinking is this:  some work was done to
>     build
>     out the working copy that serves as the source of your copy.  That's
>     work
>     that might need to be done again someday.  It would be nice to
>     preserve that
>     work a non-destructive fashion:  to remember the sparse configuration of
>     your tree so that it can be "replayed" later, or perhaps by someone
>     else.
>     Well, what if you could marshal that sparse configuration into the
>     repository in some fashion so that it could be applied to tags
>     created from
>     your working copy?  What complications or problems does this
>     introduce in
>     your situation?  What problems does it alleviate?
> 
> 
>     Stas Cherkassky wrote:
>     > Julian,
>     >
>     > thanks you for your reply.
>     > Your approach may technically work, but, I am afraid, it's not
>     realistic in
>     > real large project.
>     >
>     > The [everything except dirA and dirB] part may be quite
>     complicated for
>     > large sparse tree.
>     >
>     > The second approach (multiple svn cp WC URL) is also cumbersome
>     and requires
>     > multiple commits.
>     > It also won't work if some files were already tagged before (you
>     can't svn
>     > cp over existing file).
>     >
>     > Both approaches are not easy to automate (too many cases to consider).
>     > Just consider, what will happen 2nd time, when user B wants to
>     update the
>     > tag in his WC. Some files were tagged before, some  - not. They
>     may be in
>     > the same dir, deep inside the tree.
>     >
>     > Just compare it to 'cvs tag -R -F tagX myproj'. The -F switch
>     would tag
>     > untagged files, and move tag on tagged files. Whatever files I
>     have in my
>     > WC. In one simple command.
>     >
>     > You may point our (correctly) that this feature was not requested
>     or was not
>     > considered important enough - after all it's open source
>     community, and
>     > developers do what they think is important and interesting to
>     them.  And
>     > this is all correct. Incorrect is to claim that SVN does have the
>     ability.
>     > Even if it technically does, its absolutely unusable (all IMHO, of
>     course).
>     > The only real solution that I can see, it to have some switch for
>     'svn cp'
>     > that will do exactly what's needed - including override files
>     already under
>     > the ^/tags/ dir..
>     >
>     > thanks,
>     > Stas
>     >
>     >
>     >
>     > On Wed, Jan 13, 2010 at 8:00 PM, Julian Foad
>     <julianfoad@btopenworld.com <ma...@btopenworld.com>>wrote:
>     >
>     >> Hi Stas.
>     >>
>     >> Your question raises an important issue: "What are the semantics of
>     >> sparse directories?" You expected "svn copy" to copy just the
>     parts that
>     >> are present in the WC, whereas in fact it copies the whole
>     logical tree.
>     >> There is a reason for that, but I think it is not clear how a user
>     >> should expect each Subversion command to behave.
>     >>
>     >> In this email I will just address how it works at the moment and
>     how to
>     >> achieve the tagging that you wanted. I will start a separate email
>     >> thread to discuss the semantics of sparse directories.
>     >>
>     >>
>     >> Stas Cherkassky wrote:
>     >>> I don't want to remove parts of the tree from the repository.
>     >>> What I want is, effectively, a way to independently mark separate
>     >>> parts of the tree with the same tag.
>     >>> (I tried to explain it in my example). I don't see how your method
>     >>> (using svn rm) does it.
>     >> I'll explain my method using "svn rm" first, and then the method
>     using
>     >> "svn copy" with multiple source arguments.
>     >>
>     >> You don't exactly want to remove parts from the repository, but
>     one way
>     >> of thinking about what you want to create in the repository (the
>     tagged
>     >> data) is a copy of your normal working tree except with some parts
>     >> removed. A common and flexible way to create a tree in a Subversion
>     >> repository is to create it in a WC and then commit (or in this case
>     >> copy) it. That's what my suggestion was for.
>     >>
>     >> More precisely,
>     >>
>     >>  svn rm myproj/[everything except dirA and dirB]
>     >>  svn copy myproj ^/tags/rel_X/
>     >>
>     >> And then if you want to restore your WC to how it was, you can revert
>     >> the parts that are scheduled for deletion:
>     >>
>     >>  svn revert -R myproj/[everything except dirA and dirB]
>     >>
>     >>
>     >> Another way of thinking about the task is to ask Subversion to
>     copy only
>     >> the parts you want:
>     >>
>     >>  $ svn copy --parents myproj/dirA myproj/dirB ^/tags/rel_X/myproj/
>     >>  Committed revision 2.
>     >>
>     >>  $ svn log -vq -c2
>     >>  ------------------------------------------------------------------
>     >>  r2 | julianfoad | 2010-01-08 10:37:34 +0000 (Fri, 08 Jan 2010)
>     >>  Changed paths:
>     >>     A /tags/rel_X/myproj
>     >>     A /tags/rel_X/myproj/dirA (from /myproj/dirA:1)
>     >>     A /tags/rel_X/myproj/dirB (from /myproj/dirB:1)
>     >>  ------------------------------------------------------------------
>     >>
>     >> There is a limitation with this form of the copy command. With a
>     single
>     >> "svn copy" command, all the specified source objects (dirA and
>     dirB in
>     >> this case) are created as direct children of the target
>     directory. There
>     >> is no way to extend that example to make myproj/dirC/foo go to
>     >> ^/tags/rel_X/myproj/dirC/foo at the same time as copying dirA and
>     dirB.
>     >> However, you can extend a tag by making a further commit to it with a
>     >> second copy command, unless committing to a tag is disabled in your
>     >> repository. So you could do:
>     >>
>     >>  $ svn mkdir ^/tags/rel_X/myproj      -m "New tag. Incomplete."
>     >>  $ svn copy dirA ^/tags/rel_X/myproj/ -m "Incomplete."
>     >>  $ svn copy dirB ^/tags/rel_X/myproj/ -m "Incomplete."
>     >>  $ svn copy --parents dirC/foo ^/tags/rel_X/myproj/dirC/foo
>     >>             -m "Completed tag for rel_X of myproj."
>     >>
>     >>
>     >>> If a tag is a property of a file (like in CVS), it's natural
>     operation :
>     >>> cvs tag -R dirA -t relX    # tags whatever files and versions I
>     have in
>     >> WC
>     >>> and then
>     >>> cvs tag -R dirB -t relX
>     >>> In the end I have both dirA and dirB tagged with same tag relX.
>     >> You can certainly do the equivalent thing in Subversion: it is
>     the "svn
>     >> copy" method that I described above.
>     >>
>     >>> If a tag is an object (like in Perforce), and it's property is a
>     list
>     >>> of files/versions it applies to, it's also easily achievable in very
>     >>> similar manner - 2nd p4 tag call would just add more files to that
>     >>> property of the tag object.
>     >> That is also like the "svn copy" method that I described above.
>     >>
>     >>> With SVN's implementation of tags (logical directory in repo), it
>     >>> would also be possible, had 'svn copy' behave like I suggested.
>     >> That is true.
>     >>
>     >>> IMHO, this way is more intuitive - that's what unix copy command
>     would
>     >> do.
>     >>
>     >> The Unix copy command is not a fair comparison, because a normal Unix
>     >> directory is not a temporary working representation of the real data
>     >> stored on the server, and so does not have a way to mark certain
>     files
>     >> and directories as "excluded from the local working copy". Or,
>     looking
>     >> at it another way, if you mark some of the directory's children as
>     >> hidden (by renaming them to start with a dot in Unix), then a copy of
>     >> the whole directory will still include those hidden items.
>     >>
>     >>
>     >> - Julian
>     >>
>     >>
>     >>
>     >
>     >
> 
> 
>     --
>     C. Michael Pilato <cmpilato@collab.net <ma...@collab.net>>
>     CollabNet   <>   www.collab.net <http://www.collab.net>   <>  
>     Distributed Development On Demand
> 
> 
> 
> 
> -- 
> Stas Cherkassky
> email:   scherkas@gmail.com <ma...@gmail.com>
> phone:  +972-54-4261959
> 


-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand


Re: suspected bug report or feature request - tagging a sparse directory

Posted by Stas Cherkassky <sc...@gmail.com>.
Michael,

I think you are right - Subversion just wasn't designed for this
methodology.
But I can't believe SVN users don't have the same problem. I wonder how they
solve it ...
In other words, the sparse tag is just a mean to achieve what I need, which
is a communication
between sub-module designers and integrator. How is this supposed to be done
in SVN ?
How does integrator knows which version of each sub-module to take as a
release-candidate
for next integration ?

The "view attached to tag" that you suggested is interesting (it resembles a
bit a tag implementation
in Perforce, where tag is an object which knows all files/versions it's put
on - which, IMHO is the best),
... but only resembles. In my case there is no special goal to reproduce
that partial tree. I need to collect a few sub-modules to build a module
(this is hierarchical, the module may in turn be only part of the project).
And, as far as I understand, such "view" attached to a tag would not help me
to achieve this goal.

thanks,
Stas

On Thu, Jan 14, 2010 at 6:39 PM, C. Michael Pilato <cm...@collab.net>wrote:

> Stas, you are quite right that the messaging here is important, if only to
> avoid hurt feelings.  Your use-case is a valid one, and we'd all do well
> not
> to discount it.  But Subversion simply wasn't designed to accommodate it.
> And this isn't even a small "oops we overlooked it" thing.  This is a
> fundamental difference between the tagging and branching paradigms in CVS
> and Subversion -- a difference introduced intentionally when Subversion was
> conceived.
>
> I wonder:  if you could attach to your tag a view specification that, upon
> checkout of the tag, would "kick in" and cause a working copy to be
> constructed sparsely as defined in that specification, would your concerns
> be alleviated?  My line of thinking is this:  some work was done to build
> out the working copy that serves as the source of your copy.  That's work
> that might need to be done again someday.  It would be nice to preserve
> that
> work a non-destructive fashion:  to remember the sparse configuration of
> your tree so that it can be "replayed" later, or perhaps by someone else.
> Well, what if you could marshal that sparse configuration into the
> repository in some fashion so that it could be applied to tags created from
> your working copy?  What complications or problems does this introduce in
> your situation?  What problems does it alleviate?
>
>
> Stas Cherkassky wrote:
> > Julian,
> >
> > thanks you for your reply.
> > Your approach may technically work, but, I am afraid, it's not realistic
> in
> > real large project.
> >
> > The [everything except dirA and dirB] part may be quite complicated for
> > large sparse tree.
> >
> > The second approach (multiple svn cp WC URL) is also cumbersome and
> requires
> > multiple commits.
> > It also won't work if some files were already tagged before (you can't
> svn
> > cp over existing file).
> >
> > Both approaches are not easy to automate (too many cases to consider).
> > Just consider, what will happen 2nd time, when user B wants to update the
> > tag in his WC. Some files were tagged before, some  - not. They may be in
> > the same dir, deep inside the tree.
> >
> > Just compare it to 'cvs tag -R -F tagX myproj'. The -F switch would tag
> > untagged files, and move tag on tagged files. Whatever files I have in my
> > WC. In one simple command.
> >
> > You may point our (correctly) that this feature was not requested or was
> not
> > considered important enough - after all it's open source community, and
> > developers do what they think is important and interesting to them.  And
> > this is all correct. Incorrect is to claim that SVN does have the
> ability.
> > Even if it technically does, its absolutely unusable (all IMHO, of
> course).
> > The only real solution that I can see, it to have some switch for 'svn
> cp'
> > that will do exactly what's needed - including override files already
> under
> > the ^/tags/ dir..
> >
> > thanks,
> > Stas
> >
> >
> >
> > On Wed, Jan 13, 2010 at 8:00 PM, Julian Foad <julianfoad@btopenworld.com
> >wrote:
> >
> >> Hi Stas.
> >>
> >> Your question raises an important issue: "What are the semantics of
> >> sparse directories?" You expected "svn copy" to copy just the parts that
> >> are present in the WC, whereas in fact it copies the whole logical tree.
> >> There is a reason for that, but I think it is not clear how a user
> >> should expect each Subversion command to behave.
> >>
> >> In this email I will just address how it works at the moment and how to
> >> achieve the tagging that you wanted. I will start a separate email
> >> thread to discuss the semantics of sparse directories.
> >>
> >>
> >> Stas Cherkassky wrote:
> >>> I don't want to remove parts of the tree from the repository.
> >>> What I want is, effectively, a way to independently mark separate
> >>> parts of the tree with the same tag.
> >>> (I tried to explain it in my example). I don't see how your method
> >>> (using svn rm) does it.
> >> I'll explain my method using "svn rm" first, and then the method using
> >> "svn copy" with multiple source arguments.
> >>
> >> You don't exactly want to remove parts from the repository, but one way
> >> of thinking about what you want to create in the repository (the tagged
> >> data) is a copy of your normal working tree except with some parts
> >> removed. A common and flexible way to create a tree in a Subversion
> >> repository is to create it in a WC and then commit (or in this case
> >> copy) it. That's what my suggestion was for.
> >>
> >> More precisely,
> >>
> >>  svn rm myproj/[everything except dirA and dirB]
> >>  svn copy myproj ^/tags/rel_X/
> >>
> >> And then if you want to restore your WC to how it was, you can revert
> >> the parts that are scheduled for deletion:
> >>
> >>  svn revert -R myproj/[everything except dirA and dirB]
> >>
> >>
> >> Another way of thinking about the task is to ask Subversion to copy only
> >> the parts you want:
> >>
> >>  $ svn copy --parents myproj/dirA myproj/dirB ^/tags/rel_X/myproj/
> >>  Committed revision 2.
> >>
> >>  $ svn log -vq -c2
> >>  ------------------------------------------------------------------
> >>  r2 | julianfoad | 2010-01-08 10:37:34 +0000 (Fri, 08 Jan 2010)
> >>  Changed paths:
> >>     A /tags/rel_X/myproj
> >>     A /tags/rel_X/myproj/dirA (from /myproj/dirA:1)
> >>     A /tags/rel_X/myproj/dirB (from /myproj/dirB:1)
> >>  ------------------------------------------------------------------
> >>
> >> There is a limitation with this form of the copy command. With a single
> >> "svn copy" command, all the specified source objects (dirA and dirB in
> >> this case) are created as direct children of the target directory. There
> >> is no way to extend that example to make myproj/dirC/foo go to
> >> ^/tags/rel_X/myproj/dirC/foo at the same time as copying dirA and dirB.
> >> However, you can extend a tag by making a further commit to it with a
> >> second copy command, unless committing to a tag is disabled in your
> >> repository. So you could do:
> >>
> >>  $ svn mkdir ^/tags/rel_X/myproj      -m "New tag. Incomplete."
> >>  $ svn copy dirA ^/tags/rel_X/myproj/ -m "Incomplete."
> >>  $ svn copy dirB ^/tags/rel_X/myproj/ -m "Incomplete."
> >>  $ svn copy --parents dirC/foo ^/tags/rel_X/myproj/dirC/foo
> >>             -m "Completed tag for rel_X of myproj."
> >>
> >>
> >>> If a tag is a property of a file (like in CVS), it's natural operation
> :
> >>> cvs tag -R dirA -t relX    # tags whatever files and versions I have in
> >> WC
> >>> and then
> >>> cvs tag -R dirB -t relX
> >>> In the end I have both dirA and dirB tagged with same tag relX.
> >> You can certainly do the equivalent thing in Subversion: it is the "svn
> >> copy" method that I described above.
> >>
> >>> If a tag is an object (like in Perforce), and it's property is a list
> >>> of files/versions it applies to, it's also easily achievable in very
> >>> similar manner - 2nd p4 tag call would just add more files to that
> >>> property of the tag object.
> >> That is also like the "svn copy" method that I described above.
> >>
> >>> With SVN's implementation of tags (logical directory in repo), it
> >>> would also be possible, had 'svn copy' behave like I suggested.
> >> That is true.
> >>
> >>> IMHO, this way is more intuitive - that's what unix copy command would
> >> do.
> >>
> >> The Unix copy command is not a fair comparison, because a normal Unix
> >> directory is not a temporary working representation of the real data
> >> stored on the server, and so does not have a way to mark certain files
> >> and directories as "excluded from the local working copy". Or, looking
> >> at it another way, if you mark some of the directory's children as
> >> hidden (by renaming them to start with a dot in Unix), then a copy of
> >> the whole directory will still include those hidden items.
> >>
> >>
> >> - Julian
> >>
> >>
> >>
> >
> >
>
>
> --
> C. Michael Pilato <cm...@collab.net>
> CollabNet   <>   www.collab.net   <>   Distributed Development On Demand
>
>


-- 
Stas Cherkassky
email:   scherkas@gmail.com
phone:  +972-54-4261959

Re: suspected bug report or feature request - tagging a sparse directory

Posted by "C. Michael Pilato" <cm...@collab.net>.
Stas, you are quite right that the messaging here is important, if only to
avoid hurt feelings.  Your use-case is a valid one, and we'd all do well not
to discount it.  But Subversion simply wasn't designed to accommodate it.
And this isn't even a small "oops we overlooked it" thing.  This is a
fundamental difference between the tagging and branching paradigms in CVS
and Subversion -- a difference introduced intentionally when Subversion was
conceived.

I wonder:  if you could attach to your tag a view specification that, upon
checkout of the tag, would "kick in" and cause a working copy to be
constructed sparsely as defined in that specification, would your concerns
be alleviated?  My line of thinking is this:  some work was done to build
out the working copy that serves as the source of your copy.  That's work
that might need to be done again someday.  It would be nice to preserve that
work a non-destructive fashion:  to remember the sparse configuration of
your tree so that it can be "replayed" later, or perhaps by someone else.
Well, what if you could marshal that sparse configuration into the
repository in some fashion so that it could be applied to tags created from
your working copy?  What complications or problems does this introduce in
your situation?  What problems does it alleviate?


Stas Cherkassky wrote:
> Julian,
> 
> thanks you for your reply.
> Your approach may technically work, but, I am afraid, it's not realistic in
> real large project.
> 
> The [everything except dirA and dirB] part may be quite complicated for
> large sparse tree.
> 
> The second approach (multiple svn cp WC URL) is also cumbersome and requires
> multiple commits.
> It also won't work if some files were already tagged before (you can't svn
> cp over existing file).
> 
> Both approaches are not easy to automate (too many cases to consider).
> Just consider, what will happen 2nd time, when user B wants to update the
> tag in his WC. Some files were tagged before, some  - not. They may be in
> the same dir, deep inside the tree.
> 
> Just compare it to 'cvs tag -R -F tagX myproj'. The -F switch would tag
> untagged files, and move tag on tagged files. Whatever files I have in my
> WC. In one simple command.
> 
> You may point our (correctly) that this feature was not requested or was not
> considered important enough - after all it's open source community, and
> developers do what they think is important and interesting to them.  And
> this is all correct. Incorrect is to claim that SVN does have the ability.
> Even if it technically does, its absolutely unusable (all IMHO, of course).
> The only real solution that I can see, it to have some switch for 'svn cp'
> that will do exactly what's needed - including override files already under
> the ^/tags/ dir..
> 
> thanks,
> Stas
> 
> 
> 
> On Wed, Jan 13, 2010 at 8:00 PM, Julian Foad <ju...@btopenworld.com>wrote:
> 
>> Hi Stas.
>>
>> Your question raises an important issue: "What are the semantics of
>> sparse directories?" You expected "svn copy" to copy just the parts that
>> are present in the WC, whereas in fact it copies the whole logical tree.
>> There is a reason for that, but I think it is not clear how a user
>> should expect each Subversion command to behave.
>>
>> In this email I will just address how it works at the moment and how to
>> achieve the tagging that you wanted. I will start a separate email
>> thread to discuss the semantics of sparse directories.
>>
>>
>> Stas Cherkassky wrote:
>>> I don't want to remove parts of the tree from the repository.
>>> What I want is, effectively, a way to independently mark separate
>>> parts of the tree with the same tag.
>>> (I tried to explain it in my example). I don't see how your method
>>> (using svn rm) does it.
>> I'll explain my method using "svn rm" first, and then the method using
>> "svn copy" with multiple source arguments.
>>
>> You don't exactly want to remove parts from the repository, but one way
>> of thinking about what you want to create in the repository (the tagged
>> data) is a copy of your normal working tree except with some parts
>> removed. A common and flexible way to create a tree in a Subversion
>> repository is to create it in a WC and then commit (or in this case
>> copy) it. That's what my suggestion was for.
>>
>> More precisely,
>>
>>  svn rm myproj/[everything except dirA and dirB]
>>  svn copy myproj ^/tags/rel_X/
>>
>> And then if you want to restore your WC to how it was, you can revert
>> the parts that are scheduled for deletion:
>>
>>  svn revert -R myproj/[everything except dirA and dirB]
>>
>>
>> Another way of thinking about the task is to ask Subversion to copy only
>> the parts you want:
>>
>>  $ svn copy --parents myproj/dirA myproj/dirB ^/tags/rel_X/myproj/
>>  Committed revision 2.
>>
>>  $ svn log -vq -c2
>>  ------------------------------------------------------------------
>>  r2 | julianfoad | 2010-01-08 10:37:34 +0000 (Fri, 08 Jan 2010)
>>  Changed paths:
>>     A /tags/rel_X/myproj
>>     A /tags/rel_X/myproj/dirA (from /myproj/dirA:1)
>>     A /tags/rel_X/myproj/dirB (from /myproj/dirB:1)
>>  ------------------------------------------------------------------
>>
>> There is a limitation with this form of the copy command. With a single
>> "svn copy" command, all the specified source objects (dirA and dirB in
>> this case) are created as direct children of the target directory. There
>> is no way to extend that example to make myproj/dirC/foo go to
>> ^/tags/rel_X/myproj/dirC/foo at the same time as copying dirA and dirB.
>> However, you can extend a tag by making a further commit to it with a
>> second copy command, unless committing to a tag is disabled in your
>> repository. So you could do:
>>
>>  $ svn mkdir ^/tags/rel_X/myproj      -m "New tag. Incomplete."
>>  $ svn copy dirA ^/tags/rel_X/myproj/ -m "Incomplete."
>>  $ svn copy dirB ^/tags/rel_X/myproj/ -m "Incomplete."
>>  $ svn copy --parents dirC/foo ^/tags/rel_X/myproj/dirC/foo
>>             -m "Completed tag for rel_X of myproj."
>>
>>
>>> If a tag is a property of a file (like in CVS), it's natural operation :
>>> cvs tag -R dirA -t relX    # tags whatever files and versions I have in
>> WC
>>> and then
>>> cvs tag -R dirB -t relX
>>> In the end I have both dirA and dirB tagged with same tag relX.
>> You can certainly do the equivalent thing in Subversion: it is the "svn
>> copy" method that I described above.
>>
>>> If a tag is an object (like in Perforce), and it's property is a list
>>> of files/versions it applies to, it's also easily achievable in very
>>> similar manner - 2nd p4 tag call would just add more files to that
>>> property of the tag object.
>> That is also like the "svn copy" method that I described above.
>>
>>> With SVN's implementation of tags (logical directory in repo), it
>>> would also be possible, had 'svn copy' behave like I suggested.
>> That is true.
>>
>>> IMHO, this way is more intuitive - that's what unix copy command would
>> do.
>>
>> The Unix copy command is not a fair comparison, because a normal Unix
>> directory is not a temporary working representation of the real data
>> stored on the server, and so does not have a way to mark certain files
>> and directories as "excluded from the local working copy". Or, looking
>> at it another way, if you mark some of the directory's children as
>> hidden (by renaming them to start with a dot in Unix), then a copy of
>> the whole directory will still include those hidden items.
>>
>>
>> - Julian
>>
>>
>>
> 
> 


-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand


Re: suspected bug report or feature request - tagging a sparse directory

Posted by Stas Cherkassky <sc...@gmail.com>.
Julian,

thanks you for your reply.
Your approach may technically work, but, I am afraid, it's not realistic in
real large project.

The [everything except dirA and dirB] part may be quite complicated for
large sparse tree.

The second approach (multiple svn cp WC URL) is also cumbersome and requires
multiple commits.
It also won't work if some files were already tagged before (you can't svn
cp over existing file).

Both approaches are not easy to automate (too many cases to consider).
Just consider, what will happen 2nd time, when user B wants to update the
tag in his WC. Some files were tagged before, some  - not. They may be in
the same dir, deep inside the tree.

Just compare it to 'cvs tag -R -F tagX myproj'. The -F switch would tag
untagged files, and move tag on tagged files. Whatever files I have in my
WC. In one simple command.

You may point our (correctly) that this feature was not requested or was not
considered important enough - after all it's open source community, and
developers do what they think is important and interesting to them.  And
this is all correct. Incorrect is to claim that SVN does have the ability.
Even if it technically does, its absolutely unusable (all IMHO, of course).
The only real solution that I can see, it to have some switch for 'svn cp'
that will do exactly what's needed - including override files already under
the ^/tags/ dir..

thanks,
Stas



On Wed, Jan 13, 2010 at 8:00 PM, Julian Foad <ju...@btopenworld.com>wrote:

> Hi Stas.
>
> Your question raises an important issue: "What are the semantics of
> sparse directories?" You expected "svn copy" to copy just the parts that
> are present in the WC, whereas in fact it copies the whole logical tree.
> There is a reason for that, but I think it is not clear how a user
> should expect each Subversion command to behave.
>
> In this email I will just address how it works at the moment and how to
> achieve the tagging that you wanted. I will start a separate email
> thread to discuss the semantics of sparse directories.
>
>
> Stas Cherkassky wrote:
> > I don't want to remove parts of the tree from the repository.
> > What I want is, effectively, a way to independently mark separate
> > parts of the tree with the same tag.
> > (I tried to explain it in my example). I don't see how your method
> > (using svn rm) does it.
>
> I'll explain my method using "svn rm" first, and then the method using
> "svn copy" with multiple source arguments.
>
> You don't exactly want to remove parts from the repository, but one way
> of thinking about what you want to create in the repository (the tagged
> data) is a copy of your normal working tree except with some parts
> removed. A common and flexible way to create a tree in a Subversion
> repository is to create it in a WC and then commit (or in this case
> copy) it. That's what my suggestion was for.
>
> More precisely,
>
>  svn rm myproj/[everything except dirA and dirB]
>  svn copy myproj ^/tags/rel_X/
>
> And then if you want to restore your WC to how it was, you can revert
> the parts that are scheduled for deletion:
>
>  svn revert -R myproj/[everything except dirA and dirB]
>
>
> Another way of thinking about the task is to ask Subversion to copy only
> the parts you want:
>
>  $ svn copy --parents myproj/dirA myproj/dirB ^/tags/rel_X/myproj/
>  Committed revision 2.
>
>  $ svn log -vq -c2
>  ------------------------------------------------------------------
>  r2 | julianfoad | 2010-01-08 10:37:34 +0000 (Fri, 08 Jan 2010)
>  Changed paths:
>     A /tags/rel_X/myproj
>     A /tags/rel_X/myproj/dirA (from /myproj/dirA:1)
>     A /tags/rel_X/myproj/dirB (from /myproj/dirB:1)
>  ------------------------------------------------------------------
>
> There is a limitation with this form of the copy command. With a single
> "svn copy" command, all the specified source objects (dirA and dirB in
> this case) are created as direct children of the target directory. There
> is no way to extend that example to make myproj/dirC/foo go to
> ^/tags/rel_X/myproj/dirC/foo at the same time as copying dirA and dirB.
> However, you can extend a tag by making a further commit to it with a
> second copy command, unless committing to a tag is disabled in your
> repository. So you could do:
>
>  $ svn mkdir ^/tags/rel_X/myproj      -m "New tag. Incomplete."
>  $ svn copy dirA ^/tags/rel_X/myproj/ -m "Incomplete."
>  $ svn copy dirB ^/tags/rel_X/myproj/ -m "Incomplete."
>  $ svn copy --parents dirC/foo ^/tags/rel_X/myproj/dirC/foo
>             -m "Completed tag for rel_X of myproj."
>
>
> > If a tag is a property of a file (like in CVS), it's natural operation :
> > cvs tag -R dirA -t relX    # tags whatever files and versions I have in
> WC
> > and then
> > cvs tag -R dirB -t relX
> > In the end I have both dirA and dirB tagged with same tag relX.
>
> You can certainly do the equivalent thing in Subversion: it is the "svn
> copy" method that I described above.
>
> > If a tag is an object (like in Perforce), and it's property is a list
> > of files/versions it applies to, it's also easily achievable in very
> > similar manner - 2nd p4 tag call would just add more files to that
> > property of the tag object.
>
> That is also like the "svn copy" method that I described above.
>
> > With SVN's implementation of tags (logical directory in repo), it
> > would also be possible, had 'svn copy' behave like I suggested.
>
> That is true.
>
> > IMHO, this way is more intuitive - that's what unix copy command would
> do.
>
> The Unix copy command is not a fair comparison, because a normal Unix
> directory is not a temporary working representation of the real data
> stored on the server, and so does not have a way to mark certain files
> and directories as "excluded from the local working copy". Or, looking
> at it another way, if you mark some of the directory's children as
> hidden (by renaming them to start with a dot in Unix), then a copy of
> the whole directory will still include those hidden items.
>
>
> - Julian
>
>
>


-- 
Stas Cherkassky
email:   scherkas@gmail.com
phone:  +972-54-4261959

Re: suspected bug report or feature request - tagging a sparse directory

Posted by Julian Foad <ju...@btopenworld.com>.
Hi Stas.

Your question raises an important issue: "What are the semantics of
sparse directories?" You expected "svn copy" to copy just the parts that
are present in the WC, whereas in fact it copies the whole logical tree.
There is a reason for that, but I think it is not clear how a user
should expect each Subversion command to behave.

In this email I will just address how it works at the moment and how to
achieve the tagging that you wanted. I will start a separate email
thread to discuss the semantics of sparse directories.


Stas Cherkassky wrote:
> I don't want to remove parts of the tree from the repository.
> What I want is, effectively, a way to independently mark separate
> parts of the tree with the same tag.
> (I tried to explain it in my example). I don't see how your method
> (using svn rm) does it.

I'll explain my method using "svn rm" first, and then the method using
"svn copy" with multiple source arguments.

You don't exactly want to remove parts from the repository, but one way
of thinking about what you want to create in the repository (the tagged
data) is a copy of your normal working tree except with some parts
removed. A common and flexible way to create a tree in a Subversion
repository is to create it in a WC and then commit (or in this case
copy) it. That's what my suggestion was for.

More precisely,

  svn rm myproj/[everything except dirA and dirB]
  svn copy myproj ^/tags/rel_X/

And then if you want to restore your WC to how it was, you can revert
the parts that are scheduled for deletion:

  svn revert -R myproj/[everything except dirA and dirB]


Another way of thinking about the task is to ask Subversion to copy only
the parts you want:

  $ svn copy --parents myproj/dirA myproj/dirB ^/tags/rel_X/myproj/
  Committed revision 2.

  $ svn log -vq -c2
  ------------------------------------------------------------------
  r2 | julianfoad | 2010-01-08 10:37:34 +0000 (Fri, 08 Jan 2010)
  Changed paths:
     A /tags/rel_X/myproj
     A /tags/rel_X/myproj/dirA (from /myproj/dirA:1)
     A /tags/rel_X/myproj/dirB (from /myproj/dirB:1)
  ------------------------------------------------------------------

There is a limitation with this form of the copy command. With a single
"svn copy" command, all the specified source objects (dirA and dirB in
this case) are created as direct children of the target directory. There
is no way to extend that example to make myproj/dirC/foo go to
^/tags/rel_X/myproj/dirC/foo at the same time as copying dirA and dirB.
However, you can extend a tag by making a further commit to it with a
second copy command, unless committing to a tag is disabled in your
repository. So you could do:

  $ svn mkdir ^/tags/rel_X/myproj      -m "New tag. Incomplete."
  $ svn copy dirA ^/tags/rel_X/myproj/ -m "Incomplete."
  $ svn copy dirB ^/tags/rel_X/myproj/ -m "Incomplete."
  $ svn copy --parents dirC/foo ^/tags/rel_X/myproj/dirC/foo
             -m "Completed tag for rel_X of myproj."


> If a tag is a property of a file (like in CVS), it's natural operation :
> cvs tag -R dirA -t relX    # tags whatever files and versions I have in WC
> and then
> cvs tag -R dirB -t relX
> In the end I have both dirA and dirB tagged with same tag relX.

You can certainly do the equivalent thing in Subversion: it is the "svn
copy" method that I described above.

> If a tag is an object (like in Perforce), and it's property is a list
> of files/versions it applies to, it's also easily achievable in very
> similar manner - 2nd p4 tag call would just add more files to that
> property of the tag object.

That is also like the "svn copy" method that I described above.

> With SVN's implementation of tags (logical directory in repo), it
> would also be possible, had 'svn copy' behave like I suggested.

That is true.

> IMHO, this way is more intuitive - that's what unix copy command would do.

The Unix copy command is not a fair comparison, because a normal Unix
directory is not a temporary working representation of the real data
stored on the server, and so does not have a way to mark certain files
and directories as "excluded from the local working copy". Or, looking
at it another way, if you mark some of the directory's children as
hidden (by renaming them to start with a dot in Unix), then a copy of
the whole directory will still include those hidden items.


- Julian