You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Steve Johnson <st...@equilibrium.com> on 2005/09/08 20:03:27 UTC
Checkout filtering wanted
Greetings,
We've been playing with Subversion for a few months, and have just
taken the plunge and converted over from CVS. Svn is generally a
huge improvement over CVS. We do, however, have a few key features
that we'd like to see implemented. I'd like to discuss the first of
these...
I've perused the mailing list and have read discussion threads
regarding a number of requests/needs for checkout filtering. From
what I've seen, however, none of these discussions led to any real
interest in actually implementing a filtering mechanism. I'm
wondering why this is, and I'm hoping to be able to change this.
Subversion's arbitrary properties would seem to me to lead naturally
to a filtering mechanism. What I want to see implemented is:
checkout --filter '<name><op><value>'
where <name> is a subversion property name, <value> is a value to
compare with the value of the <name> property at each node of a tree,
and <op> defines the comparison. Only when the comparison succeeded
on a versioned object would that object be checked out. For our
purposes, this check could be done only on directories. For
generality, I could see extending it to files as well. The 'update'
action would have to take this argument as well.
For example (and to illustrate one of my company's needs), others
have asked for a way to do a checkout that avoids directories of code
specific to a platform other than the one currently targeted. We
currently support 6 platforms, and might add others. Having
developers pull down all the code and prebuilt libraries for all 6
platforms is wasteful of both time and disk space. Some have
mentioned the work-around of separating all such code into totally
separate major subtrees. Don't most of you agree that this is
absurd? Good coding practices dictate organizing code by module, not
by platform. So it really does seem like a natural thing to want/
need to avoid irrelevant platform-specific directories sprinkled
throughout a source tree.
Given the filtering enhancement mentioned above, one could simply tag
their platform specific directories with say a "platform" property,
and then do:
checkout --filter 'platform=Mac'
to get a source tree applicable for building the mac version of a
product. For this to be really useful in this situation, the
nonexistence of the referenced property should be treated as a match
so that untagged directories are always grabbed. I guess it would be
nice to have the filter op able to work either way.
I would also use this feature for optimizing particular product
builds. Areas of the tree needed only to build specific products
could similarly be tagged appropriately. So, for example, to grab a
tree with only the files sufficient to build DeBabelizer (that's our
product) on Macintosh, you'd do:
checkout --filter 'platform=Mac, product=Debab'
or something similar.
I want this feature so bad, I'm thinking of implementing it
myself!!! There must be others out there that would greatly benefit
by this feature. What do others think? Is there any downside to
such a feature? Am I forgetting something that would make this
feature "evil"?
Thanks in advance for any valuable discussion that might ensue.
Steve Johnson
Equilibrium
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Checkout filtering wanted
Posted by Ben Collins-Sussman <su...@collab.net>.
On Sep 8, 2005, at 1:49 PM, Steve Johnson wrote:
> Hi Ben,
>
> Without knowing the details...Yes yes, it probably would. I'm
> salivating already! Is there a feature entry in the database for
> that work? I looked for such a feature, but didn't manage to find
> one.
>
> I'm glad to hear that -nonrecursive will get fixed. (When??)
Ask Ben Reser about it on the dev@ list. Maybe he needs some
prodding. :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Checkout filtering wanted
Posted by Steve Johnson <st...@equilibrium.com>.
Hi Ben,
Without knowing the details...Yes yes, it probably would. I'm
salivating already! Is there a feature entry in the database for that
work? I looked for such a feature, but didn't manage to find one.
I'm glad to hear that -nonrecursive will get fixed. (When??) With
that fix, one could then write external scripts to effect the
behavior I want. Still, I'd much rather see this as a solid feature
in svn.
Thanks,
S
>
> On Sep 8, 2005, at 1:03 PM, Steve Johnson wrote:
> >
> > I want this feature so bad, I'm thinking of implementing it
> > myself!!! There must be others out there that would greatly
> > benefit by this feature. What do others think? Is there any
> > downside to such a feature? Am I forgetting something that would
> > make this feature "evil"?
>
> Somebody is currently working on making 'svn checkout --nonrecursive'
> work as expected, and then creating a UI for explicitly including or
> excluding subdirectories as needed. That would solve the same
> problem, wouldn't it?
>
>
>
> --
> www.collab.net <> CollabNet | Distributed Development On Demand
>
>
>
>
Re: Checkout filtering wanted
Posted by Ximon Eighteen <xi...@int.greenpeace.org>.
> The property based filtering I propose seems to me to be a more
> powerful, and yet also simpler and more elegant a solution. It would
> be capable of doing complex filtering without the need to write
> external tools. The name-based mechanism could even be folded into the
> --filter mechanism by having a "meta property" that specified the
> object's name or path, thereby allowing queries on names as well.
I also like the fact that the property stays with the repository though I
haven't come up with a use case for this that couldn't be done some other way
yet. I guess I like it because it makes the repository self documenting.
--
-----------------------------------------------------------------------
Ximon Eighteen [desk: +31 (0)20 718 2134 reception: +31 (0)20 718 2000]
IT, Greenpeace International System Support Engineer
Ottho Heldringstraat 5, 1066 AZ Amsterdam & Application Developer
-----------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Checkout filtering wanted
Posted by Steve Johnson <st...@equilibrium.com>.
I agree with Ximon. Again, not knowing anything about the feature
other than what Ben mentioned...having to explicitly specify
directories to check out will either a) be used only for very trivial
and temporary filtering, or b) be something that would be utilized by
a wrapper mechanism that would somehow create on the fly the list of
directories to check out based on external criteria (often utilizing
svn's properties).
The property based filtering I propose seems to me to be a more
powerful, and yet also simpler and more elegant a solution. It would
be capable of doing complex filtering without the need to write
external tools. The name-based mechanism could even be folded into
the --filter mechanism by having a "meta property" that specified the
object's name or path, thereby allowing queries on names as well.
Per Ben's suggestion, I'm going to try to speak with Ben Reser about
this.
Steve
> > Somebody is currently working on making 'svn checkout --
> nonrecursive'
> > work as expected, and then creating a UI for explicitly including or
> > excluding subdirectories as needed. That would solve the same
> problem,
> > wouldn't it?
>
> I think it would solve the problem, but in a less useful way. Let
> me see if
> you agree with my thinking.
>
> With the proposal for --filter all that the client performing the
> checkout
> needs to know is that they want a mac osx version for instance.
>
> With the solution you refer to isn't it true that the client would
> need to
> fully specify the set of directories to include and exclude, and that
> therefore introduces potential for checking out the wrong set of
> directories
> for a given platform?
>
> If the 'UI for explicitly...' allows the client to pull the set of
> directories
> from a subversion property somehow then that would be closer to
> what Steve
> Johnson asked for but would not indicate what the directory set is
> for whereas
> the simple property 'arch=i386' requires less knowledge of the
> repository
> contents and labels the directory/file set into the bargain.
>
> Hmmm, just realised. The non recursive UI solution could work if it
> means that
> the set of directories/files could be specified by a script and the
> clients
> run the script or reference a file containing the required paths,
> and the
> file/script came from the repository and was named something like
> mac_osx.sh
> or mac_osx.paths or some such.
>
> Ximon
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
Re: Checkout filtering wanted
Posted by Ximon Eighteen <xi...@int.greenpeace.org>.
> Somebody is currently working on making 'svn checkout --nonrecursive'
> work as expected, and then creating a UI for explicitly including or
> excluding subdirectories as needed. That would solve the same problem,
> wouldn't it?
I think it would solve the problem, but in a less useful way. Let me see if
you agree with my thinking.
With the proposal for --filter all that the client performing the checkout
needs to know is that they want a mac osx version for instance.
With the solution you refer to isn't it true that the client would need to
fully specify the set of directories to include and exclude, and that
therefore introduces potential for checking out the wrong set of directories
for a given platform?
If the 'UI for explicitly...' allows the client to pull the set of directories
from a subversion property somehow then that would be closer to what Steve
Johnson asked for but would not indicate what the directory set is for whereas
the simple property 'arch=i386' requires less knowledge of the repository
contents and labels the directory/file set into the bargain.
Hmmm, just realised. The non recursive UI solution could work if it means that
the set of directories/files could be specified by a script and the clients
run the script or reference a file containing the required paths, and the
file/script came from the repository and was named something like mac_osx.sh
or mac_osx.paths or some such.
Ximon
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Checkout filtering wanted
Posted by Ben Collins-Sussman <su...@collab.net>.
On Sep 8, 2005, at 1:03 PM, Steve Johnson wrote:
>
> I want this feature so bad, I'm thinking of implementing it
> myself!!! There must be others out there that would greatly
> benefit by this feature. What do others think? Is there any
> downside to such a feature? Am I forgetting something that would
> make this feature "evil"?
Somebody is currently working on making 'svn checkout --nonrecursive'
work as expected, and then creating a UI for explicitly including or
excluding subdirectories as needed. That would solve the same
problem, wouldn't it?
--
www.collab.net <> CollabNet | Distributed Development On Demand
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Checkout filtering wanted
Posted by Steve Johnson <st...@equilibrium.com>.
Hi Ximon,
Thanks for the feedback. The issues you raise are right on. Many of
them were going through my head as I wrote my email. I figured I was
just laying the basic idea out there, and that the details could come
later. In fact, I wonder if others more familiar with svn might be
better suited than I to spec the exact feature. I mainly just know
what behavior I need from the system.
I'm anxious to learn more about the feature Ben sited in his
response. Maybe something that will solve this problem is already in
the works!
Take care,
Steve
> Steve Johnson wrote:
> > Greetings,
> >
> > We've been playing with Subversion for a few months, and have just
> > taken the plunge and converted over from CVS. Svn is generally
> a huge
> > improvement over CVS. We do, however, have a few key features that
> > we'd like to see implemented. I'd like to discuss the first of
> these...
> >
> > I've perused the mailing list and have read discussion threads
> > regarding a number of requests/needs for checkout filtering.
> From what
> > I've seen, however, none of these discussions led to any real
> interest
> > in actually implementing a filtering mechanism. I'm wondering
> why this
> > is, and I'm hoping to be able to change this.
> >
> > Subversion's arbitrary properties would seem to me to lead
> naturally to
> > a filtering mechanism. What I want to see implemented is:
> >
> > checkout --filter '<name><op><value>'
> >
> > where <name> is a subversion property name, <value> is a value to
> > compare with the value of the <name> property at each node of a
> tree,
> > and <op> defines the comparison. Only when the comparison
> succeeded on
> > a versioned object would that object be checked out. For our
> purposes,
> > this check could be done only on directories. For generality, I
> could
> > see extending it to files as well. The 'update' action would
> have to
> > take this argument as well.
> >
> > For example (and to illustrate one of my company's needs),
> others have
> > asked for a way to do a checkout that avoids directories of code
> > specific to a platform other than the one currently targeted. We
> > currently support 6 platforms, and might add others. Having
> developers
> > pull down all the code and prebuilt libraries for all 6
> platforms is
> > wasteful of both time and disk space. Some have mentioned the
> > work-around of separating all such code into totally separate major
> > subtrees. Don't most of you agree that this is absurd? Good
> coding
> > practices dictate organizing code by module, not by platform.
> So it
> > really does seem like a natural thing to want/ need to avoid
> irrelevant
> > platform-specific directories sprinkled throughout a source tree.
> >
> > Given the filtering enhancement mentioned above, one could simply
> tag
> > their platform specific directories with say a "platform"
> property, and
> > then do:
> >
> > checkout --filter 'platform=Mac'
> >
> > to get a source tree applicable for building the mac version of a
> > product. For this to be really useful in this situation, the
> > nonexistence of the referenced property should be treated as a
> match so
> > that untagged directories are always grabbed. I guess it would
> be nice
> > to have the filter op able to work either way.
> >
> > I would also use this feature for optimizing particular product
> > builds. Areas of the tree needed only to build specific
> products could
> > similarly be tagged appropriately. So, for example, to grab a tree
> > with only the files sufficient to build DeBabelizer (that's our
> > product) on Macintosh, you'd do:
> >
> > checkout --filter 'platform=Mac, product=Debab'
>
> I like the suggestion at first glance. You'd need to specify how
> this works in
> a little more detail though.
>
> Suggestions:
>
> 1. To match when the property does not exist use a command line
> option, e.g.
> --filter-match-missing-prop=yes|[no]. Default to no.
>
> 2. Do not use a comma between match specifications as it is not
> clear if the
> match specifications should be and'd or or'd. I'd expand the
> definition of the
> syntax to something like:-
>
> name := any valid property name, in quotes ("" or '') if
> includes spaces
> op := one of ==, !=, <, > (how do comparisons work? string?
> numeric?)
> value := a string value, in quotes if necessary
> combine:= one of &&, ||
> match := <name><op><value>
> filter := <match>(<combine><match>)*
>
> 3. No magic on commit, newly added files/directories must manually
> have
> properties set on them. This means it is possible to checkout using
> a filter
> and then commit files/dirs that don't match the filter. Does this
> matter? If
> so perhaps add a --enforce-filter-match option to commit?
>
> 4. The filter value should allow shell expansion on the client
> before being
> interpreted thereby allowing something like --filter 'arch=
> $HOSTTYPE' (on my
> system $HOSTTYPE is i386). I don't know if it's even possible to
> prevent
> client side expansion of shell variables so maybe this is a moot
> point.
>
> Ximon
>
>