You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Karl Fogel <kf...@newton.ch.collab.net> on 2002/05/17 20:52:20 UTC
On the Infinitude of Module Solutions.
I've renamed this thread; it had been "Re: svn commit: rev 1957 -
trunk/subversion/include trunk/subversion/libsvn_wc", which wasn't
exactly eye-catching :-).
I'd like to propose a way out of our module system dilemma. Right
now, the two horns of the dilemma are:
1) Write a description-based module system, where a description
file (or property, whatever) provides a list of external
"things" that are included in the module. Checking out the
description results in the external things being checked out
too; updates and commits account for them, insert hand-waving
here, to be supplemented by concrete description below.
...OR...
2) Make a first-class "reference node" type, recognized by both
filesystem and client, that points to external things. When you
check out a reference, it drags in the thing it points to, while
still remaining aware that its a reference (so that if the
reference changes, all the appropriate changes are made).
Updates and commits account for them. Insert hand-waving here,
to be supplemented by more hand-waving below.
These two solutions are not mutually exclusive. There's no reason we
couldn't implement both, except that it might cause some confusion for
users who have to decide which way to do a module.
The reference node solution has a certain geek-elegance appeal. I
find myself instantly attracted to it. It's simple to explain the
concept, and it's easy to see, at a high level, *how* to do almost
anything one might want to do with modules via this method. Indeed, I
must say outright that I favor the reference solution hands-down.
So why am I about to start on solution number 1? :-)
A few reasons. First, the simplicity of references is deceptive.
They are conceptually easy, yes, but good concrete interfaces are
actually hard to come up with. One always has to distinguish between
acting on the reference versus acting on the thing it refers to, and
its easy to end up with inconsistent situations (especially at
reference creation time, and reference changing time). Here I'm going
to hand-wave a bit, because I don't have time to write up examples of
all these situations. But you can come up with the problems yourself
pretty easily. The recipe is simple: first come up with what *seems*
like a good interface for reference creation and modification, and
then try to think of situations where it breaks down. You'll find
they're very easy to imagine :-). (Unfortunately, not all of the
discussion about this has happened on the list; some has been on IRC,
some has been on phone calls. So these objections might look like
they're coming out of the blue sky, when in fact they're products of
discussion with others.)
A second reason: references involve changes to both server and client
side. Creating a new node type involves lots of little changes all
over the code. Not a showstopper, just very time-consuming.
Finally, I find that when I talk to people about how modules will be
used, I get (roughly speaking) more different answers than there are
particles in the universe. So I don't think that we're going to fully
nail this the first time, whether we use references or descriptions.
Given that, I want to start with descriptions, since they're easier to
implement, and have much less drastic repercussions on the rest of
Subversion.
Think of solution (1) as training wheels for solution (2). I don't
believe we can do a good job of (2) in a schedulable way, because it
involves huge unresolved questions about both interfaces and
semantics. Those questions will be partially resolved through
discussion and thought, and partly through experience -- experience
which using (1) can help us acquire. In the meantime, (1) gives us a
useable module system, perhas one that will (in practice) suffice for
most purposes.
I honestly don't know whether (1) will end up being throwaway code
someday. If references eventually turn out to be everything we could
hope for, then it will. But that's not at all a sure thing. I think
there's a possibility that (1) will turn out to satisfy most needs.
Please remember that this is *not* a vote against references. Anyone
who wants to start work on a reference node system right now should do
so. There's no reason the two can't develop in parallel. But I
personally am not planning to devote coding time to it before Alpha
(and therefore not before 1.0). It needs a LOT more discussion and
design cycles... And in the meantime, we need some sort of module
solution.
Here is some more detail about (1):
It's basically what's described in issue #517, except Mike Pilato
suggested the following modification: instead of a file, have a
svn:references
(or "svn:external-sources" or whatever) property that can be put on a
directory. The value of the property is a description of additional
items to be checked out into subdirectories (the format of the
description is the same as the file format described in issue #517).
So during a checkout, if you encounter this property set on directory
"A/", then each item listed in the property's value will be checked
out as a disconnected subdir of A/.
Here, "disconnected" means that the subdirs do not get entries in
"A/.svn/entries". They're simply there. Whether commits and updates
recurse into such subdirs is something we can decide (maybe
conditionally, if they're listed in the property). When the property
changes, the subdirs change too (though paths to local mods remain, as
per usual). We can either interpret the property at propset time, or
(better) at update time, with commits blocking until the property has
been interpreted.
(This is still not as concrete a description as I would like, but
there are physical barriers to how much writing one can do in a day
:-) .)
Anyway. This supports multiple-repository modules (something that I
think *is* going to be important right away) simply. It adds no new
node types to Subversion. It doesn't affect the server code. It's
easy to implement and gets us useful functionality right away.
I've always been a little wary of the "worse is better" philosophy,
but this is one case where I think it will help us to do the worse
thing first. References might be super-elegant, but there's just no
way we can have them done by Alpha without sacrificing either time or
design quality.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Re: On the Infinitude of Module Solutions.
Posted by Bill Tutt <ra...@lyra.org>.
Just as an FYI, the short note I wrote earlier was not to indicate
agreement (or indeed disagreement) with Mike's new idea of how to store
the data, it was more agreeing that due to schedule/risk issues that not
having a refernce node is a valid choice to make for 1.0. (Seeing as
that touching schema, etc. is a very annoying and very possibly
destabalizing change to make now.)
I hope to compose a more detailed reply to Karl and Greg's emails
sometime tonight.
Bill
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: On the Infinitude of Module Solutions.
Posted by Greg Stein <gs...@lyra.org>.
On Fri, May 17, 2002 at 03:52:20PM -0500, Karl Fogel wrote:
>...
> It's basically what's described in issue #517, except Mike Pilato
> suggested the following modification: instead of a file, have a
>
> svn:references
>
> (or "svn:external-sources" or whatever) property that can be put on a
> directory. The value of the property is a description of additional
> items to be checked out into subdirectories (the format of the
Right. So call it 'svn:extra-items' or something. svn:references is way to
geeky. It does not describe its end-user functionality.
> description is the same as the file format described in issue #517).
> So during a checkout, if you encounter this property set on directory
> "A/", then each item listed in the property's value will be checked
> out as a disconnected subdir of A/.
>
> Here, "disconnected" means that the subdirs do not get entries in
> "A/.svn/entries". They're simply there. Whether commits and updates
> recurse into such subdirs is something we can decide (maybe
I don't think we need to worry about the behavior at all. The behavior is
defined by our mixed-repository working copy model.
In short, "do the checkouts, the working copy will define the relationships
between the directories." There is no *new* behavior [beyond the behavior we
state for mixed-reps].
For example, I would think that an 'svn update' will update all working
copies under '.', whether they are from one repository or many. Note that
this is exactly the same behavior as CVS: the dirs get "hooked into" their
parents, and an update or commit will just incorporate them into their
operation.
Don't make it configurable whether it gets hooked or not. That will just
cause users no end of confusion (and this happens to me occasionally, where
a CVS working copy doesn't get hooked into its parent; thankfully, I know
how to edit CVS/Entries to manually hook it).
>...
> Anyway. This supports multiple-repository modules (something that I
> think *is* going to be important right away) simply. It adds no new
It absolutely will need to. I think of projects as one-per-repository, but
many other people on this list believe many-projects-per-repos. That isn't a
problem, and Subversion will support both models quite easily (it is just
policy for the repository/project admins).
BUT! (and you knew that was coming...)
What about the "cross-ownership" boundary? Subversion is composed using a
module description like this:
trunk http://svn.collab.net/repos/svn/trunk
trunk/apr http://svn.apache.org/repos/apr/apr/trunk
trunk/apr-util http://svn.apache.org/repos/apr/apr-util/trunk
trunk/neon http://svn.webdav.org/repos/neon/trunk
You're looking at three different realms of ownership. For our example
population of 1, we're batting 1.000 for multi-repository modules.
I will also note that the "reference node" concept cannot support module
descriptions similar to above without a TON of work. The "place URL into
REF1/FOO" isn't possible. You would have to have ownership of the target of
REF1. In the specific example above, we could define the references to
apr(-util) and Neon within svn/trunk. However, consider the case where some
people prefer to have 'db' in their working copy. There is no way [in my
world] where I'd support adding that reference to our mainline trunk. Thus,
those developers would be screwed, unless they could define a target for
trunk/db/ (which the "description file" approach can, but the refnode style
cannot).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: On the Infinitude of Module Solutions.
Posted by cm...@collab.net.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> > Even though being able to have an actual tag concept tempts me greatly.
> > :)
>
> Keep that stick out of my anthill, man! :-)
No, let him stick it in here...we just won't tell him that we're
actually termites!
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: On the Infinitude of Module Solutions.
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
"Bill Tutt" <ra...@lyra.org> writes:
> No arguments from this end of the continent. I even expected you to say
> something that. The beauty of the parsed file format is that the
> command line can still preserve the same interface in V2.0 even if the
> schema now has a reference node concept.
Cool; hoped you'd see it that way.
> Even though being able to have an actual tag concept tempts me greatly.
> :)
Keep that stick out of my anthill, man! :-)
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: On the Infinitude of Module Solutions.
Posted by Bill Tutt <ra...@lyra.org>.
No arguments from this end of the continent. I even expected you to say
something that. The beauty of the parsed file format is that the
command line can still preserve the same interface in V2.0 even if the
schema now has a reference node concept.
Even though being able to have an actual tag concept tempts me greatly.
:)
Keeper of the "A copy is not a branch" flame war,
Bill
----
Do you want a dangerous fugitive staying in your flat?
No.
Well, don't upset him and he'll be a nice fugitive staying in your flat.
> -----Original Message-----
> From: Karl Fogel [mailto:kfogel@newton.ch.collab.net]
> Sent: Friday, May 17, 2002 1:52 PM
> To: Bill Tutt
> Cc: peter.westlake@arm.com; dev@subversion.tigris.org
> Subject: On the Infinitude of Module Solutions.
>
> I've renamed this thread; it had been "Re: svn commit: rev 1957 -
> trunk/subversion/include trunk/subversion/libsvn_wc", which wasn't
> exactly eye-catching :-).
>
> I'd like to propose a way out of our module system dilemma. Right
> now, the two horns of the dilemma are:
>
> 1) Write a description-based module system, where a description
> file (or property, whatever) provides a list of external
> "things" that are included in the module. Checking out the
> description results in the external things being checked out
> too; updates and commits account for them, insert hand-waving
> here, to be supplemented by concrete description below.
>
> ...OR...
>
> 2) Make a first-class "reference node" type, recognized by both
> filesystem and client, that points to external things. When you
> check out a reference, it drags in the thing it points to, while
> still remaining aware that its a reference (so that if the
> reference changes, all the appropriate changes are made).
> Updates and commits account for them. Insert hand-waving here,
> to be supplemented by more hand-waving below.
>
> These two solutions are not mutually exclusive. There's no reason we
> couldn't implement both, except that it might cause some confusion for
> users who have to decide which way to do a module.
>
> The reference node solution has a certain geek-elegance appeal. I
> find myself instantly attracted to it. It's simple to explain the
> concept, and it's easy to see, at a high level, *how* to do almost
> anything one might want to do with modules via this method. Indeed, I
> must say outright that I favor the reference solution hands-down.
>
> So why am I about to start on solution number 1? :-)
>
> A few reasons. First, the simplicity of references is deceptive.
> They are conceptually easy, yes, but good concrete interfaces are
> actually hard to come up with. One always has to distinguish between
> acting on the reference versus acting on the thing it refers to, and
> its easy to end up with inconsistent situations (especially at
> reference creation time, and reference changing time). Here I'm going
> to hand-wave a bit, because I don't have time to write up examples of
> all these situations. But you can come up with the problems yourself
> pretty easily. The recipe is simple: first come up with what *seems*
> like a good interface for reference creation and modification, and
> then try to think of situations where it breaks down. You'll find
> they're very easy to imagine :-). (Unfortunately, not all of the
> discussion about this has happened on the list; some has been on IRC,
> some has been on phone calls. So these objections might look like
> they're coming out of the blue sky, when in fact they're products of
> discussion with others.)
>
> A second reason: references involve changes to both server and client
> side. Creating a new node type involves lots of little changes all
> over the code. Not a showstopper, just very time-consuming.
>
> Finally, I find that when I talk to people about how modules will be
> used, I get (roughly speaking) more different answers than there are
> particles in the universe. So I don't think that we're going to fully
> nail this the first time, whether we use references or descriptions.
> Given that, I want to start with descriptions, since they're easier to
> implement, and have much less drastic repercussions on the rest of
> Subversion.
>
> Think of solution (1) as training wheels for solution (2). I don't
> believe we can do a good job of (2) in a schedulable way, because it
> involves huge unresolved questions about both interfaces and
> semantics. Those questions will be partially resolved through
> discussion and thought, and partly through experience -- experience
> which using (1) can help us acquire. In the meantime, (1) gives us a
> useable module system, perhas one that will (in practice) suffice for
> most purposes.
>
> I honestly don't know whether (1) will end up being throwaway code
> someday. If references eventually turn out to be everything we could
> hope for, then it will. But that's not at all a sure thing. I think
> there's a possibility that (1) will turn out to satisfy most needs.
>
> Please remember that this is *not* a vote against references. Anyone
> who wants to start work on a reference node system right now should do
> so. There's no reason the two can't develop in parallel. But I
> personally am not planning to devote coding time to it before Alpha
> (and therefore not before 1.0). It needs a LOT more discussion and
> design cycles... And in the meantime, we need some sort of module
> solution.
>
> Here is some more detail about (1):
>
> It's basically what's described in issue #517, except Mike Pilato
> suggested the following modification: instead of a file, have a
>
> svn:references
>
> (or "svn:external-sources" or whatever) property that can be put on a
> directory. The value of the property is a description of additional
> items to be checked out into subdirectories (the format of the
> description is the same as the file format described in issue #517).
> So during a checkout, if you encounter this property set on directory
> "A/", then each item listed in the property's value will be checked
> out as a disconnected subdir of A/.
>
> Here, "disconnected" means that the subdirs do not get entries in
> "A/.svn/entries". They're simply there. Whether commits and updates
> recurse into such subdirs is something we can decide (maybe
> conditionally, if they're listed in the property). When the property
> changes, the subdirs change too (though paths to local mods remain, as
> per usual). We can either interpret the property at propset time, or
> (better) at update time, with commits blocking until the property has
> been interpreted.
>
> (This is still not as concrete a description as I would like, but
> there are physical barriers to how much writing one can do in a day
> :-) .)
>
> Anyway. This supports multiple-repository modules (something that I
> think *is* going to be important right away) simply. It adds no new
> node types to Subversion. It doesn't affect the server code. It's
> easy to implement and gets us useful functionality right away.
>
> I've always been a little wary of the "worse is better" philosophy,
> but this is one case where I think it will help us to do the worse
> thing first. References might be super-elegant, but there's just no
> way we can have them done by Alpha without sacrificing either time or
> design quality.
>
> -K
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org