You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Branko Čibej <br...@xbc.nu> on 2001/12/29 11:12:09 UTC
We have competition ...
-------- Original Message --------
Subject: Re: version control
Date: Thu, 27 Dec 2001 11:12:01 -0800 (PST)
From: Tom Lord <lo...@emf.net>
To: gcc@gcc.gnu.org
I hope not to abuse gcc@gcc.gnu.org as a revision control discussion
list, but Florian's comment touches on an aspect of `arch' that
I thought a lot about GCC while writing. In other words, this feature
was designed partly in response to watching how CVS is used by GCC
developers:
Florian Weimer:
>Tom Lord <lo...@emf.net> writes:
>> There are alternatives to Subversion [which also handle
>> file and directory renaming well]. See:
>>
>> http://www.regexps.com/super-secret/arch.html
>>
>> arch features distributed revision control (e.g. form
>> branches in your own private repository) and features
>> for merging that are fancier than what Subversion offers.
> Aegis offers something in this direction, too, although it lacks
> automation of repository synchronization.
arch doesn't "synchronize" repositories. Instead, there is a global
(worldwide) name-space of branches and revisions. Suppose that the
main-line of GCC development is in repository A. Any developer can create
a repository B and start a branch in B that just points to (not copies)
an ancestor from repository A. While working in B, you can update against
A, for example. From a user interface perspective, A and B appear to be
one big repository.
The existence of B imposes no administrative overhead on the owner of A.
The branching is symmetric: if B is made readable via FTP, users of A can
update against B, too.
If several developers do this in parallel, creating repositories, C, D,
E,...
all of those repositories support mutual branching and merging too.
I've also tried to make sure that the drudge-work of merging can be
farmed out and shared. A developer working on B can prepare (in B) the
merge of A and B in such a way that:
1. Whoever gets to review and commit changes from B to A
will have a clean (conflictless) merge. This isn't magic,
conflicts are conflicts, but the point is that the
activity of resolving conflicts can be neatly separated from the
activity of modifying the main branch.
2. The patch set for that merge comes in an easy-to-review
format (there's even a trivial but very handy web interface
for browsing the diffs between two revisions).
As I recall, Aegis' primary claim to fame is process automation: you
can define and impose check-in pre-conditions (e.g. that sources must
pass certain tests). arch doesn't directly provide that -- but such a
feature could easily be layered on top of arch (a few K of code).
arch's virtues for that purpose are (in theory) reliability and a nice
shell interface: repositories don't get stuck in states requiring someone
to fix them by hand; everything has a nice CLI with reasonable input
and output formats.
What I've thought might make sense is to (further) automate many of
the duties of the SC and release manager: (further) automate snapshot
and release cutting; automate distributed platform testing and
reporting; create a web-visible RC-driven queue of patches needing
approval; etc. Such automation looks simple to do, though getting the
details right for GCC would require some interaction with SC members
and other developers (and I am not entirely up-to-date in my knowledge
of where GCC process automation currently stands).
-t
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: More competition, and a specification rant. Was Re: We have competition ...
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Mark, when Stellation (& its specs) have gone irrevocably open source,
can you let us know at this list? We'll want to take a look at the Z
spec, and other things no doubt.
Thanks,
-Karl
"Mark C. Chu-Carroll" <mc...@watson.ibm.com> writes:
> A lot of what they're doing sounds quite similar to what we're working
> on in our project. (Where we translates as the Stellation project, your
> friendly competitors who used to be called Synedra, and before that
> Coven. Yes, we've been renamed again. This time we went through the
> legal process of formally clearing the name, so we're absolutely never
> going to rename the project again. Really.)
>
> And we are going to go open-source, probably sometime in February. (We'd
> hoped to do it a lot earlier, but getting through the politics to get
> permission to open took a lot longer than we'd expected.)
>
> Anyway... The basic idea of lazy distributed branching, which is what it
> sounds like Arch is trying to do, is something that Stellation will also
> do. (And we have a spec written in Z that describes branching and
> reconciliation.) It's a really useful, important notion. When you're
> working on a big project, particularly with project-oriented versioning
> (which we all - arch, Subversion, and Stellation are implementing in
> some form), it's very difficult to properly coordinate work between
> widely distributed people. Getting branching right is the first step,
> but requiring a single global repository is still a serious problem.
> Distributed repository replicas, with lazy branching and automated
> reconciliation and merging makes a dramatic difference in how well the
> system works. I'd strongly encourage you guys to take a look at the code
> for Arch and Stellation (when we get it out), and put something like
> that in to SubVersion.
>
> I'll also take this opportunity to push one of my personal agendas... If
> you're going to implement something like replication, take the time to
> learn a specification language, and *write the spec*. Things like
> replication/reconciliation are incredibly complicated, and it's really
> easy to get things wrong. Without a really thorough understanding of
> exactly what you want the system to do, you can't get it right; and with
> something as complicated as replication/reconciliation, it's close to
> impossible to be sure that you really do understand it unless you've
> worked it through. For Stellation, I've got a specification of the base
> system in Z (an excellent specification language for describing systems
> that are predominantly transformation based). I've got a *partial* spec
> of replication written, and I'm working on finishing it.
>
> I know that proposing the idea of writing formal mathematical
> specifications is not exactly a popular notion, but it really makes a
> difference. The specifications that I wrote saved me *at least* three
> months of work; quite possible a lot longer.
>
> (For an example of the problems we saved... We're implemented using an
> RDB (postgresql) for storage. Our original design was a multi-layer
> filesystem. We were going to implement a simple filesystem using the
> database. Then we were going to use the DBFS to implement a versioned
> filesystem. Then we were going to build the real system over the version
> filesystem. First, It turned out that the method we were planning on
> using for mapping branches and merges onto the versioned filesystem was
> wrong; second,we realized that the versioned filesystem model that we
> were using would cause consistency errors when trying to merge certain
> kinds of changes; and third, we were creating redundant data in the name
> of doing good merges that made it *harder* to get merges correct.)
>
> It would have taken months of effort to correct these problems after we
> implemented them.
>
> <MC>
>
> On Sat, 2001-12-29 at 06:12, Branko Èibej wrote:
> >
> >
> > -------- Original Message --------
> > Subject: Re: version control
> > Date: Thu, 27 Dec 2001 11:12:01 -0800 (PST)
> > From: Tom Lord <lo...@emf.net>
> > To: gcc@gcc.gnu.org
> >
> >
> > I hope not to abuse gcc@gcc.gnu.org as a revision control discussion
> > list, but Florian's comment touches on an aspect of `arch' that
> > I thought a lot about GCC while writing. In other words, this feature
> > was designed partly in response to watching how CVS is used by GCC
> > developers:
> >
> > Florian Weimer:
> > >Tom Lord <lo...@emf.net> writes:
> >
> > >> There are alternatives to Subversion [which also handle
> > >> file and directory renaming well]. See:
> > >>
> > >> http://www.regexps.com/super-secret/arch.html
> > >>
> > >> arch features distributed revision control (e.g. form
> > >> branches in your own private repository) and features
> > >> for merging that are fancier than what Subversion offers.
> >
> > > Aegis offers something in this direction, too, although it lacks
> > > automation of repository synchronization.
> >
> > arch doesn't "synchronize" repositories. Instead, there is a global
> > (worldwide) name-space of branches and revisions. Suppose that the
> > main-line of GCC development is in repository A. Any developer can create
> > a repository B and start a branch in B that just points to (not copies)
> > an ancestor from repository A. While working in B, you can update against
> > A, for example. From a user interface perspective, A and B appear to be
> > one big repository.
> >
> > The existence of B imposes no administrative overhead on the owner of A.
> >
> > The branching is symmetric: if B is made readable via FTP, users of A can
> > update against B, too.
> >
> > If several developers do this in parallel, creating repositories, C, D,
> > E,...
> > all of those repositories support mutual branching and merging too.
> >
> > I've also tried to make sure that the drudge-work of merging can be
> > farmed out and shared. A developer working on B can prepare (in B) the
> > merge of A and B in such a way that:
> >
> > 1. Whoever gets to review and commit changes from B to A
> > will have a clean (conflictless) merge. This isn't magic,
> > conflicts are conflicts, but the point is that the
> > activity of resolving conflicts can be neatly separated from the
> > activity of modifying the main branch.
> >
> > 2. The patch set for that merge comes in an easy-to-review
> > format (there's even a trivial but very handy web interface
> > for browsing the diffs between two revisions).
> >
> > As I recall, Aegis' primary claim to fame is process automation: you
> > can define and impose check-in pre-conditions (e.g. that sources must
> > pass certain tests). arch doesn't directly provide that -- but such a
> > feature could easily be layered on top of arch (a few K of code).
> > arch's virtues for that purpose are (in theory) reliability and a nice
> > shell interface: repositories don't get stuck in states requiring someone
> > to fix them by hand; everything has a nice CLI with reasonable input
> > and output formats.
> >
> > What I've thought might make sense is to (further) automate many of
> > the duties of the SC and release manager: (further) automate snapshot
> > and release cutting; automate distributed platform testing and
> > reporting; create a web-visible RC-driven queue of patches needing
> > approval; etc. Such automation looks simple to do, though getting the
> > details right for GCC would require some interaction with SC members
> > and other developers (and I am not entirely up-to-date in my knowledge
> > of where GCC process automation currently stands).
> >
> > -t
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> > For additional commands, e-mail: dev-help@subversion.tigris.org
> --
> Mark Craig Chu-Carroll, IBM T.J. Watson Research Center
> <mc...@watson.ibm.com>
> *** The (recently renamed) Stellation project:
> *** http://domino.research.ibm.com/synedra/synedra.nsf
> *** GPG Public key available at keyserver.net
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
More competition, and a specification rant. Was Re: We have
competition ...
Posted by "Mark C. Chu-Carroll" <mc...@watson.ibm.com>.
A lot of what they're doing sounds quite similar to what we're working
on in our project. (Where we translates as the Stellation project, your
friendly competitors who used to be called Synedra, and before that
Coven. Yes, we've been renamed again. This time we went through the
legal process of formally clearing the name, so we're absolutely never
going to rename the project again. Really.)
And we are going to go open-source, probably sometime in February. (We'd
hoped to do it a lot earlier, but getting through the politics to get
permission to open took a lot longer than we'd expected.)
Anyway... The basic idea of lazy distributed branching, which is what it
sounds like Arch is trying to do, is something that Stellation will also
do. (And we have a spec written in Z that describes branching and
reconciliation.) It's a really useful, important notion. When you're
working on a big project, particularly with project-oriented versioning
(which we all - arch, Subversion, and Stellation are implementing in
some form), it's very difficult to properly coordinate work between
widely distributed people. Getting branching right is the first step,
but requiring a single global repository is still a serious problem.
Distributed repository replicas, with lazy branching and automated
reconciliation and merging makes a dramatic difference in how well the
system works. I'd strongly encourage you guys to take a look at the code
for Arch and Stellation (when we get it out), and put something like
that in to SubVersion.
I'll also take this opportunity to push one of my personal agendas... If
you're going to implement something like replication, take the time to
learn a specification language, and *write the spec*. Things like
replication/reconciliation are incredibly complicated, and it's really
easy to get things wrong. Without a really thorough understanding of
exactly what you want the system to do, you can't get it right; and with
something as complicated as replication/reconciliation, it's close to
impossible to be sure that you really do understand it unless you've
worked it through. For Stellation, I've got a specification of the base
system in Z (an excellent specification language for describing systems
that are predominantly transformation based). I've got a *partial* spec
of replication written, and I'm working on finishing it.
I know that proposing the idea of writing formal mathematical
specifications is not exactly a popular notion, but it really makes a
difference. The specifications that I wrote saved me *at least* three
months of work; quite possible a lot longer.
(For an example of the problems we saved... We're implemented using an
RDB (postgresql) for storage. Our original design was a multi-layer
filesystem. We were going to implement a simple filesystem using the
database. Then we were going to use the DBFS to implement a versioned
filesystem. Then we were going to build the real system over the version
filesystem. First, It turned out that the method we were planning on
using for mapping branches and merges onto the versioned filesystem was
wrong; second,we realized that the versioned filesystem model that we
were using would cause consistency errors when trying to merge certain
kinds of changes; and third, we were creating redundant data in the name
of doing good merges that made it *harder* to get merges correct.)
It would have taken months of effort to correct these problems after we
implemented them.
<MC>
On Sat, 2001-12-29 at 06:12, Branko Čibej wrote:
>
>
> -------- Original Message --------
> Subject: Re: version control
> Date: Thu, 27 Dec 2001 11:12:01 -0800 (PST)
> From: Tom Lord <lo...@emf.net>
> To: gcc@gcc.gnu.org
>
>
> I hope not to abuse gcc@gcc.gnu.org as a revision control discussion
> list, but Florian's comment touches on an aspect of `arch' that
> I thought a lot about GCC while writing. In other words, this feature
> was designed partly in response to watching how CVS is used by GCC
> developers:
>
> Florian Weimer:
> >Tom Lord <lo...@emf.net> writes:
>
> >> There are alternatives to Subversion [which also handle
> >> file and directory renaming well]. See:
> >>
> >> http://www.regexps.com/super-secret/arch.html
> >>
> >> arch features distributed revision control (e.g. form
> >> branches in your own private repository) and features
> >> for merging that are fancier than what Subversion offers.
>
> > Aegis offers something in this direction, too, although it lacks
> > automation of repository synchronization.
>
> arch doesn't "synchronize" repositories. Instead, there is a global
> (worldwide) name-space of branches and revisions. Suppose that the
> main-line of GCC development is in repository A. Any developer can create
> a repository B and start a branch in B that just points to (not copies)
> an ancestor from repository A. While working in B, you can update against
> A, for example. From a user interface perspective, A and B appear to be
> one big repository.
>
> The existence of B imposes no administrative overhead on the owner of A.
>
> The branching is symmetric: if B is made readable via FTP, users of A can
> update against B, too.
>
> If several developers do this in parallel, creating repositories, C, D,
> E,...
> all of those repositories support mutual branching and merging too.
>
> I've also tried to make sure that the drudge-work of merging can be
> farmed out and shared. A developer working on B can prepare (in B) the
> merge of A and B in such a way that:
>
> 1. Whoever gets to review and commit changes from B to A
> will have a clean (conflictless) merge. This isn't magic,
> conflicts are conflicts, but the point is that the
> activity of resolving conflicts can be neatly separated from the
> activity of modifying the main branch.
>
> 2. The patch set for that merge comes in an easy-to-review
> format (there's even a trivial but very handy web interface
> for browsing the diffs between two revisions).
>
> As I recall, Aegis' primary claim to fame is process automation: you
> can define and impose check-in pre-conditions (e.g. that sources must
> pass certain tests). arch doesn't directly provide that -- but such a
> feature could easily be layered on top of arch (a few K of code).
> arch's virtues for that purpose are (in theory) reliability and a nice
> shell interface: repositories don't get stuck in states requiring someone
> to fix them by hand; everything has a nice CLI with reasonable input
> and output formats.
>
> What I've thought might make sense is to (further) automate many of
> the duties of the SC and release manager: (further) automate snapshot
> and release cutting; automate distributed platform testing and
> reporting; create a web-visible RC-driven queue of patches needing
> approval; etc. Such automation looks simple to do, though getting the
> details right for GCC would require some interaction with SC members
> and other developers (and I am not entirely up-to-date in my knowledge
> of where GCC process automation currently stands).
>
> -t
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
--
Mark Craig Chu-Carroll, IBM T.J. Watson Research Center
<mc...@watson.ibm.com>
*** The (recently renamed) Stellation project:
*** http://domino.research.ibm.com/synedra/synedra.nsf
*** GPG Public key available at keyserver.net
Re: We have competition ...
Posted by Ben Collins-Sussman <su...@collab.net>.
> >Tom Lord <lo...@emf.net> writes:
>
> >> There are alternatives to Subversion [which also handle
> >> file and directory renaming well]. See:
> >>
> >> http://www.regexps.com/super-secret/arch.html
> >>
> >> arch features distributed revision control (e.g. form
> >> branches in your own private repository) and features
> >> for merging that are fancier than what Subversion offers.
Wow, that URL has quite a detailed specification!
It sounds pretty neat.
Where's the code?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org