[merge tracking] Problems with merge info availability and merge interface

[Daniel Rall <dl...@collab.net>]

While working on implementing the merge tracking design, we've run
across a few areas where the design and the existing system aren't
great fits.  I've outlined a few of them below -- some Words of Wisdom
on how to proceed would be welcome.

Note: The most complete solution to many of these problems is to keep
a full, sync'd copy of all merge info in both WC and repository.
Considering that this would move Subversion from a centralized
repository towards a more distributed system, this is not something we
want to pursue (at least, not at this time). [1]

The complete TODO list of outstanding problems and tasks is tracked
here:
http://svn.collab.net/repos/svn/branches/merge-tracking/TODO


WC -> WC copy/move operations
-----------------------------
Such operations are important for refactoring tools (e.g. Eclipse,
IDEA, etc.).  Retaining support for operations offline is quite useful
in this regard (as you can still run'em on the train, when your
repository is down, etc.).

Preserving offline operation is difficult without replicating merge
info to the client's WC, and keeping that merge info sync'd with the
repository.  Usually, the client must contact the repository to get
the merge info -- both explicit/inherited, and reflected from all
revisions of the copy source at its current path -- it needs for the
parents of the copy source in order to have the complete information
that needs to be written about the children.  The formula's something
like:

  dest merge info = src merge info + revs of src at path

We need to use this formula on the client side because subsequent
merges may change the merge info for the destination path before a
commit to the repository can set the merge info reflected by the
copy's source path.

If the the copy source has no merge info, and the destination path is
a direct parent of the source path, then we don't need to move any
merge info (since it will still be gotten from the same parent), but
we still need to do history tracing to find the oldest revision at
which the copy source has been at its current path, so that it can be
included in the merge info for the destination.

This is not a correctness issue, but rather one of maintaining
disconnected operation in the face of merge info.


URL -> URL -> WC merge operations (merge test #19)
--------------------------------------------------
A useful operation which takes the difference between URL1@rX and
URL2@rY, and applies the delta between those trees to a WC.

The operation currently comes down as one big text delta between the
files.  We can't tell what revs it actually contains, nor how they
relate to the WC's merge info.  Because of this, we can't calculate
which portions of the delta to apply, and which to filter out (because
they're already represented by our WC).

We can preserve the existing behavior without using merge info (as a
worst case), but this is rather at odds with the goal of introducing
merge tracking, so we'd like to do something better.

One option is to send the target paths, plus any WC merge info
changes, to the repository when procuring the editor, and let the
editor filter out the merges which are already represented by the WC.


WC notifications from a merge operation
---------------------------------------
These notifications are received as callbacks indicating that an
interesting action has occurred on a path.

For merge tracking purposes, (at least) skips and conflicts must be
handled, both of which affect merge info set on the WC as the result
of a merge.  Conflicts may also involve entering a conflict resolution
mode, or necessitate only partial application of the requested merge
range to conflicted files.

Notifications of all types generate output.  Merging a set of revision
ranges in a loop may cause multiple notifications for a single WC
item.  This is a rather large difference from existing command-line
client output (and libsvn_client API behavior).

This is not a problem so much as question of what we want to do from
an interface perspective.



[1] While we could explore the option of sync'ing mergeinfo between WC
and repository, comparision to other VC systems suggests that this
would likely require additional restrictions on WCs (e.g. no separable
WCs unless we have one merge info DB for each directory, which would
be horrendous), and quite a bit of work.  It would be difficult to
make such a change in behavior compatible with previous versions of
Subversion.

Re: [merge tracking] Problems with merge info availability and merge interface

[Daniel Rall <dl...@collab.net>]

On Fri, 18 Aug 2006, Karl Fogel wrote:

> Daniel Rall <dl...@collab.net> writes:
> > Note: The most complete solution to many of these problems is to keep
> > a full, sync'd copy of all merge info in both WC and repository.
> > Considering that this would move Subversion from a centralized
> > repository towards a more distributed system, this is not something we
> > want to pursue (at least, not at this time). [1]
> 
> I don't have good answers for the problems you posed (sorry, just a
> time issue at the moment), but I did want to question the article of
> faith stated above.
> 
> Should a solution be rejected because it tends in the direction of a
> distributed system?  If solutions to current problems push Subversion
> in that direction, it might mean that's a useful direction to go.  And
> nothing will force us to go farther in that direction than we choose
> to, so why worry? :-)

Garrett responded:

> It's certainly an interesting idea...

If at all possible, I would prefer to avoid implementing distributed
repositories in conjunction with Merge Tracking, which is already a
very complex topic in its own right.

Karl wrote:

> >(The reasons given in [1] are understandable... at some point we may
> >have to ditch the detachable working copies, handy though they are.)

Garrett responded:

> Honestly, I'm starting to think that ditching detachable working
> copies is a reasonable step to take.  An awful lot of good ideas break
> down once you have them, and considering what the tradeoffs start to
> look like, I'm not sure it's worth it to keep the feature.

While I think it's possible to implement some degree of Merge Tracking
support on top of the detachable WCs concept, I am coming to the
conclusion that it'll end up being both *extremely* painful (due to
needless complexity in the WC code), and not nearly as complete as it
could otherwise be (e.g. it'll be difficult to retain merge info after
a merge without a commit, and a subsequent detaching of a WC which was
affected by the merge but didn't have the merge info recorded on it).


> (The reasons given in [1] are understandable... at some point we may
> have to ditch the detachable working copies, handy though they are.)
> 
> Gratuitous meta-comment courtesy of,
> -K
> 
> > The complete TODO list of outstanding problems and tasks is tracked
> > here:
> > http://svn.collab.net/repos/svn/branches/merge-tracking/TODO
> >
> >
> > WC -> WC copy/move operations
> > -----------------------------
> > Such operations are important for refactoring tools (e.g. Eclipse,
> > IDEA, etc.).  Retaining support for operations offline is quite useful
> > in this regard (as you can still run'em on the train, when your
> > repository is down, etc.).
> >
> > Preserving offline operation is difficult without replicating merge
> > info to the client's WC, and keeping that merge info sync'd with the
> > repository.  Usually, the client must contact the repository to get
> > the merge info -- both explicit/inherited, and reflected from all
> > revisions of the copy source at its current path -- it needs for the
> > parents of the copy source in order to have the complete information
> > that needs to be written about the children.  The formula's something
> > like:
> >
> >   dest merge info = src merge info + revs of src at path
> >
> > We need to use this formula on the client side because subsequent
> > merges may change the merge info for the destination path before a
> > commit to the repository can set the merge info reflected by the
> > copy's source path.
> >
> > If the the copy source has no merge info, and the destination path is
> > a direct parent of the source path, then we don't need to move any
> > merge info (since it will still be gotten from the same parent), but
> > we still need to do history tracing to find the oldest revision at
> > which the copy source has been at its current path, so that it can be
> > included in the merge info for the destination.
> >
> > This is not a correctness issue, but rather one of maintaining
> > disconnected operation in the face of merge info.
> >
> >
> > URL -> URL -> WC merge operations (merge test #19)
> > --------------------------------------------------
> > A useful operation which takes the difference between URL1@rX and
> > URL2@rY, and applies the delta between those trees to a WC.
> >
> > The operation currently comes down as one big text delta between the
> > files.  We can't tell what revs it actually contains, nor how they
> > relate to the WC's merge info.  Because of this, we can't calculate
> > which portions of the delta to apply, and which to filter out (because
> > they're already represented by our WC).
> >
> > We can preserve the existing behavior without using merge info (as a
> > worst case), but this is rather at odds with the goal of introducing
> > merge tracking, so we'd like to do something better.
> >
> > One option is to send the target paths, plus any WC merge info
> > changes, to the repository when procuring the editor, and let the
> > editor filter out the merges which are already represented by the WC.
> >
> >
> > WC notifications from a merge operation
> > ---------------------------------------
> > These notifications are received as callbacks indicating that an
> > interesting action has occurred on a path.
> >
> > For merge tracking purposes, (at least) skips and conflicts must be
> > handled, both of which affect merge info set on the WC as the result
> > of a merge.  Conflicts may also involve entering a conflict resolution
> > mode, or necessitate only partial application of the requested merge
> > range to conflicted files.
> >
> > Notifications of all types generate output.  Merging a set of revision
> > ranges in a loop may cause multiple notifications for a single WC
> > item.  This is a rather large difference from existing command-line
> > client output (and libsvn_client API behavior).
> >
> > This is not a problem so much as question of what we want to do from
> > an interface perspective.
> >
> >
> >
> > [1] While we could explore the option of sync'ing mergeinfo between WC
> > and repository, comparision to other VC systems suggests that this
> > would likely require additional restrictions on WCs (e.g. no separable
> > WCs unless we have one merge info DB for each directory, which would
> > be horrendous), and quite a bit of work.  It would be difficult to
> > make such a change in behavior compatible with previous versions of
> > Subversion.

Re: [merge tracking] Problems with merge info availability and merge interface

[Garrett Rooney <ro...@electricjellyfish.net>]

On 8/18/06, Karl Fogel <kf...@google.com> wrote:
> Daniel Rall <dl...@collab.net> writes:
> > Note: The most complete solution to many of these problems is to keep
> > a full, sync'd copy of all merge info in both WC and repository.
> > Considering that this would move Subversion from a centralized
> > repository towards a more distributed system, this is not something we
> > want to pursue (at least, not at this time). [1]
>
> I don't have good answers for the problems you posed (sorry, just a
> time issue at the moment), but I did want to question the article of
> faith stated above.
>
> Should a solution be rejected because it tends in the direction of a
> distributed system?  If solutions to current problems push Subversion
> in that direction, it might mean that's a useful direction to go.  And
> nothing will force us to go farther in that direction than we choose
> to, so why worry? :-)

It's certainly an interesting idea...

> (The reasons given in [1] are understandable... at some point we may
> have to ditch the detachable working copies, handy though they are.)

Honestly, I'm starting to think that ditching detachable working
copies is a reasonable step to take.  An awful lot of good ideas break
down once you have them, and considering what the tradeoffs start to
look like, I'm not sure it's worth it to keep the feature.

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: [merge tracking] Problems with merge info availability and merge interface

[Karl Fogel <kf...@google.com>]

Daniel Rall <dl...@collab.net> writes:
> Note: The most complete solution to many of these problems is to keep
> a full, sync'd copy of all merge info in both WC and repository.
> Considering that this would move Subversion from a centralized
> repository towards a more distributed system, this is not something we
> want to pursue (at least, not at this time). [1]

I don't have good answers for the problems you posed (sorry, just a
time issue at the moment), but I did want to question the article of
faith stated above.

Should a solution be rejected because it tends in the direction of a
distributed system?  If solutions to current problems push Subversion
in that direction, it might mean that's a useful direction to go.  And
nothing will force us to go farther in that direction than we choose
to, so why worry? :-)

(The reasons given in [1] are understandable... at some point we may
have to ditch the detachable working copies, handy though they are.)

Gratuitous meta-comment courtesy of,
-K

> The complete TODO list of outstanding problems and tasks is tracked
> here:
> http://svn.collab.net/repos/svn/branches/merge-tracking/TODO
>
>
> WC -> WC copy/move operations
> -----------------------------
> Such operations are important for refactoring tools (e.g. Eclipse,
> IDEA, etc.).  Retaining support for operations offline is quite useful
> in this regard (as you can still run'em on the train, when your
> repository is down, etc.).
>
> Preserving offline operation is difficult without replicating merge
> info to the client's WC, and keeping that merge info sync'd with the
> repository.  Usually, the client must contact the repository to get
> the merge info -- both explicit/inherited, and reflected from all
> revisions of the copy source at its current path -- it needs for the
> parents of the copy source in order to have the complete information
> that needs to be written about the children.  The formula's something
> like:
>
>   dest merge info = src merge info + revs of src at path
>
> We need to use this formula on the client side because subsequent
> merges may change the merge info for the destination path before a
> commit to the repository can set the merge info reflected by the
> copy's source path.
>
> If the the copy source has no merge info, and the destination path is
> a direct parent of the source path, then we don't need to move any
> merge info (since it will still be gotten from the same parent), but
> we still need to do history tracing to find the oldest revision at
> which the copy source has been at its current path, so that it can be
> included in the merge info for the destination.
>
> This is not a correctness issue, but rather one of maintaining
> disconnected operation in the face of merge info.
>
>
> URL -> URL -> WC merge operations (merge test #19)
> --------------------------------------------------
> A useful operation which takes the difference between URL1@rX and
> URL2@rY, and applies the delta between those trees to a WC.
>
> The operation currently comes down as one big text delta between the
> files.  We can't tell what revs it actually contains, nor how they
> relate to the WC's merge info.  Because of this, we can't calculate
> which portions of the delta to apply, and which to filter out (because
> they're already represented by our WC).
>
> We can preserve the existing behavior without using merge info (as a
> worst case), but this is rather at odds with the goal of introducing
> merge tracking, so we'd like to do something better.
>
> One option is to send the target paths, plus any WC merge info
> changes, to the repository when procuring the editor, and let the
> editor filter out the merges which are already represented by the WC.
>
>
> WC notifications from a merge operation
> ---------------------------------------
> These notifications are received as callbacks indicating that an
> interesting action has occurred on a path.
>
> For merge tracking purposes, (at least) skips and conflicts must be
> handled, both of which affect merge info set on the WC as the result
> of a merge.  Conflicts may also involve entering a conflict resolution
> mode, or necessitate only partial application of the requested merge
> range to conflicted files.
>
> Notifications of all types generate output.  Merging a set of revision
> ranges in a loop may cause multiple notifications for a single WC
> item.  This is a rather large difference from existing command-line
> client output (and libsvn_client API behavior).
>
> This is not a problem so much as question of what we want to do from
> an interface perspective.
>
>
>
> [1] While we could explore the option of sync'ing mergeinfo between WC
> and repository, comparision to other VC systems suggests that this
> would likely require additional restrictions on WCs (e.g. no separable
> WCs unless we have one merge info DB for each directory, which would
> be horrendous), and quite a bit of work.  It would be difficult to
> make such a change in behavior compatible with previous versions of
> Subversion.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org