Move tracking - a call to action

[Julian Foad <ju...@btopenworld.com>]

At the Hackathon I talked to a bunch of us about how I believe we need to start implementing move tracking, and my overview of a plan, and there was wide agreement from those present.

I am proposing that was tackle move tracking now.


Briefly
=======

It comes from my wanting to make merging better.  In thinking about better merging, it became clear to me that we need to track moves so that we can unambiguously match up "the same" node in different branches, or in different revisions.  While we can *infer* moves by looking for pairs of copy and delete in various scenarios, we need a way to track and transmit the results of that inference, since it would be crazy to embed that logic within the merge code.  In other words, we need at least a move-aware diff interface to use in merging.

If we're going to build a move-aware diff (or tree editor) interface, then rather than try to redesign merging at the same time, it is going to be both easier and more beneficial to first track moves around the basic work-flow cycle:

          ---> commit --->

  [ WC ]                    [ repo ]
          <--- update <---


After all,

  * Much of the WC part is already working in v1.8.

  * Commit and update both transmit the tree changes through an editor.
    I have ideas about various ways to make a move-aware editor, ranging
    from transmitting move info transparently over Ev1 in entry-props,
    through to embracing Ev2.

  * We already have ideas about how the repository can store and replay
    moves.


When that basic work cycle supports moves, that will by itself create a huge leap in usability, that can be released by itself.  After that, the other big components, primarily:

  * merge

  * diff and patch

can be upgraded to support move tracking in a later release.

Most importantly, we already have lots of developer interest.  Stefan Sperling has volunteered to work on some of the WC implementation, Ivan volunteered to work on protocol upgrades, and Brane and Stefan Fuhrmann are interested in the semantics and the repository side, among others.  I am planning to work on the editor interface and whatever else is necessary, and particularly to work towards making use of move tracking in merging.


Compatibility and Incremental development
=========================================

The key component is a move-aware edit drive, with the ability to convert between old and new edit drives in both directions.  Call it Ev2 and the shims if you like.  With that, we can upgrade the system one piece at a time.  For example, we could implement storage of moves in the repository, and upgrade 'commit' to store moves, before we implement any way to retrieve the move info from the repository other than as copy-and-delete.  That particular example is of no use to users, but very helpful for development.

We will preserve backward compatibility between
move-aware clients and move-unaware repositories, and between
move-aware repositories and move-unaware clients.  In all cases,
simple compatibility will be available by falling back to
copy-and-delete.  In some cases, heuristic detection of moves may be
offered as an option.


How to Proceed?
===============

There will be a lot of things to discuss in depth.  I suggest we discuss the general plan and in this thread.  For specific components, such as the move-aware editor or the repository storage, let's start separate threads please.

I am writing down more details about several areas on the Wiki, as I go along.  This is my project overview page:

  <https://wiki.apache.org/subversion/MoveDev/MoveDev>

I see there are already some passionate feedback comments in there, which is great :-)

- Julian


--
Join WANdisco's free daily demo sessions on Scaling Subversion for the Enterprise
<http://www.wandisco.com/training/webinars>

Re: Move tracking - a call to action

[Stefan Sperling <st...@elego.de>]

On Fri, Jun 21, 2013 at 09:00:11AM +0000, Markus Schaber wrote:
> Hi, Julian,
> 
> 
> Von: Julian Foad [mailto:julianfoad@btopenworld.com]
> 
> > [...]
> > We will preserve backward compatibility between move-aware clients and move-
> > unaware repositories, and between move-aware repositories and move-unaware
> > clients.  In all cases, simple compatibility will be available by falling
> > back to copy-and-delete.  In some cases, heuristic detection of moves may be
> > offered as an option.
> 
> It might be useful to provide an API to hook into that heuristic - clients
> embedded into IDEs may have more semantic knowledge about object identity. 
> For example, CODESYS objects always contain a (per project) unique GUID, and 
> certain groups of files (Form.cs, Form.Designer.cs, Form.resx) are always 
> renamed or moved together in Visual Studio.

With a heuristic, it's quite possible that it will be up to the client
or IDE integration to ask the user about incoming moves. Most definitely
in cases where the move is ambiguous when provided as copy+delete, such as
  svn cp A B; svn cp A C; svn rm A; svn commit
which is currently indistinguishable from 
  svn cp A B; svn mv A C; svn commit
and
  svn cp A C; svn mv A B; svn commit

In situations like this, the 'svn' client might decide to ask the user
to pick one of the possible moves, or treat them all as copy+delete.

It is possible that no user interaction will be required at all in
your case. I don't think we'll need a special API feature for hooking
into the heuristic. The client will drive conflict resolution, and
the client/wc libraries will just do what the client tells them to do.
We need to make sure that clients receive all information about the
conflict they need to make an informed decision, of course.

AW: Move tracking - a call to action

[Markus Schaber <m....@codesys.com>]

Hi, Julian,


Von: Julian Foad [mailto:julianfoad@btopenworld.com]

> [...]
> We will preserve backward compatibility between move-aware clients and move-
> unaware repositories, and between move-aware repositories and move-unaware
> clients.  In all cases, simple compatibility will be available by falling
> back to copy-and-delete.  In some cases, heuristic detection of moves may be
> offered as an option.

It might be useful to provide an API to hook into that heuristic - clients
embedded into IDEs may have more semantic knowledge about object identity. 
For example, CODESYS objects always contain a (per project) unique GUID, and 
certain groups of files (Form.cs, Form.Designer.cs, Form.resx) are always 
renamed or moved together in Visual Studio.


Best regards

Markus Schaber

CODESYS(r) a trademark of 3S-Smart Software Solutions GmbH

Inspiring Automation Solutions

3S-Smart Software Solutions GmbH
Dipl.-Inf. Markus Schaber | Product Development Core Technology
Memminger Str. 151 | 87439 Kempten | Germany
Tel. +49-831-54031-979 | Fax +49-831-54031-50

E-Mail: m.schaber@codesys.com | Web: http://www.codesys.com | CODESYS store: http://store.codesys.com
CODESYS forum: http://forum.codesys.com

Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915