You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Julian Foad <ju...@btopenworld.com> on 2012/08/03 21:25:58 UTC
Enabling symmetric merge, and UI details
Hi, merge fans.
It's time to lose the temporary '--symmetric' option and make it the default mode in 'svn'.
Some people have been kind enough to try using the 'symmetric' mode and report back, mainly Paul to whom I am most grateful. I am aware of two current issues ([1],[2]) which I am investigating and they are looking simple enough to be resolved soon. Please help me flag up any other issues.
Before we consider the details of the UI changes, I thought back to how the user base would see 'symmetric merge' in the context of the history of Subversion's merge tracking, and wrote a brief overview at
<http://wiki.apache.org/subversion/SymmetricMerge#The_User.27s_Perspective>.
UI CHANGES
I think, and please tell me if you disagree, that we will make the best progression with a UI like this, which the attached patch [6] roughly implements for illustration:
* Use the 'symmetric'/'auto'[3] mode for any merge that was previously a reintegrate or a non-reintegrate 'sync' merge. That is, when there is:
- just one source;
- no start revision;
- (an end revision may be specified, which can be done without a start rev by specifying peg-rev syntax: SOURCE@END_REV);
- no incompatible options (such as --ignore-ancestry).
* If the code chooses a reintegrate-like merge, it will apply the strict WC checks like reintegrate does in 1.7.
* Make the '--reintegrate' option mean the user believes a reintegrate-style merge is needed, so check that the previous merge is as expected (that it, in the opposite direction) and bail out if not.
* Mark the '--reintegrate' option as deprecated, as the reason for its existence has gone.
* Delete the '--symmetric' option; it was only there for testing and has not been released.
WHAT WILL NOT CHANGE
Two specific things to call out:
* A merge specifying '--reintegrate' merge will not change in any scenario where 'merge --reintegrate' is the user's best option today.
* A merge specifying '-r X:Y' will not change. This requests a cherry-pick merge, using merge tracking to avoid re-merging any revs in the range that are already merged. Currently, if 'X' is 1 or a low enough number such that there are no unmerged revisions before 'X', then Subversion performs exactly the same kind of merge as when 'X' is not specified, which we call a 'sync' merge. I believe we could make a later enhancement to automatically use 'reintegrate' mode if that would be the correct thing to do for merging the requested range.
OPTIONS NOT ALLOWED WITH --REINTEGRATE
The following options are not allowed with '--reintegrate' in 1.7 but are allowed for non-reintegrate merge-tracking merges:
--record-only
--depth
--force
--allow-mixed-revisions
My recommendation for these options is that if we choose a reintegrate-style merge then we error out, saying "the required merge is reintegrate-like[4], and option --X is not supported for this kind of merge". Otherwise we handle the option just like in 1.7.
Please let me know if this seems good, while I go and adjust the help text.
- Julian
[1] Unexpected revision number in the "Merging r6 through r9" notification, <http://svn.haxx.se/dev/archive-2012-08/0005.shtml>.
[2] Behaviour when the root has not been merged, a subtree has, and now we merge the root in the other direction, <http://svn.haxx.se/dev/archive-2012-08/0008.shtml>.
[3] Every time I say 'symmetric', I try to think of a better name. It's not symmetric, it's just based on a symmetric model. 'automatic'? 'bidirectional'?
[4] Need to choose terminology for a reintegrate-style merge, for the UI. "reversing direction"?
[5] Wiki page, <http://wiki.apache.org/subversion/SymmetricMerge>.
[6] Attachment, 'enable-symmetric-merge-1.patch'.
[7] GUIs and other clients can switch to the new svn_client API when
they are ready to do so; here I'm just talking about our 'svn' client.
--
Subversion Live 2012 - 2 Days: training, networking, demos & more! http://bit.ly/NgUaRi
Certified & Supported Apache Subversion Downloads: http://www.wandisco.com/subversion/download
Re: Enabling symmetric merge, and UI details
Posted by Julian Foad <ju...@btopenworld.com>.
Neels J Hofmeyr wrote:
> On 2012-08-03 21:25, Julian Foad wrote:
>> It's time to lose the temporary '--symmetric' option and make
>> it the default mode in 'svn'.
>
> sounds fantastic :)
>
>> <http://wiki.apache.org/subversion/SymmetricMerge#The_User.27s_Perspective>.
>
> A good read, but I can't find that "[5]" reference:
> "...we now have a better model of what merge tracking means [5]."
Ah... In my head I have a better model of what merge tracking means. I don't really have a good write-up to refer to. This section <http://wiki.apache.org/subversion/SymmetricMerge#Symmetric_Merging_and_Complete_Merges> and this section <wiki.apache.org/subversion/MergeTrackingIdeas#Logical_Change_Tracking> are the closest I have.
- Julian
> ~Neels
>
> </random_dev_reply_by_neels>
>
Re: Enabling symmetric merge, and UI details
Posted by Neels J Hofmeyr <ne...@elego.de>.
On 2012-08-03 21:25, Julian Foad wrote:
> It's time to lose the temporary '--symmetric' option and make it the default mode in 'svn'.
sounds fantastic :)
> <http://wiki.apache.org/subversion/SymmetricMerge#The_User.27s_Perspective>.
A good read, but I can't find that "[5]" reference:
"...we now have a better model of what merge tracking means [5]."
~Neels
</random_dev_reply_by_neels>
Re: Enabling symmetric merge, and UI details
Posted by Julian Foad <ju...@btopenworld.com>.
I (Julian Foad) wrote on 6 August 2012:
> OK, enabled in 1369896.
>
> --reintegrate forces a 1.7 reintegrate, unconditionally, at the moment; I
> haven't put a deprecation warning in there.
>
> Please let me know if anyone sees unexpected results from merge now.
>
> TODO:
>
> * Revise 'svn merge' help text.
I made an initial major update in r1370818. It needs more.
> * Checks for 'no local mods' and 'no switched subtrees'
> aren't currently being performed when they should; it needs to wait till
> after working out which kind of merge is required, and then do the strict checks
> iff it's reintegrate-like.
Fixed in r1369923.
> * Check foreign-repos merges don't try to use the symmetric merge code path.
Checked; it's fine.
> * Add a JavaHL API version of svn_client_merge_symmetric().
TODO.
> * ambiguity: reintegrate or not (merge_symmetric_tests-18)
> Email thread "Symmetric merge and subtrees".
TODO.
> * Issue #4217 -- reported rev unexpected (merge_tests-78)
> Email thread "Symmetric merge and deleted subtrees".
I'm happy with that, thanks to Paul explaining the details to me. I'll update the test expectation and close the issue.
- Julian
Re: Enabling symmetric merge, and UI details
Posted by Branko Čibej <br...@wandisco.com>.
On 06.08.2012 20:23, Stefan Sperling wrote:
> On Mon, Aug 06, 2012 at 07:09:11PM +0100, Julian Foad wrote:
>> --reintegrate forces a 1.7 reintegrate, unconditionally, at the moment; I haven't put a deprecation warning in there.
> In my opinion it would be best to try to keep --reintegrate and its
> current behaviour around until 2.0, with a deprecation warning.
"2.0" is effectively "forever." Is that really necessary if at some
point the merge algorithm becomes truly correct and doesn't require
--reintegrate hacks?
> If we change the behaviour of --reintegrate we might break existing scripts.
We've broken scripts any number of times already. IMO --reintegrate just
papers over a bug. The idea is to fix the underlying bug. I see nothing
in our versioning rules that requires bug-for-bug backwards compatibility.
-- Brane
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: Enabling symmetric merge, and UI details
Posted by Stefan Sperling <st...@elego.de>.
On Mon, Aug 06, 2012 at 07:09:11PM +0100, Julian Foad wrote:
> OK, enabled in 1369896.
This is awesome progress. No time to try it right now but I will
definitely try this. Thanks!
> --reintegrate forces a 1.7 reintegrate, unconditionally, at the moment; I haven't put a deprecation warning in there.
In my opinion it would be best to try to keep --reintegrate and its
current behaviour around until 2.0, with a deprecation warning.
If we change the behaviour of --reintegrate we might break existing scripts.
Re: Enabling symmetric merge, and UI details
Posted by Julian Foad <ju...@btopenworld.com>.
OK, enabled in 1369896.
--reintegrate forces a 1.7 reintegrate, unconditionally, at the moment; I haven't put a deprecation warning in there.
Please let me know if anyone sees unexpected results from merge now.
TODO:
* Revise 'svn merge' help text.
* Checks for 'no local mods' and 'no switched subtrees' aren't currently being performed when they should; it needs to wait till after working out which kind of merge is required, and then do the strict checks iff it's reintegrate-like.
* Check foreign-repos merges don't try to use the symmetric merge code path.
* Add a JavaHL API version of svn_client_merge_symmetric().
* ambiguity: reintegrate or not (merge_symmetric_tests-18)
Email thread "Symmetric merge and subtrees".
* Issue #4217 -- reported rev unexpected (merge_tests-78)
Email thread "Symmetric merge and deleted subtrees".
- Julian
----- Original Message -----
> From: Julian Foad <ju...@btopenworld.com>
> To: Branko Čibej <br...@wandisco.com>
> Cc: "dev@subversion.apache.org" <de...@subversion.apache.org>
> Sent: Monday, 6 August 2012, 10:23
> Subject: Re: Enabling symmetric merge, and UI details
>
> Branko Čibej wrote on 3 August 2012:
>
>> On 03.08.2012 21:51, Mark Phippard wrote:
>>> On Fri, Aug 3, 2012 at 3:25 PM, Julian Foad wrote:
>>>> * Make the '--reintegrate' option mean the user believes
>>>> a reintegrate-style merge is needed, so check that the
>>>> previous merge is as expected (that it, in the opposite
>>>> direction) and bail out if not.
>>>
>>> Why do you want to do this? Just to educate the user that they do not
>>> need reintegrate? It does not seem like it would be necessary. What
>>> happens today if the user does this? I guess if --reintegrate does
>>> the wrong thing today then this makes sense. But if it basically
>>> still gives a valid result why bother to make a behavior change in
>>> what is a deprecated option anyway?
>>
>> I concur. Our goal is to make the merge algorithm truly symmetric within
>> the next release or two, inshallah. In order to get there, the behaviour
>> of "svn merge" will certainly change in some scenarios, but we
> obviously
>> expect that the results will become more correct. As long as we can
>> argue that any such changes in 1.8 are simply incremental adjustments on
>> the way towards that goal, it seems a waste of effort to maintain any
>> kind of special meaning for the --reintegrate option.
>>
>> IMO the only thing --reintegrate should do in 1.8 is to issue a warning
>> about it being deprecated; and in 1.9 it we can silently ignore it.
>
> You mean --reintegrate should cause 'svn' to do a 1.7 reintegrate merge
> (and issue a deprecation warning)? I don't have an opinion on the warning,
> but doing an old-style reintegrate makes sense: there is no need to make the
> behaviour of that option more complex like I was suggesting.
>
> - Julian
Re: Enabling symmetric merge, and UI details
Posted by Julian Foad <ju...@btopenworld.com>.
Branko Čibej wrote on 3 August 2012:
> On 03.08.2012 21:51, Mark Phippard wrote:
>> On Fri, Aug 3, 2012 at 3:25 PM, Julian Foad wrote:
>>> * Make the '--reintegrate' option mean the user believes
>>> a reintegrate-style merge is needed, so check that the
>>> previous merge is as expected (that it, in the opposite
>>> direction) and bail out if not.
>>
>> Why do you want to do this? Just to educate the user that they do not
>> need reintegrate? It does not seem like it would be necessary. What
>> happens today if the user does this? I guess if --reintegrate does
>> the wrong thing today then this makes sense. But if it basically
>> still gives a valid result why bother to make a behavior change in
>> what is a deprecated option anyway?
>
> I concur. Our goal is to make the merge algorithm truly symmetric within
> the next release or two, inshallah. In order to get there, the behaviour
> of "svn merge" will certainly change in some scenarios, but we obviously
> expect that the results will become more correct. As long as we can
> argue that any such changes in 1.8 are simply incremental adjustments on
> the way towards that goal, it seems a waste of effort to maintain any
> kind of special meaning for the --reintegrate option.
>
> IMO the only thing --reintegrate should do in 1.8 is to issue a warning
> about it being deprecated; and in 1.9 it we can silently ignore it.
You mean --reintegrate should cause 'svn' to do a 1.7 reintegrate merge (and issue a deprecation warning)? I don't have an opinion on the warning, but doing an old-style reintegrate makes sense: there is no need to make the behaviour of that option more complex like I was suggesting.
- Julian
Re: Enabling symmetric merge, and UI details
Posted by Branko Čibej <br...@wandisco.com>.
On 03.08.2012 21:51, Mark Phippard wrote:
> On Fri, Aug 3, 2012 at 3:25 PM, Julian Foad <ju...@btopenworld.com> wrote:
>
>> It's time to lose the temporary '--symmetric' option and make it the default mode in 'svn'.
> Sounds good. Just a couple of questions.
>
>> * Make the '--reintegrate' option mean the user believes a reintegrate-style merge is needed, so check that the
>> previous merge is as expected (that it, in the opposite direction) and bail out if not.
> Why do you want to do this? Just to educate the user that they do not
> need reintegrate? It does not seem like it would be necessary. What
> happens today if the user does this? I guess if --reintegrate does
> the wrong thing today then this makes sense. But if it basically
> still gives a valid result why bother to make a behavior change in
> what is a deprecated option anyway?
I concur. Our goal is to make the merge algorithm truly symmetric within
the next release or two, inshallah. In order to get there, the behaviour
of "svn merge" will certainly change in some scenarios, but we obviously
expect that the results will become more correct. As long as we can
argue that any such changes in 1.8 are simply incremental adjustments on
the way towards that goal, it seems a waste of effort to maintain any
kind of special meaning for the --reintegrate option.
IMO the only thing --reintegrate should do in 1.8 is to issue a warning
about it being deprecated; and in 1.9 it we can silently ignore it.
-- Brane
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: Enabling symmetric merge, and UI details
Posted by Julian Foad <ju...@btopenworld.com>.
Mark Phippard wrote:
> On Fri, Aug 3, 2012 at 4:54 PM, Julian Foad wrote:
>> Today, if the user gives the --reintegrate option when a
>> non-reintegrate merge is the appropriate one based on past
>> merges, Subversion goes through the motions of a reintegrate
>> merge and produces the wrong result. [...]
>
> If it does the wrong thing today in this situation, then I am in favor
> of your proposal.
>
>>> Do you plan on adding a new mergeSync API to JavaHL or just have the
>>> JavaHL C++ code call the new API when the RevisionRange is passed as I
>>> noted above? I would be fine with the latter as I do not think it
>>> introduces any unexpected new behaviors. There is already a specific
>>> mergeReintegrate JavaHL API.
>>
>> I would prefer to add a new API to JavaHL, as the current merge
>> API is already way too overloaded with variations of behaviour in
>> my opinion.
>
> That is OK with me. Based on the existing signature I mentioned, it
> seems like the only option you would drop is the RevisionRange
> argument. I think when Hyrum cleaned up the JavaHL methods he just
> preferred to not have as many subtle variants of the method.
>
> Regardless which option you choose, I just wanted to be sure there was
> some way we can use the new API from JavaHL. Adjusting Subclipse to
> use the right method will be trivial.
>
> A couple of other comments:
>
> You do not mention explicit 2-URL merges but I assume those will be unchanged?
Correct: 2-URL merges are unchanged.
> You do not mention foreign repository merges. Perhaps the wiki does?
They are non-merge-tracking merges, and so should be unchanged. I will check whether that's correctly implemented.
- Julian
Re: Enabling symmetric merge, and UI details
Posted by Mark Phippard <ma...@gmail.com>.
On Fri, Aug 3, 2012 at 4:54 PM, Julian Foad <ju...@btopenworld.com> wrote:
>> It does not seem like it would be necessary. What
>> happens today if the user does this? I guess if --reintegrate does
>> the wrong thing today then this makes sense. But if it basically
>> still gives a valid result why bother to make a behavior change in
>> what is a deprecated option anyway?
>
> Today, if the user gives the --reintegrate option when a non-reintegrate merge is the appropriate one based on past
> merges, Subversion goes through the motions of a reintegrate merge and produces the wrong result. (Wrong in the
> sense that it doesn't properly merge the sets of unique changes from the two branches, not that it doesn't do
> exactly what we taught it to do.)
If it does the wrong thing today in this situation, then I am in favor
of your proposal.
>> Do you plan on adding a new mergeSync API to JavaHL or just have the
>> JavaHL C++ code call the new API when the RevisionRange is passed as I
>> noted above? I would be fine with the latter as I do not think it
>> introduces any unexpected new behaviors. There is already a specific
>> mergeReintegrate JavaHL API.
>
> I would prefer to add a new API to JavaHL, as the current merge API is already way too overloaded with variations
> of behaviour in my opinion.
That is OK with me. Based on the existing signature I mentioned, it
seems like the only option you would drop is the RevisionRange
argument. I think when Hyrum cleaned up the JavaHL methods he just
preferred to not have as many subtle variants of the method.
Regardless which option you choose, I just wanted to be sure there was
some way we can use the new API from JavaHL. Adjusting Subclipse to
use the right method will be trivial.
A couple of other comments:
You do not mention explicit 2-URL merges but I assume those will be unchanged?
You do not mention foreign repository merges. Perhaps the wiki does?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Enabling symmetric merge, and UI details
Posted by Julian Foad <ju...@btopenworld.com>.
Mark Phippard wrote:
> On Fri, Aug 3, 2012 at 3:25 PM, Julian Foad wrote:
>> It's time to lose the temporary '--symmetric' option and make
>> it the default mode in 'svn'.
>
> Sounds good. Just a couple of questions.
>
>> * Make the '--reintegrate' option mean the user believes a
>> reintegrate-style merge is needed, so check that the
>> previous merge is as expected (that it, in the opposite
>> direction) and bail out if not.
>
> Why do you want to do this? Just to educate the user that they do not
> need reintegrate?
I was thinking the main alternative would be to silently ignore it, and that seems unnecessarily misleading. But I guess you are thinking the main alternative is to keep it doing exactly what it does today. I don't have a strong objection to that, but I feel we can do a better service for the user by reporting if the user seems to have made a mistake, because it seems to me that a significant cause of merge problems in the real world is users making a mistake (forgetting which WC they're in, for example).
> It does not seem like it would be necessary. What
> happens today if the user does this? I guess if --reintegrate does
> the wrong thing today then this makes sense. But if it basically
> still gives a valid result why bother to make a behavior change in
> what is a deprecated option anyway?
Today, if the user gives the --reintegrate option when a non-reintegrate merge is the appropriate one based on past merges, Subversion goes through the motions of a reintegrate merge and produces the wrong result. (Wrong in the sense that it doesn't properly merge the sets of unique changes from the two branches, not that it doesn't do exactly what we taught it to do.)
>> * A merge specifying '-r X:Y' will not change. This
>> requests a cherry-pick merge, using merge tracking to avoid
>> re-merging any revs in the range that are already merged.
>> Currently, if 'X' is 1 or a low enough number such that
>> there are no unmerged revisions before 'X', then Subversion
>> performs exactly the same kind of merge as when 'X'
>> is not specified, which we call a 'sync' merge. I believe we
>> could make a later enhancement to automatically use
>> 'reintegrate' mode if that would be the correct thing to do
>> for merging the requested range.
>
> My question here is at the JavaHL API level.
>
> Today, the API that we call to do a sync-style merge is the same as
> the one we call to merge revision ranges. Specifically, here is the
> signature:
>
> /**
> * Merge set of revisions into a new local path.
> * @param path path or url
> * @param pegRevision revision to interpret path
> * @param revisions revisions to merge
> * @param localPath target local path
> * @param force overwrite local changes
> * @param depth how deep to traverse into subdirectories
> * @param ignoreAncestry ignore if files are not related
> * @param dryRun do not change anything
> * @param recordOnly record mergeinfo but do not run merge
> * @throws ClientException
> */
> void merge(String path, Revision pegRevision,
> List<RevisionRange> revisions,
> String localPath, boolean force, Depth depth,
> boolean ignoreAncestry, boolean dryRun, boolean recordOnly)
> throws ClientException;
>
> When we want to a sync-style merge we pass a single RevisionRange
> object in the List with values of 0 for the Start and End revision. I
> see the following note:
>
>> [7] GUIs and other clients can switch to the new svn_client API when
>> they are ready to do so; here I'm just talking about our 'svn'
> client.
>
> Do you plan on adding a new mergeSync API to JavaHL or just have the
> JavaHL C++ code call the new API when the RevisionRange is passed as I
> noted above? I would be fine with the latter as I do not think it
> introduces any unexpected new behaviors. There is already a specific
> mergeReintegrate JavaHL API.
I would prefer to add a new API to JavaHL, as the current merge API is already way too overloaded with variations of behaviour in my opinion.
- Julian
Re: Enabling symmetric merge, and UI details
Posted by Mark Phippard <ma...@gmail.com>.
On Fri, Aug 3, 2012 at 3:25 PM, Julian Foad <ju...@btopenworld.com> wrote:
> It's time to lose the temporary '--symmetric' option and make it the default mode in 'svn'.
Sounds good. Just a couple of questions.
> * Make the '--reintegrate' option mean the user believes a reintegrate-style merge is needed, so check that the
> previous merge is as expected (that it, in the opposite direction) and bail out if not.
Why do you want to do this? Just to educate the user that they do not
need reintegrate? It does not seem like it would be necessary. What
happens today if the user does this? I guess if --reintegrate does
the wrong thing today then this makes sense. But if it basically
still gives a valid result why bother to make a behavior change in
what is a deprecated option anyway?
> * A merge specifying '-r X:Y' will not change. This requests a cherry-pick merge, using merge tracking to avoid
> re-merging any revs in the range that are already merged. Currently, if 'X' is 1 or a low enough number such that
> there are no unmerged revisions before 'X', then Subversion performs exactly the same kind of merge as when 'X'
> is not specified, which we call a 'sync' merge. I believe we could make a later enhancement to automatically use
> 'reintegrate' mode if that would be the correct thing to do for merging the requested range.
My question here is at the JavaHL API level.
Today, the API that we call to do a sync-style merge is the same as
the one we call to merge revision ranges. Specifically, here is the
signature:
/**
* Merge set of revisions into a new local path.
* @param path path or url
* @param pegRevision revision to interpret path
* @param revisions revisions to merge
* @param localPath target local path
* @param force overwrite local changes
* @param depth how deep to traverse into subdirectories
* @param ignoreAncestry ignore if files are not related
* @param dryRun do not change anything
* @param recordOnly record mergeinfo but do not run merge
* @throws ClientException
*/
void merge(String path, Revision pegRevision, List<RevisionRange> revisions,
String localPath, boolean force, Depth depth,
boolean ignoreAncestry, boolean dryRun, boolean recordOnly)
throws ClientException;
When we want to a sync-style merge we pass a single RevisionRange
object in the List with values of 0 for the Start and End revision. I
see the following note:
> [7] GUIs and other clients can switch to the new svn_client API when
> they are ready to do so; here I'm just talking about our 'svn' client.
Do you plan on adding a new mergeSync API to JavaHL or just have the
JavaHL C++ code call the new API when the RevisionRange is passed as I
noted above? I would be fine with the latter as I do not think it
introduces any unexpected new behaviors. There is already a specific
mergeReintegrate JavaHL API.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/