You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David Moon <da...@versata.com> on 2013/01/14 17:49:02 UTC
Subversion merge creates bogus tree conflicts
Subversion creates an unnecessary tree conflict when two branches of
a directory are merged. See the transcript below. Since the
contents of the directory do not conflict between the two branches,
it should just add both subdirectories to the directory and not
report any conflict. In my opinion, that should be the default merge
behavior, but there isn't even any way using svn resolve to tell it
to do that! The only thing svn resolve can do is lose the incoming
change.
The unnecessary tree conflict does not always happen, based on I know
not what. But the transcript below seems to be a reliable way to
reproduce it.
This transcript is boiled down to a simple test case. In the real
thing there were too many of these tree conflicts to fix by hand, so
fixing it by hand is not a solution even if I could figure out how to
fix it by hand.
$REPOS is file:/// something.
> svn --version
svn, version 1.7.8 (r1419691)
compiled Dec 12 2012, 14:18:28
> svn mkdir --parents $REPOS/test1/branches/B1/D1/D2/A -m xx
> svn mkdir --parents $REPOS/test1/tags/T1/D1/D2/B -m xx
> svn co $REPOS/test1/branches/B1/D1/D2/A
> echo test > A/test1.txt
> cd A; svn add test1.txt
> svn commit -m xx
> cd ..
> svn co $REPOS/test1/tags/T1/D1/D2/B
> echo test > B/test2.txt
> cd B; svn add test2.txt
> svn commit -m xx
> cd ..
> svn copy $REPOS/test1/tags/T1 $REPOS/test1/branches/B2 -m xx
> svn list -R $REPOS/test1/branches/B1
D1/
D1/D2/
D1/D2/A/
D1/D2/A/test1.txt
> svn list -R $REPOS/test1/branches/B2
D1/
D1/D2/
D1/D2/B/
D1/D2/B/test2.txt
> svn co $REPOS/test1/branches/B2
> cd B2
> svn merge $REPOS/test1/branches/B1
--- Merging r167502 through r167505 into '.':
C D1/D2/A
--- Recording mergeinfo for merge of r167502 through r167505 into '.':
U .
Summary of conflicts:
Tree conflicts: 1
> svn status
M .
! C D1/D2/A
> local delete, incoming edit upon merge
Summary of conflicts:
Tree conflicts: 1
> ls D1/D2
B/
Note: ls D1/D2 should say A/ B/ and there should be no tree conflict.
In the real thing, the svn status explanation of the tree conflict is
usually "local add, incoming add upon merge." I can't explain why
the simple test case says local delete instead. I don't know if this
matters.
I can't say that Subversion is not behaving according to the
documentation, because the documentation at http://svnbook.red-
bean.com/en/1.7/index.html is extremely vague about this aspect of
Subversion. But it's not behaving in a useful way.
Environmental data:
operating system - Red Hat Enterprise Linux Server release 5.3 (64
bit). Also fails in RHEL 5.5.
The release and/or revision of Subversion - 1.7.8
The compiler and configuration options you built Subversion with -
downloaded from Collabnet, I did not build it
Any private modifications you made to your Subversion - none
The version of Berkeley DB you're running Subversion with, if any -
using FSFS
Anything else that could possibly be relevant. Err on the side of too
much information, rather than too little. - I can't think of anything
else.
I hope this bug report is useful to you. I can't wait for a fix, so
I am changing to a different strategy that does not depend on
Subversion merge.
--Dave Moon
Re: Subversion merge creates bogus tree conflicts
Posted by Julian Foad <ju...@btopenworld.com>.
David Moon wrote:
> Julian Foad wrote:
>>> Hmm, I wonder if the directory not existing on the target means this is actually
>>> a simple bug, not really some complicated problem with merging. Why wasn't
>>> the parent directory of a file being added created automatically, as if svn
>>> merge took a --parent argument? When would that be harmful?
>>
>> It's very complex -- in general (different merge scenarios) the delete
>> might have been a conscious user action committed in the target branch, and
>> might be a move away rather than a plain delete, since svn still lacks move
>> tracking. We can't assume that just recreating the dir is the right thing,
>> although it should be offered as an option in the conflict resolver.
>
> I assumed, perhaps incorrectly, that svn knew the difference between something
> being deleted and something never having existed. In other words, if the
> directory had existed and then been deleted or moved, the list of changes being
> merged would have included a delete. But if the directory never existed before
> it would not be mentioned in the list of changes being merged.
>
> Just trying to be clear and unambiguous.
We're talking about the scenario where an incoming change is a file being added inside a directory which doesn't exist on the target. The "delete" isn't an incoming change (a change that happened in the merge source), rather "delete" is how we are describing the apparent difference between the old side of the merge source (where there is an 'A') and the target (where there isn't). "delete" is a misleading word for the UI to use in this case, for sure. And as we're talking about a manual merge, we are merging branches with no common ancestry, so we can't really say in this case that "'A' never existed" nor that "'A' did exist and was deleted"; rather we have two separate lines of history and 'A' existed in one of them.
Of course the Subversion repository knows what existed where and when, but what's complex is figuring out how to interpret and present the relationships in terms of doing sensible things in all the combinations that a merge may encounter.
And I haven't even mentioned the fact that some users want strict conflict detection ("I have arranged my work flow such that I expect a smooth merge, so notify me if anything seems not to match up") while in other situations users want lenient conflict detection ("just create the result that includes the changes I most likely want and automatically omit duplicates, create missing dirs and so on; don't mark conflicts when there is an alternative"). In that regard I want to build in a switch to select either mode, but we don't have that yet.
- Julian
Re: Subversion merge creates bogus tree conflicts
Posted by David Moon <da...@versata.com>.
On Jan 15, 2013, at 5:29 PM, Ben Reser wrote:
> I've opened the issue #4292 on this.
> http://subversion.tigris.org/issues/show_bug.cgi?id=4292
Thanks.
> Note that resolving that problem won't help your particular case
> because it'd also mean that D1/D2's creation would be attempted by the
> merge and then you'd receive a different tree conflict on that.
>
> Ultimately a fully better user experience with this would require tree
> conflict resolution handling. In this case being able to say "give me
> the union of the trees and the union of the properties on the
> directories of the trees," would be helpful to you.
Agreed.
--Dave Moon
Re: Subversion merge creates bogus tree conflicts
Posted by Ben Reser <be...@reser.org>.
I've opened the issue #4292 on this.
http://subversion.tigris.org/issues/show_bug.cgi?id=4292
Note that resolving that problem won't help your particular case
because it'd also mean that D1/D2's creation would be attempted by the
merge and then you'd receive a different tree conflict on that.
Ultimately a fully better user experience with this would require tree
conflict resolution handling. In this case being able to say "give me
the union of the trees and the union of the properties on the
directories of the trees," would be helpful to you.
Re: Subversion merge creates bogus tree conflicts
Posted by Ben Reser <be...@reser.org>.
On Tue, Jan 15, 2013 at 11:16 AM, Ben Reser <be...@reser.org> wrote:
> Here's the same as above just as a script to demonstrate:
> [[[
> svn mkdir --parents $REPOS/test1/branches/B1/A -m xx
> svn co $REPOS/test1/branches/B1
> echo test > B1/A/test.txt
> svn add B1/A/test.txt
> svn commit -m xx B1
> svn cp $REPOS/test1/branches/B1 $REPOS/test1/branches/B2 -m xx
> svn co $REPOS/test1/branches/B2
> echo test2 >> B2/A/test.txt
> svn commit B2 -m xx
> svn co $REPOS/test1/branches/B1
> cd B1
> svn rm --force A
> svn commit -m xx
> svn merge -r0:HEAD $REPOS/test1/branches/B2
> ]]]
and that last merge really can be just:
svn merge $REPOS/test1/branches/B2
But that's what I get for copying and pasting too much.
Re: Subversion merge creates bogus tree conflicts
Posted by Ben Reser <be...@reser.org>.
On Tue, Jan 15, 2013 at 10:32 AM, David Moon <da...@versata.com> wrote:
> I assumed, perhaps incorrectly, that svn knew the difference between
> something being deleted and something never having existed. In other words,
> if the directory had existed and then been deleted or moved, the list of
> changes being merged would have included a delete. But if the directory
> never existed before it would not be mentioned in the list of changes being
> merged.
Julian was trying to suggest the following scenario.
B1 and B2 are related branches.
B1/A/test.txt and B2/A/test.txt exist
B1 deletes A
B2/A/test.txt is changed
You now merge B2 back onto B1.
The delete is not among the list of changes being merged. Since the
list of changes being merged are coming from B1.
There are a couple of potential solutions we could recreate A and
test.txt, we could also ignore everything in A. But we can't make
really make a decision for the user because we don't really know their
intent.
What's going to happen right now in this case is a tree conflict. As
should be obvious from this thread, we don't have resolution of tree
conflicts handled very well yet. What we should ultimately gain here
is an interactive resolution of the problem much as we've gained for
file conflicts, where you can choose one of the resolutions.
Here's the same as above just as a script to demonstrate:
[[[
svn mkdir --parents $REPOS/test1/branches/B1/A -m xx
svn co $REPOS/test1/branches/B1
echo test > B1/A/test.txt
svn add B1/A/test.txt
svn commit -m xx B1
svn cp $REPOS/test1/branches/B1 $REPOS/test1/branches/B2 -m xx
svn co $REPOS/test1/branches/B2
echo test2 >> B2/A/test.txt
svn commit B2 -m xx
svn co $REPOS/test1/branches/B1
cd B1
svn rm --force A
svn commit -m xx
svn merge -r0:HEAD $REPOS/test1/branches/B2
]]]
Re: Subversion merge creates bogus tree conflicts
Posted by David Moon <da...@versata.com>.
On Jan 15, 2013, at 1:30 PM, Julian Foad wrote:
>> Hmm, I wonder if the directory not existing on the target means
>> this is actually
>> a simple bug, not really some complicated problem with merging.
>> Why wasn't
>> the parent directory of a file being added created automatically,
>> as if svn
>> merge took a --parent argument? When would that be harmful?
>
> It's very complex -- in general (different merge scenarios) the
> delete might have been a conscious user action committed in the
> target branch, and might be a move away rather than a plain delete,
> since svn still lacks move tracking. We can't assume that just
> recreating the dir is the right thing, although it should be
> offered as an option in the conflict resolver.
I assumed, perhaps incorrectly, that svn knew the difference between
something being deleted and something never having existed. In other
words, if the directory had existed and then been deleted or moved,
the list of changes being merged would have included a delete. But
if the directory never existed before it would not be mentioned in
the list of changes being merged.
Just trying to be clear and unambiguous.
--Dave Moon
Re: Subversion merge creates bogus tree conflicts
Posted by Julian Foad <ju...@btopenworld.com>.
David Moon wrote:
> On Jan 15, 2013, at 12:06 PM, Julian Foad wrote:
>> but I at this point I still think the use case is invalid. Or, put another
>> way, the use case is certainly something people might want to do from time
>> to time but the "merge" command is not the tool to do it. Rather it
>> requires some manual steps to set up a starting point from which "merge"
>> can then do the rest.
>
> Fair enough, but there is no clue in existing documentation what those manual
> steps might be.
>
> As a user, I think it would be easier if the special fixing up had to be done
> after the merge had reported conflicts rather than before. But there don't
> seem to be the same svn resolve features for directory merging as for file
> merging.
Yup, this whole area needs better documentation and even more importantly better design and implementation. Anybody reading this who might be able to help in any way, please get in touch!
>> Two adds of different names to the same directory do not conflict. The
>> conflict you saw is on attempting to add a file into a directory ('A')
>> that didn't exist on the target.
>
> I am sure you are correct, but it wasn't clear from the output of svn merge
> and svn status. This can be regarded as another bug report: the error messages
> for tree conflict (in svn 1.7.8, I don't know about newer versions) range
> from misleading to incomprehensible. Designing good error messages is not an
> easy problem, as I well know myself, so please take that as a potential area for
> improvement, not as criticism.
OK, feedback taken.
> Hmm, I wonder if the directory not existing on the target means this is actually
> a simple bug, not really some complicated problem with merging. Why wasn't
> the parent directory of a file being added created automatically, as if svn
> merge took a --parent argument? When would that be harmful?
It's very complex -- in general (different merge scenarios) the delete might have been a conscious user action committed in the target branch, and might be a move away rather than a plain delete, since svn still lacks move tracking. We can't assume that just recreating the dir is the right thing, although it should be offered as an option in the conflict resolver.
> But then, what does the "local add, incoming add upon merge" error
> mean if not a conflict of adding different names to the same directory? Sorry I
> don't have a simple test case to provoke that error.
It means the thing (at a specific path) didn't exist in the "old" side of the incoming change, and is being added by the incoming change of the merge, and also exists in the target already (apparently having been "added" since it wasn't present in the assumed common ancestor).
> Again, I don't need a solution myself. I only reported this as a way of
> giving back to the Subversion community by suggesting an ease of use
> improvement.
Thank you; I do appreciate it takes time and effort to do so, and it is valuable to us.
- Julian
Re: Subversion merge creates bogus tree conflicts
Posted by David Moon <da...@versata.com>.
On Jan 15, 2013, at 12:06 PM, Julian Foad wrote:
> You didn't state what nodes were already present at the start of
> your test case; I'm assuming 'test1' was not present and so 'B1'
> was created by the first commit shown, and 'B2' originated as 'T1'
> which was created in the second commit shown.
Yes, test1 was created by the test case. I should have said that.
> but I at this point I still think the use case is invalid. Or, put
> another way, the use case is certainly something people might want
> to do from time to time but the "merge" command is not the tool to
> do it. Rather it requires some manual steps to set up a starting
> point from which "merge" can then do the rest.
Fair enough, but there is no clue in existing documentation what
those manual steps might be.
As a user, I think it would be easier if the special fixing up had to
be done after the merge had reported conflicts rather than before.
But there don't seem to be the same svn resolve features for
directory merging as for file merging.
> Two adds of different names to the same directory do not conflict.
> The conflict you saw is on attempting to add a file into a
> directory ('A') that didn't exist on the target.
I am sure you are correct, but it wasn't clear from the output of svn
merge and svn status. This can be regarded as another bug report:
the error messages for tree conflict (in svn 1.7.8, I don't know
about newer versions) range from misleading to incomprehensible.
Designing good error messages is not an easy problem, as I well know
myself, so please take that as a potential area for improvement, not
as criticism.
Hmm, I wonder if the directory not existing on the target means this
is actually a simple bug, not really some complicated problem with
merging. Why wasn't the parent directory of a file being added
created automatically, as if svn merge took a --parent argument?
When would that be harmful?
But then, what does the "local add, incoming add upon merge" error
mean if not a conflict of adding different names to the same
directory? Sorry I don't have a simple test case to provoke that error.
Again, I don't need a solution myself. I only reported this as a way
of giving back to the Subversion community by suggesting an ease of
use improvement.
--Dave Moon
Re: Subversion merge creates bogus tree conflicts
Posted by Julian Foad <ju...@btopenworld.com>.
David Moon wrote:
> On Jan 15, 2013, at 8:38 AM, Julian Foad wrote:
>> I think the main point is you need to understand that
>>
>> "svn merge" doesn't calculate the union of two sets of nodes, it combines
>> two sets of *changes* that have been made since a common starting point.
>>
>> You might want to read the section on merging in The Book
> <http://svnbook.red-bean.com/>.
>
> I do understand that, and I did read that.
OK David, I'm sorry for the wrong assumption.
Now, having read through the whole of this email, let me first summarize, and then respond to some particular points.
I agree Subversion's merge should generally work better than it does in scenarios similar to the one you presented. There's a lot of work to do in this area (merging, tree conflicts, conflict resolution). It is my main focus these days.
Your test case attempts to use an automatic merge command to merge unrelated branches (see below re. "unrelated"), and this is not expected to work, and in 1.8 it will in fact throw an "unrelated branches" error. That led Ben and me to conclude your report is invalid. However, by adding a revision range to the merge command we can request a non-automatic merge, which can be appropriate in cases similar to yours. But when I try that with your case, I still get the unexpected conflict -- see below.
Finally, migration from ClearCase is notoriously difficult to do right. The output of such a migration tool cannot be considered /a priori/ to be valid or reasonable Subversion usage.
Now some detailed responses.
> In this case the common starting point is revision zero, i.e. the initial empty
> state of the repository. I do understand that most Subversion merge use cases
> have a non-empty common starting point, but this case does not.
For the purpose of automatic merging -- that is, using merge tracking to figure out what to merge when you don't specify the revisions -- Subversion assumes there is a common ancestor of the *node* (file or directory) that you're merging. The empty state of the repository (the root directory at revision zero) doesn't serve this purpose, because (in your test case) the directories you are merging are independently created *subdirectories* of the root directory, they are not *branches* of the root directory.
The node I'm talking about is the root node of the merge: that is, in your test case, the directory 'B1' (source branch) and 'B2' (the target branch).
You didn't state what nodes were already present at the start of your test case; I'm assuming 'test1' was not present and so 'B1' was created by the first commit shown, and 'B2' originated as 'T1' which was created in the second commit shown.
(If, however, 'B1' and 'T1' did already exist at the beginning of the test case, and had a common ancestor, then an automatic merge would be valid and the analysis would be a bit different.)
> You can excuse Subversion's behavior by saying the two directories named D2
> were created independently, so they can't be merged, but that doesn't
> make the current behavior useful. Anyway the two directories D1 in my testcase
> were also created independently, yet Subversion was able to merge them. Is it
> really right that independently created directories with identical sets of child
> names can be merged, but independently created directories with disjoint sets of
> child names cannot be merged?
Like I said, we're not merging directories, we're merging changes. The distinction is important. More precisely, in this kind of merge, we're merging a change into an existing tree. What change is Subversion trying to merge into the directory D1 in the target branch, and what change into D2? If I run your commands starting with an empty repository, I can see exactly what happens:
[[[
+ svn mkdir --parents $REPOS/test1/branches/B1/D1/D2/A -m xx
Committed revision 1.
+ svn mkdir --parents $REPOS/test1/tags/T1/D1/D2/B -m xx
Committed revision 2.
...
+ cd B2
+ svn merge -r0:HEAD $REPOS/test1/branches/B1
--- Merging r2 through r5 into '.':
C D1/D2/A
--- Recording mergeinfo for merge of r2 through r5 into '.':
U .
Summary of conflicts:
Tree conflicts: 1
]]]
The "--- Merging r2 through r5" line shows that Subversion decides the first change it needs to merge into the target is r2. Notably, it does not try to merge r1, which included the *creation* of 'B1' and of 'A'. The first change within B1 that it tries to merge is a change *inside* A (the creation of 'test1.txt'), and this change can't be merged into the target because the target has no 'A' inside which to make that change.
That is why you see a tree conflict described as 'local delete, incoming edit'. It appears that the directory 'A', which was expected to exist in the target already before change r2 can be merged, seems to be missing from the target and so (because we assume you started from a common state) appears to have been 'deleted'.
I'm not entirely sure whether Subversion should be choosing to merge a bit more -- perhaps "all of revision r1 except for the creation of the branch-root 'B1'" -- but I at this point I still think the use case is invalid. Or, put another way, the use case is certainly something people might want to do from time to time but the "merge" command is not the tool to do it. Rather it requires some manual steps to set up a starting point from which "merge" can then do the rest.
> On Jan 14, 2013, at 10:03 PM, Ben Reser wrote:
>> Looks to me like you're using the merge command wrong.
>>
>> 1) You're using the sync merge format of the 1.7.x merge command,
>> however the two branches you're merging (B1 and B2) have no common
>> ancestor. Since B2 was copied from T1 which you independently
>> created, there's really no way for the command to know what to do.
>> The fact that it does anything can be considered a bug
>
> I would consider it a bug that svn merge didn't recognize that the initial
> empty state of the repository was the common ancestor, so it needed to merge all
> the changes on both branches. You might disagree.
As explained above, Subversion has a stricter definition of node ancestry.
>> 2) If what you're really intending to do here is a cherry-pick merge
>> of the changes that were made on B1 (adding A and test1.txt) then....
>
> Isn't it strange to you that you tried a bunch of variations and some of
> them worked and others didn't? It seems strange to me. Maybe that's
> because I don't understand Subversion well enough to see a pattern in what
> works and what doesn't.
Ben's demonstration didn't seem strange to me, it was spot on as an aid to understanding (rather than as a practical solution). He was showing the kind of steps that are needed to go from a pair of unrelated branches to a point where you can use the "merge" command to do the rest. It might make more sense to you now in retrospect if you've followed what I wrote above.
>> Your example use case seems so far from any realistic
>> scenario that it's hard to envision what you're actually doing here.
>
> I didn't want to distract you by providing information about anything other
> than the boiled-down test case, but maybe that was a mistake. The real-life
> situation involves migration from ClearCase using ClearVision's migrate2svn
> tool. Because of the way that tool works, elements which were the same object
> in ClearCase can be created multiple times independently in Subversion, without
> telling Subversion that they are related, other than having the same name in
> different branches. In addition, that tool can create partially populated
> branches which then need to be merged together to create a fully populated
> branch. It was that final merge that failed with tree conflicts. So it really
> is a realistic scenario if you think migration from ClearCase to Subversion is
> realistic.
It is the migration tool's job to convert the concepts of branches and everything else from ClearCase to Subversion. This is notoriously hard for CC -> SVN in particular, and most of the available tools are woefully inadequate in general. I spoke recently to someone who to the best of my knowledge is at the state of the art in CC->SVN tooling, having solved most of these kind of issues in many large migrations; ask about that in a separate email thread and I'll try to find his details if you're interested.
>> I can understand the argument that maybe we should handle this
>> directory creation conflict a little more gracefully, but it does seem
>> to be a legitimate tree conflict.
>
> If one of the two things being merged created a directory and the other deleted
> the same directory, I would call that a tree conflict. But I don't see how
> two adds to a directory can be a conflict; if the names being added are
> different, there is no conflict. If the names being added are the same, merge
> the children. However, I don't understand Subversion's definition of
> tree conflict. http://svnbook.red-bean.com/en/1.7 doesn't define it, and
> the example it gives involves rename vs. edit conflict, not add vs. add
> conflict.
Two adds of different names to the same directory do not conflict. The conflict you saw is on attempting to add a file into a directory ('A') that didn't exist on the target.
> I don't really want to have an argument. I am just trying to point out that
> Subversion's behavior is not useful in this case. I am also not sure
> Subversion merge is behaving consistently in all cases. It's the Subversion
> development community's choice whether to improve the behavior or leave it
> like it is.
OK. I am (and we are) certainly aware that there is a lot of inconsistency and lack of simple useful "just do the right thing" behaviour in this area. We are working on it.
Thank you for taking the time to provide this feedback.
- Julian
> And before I forget, thank you both for the quick and thoughtful responses.
>
> --Dave Moon
Re: Subversion merge creates bogus tree conflicts
Posted by David Moon <da...@versata.com>.
On Jan 15, 2013, at 9:40 AM, Stefan Sperling wrote:
> What you'll need to consider is that people will respond to your
> question in at least two ways:
>
> 1) Discuss and suggest desired behaviour to address your
> particular use
> case, with an eye on improving a future version of Subversion.
>
> 2) Help you understand the limitations of the current implementation,
> and perhaps suggest ways of dealing with your use case with the
> current implementation.
Thanks for clarifying that.
I personally do not need #2, as I have already switched to a
migrate2svn use case that does not require any use of svn merge. I
probably was not clear enough about that at the end of my initial email.
--Dave Moon
Re: Subversion merge creates bogus tree conflicts
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Jan 15, 2013 at 09:18:15AM -0500, David Moon wrote:
> Is it really right that
> independently created directories with identical sets of child names
> can be merged, but independently created directories with disjoint
> sets of child names cannot be merged?
No, of course it isn't. What Subversion should really be doing here
is offering functionality which automates tree conflict resolution
to the point where the user can easily trigger the behaviour best
suited for the particular use case at hand.
However, there is the idealised behaviour the community would like to
see in the future, and there is the current behaviour of the implementation.
The current behaviour has been derived from design goals that have long
been reached (supersede CVS), and as technical demands on Subversion
increase (do lots of refactoring and branching and merging, which nobody
really did in CVS) we're finding more and more cases where Subversion's
current behaviour is insufficient.
Part of the process of improving Subversion's behaviour are discussions
such as the one you have raised. This is very important to allow us to
grow the tool into a direction the userbase will be happier with.
What you'll need to consider is that people will respond to your
question in at least two ways:
1) Discuss and suggest desired behaviour to address your particular use
case, with an eye on improving a future version of Subversion.
2) Help you understand the limitations of the current implementation,
and perhaps suggest ways of dealing with your use case with the
current implementation.
Please try not to confuse a response that attempts to help you do
point 2 with a response that discusses point 1.
I believe Julian is trying to help you to address point 2, to allow
you to fix your problem with the current implementation somehow.
You seem to be responding to that with a question that belongs into a
discussion of point 1.
Just making sure you're not talking past each other :)
Re: Subversion merge creates bogus tree conflicts
Posted by David Moon <da...@versata.com>.
On Jan 15, 2013, at 8:38 AM, Julian Foad wrote:
> I think the main point is you need to understand that
>
> "svn merge" doesn't calculate the union of two sets of nodes, it
> combines
> two sets of *changes* that have been made since a common starting
> point.
>
> You might want to read the section on merging in The Book <http://
> svnbook.red-bean.com/>.
I do understand that, and I did read that.
In this case the common starting point is revision zero, i.e. the
initial empty state of the repository. I do understand that most
Subversion merge use cases have a non-empty common starting point,
but this case does not.
You can excuse Subversion's behavior by saying the two directories
named D2 were created independently, so they can't be merged, but
that doesn't make the current behavior useful. Anyway the two
directories D1 in my testcase were also created independently, yet
Subversion was able to merge them. Is it really right that
independently created directories with identical sets of child names
can be merged, but independently created directories with disjoint
sets of child names cannot be merged?
On Jan 14, 2013, at 10:03 PM, Ben Reser wrote:
> Looks to me like you're using the merge command wrong.
>
> 1) You're using the sync merge format of the 1.7.x merge command,
> however the two branches you're merging (B1 and B2) have no common
> ancestor. Since B2 was copied from T1 which you independently
> created, there's really no way for the command to know what to do.
> The fact that it does anything can be considered a bug
I would consider it a bug that svn merge didn't recognize that the
initial empty state of the repository was the common ancestor, so it
needed to merge all the changes on both branches. You might disagree.
> 2) If what you're really intending to do here is a cherry-pick merge
> of the changes that were made on B1 (adding A and test1.txt) then....
Isn't it strange to you that you tried a bunch of variations and some
of them worked and others didn't? It seems strange to me. Maybe
that's because I don't understand Subversion well enough to see a
pattern in what works and what doesn't.
> Your example use case seems so far from any realistic
> scenario that it's hard to envision what you're actually doing here.
I didn't want to distract you by providing information about anything
other than the boiled-down test case, but maybe that was a mistake.
The real-life situation involves migration from ClearCase using
ClearVision's migrate2svn tool. Because of the way that tool works,
elements which were the same object in ClearCase can be created
multiple times independently in Subversion, without telling
Subversion that they are related, other than having the same name in
different branches. In addition, that tool can create partially
populated branches which then need to be merged together to create a
fully populated branch. It was that final merge that failed with
tree conflicts. So it really is a realistic scenario if you think
migration from ClearCase to Subversion is realistic.
> I can understand the argument that maybe we should handle this
> directory creation conflict a little more gracefully, but it does seem
> to be a legitimate tree conflict.
If one of the two things being merged created a directory and the
other deleted the same directory, I would call that a tree conflict.
But I don't see how two adds to a directory can be a conflict; if the
names being added are different, there is no conflict. If the names
being added are the same, merge the children. However, I don't
understand Subversion's definition of tree conflict. http://
svnbook.red-bean.com/en/1.7 doesn't define it, and the example it
gives involves rename vs. edit conflict, not add vs. add conflict.
I don't really want to have an argument. I am just trying to point
out that Subversion's behavior is not useful in this case. I am also
not sure Subversion merge is behaving consistently in all cases.
It's the Subversion development community's choice whether to improve
the behavior or leave it like it is.
And before I forget, thank you both for the quick and thoughtful
responses.
--Dave Moon
Re: Subversion merge creates bogus tree conflicts
Posted by Julian Foad <ju...@btopenworld.com>.
Ben Reser wrote:
> On Mon, Jan 14, 2013 at 8:49 AM, David Moon <da...@versata.com> wrote:
>> Subversion creates an unnecessary tree conflict when two branches of a
>> directory are merged. See the transcript below. Since the contents of the
>> directory do not conflict between the two branches, it should just add both
>> subdirectories to the directory and not report any conflict. In my opinion,
>> that should be the default merge behavior, but there isn't even any way
>> using svn resolve to tell it to do that! The only thing svn resolve can do
>> is lose the incoming change.
>>
>> The unnecessary tree conflict does not always happen, based on I know not
>> what. But the transcript below seems to be a reliable way to reproduce it.
>>
>> This transcript is boiled down to a simple test case. In the real thing
>> there were too many of these tree conflicts to fix by hand, so fixing it by
>> hand is not a solution even if I could figure out how to fix it by hand.
>>
>> $REPOS is file:/// something.
>>
>>> svn --version
>> svn, version 1.7.8 (r1419691)
>> compiled Dec 12 2012, 14:18:28
>>
>>> svn mkdir --parents $REPOS/test1/branches/B1/D1/D2/A -m xx
>>
>>> svn mkdir --parents $REPOS/test1/tags/T1/D1/D2/B -m xx
>>
>>> svn co $REPOS/test1/branches/B1/D1/D2/A
>>> echo test > A/test1.txt
>>> cd A; svn add test1.txt
>>> svn commit -m xx
>>> cd ..
>>
>>> svn co $REPOS/test1/tags/T1/D1/D2/B
>>> echo test > B/test2.txt
>>> cd B; svn add test2.txt
>>> svn commit -m xx
>>> cd ..
>>
>>> svn copy $REPOS/test1/tags/T1 $REPOS/test1/branches/B2 -m xx
>>
>>> svn list -R $REPOS/test1/branches/B1
>> D1/
>> D1/D2/
>> D1/D2/A/
>> D1/D2/A/test1.txt
>>> svn list -R $REPOS/test1/branches/B2
>> D1/
>> D1/D2/
>> D1/D2/B/
>> D1/D2/B/test2.txt
>>
>>> svn co $REPOS/test1/branches/B2
>>> cd B2
>>> svn merge $REPOS/test1/branches/B1
>> --- Merging r167502 through r167505 into '.':
>> C D1/D2/A
>> --- Recording mergeinfo for merge of r167502 through r167505 into '.':
>> U .
>> Summary of conflicts:
>> Tree conflicts: 1
>>> svn status
>> M .
>> ! C D1/D2/A
>> > local delete, incoming edit upon merge
>> Summary of conflicts:
>> Tree conflicts: 1
>>> ls D1/D2
>> B/
>>
>> Note: ls D1/D2 should say A/ B/ and there should be no tree conflict.
>>
>> In the real thing, the svn status explanation of the tree conflict is
>> usually "local add, incoming add upon merge." I can't explain why the
>> simple test case says local delete instead. I don't know if this matters.
>>
>> I can't say that Subversion is not behaving according to the documentation,
>> because the documentation at http://svnbook.red-bean.com/en/1.7/index.html
>> is extremely vague about this aspect of Subversion. But it's not behaving
>> in a useful way.
>>
>> I hope this bug report is useful to you. I can't wait for a fix, so I am
>> changing to a different strategy that does not depend on Subversion merge.
>
> Looks to me like you're using the merge command wrong.
>
> 1) You're using the sync merge format of the 1.7.x merge command,
> however the two branches you're merging (B1 and B2) have no common
> ancestor. Since B2 was copied from T1 which you independently
> created, there's really no way for the command to know what to do.
> The fact that it does anything can be considered a bug, I'll note that
> on trunk right now the example you give above fails with the following
> error:
> [[[
> svn: E205000: Try 'svn help merge' for more information
> svn: E205000: Source and target must be different but related branches
> svn: E205000: Source and target have no common ancestor:
> 'file:///Users/breser/moon/repo/test1/branches/B1@head' and
> '.@unspecified'
> ]]]
>
> Based on the behavior it seems that in 1.7.x the command is somehow
> trying to do a cherry-pick.
>
> 2) If what you're really intending to do here is a cherry-pick merge
> of the changes that were made on B1 (adding A and test1.txt) then you
> want to use the cherry-pick form:
> svn merge [-c M[,N...] | -r N:M ...] SOURCE[@REV] [TARGET_WCPATH]
>
> Note that you need either -c or -r to specify the range of revisions.
>
> You might be tempted to try (assuming that r1 is where A was added)
> svn merge -r0:HEAD $REPOS/test1/branches/B1
>
> However, you'll still get the conflict:
> [[[
> $ svn merge -r0:HEAD $REPOS/test1/branches/B1
> --- Merging r2 through r5 into '.':
> C D1/D2/A
> --- Recording mergeinfo for merge of r2 through r5 into '.':
> U .
> Summary of conflicts:
> Tree conflicts: 1
> ]]]
>
> and a similar status:
> [[[
> $ svn status
> M .
> ! C D1/D2/A
> > local delete, incoming edit upon merge
> Summary of conflicts:
> Tree conflicts: 1
> ]]]
>
> If I modify your example by adding the following two commands onto the front:
> [[[
> svn mkdir --parents $REPOS/test1/branches/B1/D1/D2 -m xx
> svn mkdir --parents $REPOS/test1/tags/T1/D1/D2 -m xx
> ]]]
>
> Which shifts the creation of your shared paths from the differences
> onto separate versions (and as such A isn't created until r3) then I
> get the following a successful merge by doing:
> [[[
> $ svn merge -r2:HEAD $REPOS/test1/branches/B1
> --- Merging r3 through r7 into '.':
> A D1/D2/A
> A D1/D2/A/test1.txt
> --- Recording mergeinfo for merge of r3 through r7 into '.':
> U .
>
> $ svn status
> M .
> A + D1/D2/A
>
> ]]]
>
> So what's happening here is that Subversion is conflicting on trying
> to create D1 and D2 because the creation of these directories are in
> the revision you're trying to merge.
>
> You can work around this by doing:
> svn mkdir D1/D2/A
>
> and then doing the merge as follows:
> svn merge -r0:HEAD $REPOS/test1/branches/B1/D1/D2/A D1/D2/A
>
> This works because the creation of those directories is now filtered
> out as happening below the path you're merging.
>
> It's hard to say if there's a bug here or not based on what you've
> said so far. Your example use case seems so far from any realistic
> scenario that it's hard to envision what you're actually doing here.
>
> The local delete on the status notification is a bug of some sort in
> 1.7 but it doesn't happen in trunk and so many things have changed
> between the two it's really hard to be sure why.
>
> I can understand the argument that maybe we should handle this
> directory creation conflict a little more gracefully, but it does seem
> to be a legitimate tree conflict.
>
> Hopefully someone else can chime in with a little more knowledge of
> this code than I have.
I think the main point is you need to understand that
"svn merge" doesn't calculate the union of two sets of nodes, it combines
two sets of *changes* that have been made since a common starting point.
You might want to read the section on merging in The Book <http://svnbook.red-bean.com/>.
The other point of note is that the difference between getting an "add versus add" tree conflict in your real case and a "delete versus add" conflict in your test case points to your test case not reflecting your real case -- those are two very different scenarios.
- Julian
Re: Subversion merge creates bogus tree conflicts
Posted by Ben Reser <be...@reser.org>.
On Mon, Jan 14, 2013 at 8:49 AM, David Moon <da...@versata.com> wrote:
> Subversion creates an unnecessary tree conflict when two branches of a
> directory are merged. See the transcript below. Since the contents of the
> directory do not conflict between the two branches, it should just add both
> subdirectories to the directory and not report any conflict. In my opinion,
> that should be the default merge behavior, but there isn't even any way
> using svn resolve to tell it to do that! The only thing svn resolve can do
> is lose the incoming change.
>
> The unnecessary tree conflict does not always happen, based on I know not
> what. But the transcript below seems to be a reliable way to reproduce it.
>
> This transcript is boiled down to a simple test case. In the real thing
> there were too many of these tree conflicts to fix by hand, so fixing it by
> hand is not a solution even if I could figure out how to fix it by hand.
>
> $REPOS is file:/// something.
>
>> svn --version
> svn, version 1.7.8 (r1419691)
> compiled Dec 12 2012, 14:18:28
>
>> svn mkdir --parents $REPOS/test1/branches/B1/D1/D2/A -m xx
>
>> svn mkdir --parents $REPOS/test1/tags/T1/D1/D2/B -m xx
>
>> svn co $REPOS/test1/branches/B1/D1/D2/A
>> echo test > A/test1.txt
>> cd A; svn add test1.txt
>> svn commit -m xx
>> cd ..
>
>> svn co $REPOS/test1/tags/T1/D1/D2/B
>> echo test > B/test2.txt
>> cd B; svn add test2.txt
>> svn commit -m xx
>> cd ..
>
>> svn copy $REPOS/test1/tags/T1 $REPOS/test1/branches/B2 -m xx
>
>> svn list -R $REPOS/test1/branches/B1
> D1/
> D1/D2/
> D1/D2/A/
> D1/D2/A/test1.txt
>> svn list -R $REPOS/test1/branches/B2
> D1/
> D1/D2/
> D1/D2/B/
> D1/D2/B/test2.txt
>
>> svn co $REPOS/test1/branches/B2
>> cd B2
>> svn merge $REPOS/test1/branches/B1
> --- Merging r167502 through r167505 into '.':
> C D1/D2/A
> --- Recording mergeinfo for merge of r167502 through r167505 into '.':
> U .
> Summary of conflicts:
> Tree conflicts: 1
>> svn status
> M .
> ! C D1/D2/A
> > local delete, incoming edit upon merge
> Summary of conflicts:
> Tree conflicts: 1
>> ls D1/D2
> B/
>
> Note: ls D1/D2 should say A/ B/ and there should be no tree conflict.
>
> In the real thing, the svn status explanation of the tree conflict is
> usually "local add, incoming add upon merge." I can't explain why the
> simple test case says local delete instead. I don't know if this matters.
>
> I can't say that Subversion is not behaving according to the documentation,
> because the documentation at http://svnbook.red-bean.com/en/1.7/index.html
> is extremely vague about this aspect of Subversion. But it's not behaving
> in a useful way.
>
> I hope this bug report is useful to you. I can't wait for a fix, so I am
> changing to a different strategy that does not depend on Subversion merge.
Looks to me like you're using the merge command wrong.
1) You're using the sync merge format of the 1.7.x merge command,
however the two branches you're merging (B1 and B2) have no common
ancestor. Since B2 was copied from T1 which you independently
created, there's really no way for the command to know what to do.
The fact that it does anything can be considered a bug, I'll note that
on trunk right now the example you give above fails with the following
error:
[[[
svn: E205000: Try 'svn help merge' for more information
svn: E205000: Source and target must be different but related branches
svn: E205000: Source and target have no common ancestor:
'file:///Users/breser/moon/repo/test1/branches/B1@head' and
'.@unspecified'
]]]
Based on the behavior it seems that in 1.7.x the command is somehow
trying to do a cherry-pick.
2) If what you're really intending to do here is a cherry-pick merge
of the changes that were made on B1 (adding A and test1.txt) then you
want to use the cherry-pick form:
svn merge [-c M[,N...] | -r N:M ...] SOURCE[@REV] [TARGET_WCPATH]
Note that you need either -c or -r to specify the range of revisions.
You might be tempted to try (assuming that r1 is where A was added)
svn merge -r0:HEAD $REPOS/test1/branches/B1
However, you'll still get the conflict:
[[[
$ svn merge -r0:HEAD $REPOS/test1/branches/B1
--- Merging r2 through r5 into '.':
C D1/D2/A
--- Recording mergeinfo for merge of r2 through r5 into '.':
U .
Summary of conflicts:
Tree conflicts: 1
]]]
and a similar status:
[[[
$ svn status
M .
! C D1/D2/A
> local delete, incoming edit upon merge
Summary of conflicts:
Tree conflicts: 1
]]]
If I modify your example by adding the following two commands onto the front:
[[[
svn mkdir --parents $REPOS/test1/branches/B1/D1/D2 -m xx
svn mkdir --parents $REPOS/test1/tags/T1/D1/D2 -m xx
]]]
Which shifts the creation of your shared paths from the differences
onto separate versions (and as such A isn't created until r3) then I
get the following a successful merge by doing:
[[[
$ svn merge -r2:HEAD $REPOS/test1/branches/B1
--- Merging r3 through r7 into '.':
A D1/D2/A
A D1/D2/A/test1.txt
--- Recording mergeinfo for merge of r3 through r7 into '.':
U .
$ svn status
M .
A + D1/D2/A
]]]
So what's happening here is that Subversion is conflicting on trying
to create D1 and D2 because the creation of these directories are in
the revision you're trying to merge.
You can work around this by doing:
svn mkdir D1/D2/A
and then doing the merge as follows:
svn merge -r0:HEAD $REPOS/test1/branches/B1/D1/D2/A D1/D2/A
This works because the creation of those directories is now filtered
out as happening below the path you're merging.
It's hard to say if there's a bug here or not based on what you've
said so far. Your example use case seems so far from any realistic
scenario that it's hard to envision what you're actually doing here.
The local delete on the status notification is a bug of some sort in
1.7 but it doesn't happen in trunk and so many things have changed
between the two it's really hard to be sure why.
I can understand the argument that maybe we should handle this
directory creation conflict a little more gracefully, but it does seem
to be a legitimate tree conflict.
Hopefully someone else can chime in with a little more knowledge of
this code than I have.