unexpected tree conflict on merge for same source file

[Sven Uhlig <sv...@web.de>]

Hello,

imagine the following repository structure:

/branches/prj1/first.txt
/branches/prj1/second.txt
/branches/prj2/first.txt
/branches/testing/first.txt
/branches/testing/second.txt
/trunk/first.txt

All first.txt in /branches/* are from the same /trunk/first.txt
The file was only changed in /trunk and synced in /branches/prj1/

prj1 was merged into testing

Why is there a tree conflict when I try to merge prj2 into testing? It
is the same file.

Below is a .bat file for creating the repo with some more conflicts that
we do not understand. I would be happy if someone could explain these as
well.


What I try to achieve with the additional testing branch is to have some
kind of staging before going live.

The idea is to do this for new features:
1. Create branch from the latest live system /(trunk)
2. Work on branch
3. Merge branch into testing system (/branches/testing)
4a If everything is ok merge into /trunk and delete branch
4b If something is wrong return to 2.

Best regards
Sven.


@REM testsvn.bat

cls

svn --version
@REM svn, Version 1.7.2 (r1207936)
@pause
svnadmin --version
@REM svnadmin, Version 1.7.2 (r1207936)
@pause
D:
cd \
mkdir test
cd test

svnadmin create repo
svn checkout file:///D:/test/repo wc

cd wc
mkdir branches
mkdir tags
mkdir trunk
svn add branches tags trunk
svn commit -m "basic repo structure"
svn switch --ignore-ancestry file:///D:/test/repo/trunk

svn info
@REM Working Copy Root Path: D:\test\wc
@REM URL: file:///D:/test/repo/trunk
@pause

echo .>first.txt
svn add first.txt
svn commit -m "1st"

svn copy -m "prj0" . "^/branches/prj0"

svn copy -r HEAD -m "prj1" "^/trunk" "^/branches/prj1"
svn switch "^/branches/prj1"
echo .>second.txt
svn add second.txt
svn commit -m "2nd"

svn switch "^/trunk"
svn copy -r HEAD -m "testing" "^/trunk" "^/branches/testing"

echo "change" > first.txt
svn commit -m "change"
svn copy -r HEAD -m "prj2" "^/trunk" "^/branches/prj2"

echo "change2" > first.txt
svn commit -m "change2"
svn copy -m "prj3" . "^/branches/prj3"

svn switch "^/branches/prj2"
svn merge "^/branches/prj1"
svn commit -m "prj1 into prj2"

svn switch "^/branches/prj3"
svn merge "^/branches/prj0"
@REM    C first.txt
@REM --- Recording mergeinfo for merge of r2 through r11 into '.':
@REM  U   .
@pause
svn st
@REM  M      .
@REM       C first.txt
@REM       >   local add, incoming add upon merge
@pause
svn revert -R .

svn switch "^/branches/prj1"
svn merge "^/trunk"
svn commit -m "prj1 sync"

svn switch "^/branches/testing"
svn merge "^/branches/prj1"
svn commit -m "prj1 into testing"

svn merge "^/branches/prj2"
@REM --- Recording mergeinfo for merge of r2 through r7 into '.':
@REM  U   .
@REM -- Zusammenfuehren von r8 bis r13 in '.':
@REM    C second.txt
@REM  G   .
@REM --- Recording mergeinfo for merge of r8 through r13 into '.':
@REM  U   .
@pause
svn st
@REM  M      .
@REM       C second.txt
@REM       >   local add, incoming add upon merge
@pause
svn revert -R .

cd \

Re: unexpected tree conflict on merge for same source file

[Sven Uhlig <sv...@web.de>]

Am 17.10.2012 12:00, schrieb Stefan Sperling:
> On Tue, Oct 16, 2012 at 06:35:21PM +0200, Stefan Sperling wrote:
>> Again, sticking to simple branching/merging patterns where all merges
>> happen between directly related branches makes things a lot easier.
> 
> Hi again Sven,
> 
> I've looked at this some more trying to come up with a simpler solution.

You are really into this problem. Thank you for your effort! :)

> It turns out that the root of all evil really is the mixed-revision
> copy you've made to create the first branch. If the branch is created
> by copying URLs the add vs. add tree conflict problem disappears.

As written in my first reply: The mixed-revision copies seem to be a
problem with the usage of TortoiseSVN. I will check the next time when I
create abranch using this 3rd party tool.

> Also, a 2-URL merge is not strictly required to run the merge between
> the two unrelated (or "cousin") branches. But you need to sync the
> branch you're merging from to trunk first, which moves the common
> ancestor point forward in history [...] 
> This requires a
> record-only merge to work around the mixed-revisionness of the branch
> but is otherwise straightforward.
> 
> Below is a unix shell script I used to reproduce the problem.
> See the comments inside the script for more details.
> 
> svn add $trunk/foo
> svn ci $trunk -m "add new file"
> 
> # mixed-rev copy!
> svn copy $trunk $branch1_url -m "creating branch 1"
> # doing this instead would prevent tree conflicts below:
> # svn copy $trunk_url $branch1_url -m "creating branch 1"

Doing a "svn up" before the "svn copy" in the root of the working copy
should the trick as well, right?

> # sync branch1 to trunk, moving the common ancestor used for the merge
> # beyond the creation of 'foo' to avoid the above tree conflict
> #
> 
> # r2 created 'foo' which exists on branch1 because of mixed-rev copy.
> # Mark r2 as merged into branch1 to avoid another add vs. add tree-conflict.
> svn up $branch1
> svn merge --record-only -c2 $trunk_url $branch1

Doing this requires that you know a lot about what happens in other
branches. There could be hundreds of commits in each of the branches and
trunk by multiple people over several month.

Why is it not possible for SVN to see that the source files are the same
and the trunk changed the contents that are now valid?

See also my other mail for a similar question.

Best regards
Sven.

PS: Sorry for cross referencing to other mails but I don't see an easy
option to reply to multiple mails in one new mail.

Re: unexpected tree conflict on merge for same source file

[Stefan Sperling <st...@elego.de>]

On Tue, Oct 16, 2012 at 06:35:21PM +0200, Stefan Sperling wrote:
> Again, sticking to simple branching/merging patterns where all merges
> happen between directly related branches makes things a lot easier.

Hi again Sven,

I've looked at this some more trying to come up with a simpler solution.

It turns out that the root of all evil really is the mixed-revision
copy you've made to create the first branch. If the branch is created
by copying URLs the add vs. add tree conflict problem disappears.

Also, a 2-URL merge is not strictly required to run the merge between
the two unrelated (or "cousin") branches. But you need to sync the
branch you're merging from to trunk first, which moves the common
ancestor point forward in history, beyond the creation of the file
which is causing the add vs. add tree conflict. This requires a
record-only merge to work around the mixed-revisionness of the branch
but is otherwise straightforward.

Below is a unix shell script I used to reproduce the problem.
See the comments inside the script for more details.

#!/bin/sh

set -e

cwd=`pwd`
basename=`basename $0`
scratch_area="`echo $basename | sed -e s/\.sh$//`"
repos=$scratch_area/repos
trunk=$scratch_area/trunk
branch1=$scratch_area/branch1
branch2=$scratch_area/branch2
trunk_url=file:///$cwd/$repos/trunk
branch1_url=file:///$cwd/$repos/branch1
branch2_url=file:///$cwd/$repos/branch2

set -x

rm -rf $scratch_area
mkdir -p $scratch_area

mkdir -p $trunk
echo alpha > $trunk/alpha
echo beta > $trunk/beta
mkdir $trunk/gamma
echo delta > $trunk/gamma/delta
mkdir $trunk/epsilon
echo zeta > $trunk/epsilon/zeta

svnadmin create $cwd/$repos
svn import $trunk $trunk_url -m "importing project tree"
rm -rf $trunk
svn checkout $trunk_url $trunk

echo foo > $trunk/foo
svn add $trunk/foo
svn ci $trunk -m "add new file"

# mixed-rev copy!
svn copy $trunk $branch1_url -m "creating branch 1"
# doing this instead would prevent tree conflicts below:
# svn copy $trunk_url $branch1_url -m "creating branch 1"

svn checkout $branch1_url $branch1
echo bar >> $branch1/alpha
svn ci $branch1 -m "change alpha on branch 1"

echo foo >> $trunk/foo
svn ci $trunk -m "change foo on trunk"

svn copy $trunk_url $branch2_url -m "creating branch 2"
svn checkout $branch2_url $branch2

# merge branch1 into branch2 -> tree conflict
# "local add, incoming add upon merge"
svn merge $branch1_url $branch2
svn status $branch2

# remove merged changes and tree conflict
svn revert -R $branch2

#
# sync branch1 to trunk, moving the common ancestor used for the merge
# beyond the creation of 'foo' to avoid the above tree conflict
#

# r2 created 'foo' which exists on branch1 because of mixed-rev copy.
# Mark r2 as merged into branch1 to avoid another add vs. add tree-conflict.
svn up $branch1
svn merge --record-only -c2 $trunk_url $branch1

# Now merging trunk -> branch1 works fine
svn merge $trunk_url $branch1
svn status $branch1
svn commit -m "sync branch 1 to trunk" $branch1

# Now merging branch1 -> branch2 works fine
svn update $branch2
svn merge $branch1_url $branch2
svn status $branch2
svn diff $branch2

Re: unexpected tree conflict on merge for same source file

[Stefan Sperling <st...@elego.de>]

On Tue, Oct 16, 2012 at 06:02:55PM +0200, Stefan Sperling wrote:
> If you really must merge between unrelated branches, use the 2-URL
> merge syntax instead of the 'svn merge ^/foo' short-hand syntax and
> specify revisions for each of the 2 URL arguments to merge meaningful
> changesets. For instance, say you created '^/branches/A' from trunk in
> r399, and then made some interesting changes between r410 and r450 on
> branch A.  You want to merge those changes into a completely unrelated
> branch called 'testing'. Try something like this:
> 
>   svn switch ^/branches/testing
>   svn merge ^/branches/A@410 ^/branches/prj@450 .

Sorry the above should have said:

  svn merge ^/branches/testing@410 ^/branches/A@450 .

This assumes that the delta between testing@410 and A@450 contains
only changes that are unique to branch A, which will be the case
unless you merged changes from other branches into A between r410
and 450. You can check the delta with the following command:

  svn diff ^/branches/testing@410 ^/branches/A@450

If the delta contains changes you don't want to merge, you can either
run the merge anyway and resolve conflicts (if any), or try this instead:

  svn merge ^/branches/A@410 ^/branches/A@450 .

If the above gives you tree-conflicts either resolve them as appropriate
or try again with --ignore-ancestry:

  svn merge --ignore-ancestry ^/branches/A@410 ^/branches/A@450 .

If you throw away the 'testing' branch after testing anyway, the lack of
mergeinfo recorded during the --ignore-ancestry merge might not be
a problem.

Again, sticking to simple branching/merging patterns where all merges
happen between directly related branches makes things a lot easier.

Re: unexpected tree conflict on merge for same source file

[Stefan Sperling <st...@elego.de>]

On Sun, Oct 21, 2012 at 09:53:49AM +0200, Sven Uhlig wrote:
> Am 18.10.2012 15:06, schrieb Stefan Sperling:
> > And more help is always welcome, if you're interested :)
> 
> I'm not sure if I understand the last part "if you're interested." Do
> you mean help as active member of the mailing list, as a tester/bug
> reporter or help as a developer?

Anything you have time for!
The project is open to many ways of participation.
See http://subversion.apache.org/contributing.html

Re: unexpected tree conflict on merge for same source file

[Sven Uhlig <sv...@web.de>]

Am 18.10.2012 15:06, schrieb Stefan Sperling:
> On Thu, Oct 18, 2012 at 02:10:25PM +0200, Sven Uhlig wrote:
>> Am 17.10.2012 19:12, schrieb Stefan Sperling:
>> At the time when the branches were created there was no use of shared
>> code. Later some functions seem to be unexpectedly usefull in the other
>> branch.
> 
> That happens and is sometimes not avoidable, but it also indicates poor
> planning and/or developer coordination. [...]
> It sounds
> like it is an exceptional circumstance, which means we're not talking
> about a problem you are hitting often, which is good. Is that correct?

It is an exception. But I am afraid that this can happen at any time again.

> And don't get me wrong, I'm just describing how Subversion works to
> explain why you are seeing conflicts. I'm not saying these spurious
> conflicts are a good thing.

It is good that you have said that. I was not sure if you didn't
understand the problem or if you are just describing SVN's behaviour. So
you do understand and you are just describing - that's fine.

>> I think that SVN records the wrong revisions for mergeinfo or copy
>> because the file was not changed since addition but SVN still remembers
>> a newer revision than the latest change and thus sees a conflict/difference.
> 
> It doesn't record wrong information in the sense that the information
> is inaccurate.

My idea was that if SVN recorded other revisions or information then may
be SVN could see that it is the same file.

> Yes, there are some nice improvements coming in 1.8. But I don't
> think the spurious add vs. add conflict we're talking about will be
> automatically resolved in 1.8. Maybe in 1.9 or later.
> We're currently still working on more basic problems in this area.

Well, there are work-arounds available. If everyone knows them, and if
everyone knows that there will be improvements at some time, then
everything is fine.

Unfortunately 1.9 is far away in the future :(

> Reporting issues like this helps us to see what kinds of tree conflicts
> people are hitting in practice.

Before I posted to this list, I used Google to find similar issues and I
did. So I was not sure if you'll accept yet another question for the
same SVN error message. I'm glad to see that you still reply :)

> This is very valuable and helps the project.
> And more help is always welcome, if you're interested :)

I'm not sure if I understand the last part "if you're interested." Do
you mean help as active member of the mailing list, as a tester/bug
reporter or help as a developer?

Best regards
Sven.

PS: Sorry for the late reply.

Re: unexpected tree conflict on merge for same source file

[Stefan Sperling <st...@elego.de>]

On Thu, Oct 18, 2012 at 02:10:25PM +0200, Sven Uhlig wrote:
> Am 17.10.2012 19:12, schrieb Stefan Sperling:
> > On Wed, Oct 17, 2012 at 05:30:14PM +0200, Sven Uhlig wrote:
> >> There could be
> >> changes that are commited to prj1 that are requried in prj2 because of
> >> e.g. some shared code.
> >> 
> >> I would do this with the following command:
> >>   svn merge -r A:B "^/branches/prj1"
> >> 
> >> Reintegration is not an option because the development will continue on
> >> both branches in parallel.
> > 
> > This approach sounds very chaotic to me.
> 
> For me it sounds like the "Keep Alive Dance". (See also the very bottom
> of this mail.)

No, that is about something else. The keep-alive-dance and
cherry-picking are not in the same class of merge patterns.
The keep-alive dance is only used in sync-style merge patterns,
which merge all changes from their parent branch. Which isn't
what you are doing when cherry-picking changes between cousins.

> Yes, it makes merging more difficult. But that is what cherry-pick and
> svn:mergeinfo are for? svn:mergeinfo has all the required information if
> it looks into the past.

Looking into the past can be a rather complicated affair, especially
with mixed-revision copies ;)

> > In your case, why are the branches which need to share common code not
> > branches off a branch which contains the shared code?
> 
> At the time when the branches were created there was no use of shared
> code. Later some functions seem to be unexpectedly usefull in the other
> branch.

That happens and is sometimes not avoidable, but it also indicates poor
planning and/or developer coordination. It should be the exception
rather than the norm in a well-managed project. I wasn't sure whether
or not it is an exceptional circumstance in your work flow. It sounds
like it is an exceptional circumstance, which means we're not talking
about a problem you are hitting often, which is good. Is that correct?

> > If the above won't work for some reason, maybe you shouldn't be using
> > branching/merging in the first place. Instead, you could treat the
> > shared code as an external dependency which your build system will
> > pull into the build.
> 
> I think externals are not the right solutions if only single files or
> even only some lines of source code are to be shared.

That's right.

> Just before the conflict in r71 the paths for second.txt to it's
> original source (using your revisions) look like this:
> 1. prj1/second.txt@40 is the original file
> 2. prj2/second.txt@50 comes from prj1/second.txt@49
> 3. testing/second.txt@61 comes from prj1/second.txt@60
> 
> When merging prj2/ into testing, then SVN has two files "second.txt" at
> hand:
> 1. prj2/second.txt@50
> 2. testing/second.txt@61
> 
> But if SVN looked back in the history of those two files it would see
> both are from prj1/second.txt (@49 and @60)
> 
> Looking back in the history since those two revisions r49 and r60 there
> was no change. So they are not from r49 an r60 but in fact from r40.

The problem happens because we're merging second.txt's parents.
Given the way Subversion's repository works the addition of a file is
an operation on the file's parent directory. SVN is merging prj2@70 and,
say, testing@70. It isn't merging prj2/second.txt@50 and testing/second.txt@61.

If you want to merge just second.txt you need to tell 'svn merge' that
second.txt is your merge target. We call this a "subtree merge" because
it is a merge that is anchored below an actual branch root directory.
Such merges can be useful to help resolve some tree conflicts, for example.
It might even be a viable approach to solving your problem if merging
just 'second.txt' is enough to share the common code.

And don't get me wrong, I'm just describing how Subversion works to
explain why you are seeing conflicts. I'm not saying these spurious
conflicts are a good thing.

However, I'm not sure if we really want to trace back history for every
single node involved in the merge to avoid flagging such conflicts, which
is what your proposal essentially amounts to in the general case.
That would be rather expensive.

It would be better to teach Subversion about resolving this kind of
tree conflict automatically after it has been recorded. It might be
possible to figure out whether or not such an 'add vs add' conflict
is spurious, and then auto-resolve this conflict when the conflict
resolver is running. Note that in 1.8 and later, the conflict resolver
will always run after the update or merge operation is done. It won't
run during the operation as it does in 1.7 and earlier. This was done
in preparation for future tree-conflict resolution improvements.

> I think that SVN records the wrong revisions for mergeinfo or copy
> because the file was not changed since addition but SVN still remembers
> a newer revision than the latest change and thus sees a conflict/difference.

It doesn't record wrong information in the sense that the information
is inaccurate.

> In the meantime I found the following document in the roadmap for SVN
> 1.8: "merge improvements"/"Symmetric Merging"
> 
> http://wiki.apache.org/subversion/SymmetricMerge
> 
> I think this document discusses similar problems in the sections "1.6.
> The Keep-Alive Dance" and "1.9. Symmetric Merge with Criss-Cross Merge".
> 
> Seeing the discussion over there, I hope to see some improvements in the
> merging algorithms :) Unfortunately it seems I have to use record-only
> merges until then.

Yes, there are some nice improvements coming in 1.8. But I don't
think the spurious add vs. add conflict we're talking about will be
automatically resolved in 1.8. Maybe in 1.9 or later.
We're currently still working on more basic problems in this area.

Reporting issues like this helps us to see what kinds of tree conflicts
people are hitting in practice. This is very valuable and helps the project.
And more help is always welcome, if you're interested :)

Re: unexpected tree conflict on merge for same source file

[Sven Uhlig <sv...@web.de>]

Am 17.10.2012 19:12, schrieb Stefan Sperling:
> On Wed, Oct 17, 2012 at 05:30:14PM +0200, Sven Uhlig wrote:
>> There could be
>> changes that are commited to prj1 that are requried in prj2 because of
>> e.g. some shared code.
>> 
>> I would do this with the following command:
>>   svn merge -r A:B "^/branches/prj1"
>> 
>> Reintegration is not an option because the development will continue on
>> both branches in parallel.
> 
> This approach sounds very chaotic to me.

For me it sounds like the "Keep Alive Dance". (See also the very bottom
of this mail.)

> Usually you'd isolate branches from another precisely because each
> branch should *not* see code developed on the other cousin branches.
> But at the same time you want these branches to share some code?

Sometimes only a small part of the changes in branch A is needed in
branch B because the ideas evolve for branch B. Usually I isolate
branches as suggested.

> That's a contradicition. And it makes merging harder than it needs to be.
> 
> Every time you cherry-pick changes between cousins you run a high
> risk of merge conflicts later when the cousins are merged into other
> branches.

Yes, it makes merging more difficult. But that is what cherry-pick and
svn:mergeinfo are for? svn:mergeinfo has all the required information if
it looks into the past.

> In your case, why are the branches which need to share common code not
> branches off a branch which contains the shared code?

At the time when the branches were created there was no use of shared
code. Later some functions seem to be unexpectedly usefull in the other
branch.

> If the above won't work for some reason, maybe you shouldn't be using
> branching/merging in the first place. Instead, you could treat the
> shared code as an external dependency which your build system will
> pull into the build.

I think externals are not the right solutions if only single files or
even only some lines of source code are to be shared.

>> >> svn switch "^/branches/testing"
>> >> svn merge "^/branches/prj1"
>> >> svn commit -m "prj1 into testing"
>> >>
>> >> svn merge "^/branches/prj2"
>> > 
>> > Again, you hit a similar problem here.
>> 
>> I do not understand why there is a conflict in "second.txt".
>> 
>> With svn:mergeinfo the SVN should have enough information to see that
>> the files are the same. There was no change and  the original source is
>> the same (/branches/prj1/)
> 
> You created second.txt on the prj1 branch. Let's assume you did so
> in revision 40, so that the change within prj1@40 looks like this:
> 
>   A	second.txt
> 
> ("add a file called second.txt")
> 
> You then cherry-picked this revision from prj1 into prj2, causing a commit
> that added second.txt to prj2. Let's call this changeset 'prj2@50':
> 
>   A 	second.txt (copied from, say, prj1@49)
>   Mergeinfo addition: Merged prj1:40-49
> 
> Now you merge prj1 into testing, let's say in r61. The common ancestor of
> testing and prj1 is trunk, which does not contain second.txt.
> So among other changes you are merging:
> 
>   A	second.txt (copied from, say, prj1@60)
>   Mergeinfo addition: Merged prj1:40-60
> 
> Next, you merge prj2 into testing, let's say in r71. The common ancestor
> of prj2 and testing is trunk, which does not contain second.txt.
> So the changeset being merged looks something like this:
> 
>   A	second.txt (copied from, say, prj2@70)
>   [no mergeinfo addition for prj1 see Stefan Sperlings 2nd mail]
>   [old merginfo: Merged prj1:40-60, see previous merge]
>   Mergeinfo addition: Merged prj2:50-70
> 
> This results in the add vs. add tree conflict.
> 
> How is Subversion supposed to tell that prj1:40-70 contains the same
> semantic change as prj2:50-70, the change which we cherry-picked from
> prj1 to prj2? In terms of mergeinfo the addition of second.txt happens
> in two distinct changesets: prj1@40 and prj2@50
> Mergeinfo doesn't tell us that, semantically, prj1@40 and prj2@50 contain
> the same change.
>
> And neither does tracing back history to the common ancestor of prj2 and
> testing tell us that these 'second.txt' files are related.
> The common ancestor is trunk, which has never seen 'second.txt'.

Just before the conflict in r71 the paths for second.txt to it's
original source (using your revisions) look like this:
1. prj1/second.txt@40 is the original file
2. prj2/second.txt@50 comes from prj1/second.txt@49
3. testing/second.txt@61 comes from prj1/second.txt@60

When merging prj2/ into testing, then SVN has two files "second.txt" at
hand:
1. prj2/second.txt@50
2. testing/second.txt@61

But if SVN looked back in the history of those two files it would see
both are from prj1/second.txt (@49 and @60)

Looking back in the history since those two revisions r49 and r60 there
was no change. So they are not from r49 an r60 but in fact from r40.

I think that SVN records the wrong revisions for mergeinfo or copy
because the file was not changed since addition but SVN still remembers
a newer revision than the latest change and thus sees a conflict/difference.

> The fact that this is a spurious conflict is evident to us because we
> know how the second.txt addition to prj2 came about. But what if we had
> added a different 'second.txt' file to prj2, which just happens to have
> the same name but is otherwise unrelated to the existing file in testing
> and prj1?

I can see that two independent adds with the same file name can cause
conflicts. But the files have the same source and SVN knows about it.
See my explanation above.

> Subversion cannot tell the difference between this scenario
> and the one above, so it needs to flag a tree conflict to alert us of
> the possibility that there is a genuine tree conflict.

There is a difference between "svn add" and "svn merge"/"svn copy". The
latter store mergeinfo or copy revision information.

> One key aspect here is that Subversion treats addition as a first-class
> operation. It doesn't care what the content of a newly added file is.
> If the change you are merging contains an 'A foo' change, that means
> "add foo", regardless of foo's content.

Your are right. The content is not important for this conflict. But the
difference between "svn add" and "svn merge"/"svn copy" is.

In the meantime I found the following document in the roadmap for SVN
1.8: "merge improvements"/"Symmetric Merging"

http://wiki.apache.org/subversion/SymmetricMerge

I think this document discusses similar problems in the sections "1.6.
The Keep-Alive Dance" and "1.9. Symmetric Merge with Criss-Cross Merge".

Seeing the discussion over there, I hope to see some improvements in the
merging algorithms :) Unfortunately it seems I have to use record-only
merges until then.

Best regards
Sven.

Re: unexpected tree conflict on merge for same source file

[Stefan Sperling <st...@elego.de>]

On Wed, Oct 17, 2012 at 07:12:17PM +0200, Stefan Sperling wrote:
> You then cherry-picked this revision from prj1 into prj2, causing a commit
> that added second.txt to prj2. Let's call this changeset 'prj2@50':
> 
>   A 	second.txt (copied from, say, prj1@49)
>   Mergeinfo addition: Merged prj1:40-49
> 
> Now you merge prj1 into testing, let's say in r61. The common ancestor of
> testing and prj1 is trunk, which does not contain second.txt.
> So among other changes you are merging:
> 
>   A	second.txt (copied from, say, prj1@60)
>   Mergeinfo addition: Merged prj1:40-60
> 
> Next, you merge prj2 into testing, let's say in r71. The common ancestor
> of prj2 and testing is trunk, which does not contain second.txt.
> So the changeset being merged looks something like this:
> 
>   A	second.txt (copied from, say, prj2@70)
>   Mergeinfo addition: Merged prj1:61-70

I got the above mergeinfo change wrong, of course :)
The incoming mergeinfo change for prj1 isn't prj1:61-70 but prj1:40-49,
which is a subset of prj1:40-60 already merged into testing.
So it is a no-op change.

>   Mergeinfo addition: Merged prj2:50-70

Re: unexpected tree conflict on merge for same source file

[Stefan Sperling <st...@elego.de>]

On Wed, Oct 17, 2012 at 05:30:14PM +0200, Sven Uhlig wrote:
> > Ideally, you'd reintegrate prj1 into trunk first and then merge the
> > changes into prj2 from trunk, using a sequence such as:
> 
> I guess this merge was a bad example and wrong as well. There could be
> changes that are commited to prj1 that are requried in prj2 because of
> e.g. some shared code.
> 
> I would do this with the following command:
>   svn merge -r A:B "^/branches/prj1"
> 
> Reintegration is not an option because the development will continue on
> both branches in parallel.

This approach sounds very chaotic to me.

Usually you'd isolate branches from another precisely because each
branch should *not* see code developed on the other cousin branches.
But at the same time you want these branches to share some code?
That's a contradicition. And it makes merging harder than it needs to be.

Every time you cherry-pick changes between cousins you run a high
risk of merge conflicts later when the cousins are merged into other
branches.

I mean, what you're doing is:

  svn copy ^/trunk ^/branches/branch1
  svn copy ^/trunk ^/branches/branch2

Usually, branch1 should never see code from branch2 and vice versa.
Unless one of the branches gets reintegrated into trunk, in which
case changes are transferred between the branches indirectly via
merges to and from trunk. That's the whole point of having two
distinct branches.

In your case, why are the branches which need to share common code not
branches off a branch which contains the shared code? Something like:

  svn copy ^/trunk ^/branches/shared-base-branch
  svn copy ^/branches/shared-base-branch ^/branches/branch1
  svn copy ^/branches/shared-base-branch ^/branches/branch2

Now you can develop common code on shared-base-branch, and changes
specific to branch1 and branch2 on each of those branches, respectively.
The flow of change is straightforward: Branch1 and branch2 only sync to
the shared-base-branch, which in turn syncs itself to trunk.
The shared-base-branch can be reintegrated into trunk once both branch1
and branch2 have been reintegrated into it.
This eliminates cherry-picking, thereby reducing the risk of merge
conflicts, and it also makes it easier to figure out which changes
have already been merged where.

If the above won't work for some reason, maybe you shouldn't be using
branching/merging in the first place. Instead, you could treat the
shared code as an external dependency which your build system will
pull into the build.
Pushing the problem out of the version control problem space into the
build system problem space can make sharing code a lot easier to manage.
You could also consider using an svn:external to pull the same shared
code into a subdirectory of each branch, instead of cherry-picking
changes between cousin branches.

> >> svn switch "^/branches/testing"
> >> svn merge "^/branches/prj1"
> >> svn commit -m "prj1 into testing"
> >>
> >> svn merge "^/branches/prj2"
> > 
> > Again, you hit a similar problem here.
> 
> I do not understand why there is a conflict in "second.txt".
> 
> As far as I can see it the following files are the same:
> - /branch/prj1/second.txt
> - /branch/prj2/second.txt
> 
> /branch/prj1/second.txt was added and never changed
> > svn switch "^/branches/prj1"
> > svn add second.txt
> > svn commit -m "2nd"
> 
> /branch/prj2/second.txt was also added and never changed
> > svn switch "^/branches/prj2"
> > svn merge "^/branches/prj1"
> > svn commit -m "prj1 into prj2"
> 
> With svn:mergeinfo the SVN should have enough information to see that
> the files are the same. There was no change and  the original source is
> the same (/branches/prj1/)

You created second.txt on the prj1 branch. Let's assume you did so
in revision 40, so that the change within prj1@40 looks like this:

  A	second.txt

("add a file called second.txt")

You then cherry-picked this revision from prj1 into prj2, causing a commit
that added second.txt to prj2. Let's call this changeset 'prj2@50':

  A 	second.txt (copied from, say, prj1@49)
  Mergeinfo addition: Merged prj1:40-49

Now you merge prj1 into testing, let's say in r61. The common ancestor of
testing and prj1 is trunk, which does not contain second.txt.
So among other changes you are merging:

  A	second.txt (copied from, say, prj1@60)
  Mergeinfo addition: Merged prj1:40-60

Next, you merge prj2 into testing, let's say in r71. The common ancestor
of prj2 and testing is trunk, which does not contain second.txt.
So the changeset being merged looks something like this:

  A	second.txt (copied from, say, prj2@70)
  Mergeinfo addition: Merged prj1:61-70
  Mergeinfo addition: Merged prj2:50-70

This results in the add vs. add tree conflict.

How is Subversion supposed to tell that prj1:40-70 contains the same
semantic change as prj2:50-70, the change which we cherry-picked from
prj1 to prj2? In terms of mergeinfo the addition of second.txt happens
in two distinct changesets: prj1@40 and prj2@50
Mergeinfo doesn't tell us that, semantically, prj1@40 and prj2@50 contain
the same change.

And neither does tracing back history to the common ancestor of prj2 and
testing tell us that these 'second.txt' files are related.
The common ancestor is trunk, which has never seen 'second.txt'.

The fact that this is a spurious conflict is evident to us because we
know how the second.txt addition to prj2 came about. But what if we had
added a different 'second.txt' file to prj2, which just happens to have
the same name but is otherwise unrelated to the existing file in testing
and prj1? Subversion cannot tell the difference between this scenario
and the one above, so it needs to flag a tree conflict to alert us of
the possibility that there is a genuine tree conflict.

One key aspect here is that Subversion treats addition as a first-class
operation. It doesn't care what the content of a newly added file is.
If the change you are merging contains an 'A foo' change, that means
"add foo", regardless of foo's content.

Re: unexpected tree conflict on merge for same source file

[Sven Uhlig <sv...@web.de>]

Am 16.10.2012 18:02, schrieb Stefan Sperling:
> On Tue, Oct 16, 2012 at 01:17:40PM +0200, Sven Uhlig wrote:
>> Below is a .bat file for creating the repo with some more conflicts that
>> we do not understand. I would be happy if someone could explain these as
>> well.
> 
> Please see my remarks below.

Thank you to all your response! I didn't expect to get a in-depth answer
this quickly - though I am very happy to see this rate of activity :)

>> echo .>first.txt
>> svn add first.txt
>> svn commit -m "1st"
>>
>> svn copy -m "prj0" . "^/branches/prj0"
> 
> You're copying a mixed-revision working copy here.
> Is that really what you want?
>   ^/branches/prj0 is a copy of ^/trunk@1
>   ^/branches/prj0/first.txt is a copy of ^/trunk/first.txt@2.
> See http://svnbook.red-bean.com/en/1.7/svn.basic.in-action.html#svn.basic.in-action.mixedrevs 

You are right. That is not what I want. This was just a test to see if
this will cause errors.

Usually we use TortoiseSVN to do everything with SVN and it seems this
is what happens behind the scenes of TSVN. I know this is the wrong list
to discuss this topic.

I will remove this from any further replies from me.

>> svn switch "^/branches/prj2"
>> svn merge "^/branches/prj1"
> 
> prj1 is a branch off trunk, and so is prj2.
> Which means prj1 and prj2 are not ancestrally related.

But they have the same ancestry, at least the same directory, may be not
the same revision. This is ok for my understanding.

> Ideally, you'd reintegrate prj1 into trunk first and then merge the
> changes into prj2 from trunk, using a sequence such as:

I guess this merge was a bad example and wrong as well. There could be
changes that are commited to prj1 that are requried in prj2 because of
e.g. some shared code.

I would do this with the following command:
  svn merge -r A:B "^/branches/prj1"

Reintegration is not an option because the development will continue on
both branches in parallel.

>> svn switch "^/branches/prj1"
>> svn merge "^/trunk"
>> svn commit -m "prj1 sync"
> 
> What is the purpose of this sync merge?

This is to sync the branch with the changes from trunk. For me it looks
like how it is done in the SVN book.

http://svnbook.red-bean.com/nightly/en/svn.branchmerge.basicmerging.html#svn.branchemerge.basicmerging.stayinsync

>> svn switch "^/branches/testing"
>> svn merge "^/branches/prj1"
>> svn commit -m "prj1 into testing"
>>
>> svn merge "^/branches/prj2"
> 
> Again, you hit a similar problem here.

I do not understand why there is a conflict in "second.txt".

As far as I can see it the following files are the same:
- /branch/prj1/second.txt
- /branch/prj2/second.txt

/branch/prj1/second.txt was added and never changed
> svn switch "^/branches/prj1"
> svn add second.txt
> svn commit -m "2nd"

/branch/prj2/second.txt was also added and never changed
> svn switch "^/branches/prj2"
> svn merge "^/branches/prj1"
> svn commit -m "prj1 into prj2"

With svn:mergeinfo the SVN should have enough information to see that
the files are the same. There was no change and  the original source is
the same (/branches/prj1/)

> What is the purpose of the separate 'testing' branch? Why are you not
> testing prj1 directly, reintegrate it into trunk when tests succeed,
> and then sync all your other branches to trunk to merge prj1 changes
> into them?

The tests are made in the branches directly at first. After that the
changes are merged to the testing branch to check how it looks like with
all the other changes in development.

The whole project is for a website. Because there are so many small
changes that do not need a real independet branch, there is a generic
development branch "testing" and there are no version tags.

When the small changes are OK, they are merged to trunk which represents
the live server. (instead of tags)

Changes that are not OK, may get reverted or updated later.

> If you did that, your approach of using 'svn merge ^/some-branch'
> would work fine. But make sure to use the --reintegrate option whenever
> you merge one of your branches back into trunk (fortunately, the requirement
> to use --reintegrate will go away in Subversion 1.8, but for now you must
> still use it).

See above:
Reintegration is not an option because the development will continue on
both branches in parallel.

> See the output of 'svn help merge' for more details. It has some diagrams
> to illustrate which 'svn merge' invocation is appropriate for a given
> merge scenario.

It seems there is no such scenario documented until now ;)

> If you really must merge between unrelated branches, use the 2-URL
> merge syntax

This seems to be the only solution - before looking at your third e-mail
from today, 10:00 GMT.

So the only open question in this mail is the conflict regarding
second.txt and the merge into /branches/testing/.

Best regards
Sven.

Re: unexpected tree conflict on merge for same source file

[Stefan Sperling <st...@elego.de>]

On Tue, Oct 16, 2012 at 01:17:40PM +0200, Sven Uhlig wrote:
> Below is a .bat file for creating the repo with some more conflicts that
> we do not understand. I would be happy if someone could explain these as
> well.

Please see my remarks below.

> @REM testsvn.bat
> 
> cls
> 
> svn --version
> @REM svn, Version 1.7.2 (r1207936)
> @pause
> svnadmin --version
> @REM svnadmin, Version 1.7.2 (r1207936)
> @pause
> D:
> cd \
> mkdir test
> cd test
> 
> svnadmin create repo
> svn checkout file:///D:/test/repo wc
> 
> cd wc
> mkdir branches
> mkdir tags
> mkdir trunk
> svn add branches tags trunk
> svn commit -m "basic repo structure"
> svn switch --ignore-ancestry file:///D:/test/repo/trunk
> 
> svn info
> @REM Working Copy Root Path: D:\test\wc
> @REM URL: file:///D:/test/repo/trunk
> @pause
> 
> echo .>first.txt
> svn add first.txt
> svn commit -m "1st"
> 
> svn copy -m "prj0" . "^/branches/prj0"

You're copying a mixed-revision working copy here.
Is that really what you want?
  ^/branches/prj0 is a copy of ^/trunk@1
  ^/branches/prj0/first.txt is a copy of ^/trunk/first.txt@2.
See http://svnbook.red-bean.com/en/1.7/svn.basic.in-action.html#svn.basic.in-action.mixedrevs 

Usually, you'd copy URLs to create branches, as you do on the
next line in the script:

> svn copy -r HEAD -m "prj1" "^/trunk" "^/branches/prj1"
> svn switch "^/branches/prj1"

The switch, like an update, results in a single-rev working copy.

> echo .>second.txt
> svn add second.txt
> svn commit -m "2nd"
> 
> svn switch "^/trunk"
> svn copy -r HEAD -m "testing" "^/trunk" "^/branches/testing"
> 
> echo "change" > first.txt
> svn commit -m "change"
> svn copy -r HEAD -m "prj2" "^/trunk" "^/branches/prj2"
> 
> echo "change2" > first.txt
> svn commit -m "change2"
> svn copy -m "prj3" . "^/branches/prj3"

Mixed-rev copy again. This time ^/branches/prj3 is a copy of ^/trunk@9,
and ^/branches/prj3/first.txt is a copy of ^/trunk/first.txt@8.

(I hope I'm getting these numbers right. I'm not actually running the script.)

> svn switch "^/branches/prj2"
> svn merge "^/branches/prj1"

prj1 is a branch off trunk, and so is prj2.
Which means prj1 and prj2 are not ancestrally related.

The merge you are running is equivalent to:

  svn merge ^/branches/prj1@4 ^/branches/prj1@HEAD .

The common ancestor of prj1 and prj2 is trunk@3. Because this is younger
than r2, which added first.txt, you do not get an 'incoming add vs. local
add' tree conflict on first.txt. So this merge happens to work as you expect.

Ideally, you'd reintegrate prj1 into trunk first and then merge the
changes into prj2 from trunk, using a sequence such as:

  svn switch ^/trunk
  svn merge --reintegrate ^/branches/prj1
  svn commit
  svn switch ^/branches/prj2
  svn merge ^/trunk

> svn commit -m "prj1 into prj2"
> 
> svn switch "^/branches/prj3"
> svn merge "^/branches/prj0"

Here you run into a tree conflict because the common ancestor is trunk@1,
which /branches/prj0 was copied from. Because you created prj0 as a
mixed-rev copy, first.txt has a different common ancestor: first.txt@2.

Subversion merges all eligible changes made between ^/trunk@1 and
^/branches/prj0@HEAD. This changeset includes the addition of first.txt,
which the merge attempts to apply to the merge target prj3. There is
already a node called first.txt in prj3, so you get a tree-conflict:

> @REM    C first.txt
> @REM --- Recording mergeinfo for merge of r2 through r11 into '.':
> @REM  U   .
> @pause
> svn st
> @REM  M      .
> @REM       C first.txt
> @REM       >   local add, incoming add upon merge
> @pause
> svn revert -R .
> 
> svn switch "^/branches/prj1"
> svn merge "^/trunk"
> svn commit -m "prj1 sync"

What is the purpose of this sync merge?

> svn switch "^/branches/testing"
> svn merge "^/branches/prj1"
> svn commit -m "prj1 into testing"
>
> svn merge "^/branches/prj2"

Again, you hit a similar problem here.

> @REM --- Recording mergeinfo for merge of r2 through r7 into '.':
> @REM  U   .
> @REM -- Zusammenfuehren von r8 bis r13 in '.':
> @REM    C second.txt
> @REM  G   .
> @REM --- Recording mergeinfo for merge of r8 through r13 into '.':
> @REM  U   .
> @pause
> svn st
> @REM  M      .
> @REM       C second.txt
> @REM       >   local add, incoming add upon merge
> @pause
> svn revert -R .
> 
> cd \

What is the purpose of the separate 'testing' branch? Why are you not
testing prj1 directly, reintegrate it into trunk when tests succeed,
and then sync all your other branches to trunk to merge prj1 changes
into them? If you did that, your approach of using 'svn merge ^/some-branch'
would work fine. But make sure to use the --reintegrate option whenever
you merge one of your branches back into trunk (fortunately, the requirement
to use --reintegrate will go away in Subversion 1.8, but for now you must
still use it).

Your troubles come from merging between ancestrally unrelated branches
and (probably to a lesser extent) from creating branches from mixed-revision
working copies. And that you are applying of the 'svn merge ^/foo' short-hand
notation to situations it wasn't designed for.

See the output of 'svn help merge' for more details. It has some diagrams
to illustrate which 'svn merge' invocation is appropriate for a given
merge scenario.

If you really must merge between unrelated branches, use the 2-URL
merge syntax instead of the 'svn merge ^/foo' short-hand syntax and
specify revisions for each of the 2 URL arguments to merge meaningful
changesets. For instance, say you created '^/branches/A' from trunk in
r399, and then made some interesting changes between r410 and r450 on
branch A.  You want to merge those changes into a completely unrelated
branch called 'testing'. Try something like this:

  svn switch ^/branches/testing
  svn merge ^/branches/A@410 ^/branches/prj@450 .

In some cases you might also have to use the --ignore-ancestry option to
prevent Subversion from tracing history backwards to find a common ancestor.
Note that this option prevents mergeinfo from being recorded, so I wouldn't
recommend this approach if you can avoid it.