Reintegrate merging with sparse checkouts

[Brian Neal <bg...@gmail.com>]

Hello -

My question is basically the same as this one:

http://svn.haxx.se/users/archive-2010-04/0097.shtml

We are forced to use an unwieldy and inconvenient directory structure.
Spare checkouts are a big help to us. But it seems that (in the few
times I have tried) to reintegrate a feature branch to trunk it has
failed if either of the working copies are sparse. I believe I
understand why this is; when you merge from trunk to your branch, the
merge info cannot get applied to the directories you have omitted from
your branch working copy. And when you finally go to reintegrate your
branch back to trunk, the reintegrate fails because SVN sees you have
not applied all the changes to certain directories.

Perhaps I have done this wrong. If I have the same sparse directory
structures in my trunk & branch working copies (i.e. I made them
sparse in exactly the same way), should the reintegrate merge work?

Otherwise is there a way to tell SVN that the directories I omitted
from my working copies are irrelevant and do not need to be merged?

Thank you,
BN

Re: Reintegrate merging with sparse checkouts

[Brian Neal <bg...@gmail.com>]

On Tue, May 22, 2012 at 7:31 AM, Johan Corveleyn <jc...@gmail.com> wrote:
> On Mon, May 21, 2012 at 9:00 PM, Brian Neal <bg...@gmail.com> wrote:
> [ ... ]
>> I was able to reproduce the issue, please see the attached
>> sparse-merge.txt and I captured the output and error that I see in the
>> attached file out.txt. (Note that I had to rename sparse-merge.bat to
>> sparse-merge.txt before Gmail would send it).
>>
>> I am using SVN 1.7.4 on Windows XP.
>>
>> The high level description of what is going on in the script is:
>>
>> 1. Create a repo using your sample greek tree.
>> 2. Make a sparse working copy of trunk, omitting the A\B directory (trunk_wc)
>> 3. Copy trunk to a feature branch.
>> 4. Make a sparse working copy of the feature branch (omitting A\B -- branch_wc)
>> 5. Make a full working copy of trunk (trunk_b_wc).
>> 6. Now change files in all three working copies and commit (but don't
>> create conflicts). In particular, in trunk_b_wc, change a file under
>> A\B.
>> 7. Merge trunk to the feature branch.
>> 8. Reintegrate merge the feature branch back to trunk.
>> 9. Observe that the merge fails, presumably because the changes to A\B
>> never made it to the feature branch.
>
> Ah ok, I see. It fails with:
>
> svn: E195016: Reintegrate can only be used if revisions 2 through 6
> were previously merged from file:///C:/Temp/svn-test
> /repos/trunk to the reintegrate source, but this is not the case:
>  branches/feature/A
>    Missing ranges: /trunk/A:4
>
> where revision 4 is the revision on trunk which affected A\B, which
> was missing (excluded by sparseness) from the branch-wc which you used
> to do the sync merge.
>
> I believe that this is normal behavior then, with the current
> functionality of svn: revision 4 is indeed not synced, and the branch
> must be synced completely (there must not be any gaps) for it to be
> reintegratable.
>
> OTOH, you could argue that --reintegrate should just work in this
> case, similar to the normal sync merge, by only looking at the parts
> that are present in the working copy (and using non-inheritable
> mergeinfo to mark the parts where the (reintegrate) merge was not
> performed in full). Hmmm, otherwise you'd almost always need full
> working copies to perform the merges, even if you're fine with only
> merging (and reintegrating) some parts ...
>
> I'm not really an expert on the merge logic and tracking, but I think
> this would be a useful enhancement (unless someone contradicts me
> here). So I'd say: please file an issue.

I have filed issue #4187:

http://subversion.tigris.org/issues/show_bug.cgi?id=4187

Thanks for looking the issue over Johan.

Regards,
-BN

Re: Reintegrate merging with sparse checkouts

[Johan Corveleyn <jc...@gmail.com>]

On Mon, May 21, 2012 at 9:00 PM, Brian Neal <bg...@gmail.com> wrote:
[ ... ]
> I was able to reproduce the issue, please see the attached
> sparse-merge.txt and I captured the output and error that I see in the
> attached file out.txt. (Note that I had to rename sparse-merge.bat to
> sparse-merge.txt before Gmail would send it).
>
> I am using SVN 1.7.4 on Windows XP.
>
> The high level description of what is going on in the script is:
>
> 1. Create a repo using your sample greek tree.
> 2. Make a sparse working copy of trunk, omitting the A\B directory (trunk_wc)
> 3. Copy trunk to a feature branch.
> 4. Make a sparse working copy of the feature branch (omitting A\B -- branch_wc)
> 5. Make a full working copy of trunk (trunk_b_wc).
> 6. Now change files in all three working copies and commit (but don't
> create conflicts). In particular, in trunk_b_wc, change a file under
> A\B.
> 7. Merge trunk to the feature branch.
> 8. Reintegrate merge the feature branch back to trunk.
> 9. Observe that the merge fails, presumably because the changes to A\B
> never made it to the feature branch.

Ah ok, I see. It fails with:

svn: E195016: Reintegrate can only be used if revisions 2 through 6
were previously merged from file:///C:/Temp/svn-test
/repos/trunk to the reintegrate source, but this is not the case:
  branches/feature/A
    Missing ranges: /trunk/A:4

where revision 4 is the revision on trunk which affected A\B, which
was missing (excluded by sparseness) from the branch-wc which you used
to do the sync merge.

I believe that this is normal behavior then, with the current
functionality of svn: revision 4 is indeed not synced, and the branch
must be synced completely (there must not be any gaps) for it to be
reintegratable.

OTOH, you could argue that --reintegrate should just work in this
case, similar to the normal sync merge, by only looking at the parts
that are present in the working copy (and using non-inheritable
mergeinfo to mark the parts where the (reintegrate) merge was not
performed in full). Hmmm, otherwise you'd almost always need full
working copies to perform the merges, even if you're fine with only
merging (and reintegrating) some parts ...

I'm not really an expert on the merge logic and tracking, but I think
this would be a useful enhancement (unless someone contradicts me
here). So I'd say: please file an issue.

-- 
Johan

Re: Reintegrate merging with sparse checkouts

[Brian Neal <bg...@gmail.com>]

On Mon, May 21, 2012 at 9:01 AM, Johan Corveleyn <jc...@gmail.com> wrote:
> [ Redirecting back to the list -- please always use "reply all" to
> keep the discussion on the list. More below ... ]

Sorry about that... see below.

>
> On Mon, May 21, 2012 at 3:54 PM, Brian Neal <bg...@gmail.com> wrote:
>> On Mon, May 21, 2012 at 5:15 AM, Johan Corveleyn <jc...@gmail.com> wrote:
>>> On Fri, May 18, 2012 at 9:48 PM, Brian Neal <bg...@gmail.com> wrote:
>>>> Hello -
>>>>
>>>> My question is basically the same as this one:
>>>>
>>>> http://svn.haxx.se/users/archive-2010-04/0097.shtml
>>>>
>>>> We are forced to use an unwieldy and inconvenient directory structure.
>>>> Spare checkouts are a big help to us. But it seems that (in the few
>>>> times I have tried) to reintegrate a feature branch to trunk it has
>>>> failed if either of the working copies are sparse. I believe I
>>>> understand why this is; when you merge from trunk to your branch, the
>>>> merge info cannot get applied to the directories you have omitted from
>>>> your branch working copy. And when you finally go to reintegrate your
>>>> branch back to trunk, the reintegrate fails because SVN sees you have
>>>> not applied all the changes to certain directories.
>>>>
>>>> Perhaps I have done this wrong. If I have the same sparse directory
>>>> structures in my trunk & branch working copies (i.e. I made them
>>>> sparse in exactly the same way), should the reintegrate merge work?
>>>>
>>>> Otherwise is there a way to tell SVN that the directories I omitted
>>>> from my working copies are irrelevant and do not need to be merged?
>>>
>>> Which version of svn did you use (client-side)? What do you mean by "failed"?
>>
>> SVN 1.7.4.
>>
>> "Failed" as in the reintegrate merge back to trunk was prevented from
>> happening. I regret not writing down the error message, but the gist
>> of it was SVN could not find merge info on the directories I did not
>> have in my sparse working copies. To work around it, we either have to
>> reintegrate merge sub-directories multiple times, or just remove the
>> --reintegrate flag and deal with the less than ideal results (tree
>> conflicts).
>>
>> When I get some time I will try to come up with a toy example that
>> reproduces the issue.
>
> Yes, that would be very useful.

I was able to reproduce the issue, please see the attached
sparse-merge.txt and I captured the output and error that I see in the
attached file out.txt. (Note that I had to rename sparse-merge.bat to
sparse-merge.txt before Gmail would send it).

I am using SVN 1.7.4 on Windows XP.

The high level description of what is going on in the script is:

1. Create a repo using your sample greek tree.
2. Make a sparse working copy of trunk, omitting the A\B directory (trunk_wc)
3. Copy trunk to a feature branch.
4. Make a sparse working copy of the feature branch (omitting A\B -- branch_wc)
5. Make a full working copy of trunk (trunk_b_wc).
6. Now change files in all three working copies and commit (but don't
create conflicts). In particular, in trunk_b_wc, change a file under
A\B.
7. Merge trunk to the feature branch.
8. Reintegrate merge the feature branch back to trunk.
9. Observe that the merge fails, presumably because the changes to A\B
never made it to the feature branch.

Thanks for your time and consideration.

-BN

Re: Reintegrate merging with sparse checkouts

[Johan Corveleyn <jc...@gmail.com>]

[ Redirecting back to the list -- please always use "reply all" to
keep the discussion on the list. More below ... ]

On Mon, May 21, 2012 at 3:54 PM, Brian Neal <bg...@gmail.com> wrote:
> On Mon, May 21, 2012 at 5:15 AM, Johan Corveleyn <jc...@gmail.com> wrote:
>> On Fri, May 18, 2012 at 9:48 PM, Brian Neal <bg...@gmail.com> wrote:
>>> Hello -
>>>
>>> My question is basically the same as this one:
>>>
>>> http://svn.haxx.se/users/archive-2010-04/0097.shtml
>>>
>>> We are forced to use an unwieldy and inconvenient directory structure.
>>> Spare checkouts are a big help to us. But it seems that (in the few
>>> times I have tried) to reintegrate a feature branch to trunk it has
>>> failed if either of the working copies are sparse. I believe I
>>> understand why this is; when you merge from trunk to your branch, the
>>> merge info cannot get applied to the directories you have omitted from
>>> your branch working copy. And when you finally go to reintegrate your
>>> branch back to trunk, the reintegrate fails because SVN sees you have
>>> not applied all the changes to certain directories.
>>>
>>> Perhaps I have done this wrong. If I have the same sparse directory
>>> structures in my trunk & branch working copies (i.e. I made them
>>> sparse in exactly the same way), should the reintegrate merge work?
>>>
>>> Otherwise is there a way to tell SVN that the directories I omitted
>>> from my working copies are irrelevant and do not need to be merged?
>>
>> Which version of svn did you use (client-side)? What do you mean by "failed"?
>
> SVN 1.7.4.
>
> "Failed" as in the reintegrate merge back to trunk was prevented from
> happening. I regret not writing down the error message, but the gist
> of it was SVN could not find merge info on the directories I did not
> have in my sparse working copies. To work around it, we either have to
> reintegrate merge sub-directories multiple times, or just remove the
> --reintegrate flag and deal with the less than ideal results (tree
> conflicts).
>
> When I get some time I will try to come up with a toy example that
> reproduces the issue.

Yes, that would be very useful.

I don't have any more ideas at this time (I have no experience with
(reintegrate-)merging combined with sparse working copies, I just
remembered the issues from the issue tracker).

Maybe someone else on the list has experience with that ...

-- 
Johan

Re: Reintegrate merging with sparse checkouts

[Johan Corveleyn <jc...@gmail.com>]

On Fri, May 18, 2012 at 9:48 PM, Brian Neal <bg...@gmail.com> wrote:
> Hello -
>
> My question is basically the same as this one:
>
> http://svn.haxx.se/users/archive-2010-04/0097.shtml
>
> We are forced to use an unwieldy and inconvenient directory structure.
> Spare checkouts are a big help to us. But it seems that (in the few
> times I have tried) to reintegrate a feature branch to trunk it has
> failed if either of the working copies are sparse. I believe I
> understand why this is; when you merge from trunk to your branch, the
> merge info cannot get applied to the directories you have omitted from
> your branch working copy. And when you finally go to reintegrate your
> branch back to trunk, the reintegrate fails because SVN sees you have
> not applied all the changes to certain directories.
>
> Perhaps I have done this wrong. If I have the same sparse directory
> structures in my trunk & branch working copies (i.e. I made them
> sparse in exactly the same way), should the reintegrate merge work?
>
> Otherwise is there a way to tell SVN that the directories I omitted
> from my working copies are irrelevant and do not need to be merged?

Which version of svn did you use (client-side)? What do you mean by "failed"?

I think 1.7 contains some improvements for merging in sparse working
copies [1], but merge is still not smart enough to know that the
merged changes only affect the parts of a sparse working copy which
are present (so it can record the entire revision as merged in the
mergeinfo). However, IIRC, some issues around this are currently fixed
in trunk, so should be part of the 1.8 release:

- 4056: don't record non-inheritable mergeinfo if missing subtrees are
not touched by the full-depth diff [2]
- 4057: don't record non-inheritable mergeinfo in shallow merge if
entire diff is within requested depth [3]

-- 
Johan

[1] http://subversion.apache.org/docs/release-notes/1.7.html#svn-merge-sparse-no-tree-conflict
[2] http://subversion.tigris.org/issues/show_bug.cgi?id=4056
[3] http://subversion.tigris.org/issues/show_bug.cgi?id=4057