You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Stefan Fuhrmann <st...@wandisco.com> on 2015/08/17 02:11:57 UTC
Re: detection of moved branches for svn-normalizer tool
On Wed, Jul 22, 2015 at 12:07 PM, Stefan Hett <st...@egosoft.com> wrote:
> Hi,
>
> I came across a case where svn-normalizer did remove mergeinfo for a
> branch which was still present but got renamed in one revision.
> I understand why it behaves the current way, but maybe in favor of the
> improvements done for handling moves it might also be a good idea to have
> svn-normalizer better handle these scenarios?
>
The latest tool version distinguishes between 3 different cases of
"missing" branches:
* "potential branch" - the (sub-)path never existed. That tends to happens
for old branches
which have not been sync'ed with /trunk for a long time.
* "surviving copies" - path got deleted but there are still copies of it
(or copies of copies).
Even if only parts got copied, they still count.
* "deleted with no surviving copies"
Only the last kind will be removed automatically. In the other cases, it is
up to the
user to decide whether to keep the mergeinfo or not.
> For a quick demo/test-case showing the underlying issue, I've attached a
> batch-file (for windows) demonstrating the case.
>
> As you see the resulting mergeinfo for FooProject/FooSDK is:
> /SDK/FooSDK/trunk:2-3
> /SDK/FooSDK2/tags/v2:6
> /SDK/FooSDK2/trunk:4
>
> Running svn-normalizer analyse now reports
> /SDK/FooSDK as a deleted branch (in r4) and running svn-normalizer
> normalize --remove-obsoletes would then drop this mergeinfo.
>
> Suggested behavior would be that if a branch is renamed and still present
> on trunk, it would not be removed when using svn-normalizer normalize
> --remove-obsoletes.
>
That's how it is implemented now.
-- Stefan^2.
Re: detection of moved branches for svn-normalizer tool
Posted by Julian Foad <ju...@gmail.com>.
Stefan <lu...@posteo.de> wrote:
> Julian Foad wrote:
>> Ping!... Stefan H or Stefan F, is there anything further to report on
>> this issue?
>
> I'll have to retest the current trunk against the test case to check out the
> current state. Quite busy atm though, so donno if I can squeeze it in this
> week. Will do though, unless Stefan F can say anything on the topic.
The reason I'm asking is because WANdisco has a customer with a large
repo with enough complex merge history that they're bound to run into
some sorts of issues like this, and I want to know how likely this
tool will work for them, to set their expectations appropriately.
- Julian
Re: detection of moved branches for svn-normalizer tool
Posted by Stefan <lu...@posteo.de>.
On 1/5/2016 15:56, Julian Foad wrote:
> Ping!... Stefan H or Stefan F, is there anything further to report on
> this issue?
I'll have to retest the current trunk against the test case to check out
the current state. Quite busy atm though, so donno if I can squeeze it
in this week. Will do though, unless Stefan F can say anything on the topic.
Regards,
Stefan
Re: detection of moved branches for svn-normalizer tool
Posted by Julian Foad <ju...@gmail.com>.
> On 05.01.2016, Julian Foad wrote:
>> Ping!... Stefan H or Stefan F, is there anything further to report on
>> this issue?
Stefan <lu...@posteo.de> wrote:
> I'll have to retest the current trunk against the test case to check out the
> current state. Quite busy atm though, so donno if I can squeeze it in this
> week. Will do though, unless Stefan F can say anything on the topic.
There is currently no urgency to this, but it would be interesting to
know if you do.
Stefan Fuhrmann wrote:
> I guess what you are asking is: Can this tool be used
> on production data? My answer is: Yes, but it is and
> will always be an expert tool. [...]
That's helpful. Thank you, both of you.
- Julian
Re: detection of moved branches for svn-normalizer tool
Posted by Stefan Fuhrmann <st...@apache.org>.
On 05.01.2016 15:56, Julian Foad wrote:
> Ping!... Stefan H or Stefan F, is there anything further to report on
> this issue?
I guess what you are asking is: Can this tool be used
on production data? My answer is: Yes, but it is and
will always be an expert tool.
The 'analyse' command is perfectly safe, may take a
long time to run and will produce a detailed report
plus a summary. The latter tells you what the tool
thinks it can do without human intervention and the
higher the removable fraction is, the saner your m/i
is overall.
Note: Everything is run in-memory - similarly to a
normal 'svn merge'. Projects with 100s of MB m/i
and deep histories will require several GB of RAM.
It is also possible to run the tool on a sub-tree.
This will save memory and produce shorter reports.
After that, it can become hairy and I should write
a Wiki page on that (not right away, I'm still taking
this week "off"). But the basic strategies are:
* Elide at the leaves where m/i is simply redundant
* Get rid of misaligned m/i
* Remove obsolete m/i
* Fix non-recursive m/i
* Shorten revision range lists
They will be applied iteratively in varying order.
Some are supported by svn-mergeinfo-normalizer, others
require a 'svn merge --record-only'.
> On 25 August 2015, Stefan Hett wrote:
>> On 17 August 2015, Stefan Fuhrmann wrote:
>>> On Wed, Jul 22, 2015, Stefan Hett wrote:
>>>> I came across a case where svn-normalizer did remove mergeinfo for a
>>>> branch which was still present but got renamed in one revision.
>>>> I understand why it behaves the current way, but maybe in favor of the
>>>> improvements done for handling moves it might also be a good idea to have
>>>> svn-normalizer better handle these scenarios?
>>>
>>> The latest tool version distinguishes between 3 different cases of "missing"
>>> branches:
>>>
>>> * "potential branch"
>>> - the (sub-)path never existed. That tends to happens for old branches
>>> which have not been sync'ed with /trunk for a long time.
>>> * "surviving copies"
>>> - path got deleted but there are still copies of it (or copies of copies).
>>> Even if only parts got copied, they still count.
>>> * "deleted with no surviving copies"
>>>
>>> Only the last kind will be removed automatically. In the other cases, it is up to the
>>> user to decide whether to keep the mergeinfo or not.
>>>
>>>> For a quick demo/test-case showing the underlying issue, I've attached a
>>>> batch-file (for windows) demonstrating the case.
>>>>
>>>> As you see the resulting mergeinfo for FooProject/FooSDK is:
>>>> /SDK/FooSDK/trunk:2-3
>>>> /SDK/FooSDK2/tags/v2:6
>>>> /SDK/FooSDK2/trunk:4
>>>>
>>>> Running svn-normalizer analyse now reports
>>>> /SDK/FooSDK as a deleted branch (in r4) and running svn-normalizer
>>>> normalize --remove-obsoletes would then drop this mergeinfo.
>>>>
>>>> Suggested behavior would be that if a branch is renamed and still present
>>>> on trunk, it would not be removed when using svn-normalizer normalize
>>>> --remove-obsoletes.
>>>
>>> That's how it is implemented now.
>>>
>>> -- Stefan^2.
>>
>> So if I get you right, running "svn-mergeinfo-normalizer normalize
>> --remove-obsoletes" should not remove FooSDK.
>> If so, I think I ran into a bug here. Tested on our productive environment
>> (rather than on this test-case) and it removed the entry in this case for
>> SDKs\CrashServer\trunk.
>>
>> Following is the relevant output from the analyse -v run:
>> [...]
>> Trying to elide mergeinfo from path
>> C:/[...]/src/SDKs/DrDump
>> into mergeinfo at path
>> C:/[...]/src
>> [...]
>> Removing obsolete entries ...
Trunk now correctly says "Trying to remove".
>> has SURVIVING COPIES: /SDKs/CrashServer/trunk
>> e.g.: /SDKs/DrDump/trunk
>> [...]
>> Encountered 1 missing branches:
That now reads "1 missing branch(es):"
>> [r185624, copied or moved] /SDKs/CrashServer
>> -> /SDKs/DrDump
>> [...]
>>
>> So to me the analyse output sounds correct. It detected that
>> /SDKs/CrashServer/trunk no longer exists on head but was renamed to
>> /SDKs/DrDump/trunk and therefore flagged as having surviving copies.
>> Still the normalize run seems to remove the entry.
I just went over the code again and verified that
analyze and normalize modify the mergeinfo in the
same way. The difference is that analyze will not
write the result to disk and shows a more verbose
output. Whenever surviving copies are reported,
the entry will not be removed from the mergeinfo.
That said, there may of course be a bug. It is
also conceivable that the entry got elided for a
different reason; the full analysis output for
/src/SDKs/DrDump and its parent might give a clue.
However, the "Removing obsolete entries" line
should only show after standard elision failed.
-- Stefan^2.
Re: detection of moved branches for svn-normalizer tool
Posted by Julian Foad <ju...@btopenworld.com>.
Ping!... Stefan H or Stefan F, is there anything further to report on
this issue?
- Julian
On 25 August 2015, Stefan Hett wrote:
> On 17 August 2015, Stefan Fuhrmann wrote:
>> On Wed, Jul 22, 2015, Stefan Hett wrote:
>>> I came across a case where svn-normalizer did remove mergeinfo for a
>>> branch which was still present but got renamed in one revision.
>>> I understand why it behaves the current way, but maybe in favor of the
>>> improvements done for handling moves it might also be a good idea to have
>>> svn-normalizer better handle these scenarios?
>>
>> The latest tool version distinguishes between 3 different cases of "missing"
>> branches:
>>
>> * "potential branch"
>> - the (sub-)path never existed. That tends to happens for old branches
>> which have not been sync'ed with /trunk for a long time.
>> * "surviving copies"
>> - path got deleted but there are still copies of it (or copies of copies).
>> Even if only parts got copied, they still count.
>> * "deleted with no surviving copies"
>>
>> Only the last kind will be removed automatically. In the other cases, it is up to the
>> user to decide whether to keep the mergeinfo or not.
>>
>>> For a quick demo/test-case showing the underlying issue, I've attached a
>>> batch-file (for windows) demonstrating the case.
>>>
>>> As you see the resulting mergeinfo for FooProject/FooSDK is:
>>> /SDK/FooSDK/trunk:2-3
>>> /SDK/FooSDK2/tags/v2:6
>>> /SDK/FooSDK2/trunk:4
>>>
>>> Running svn-normalizer analyse now reports
>>> /SDK/FooSDK as a deleted branch (in r4) and running svn-normalizer
>>> normalize --remove-obsoletes would then drop this mergeinfo.
>>>
>>> Suggested behavior would be that if a branch is renamed and still present
>>> on trunk, it would not be removed when using svn-normalizer normalize
>>> --remove-obsoletes.
>>
>> That's how it is implemented now.
>>
>> -- Stefan^2.
>
> So if I get you right, running "svn-mergeinfo-normalizer normalize
> --remove-obsoletes" should not remove FooSDK.
> If so, I think I ran into a bug here. Tested on our productive environment
> (rather than on this test-case) and it removed the entry in this case for
> SDKs\CrashServer\trunk.
>
> Following is the relevant output from the analyse -v run:
> [...]
> Trying to elide mergeinfo from path
> C:/[...]/src/SDKs/DrDump
> into mergeinfo at path
> C:/[...]/src
> [...]
> Removing obsolete entries ...
> has SURVIVING COPIES: /SDKs/CrashServer/trunk
> e.g.: /SDKs/DrDump/trunk
> [...]
> Encountered 1 missing branches:
> [r185624, copied or moved] /SDKs/CrashServer
> -> /SDKs/DrDump
> [...]
>
> So to me the analyse output sounds correct. It detected that
> /SDKs/CrashServer/trunk no longer exists on head but was renamed to
> /SDKs/DrDump/trunk and therefore flagged as having surviving copies.
> Still the normalize run seems to remove the entry.
>
> Am I getting something wrong her?
Re: detection of moved branches for svn-normalizer tool
Posted by Stefan Hett <st...@egosoft.com>.
Hi,
> On Wed, Jul 22, 2015 at 12:07 PM, Stefan Hett <stefan@egosoft.com
> <ma...@egosoft.com>> wrote:
>
> Hi,
>
> I came across a case where svn-normalizer did remove mergeinfo for
> a branch which was still present but got renamed in one revision.
> I understand why it behaves the current way, but maybe in favor of
> the improvements done for handling moves it might also be a good
> idea to have svn-normalizer better handle these scenarios?
>
>
> The latest tool version distinguishes between 3 different cases of
> "missing" branches:
>
> * "potential branch" - the (sub-)path never existed. That tends to
> happens for old branches
> which have not been sync'ed with /trunk for a long time.
> * "surviving copies" - path got deleted but there are still copies of
> it (or copies of copies).
> Even if only parts got copied, they still count.
> * "deleted with no surviving copies"
>
> Only the last kind will be removed automatically. In the other cases,
> it is up to the
> user to decide whether to keep the mergeinfo or not.
>
> For a quick demo/test-case showing the underlying issue, I've
> attached a batch-file (for windows) demonstrating the case.
>
> As you see the resulting mergeinfo for FooProject/FooSDK is:
> /SDK/FooSDK/trunk:2-3
> /SDK/FooSDK2/tags/v2:6
> /SDK/FooSDK2/trunk:4
>
> Running svn-normalizer analyse now reports
> /SDK/FooSDK as a deleted branch (in r4) and running svn-normalizer
> normalize --remove-obsoletes would then drop this mergeinfo.
>
> Suggested behavior would be that if a branch is renamed and still
> present on trunk, it would not be removed when using
> svn-normalizer normalize --remove-obsoletes.
>
>
> That's how it is implemented now.
>
> -- Stefan^2.
So if I get you right, running "svn-mergeinfo-normalizer normalize
--remove-obsoletes" should not remove FooSDK.
If so, I think I ran into a bug here. Tested on our productive
environment (rather than on this test-case) and it removed the entry in
this case for SDKs\CrashServer\trunk.
Following is the relevant output from the analyse -v run:
[...]
Trying to elide mergeinfo from path
C:/[...]/src/SDKs/DrDump
into mergeinfo at path
C:/[...]/src
[...]
Removing obsolete entries ...
has SURVIVING COPIES: /SDKs/CrashServer/trunk
e.g.: /SDKs/DrDump/trunk
[...]
Encountered 1 missing branches:
[r185624, copied or moved] /SDKs/CrashServer
-> /SDKs/DrDump
[...]
So to me the analyse output sounds correct. It detected that
/SDKs/CrashServer/trunk no longer exists on head but was renamed to
/SDKs/DrDump/trunk and therefore flagged as having surviving copies.
Still the normalize run seems to remove the entry.
Am I getting something wrong her?
--
Regards,
Stefan Hett