Re: svn merge fails "Out of memory - terminating application." or Tortoise runtime error

[Paul Burba <pt...@gmail.com>]

On Wed, Mar 4, 2009 at 12:54 PM, Nick Lockwood <nm...@i4free.co.nz> wrote:
> Do not know if this thread is still alive but hopefull will help somone else
> as spent ages trying to sort out same error.
>
> In short I managed to trace the problem (performing merge sub directory by
> sub directory) to a directory of approx 1200 files that had a svn:mergeinfo
> property which was blank.

Nick,

Counting the 1200 files in that one directory, what were the *total*
number of subtrees with mergeinfo in your merge target (i.e. how many
paths under the merge target had the svn:mergeinfo property set on
them, regardless of what the property's value is)?

> seen by running
> svn propget svn:mergeinfo --recursive --xml > mergeinfo.xml
>
> once removed (hope someone is not going to tell me off for sugesting the
> wrong thing here) with
> svn propdel --recursive svn:mergeinfo ./*

Most empty mergeinfo (what you referred to as "blank" above) is caused
by working copy to working copy copies/moves where the source item has
no explicit mergeinfo.  Since 1.5.5 these WC-to-WC copies no longer
create empty mergeinfo on the destination.  There are of course other
ways to produce empty mergeinfo, but this has been the most common
AFAICT.  Peter Kahn mentioned the Polarion importer; I don't know much
about that, but it is quite possible that some import tools create a
lot of subtree mergeinfo too.

> I was able to start performing branch to trunk working copy merges
> successfully after that.
>
> My theory (I am not an svn expert so take it with a grain of salt). I had
> picked up all these mergeinfo properties (probably from an earlier svn move
> (rename) restructuring exercise),

Ok, looks like the usual culprit of empty mergeinfo.  So what you did
in deleting this empty mergeifo is probably quite alright.  If the
subtree mergeinfo had been created because you were doing subtree
merges then you might have problems (by subtree merge I mean merges
where the WC target is not the root of your branch/copy).

> but when a merge was being performed there
> were so many that examining all these caused memory usage to go through the
> roof (2+ GB) which eventually caused abnormal termination. Either that or
> there might be a bug (memory leak?) handling empty merge info properties.

Experimenting with a merge target with 1500 subtrees with empty
mergeinfo I see memory usage peaking around 1.2 GB.  But this doesn't
happen until after the merge is complete, most of the time it is
around 200-400MB.  Debugging now to see if the spike occurs
post-merge, when the code sets mergeinfo to record the merge.  More
soon...

Paul

P.S. Moving this to the dev list too.

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1599139

Re: svn merge fails "Out of memory - terminating application." or Tortoise runtime error

[Paul Burba <pt...@gmail.com>]

On Wed, Apr 8, 2009 at 12:52 PM, Paul Burba <pt...@gmail.com> wrote:
> On Wed, Mar 4, 2009 at 12:54 PM, Nick Lockwood <nm...@i4free.co.nz> wrote:
>> Do not know if this thread is still alive but hopefull will help somone else
>> as spent ages trying to sort out same error.
>>
>> In short I managed to trace the problem (performing merge sub directory by
>> sub directory) to a directory of approx 1200 files that had a svn:mergeinfo
>> property which was blank.
>
> Nick,
>
> Counting the 1200 files in that one directory, what were the *total*
> number of subtrees with mergeinfo in your merge target (i.e. how many
> paths under the merge target had the svn:mergeinfo property set on
> them, regardless of what the property's value is)?
>
>> seen by running
>> svn propget svn:mergeinfo --recursive --xml > mergeinfo.xml
>>
>> once removed (hope someone is not going to tell me off for sugesting the
>> wrong thing here) with
>> svn propdel --recursive svn:mergeinfo ./*
>
> Most empty mergeinfo (what you referred to as "blank" above) is caused
> by working copy to working copy copies/moves where the source item has
> no explicit mergeinfo.  Since 1.5.5 these WC-to-WC copies no longer
> create empty mergeinfo on the destination.  There are of course other
> ways to produce empty mergeinfo, but this has been the most common
> AFAICT.  Peter Kahn mentioned the Polarion importer; I don't know much
> about that, but it is quite possible that some import tools create a
> lot of subtree mergeinfo too.
>
>> I was able to start performing branch to trunk working copy merges
>> successfully after that.
>>
>> My theory (I am not an svn expert so take it with a grain of salt). I had
>> picked up all these mergeinfo properties (probably from an earlier svn move
>> (rename) restructuring exercise),
>
> Ok, looks like the usual culprit of empty mergeinfo.  So what you did
> in deleting this empty mergeifo is probably quite alright.  If the
> subtree mergeinfo had been created because you were doing subtree
> merges then you might have problems (by subtree merge I mean merges
> where the WC target is not the root of your branch/copy).
>
>> but when a merge was being performed there
>> were so many that examining all these caused memory usage to go through the
>> roof (2+ GB) which eventually caused abnormal termination. Either that or
>> there might be a bug (memory leak?) handling empty merge info properties.
>
> Experimenting with a merge target with 1500 subtrees with empty
> mergeinfo I see memory usage peaking around 1.2 GB.  But this doesn't
> happen until after the merge is complete, most of the time it is
> around 200-400MB.  Debugging now to see if the spike occurs
> post-merge, when the code sets mergeinfo to record the merge.  More
> soon...

Created issue #3393 to track this problem, but I shouldn't have
bothered.  It turned out to be quite simple: We were not clearing an
iterpool when looping over each subtree with mergeinfo to set
mergeinfo describing the merge.  Fixed in r37119 and nominated for
backport to 1.5.x and 1.6.x.

Paul

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1601194