You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Brian Buesker <bb...@qualcomm.com> on 2005/07/13 05:20:40 UTC
Suspected Memory Leak for merge
I have a script that uses the SVN perl bindings to do many (typically
small) merges. Looking at the memory for the script in top shows the
memory usage growing significantly. I created a simple perl script that
just does the merges and nothing else, and captured the top output to a
file. The script basically does something like this:
my $client = new SVN::Client( auth => [SVN::Client::get_simple_provider(),
SVN::Client::get_username_provider(),
SVN::Client::get_ssl_server_trust_file_provider()]);
$client->checkout (...);
for my $i ( 1 .. $n ) {
$client->merge (...);
}
Attached is the output of top for approximately 25 merges. In this log,
the merges begin somewhere around the 20m mark. I wouldn't expect the
memory usage to grow that quickly when doing merges. We have seen much
worse growth than this where the RES value grows above 1 GB.
Unfortunately I can't make our repository available for testing, and I
haven't come up with an easy way to write a test script that will prep a
repository with enough things that can be merged in order to reproduce
this error. However, I am certainly willing to try any potential patches
to see if the leak is fixed.
Currently I have only tested this with 1.1.4, although I have not seen
anything in the changelogs for 1.2.x that would indicate something like
this has been fixed.
The entire details of my environment are as follows:
OS: Fedora Core 3
SVN Version:
% svn --version
svn, version 1.1.4 (r13838)
compiled Apr 5 2005, 15:49:54
Copyright (C) 2000-2004 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet
(http://www.Collab.Net/).
The following repository access (RA) modules are available:
* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
- handles 'http' schema
- handles 'https' schema
* ra_local : Module for accessing a repository on local disk.
- handles 'file' schema
* ra_svn : Module for accessing a repository using the svn network
protocol.
- handles 'svn' schema
Compiler:
$ gcc --version
gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Perl:
$ perl -v
This is perl, v5.8.5 built for i386-linux-thread-multi
Copyright 1987-2004, Larry Wall
Perl may be copied only under the terms of either the Artistic License
or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'. If you have access to the
Internet, point your browser at http://www.perl.com/, the Perl Home Page.
Berkeley DB Version: 4.2.52
OpenSSL Version:
$ rpm -q openssl
openssl-0.9.7a-40
Please let me know if you need any more information.
Thanks,
Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Suspected Memory Leak for merge
Posted by Brian Buesker <bb...@qualcomm.com>.
John Szakmeister wrote:
>Can you change that last part to be:
> my $subpool = SVN::Pool->new_default;
> for my $i ( 1 .. $n ) {
> $subpool->clear;
> $client->merge (..., $subpool);
> }
>
>
Ok, I tried doing that and now the script exits at the merge with an
exit status of 2. The merge is not done at all. No error message is
given though. Am I just doing something stupid? Sorry if it is something
stupid, but I haven't been using pools at all in my scripts.
BTW, if I print out the value of $subpool before the merge, it seems to
be valid (SVN::Pool=REF(0x995fc28)).
Here's my exact code with the URLs removed:
#!/usr/bin/perl -w
use SVN::Client;
my $client = new SVN::Client ( auth => [SVN::Client::get_simple_provider(),
SVN::Client::get_username_provider(),
SVN::Client::get_ssl_server_trust_file_provider()]);
my $subpool = SVN::Pool->new_default;
for my $i ( 1 .. 25 ) {
print STDERR "Merging in $i\n";
$subpool->clear ();
# testing memory usage when create new client each time
$client->merge ($url1, 'HEAD', $url2, 'HEAD', $wc, 1, 1, 0, 0,
$subpool);
print STDERR "Merging in $i done\n";
}
>Some of these client functions aren't as well behaved as they probably
>should be. They make allocations of out the pool passed to them, but we
>don't clear the pool. We should probably be using a subpool for
>temporary allocations in the svn_client functions, and only using pool
>parameter for data that needs to last longer.
>
>BTW, is there some way you can narrow down what caused the segfault with
>the property merge? I'd like to make that problem go away too.
>
>
I believe the merge that it failed on was one which added a new
directory which has a property set on it. Does that help at all?
Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Suspected Memory Leak for merge
Posted by John Szakmeister <jo...@szakmeister.net>.
On Wednesday 13 July 2005 15:40, Brian Buesker wrote:
> John Szakmeister wrote:
> >Can you try using the 1.2.1 release and just merging in r15233? I
> > think the segfault is an unrelated issue relating the the new
> > property merging algorithm.
>
> Yes, you are right that the seg fault was unrelated. However, taking
> 1.2.1 and merging in r15233 did not fix the problem. I have attached
> the top output for this run. The last one I sent may not have been a
> complete log (I may have ended the script early. I don't remember now),
> so I wouldn't get concerned right away that this one shows much larger
> values.
>
> Please let me know if you want me to try anything else.
Actually, I do have one more thing for you to try. You said you're using
a perl script that basically does the following:
> my $client = new SVN::Client( auth =>
[SVN::Client::get_simple_provider(),
> SVN::Client::get_username_provider(),
> SVN::Client::get_ssl_server_trust_file_provider()]);
>
> $client->checkout (...);
>
> for my $i ( 1 .. $n ) {
> $client->merge (...);
> }
>
Can you change that last part to be:
my $subpool = SVN::Pool->new_default;
for my $i ( 1 .. $n ) {
$subpool->clear;
$client->merge (..., $subpool);
}
Some of these client functions aren't as well behaved as they probably
should be. They make allocations of out the pool passed to them, but we
don't clear the pool. We should probably be using a subpool for
temporary allocations in the svn_client functions, and only using pool
parameter for data that needs to last longer.
BTW, is there some way you can narrow down what caused the segfault with
the property merge? I'd like to make that problem go away too.
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Suspected Memory Leak for merge
Posted by Brian Buesker <bb...@qualcomm.com>.
John Szakmeister wrote:
>Can you try using the 1.2.1 release and just merging in r15233? I think
>the segfault is an unrelated issue relating the the new property merging
>algorithm.
>
>
Yes, you are right that the seg fault was unrelated. However, taking
1.2.1 and merging in r15233 did not fix the problem. I have attached the
top output for this run. The last one I sent may not have been a
complete log (I may have ended the script early. I don't remember now),
so I wouldn't get concerned right away that this one shows much larger
values.
Please let me know if you want me to try anything else.
Thanks,
Brian
Re: Suspected Memory Leak for merge
Posted by John Szakmeister <jo...@szakmeister.net>.
On Wednesday 13 July 2005 11:16, Brian Buesker wrote:
> John Szakmeister wrote:
> >On Wednesday 13 July 2005 01:30, Brian Buesker wrote:
> >>Sorry, I forgot to attach the log. Here it is.
> >
> >I have proposed a change for the 1.2 branch that clears up quite a bit
> > of the memory usage in merge operations (r15233). Would you care to
> > try the current Subversion trunk/?
>
> Ok, I tried the trunk version at r15233. I couldn't go with a later rev
> of /trunk as that broke the building of the svn perl bindings (possibly
> due to changes made to support SWIG 1.3.25). However, r15233 didn't
> seem to fix the problem, as I still saw the memory usage growing. And
> in fact, after 4 merges, I got a segfault. Here is the stack trace from
> that (I replaced the path in frames #3 and #4 with just "..." as I
> didn't want to include the path in the mail. The path is indeed a valid
> path to a directory in both cases though):
Can you try using the 1.2.1 release and just merging in r15233? I think
the segfault is an unrelated issue relating the the new property merging
algorithm.
> #0 0x00efae82 in find_entry (ht=0x0, key=0xdc1a2f0, klen=8, val=0x0)
> at apr_hash.c:235
> #1 0x00efb08d in apr_hash_get (ht=0x0, key=0xdd2ef10a,
> klen=-1579507087) at apr_hash.c:297
> #2 0x0024887e in svn_wc__merge_props (state=0xbff60748,
> adm_access=0xdb436f8, name=0x0, server_baseprops=0x0,
> propchanges=0xdc17108, base_merge=0, dry_run=0, pool=0xdc170d0,
> entry_accum=0xbff606ac)
> at subversion/libsvn_wc/props.c:424
> #3 0x002484e7 in svn_wc_merge_props (state=0xbff60748,
> path=0xdc15200 "...", adm_access=0xdb436f8, baseprops=0x0,
> propchanges=0xdc17108, base_merge=0, dry_run=0, pool=0xdc170d0)
> at subversion/libsvn_wc/props.c:315
> #4 0x00acad8c in merge_props_changed (adm_access=0xdb436f8,
> state=0xbff60748, path=0xdc15200 "...", propchanges=0xdc15258,
> original_props=0x0, baton=0xbff60bc0) at
> subversion/libsvn_client/diff.c:731
> #5 0x00ad9acc in close_directory (dir_baton=0xdc151a8, pool=0xdc150c8)
> at subversion/libsvn_client/repos_diff.c:879
> #6 0x00992df6 in close_directory (dir_baton=0xdc151a0, pool=0xdc150c8)
> at subversion/libsvn_delta/cancel.c:266
> #7 0x00d21a1f in end_element (userdata=0xcde8408, state=222,
> nspace=0xdc04f28 "svn:", elt_name=0xdc33de0 "add-directory")
> at subversion/libsvn_ra_dav/fetch.c:2510
> #8 0x001de6fd in end_element () from
> /opt/or-buildtools/lib/libneon.so.24 #9 0x00f4415d in xmlParseTextDecl
> () from /usr/lib/libxml2.so.2 #10 0x00fcf617 in xmlParseChunk () from
> /usr/lib/libxml2.so.2
> #11 0x00000000 in ?? ()
>
> Please let me know if you want any more information.
>
> BTW - I have seen merges for large working copies consuming a lot of
> memory, so if r15233 is meant to fix just this case, that would still
> be nice. I don't know if the segfault is coming due to that fix or
> something else.
r15233 reduced one person's memory usage from over 2GB to 60MB, but you're
situation may be different. If you can apply that diff to a 1.2.1
release, and give it a whirl, you might have better luck. I'm hoping the
patch makes it into the next bugfix release.
As for the segfault, I think that's a result of a new property merging
algorithm. And actually, if you can provide information about the couple
of revs that produced that (what changes were involved, and how yoou
executed the command), it could be helpful in tracking down that bug.
Thanks!
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Suspected Memory Leak for merge
Posted by John Szakmeister <jo...@szakmeister.net>.
On Thursday 28 July 2005 17:15, Mike Joseph wrote:
> On Wed, Jul 13, 2005 at 02:12:47AM -0400, John Szakmeister wrote:
> > On Wednesday 13 July 2005 01:30, Brian Buesker wrote:
> > > Sorry, I forgot to attach the log. Here it is.
> >
> > I have proposed a change for the 1.2 branch that clears up quite a bit of
> > the memory usage in merge operations (r15233). Would you care to try the
> > current Subversion trunk/?
> >
> > -John
>
> I have been experiencing a similar problem with the svn client allocating
> all available memory and getting killed while attempting a large merge:
>
> 20781 mj 18 0 2301m 373m 2604 R 16.5 74.0 5:00.87 svn
>
> Note that that is indeed 2.3 GB of memory!
>
> I haven't been able to try the patch yet, but I did put together a test
> repository which can be used to reproduce the problem with any svn client:
>
> svn checkout https://svn.doze.net/memleak/trunk
> cd trunk
> svn merge https://svn.doze.net/memleak/vendor/v1 \
> https://svn.doze.net/memleak/vendor/v2
>
> I'm using the Debian (sarge) svn package:
>
> svn, version 1.1.4 (r13838)
> compiled May 13 2005, 06:29:47
>
> Please let me know if you need any other information. Thanks.
Using the latest trunk, it doesn't climb higher than 26MB on my machine. The
patch should make it into the upcoming release (1.2.2).
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Suspected Memory Leak for merge
Posted by Mike Joseph <mj...@doze.net>.
On Wed, Jul 13, 2005 at 02:12:47AM -0400, John Szakmeister wrote:
> On Wednesday 13 July 2005 01:30, Brian Buesker wrote:
> > Sorry, I forgot to attach the log. Here it is.
>
> I have proposed a change for the 1.2 branch that clears up quite a bit of
> the memory usage in merge operations (r15233). Would you care to try the
> current Subversion trunk/?
>
> -John
I have been experiencing a similar problem with the svn client allocating
all available memory and getting killed while attempting a large merge:
20781 mj 18 0 2301m 373m 2604 R 16.5 74.0 5:00.87 svn
Note that that is indeed 2.3 GB of memory!
I haven't been able to try the patch yet, but I did put together a test
repository which can be used to reproduce the problem with any svn client:
svn checkout https://svn.doze.net/memleak/trunk
cd trunk
svn merge https://svn.doze.net/memleak/vendor/v1 \
https://svn.doze.net/memleak/vendor/v2
I'm using the Debian (sarge) svn package:
svn, version 1.1.4 (r13838)
compiled May 13 2005, 06:29:47
Please let me know if you need any other information. Thanks.
-Mike
Re: Suspected Memory Leak for merge
Posted by Brian Buesker <bb...@qualcomm.com>.
John Szakmeister wrote:
>On Wednesday 13 July 2005 01:30, Brian Buesker wrote:
>
>
>>Sorry, I forgot to attach the log. Here it is.
>>
>>
>
>I have proposed a change for the 1.2 branch that clears up quite a bit of
>the memory usage in merge operations (r15233). Would you care to try the
>current Subversion trunk/?
>
>
Ok, I tried the trunk version at r15233. I couldn't go with a later rev of /trunk as that broke the building of the svn perl bindings (possibly due to changes made to support SWIG 1.3.25). However, r15233 didn't seem to fix the problem, as I still saw the memory usage growing. And in fact, after 4 merges, I got a segfault. Here is the stack trace from that (I replaced the path in frames #3 and #4 with just "..." as I didn't want to include the path in the mail. The path is indeed a valid path to a directory in both cases though):
#0 0x00efae82 in find_entry (ht=0x0, key=0xdc1a2f0, klen=8, val=0x0)
at apr_hash.c:235
#1 0x00efb08d in apr_hash_get (ht=0x0, key=0xdd2ef10a, klen=-1579507087)
at apr_hash.c:297
#2 0x0024887e in svn_wc__merge_props (state=0xbff60748, adm_access=0xdb436f8,
name=0x0, server_baseprops=0x0, propchanges=0xdc17108, base_merge=0,
dry_run=0, pool=0xdc170d0, entry_accum=0xbff606ac)
at subversion/libsvn_wc/props.c:424
#3 0x002484e7 in svn_wc_merge_props (state=0xbff60748,
path=0xdc15200 "...", adm_access=0xdb436f8, baseprops=0x0,
propchanges=0xdc17108, base_merge=0, dry_run=0, pool=0xdc170d0)
at subversion/libsvn_wc/props.c:315
#4 0x00acad8c in merge_props_changed (adm_access=0xdb436f8, state=0xbff60748,
path=0xdc15200 "...", propchanges=0xdc15258, original_props=0x0,
baton=0xbff60bc0) at subversion/libsvn_client/diff.c:731
#5 0x00ad9acc in close_directory (dir_baton=0xdc151a8, pool=0xdc150c8)
at subversion/libsvn_client/repos_diff.c:879
#6 0x00992df6 in close_directory (dir_baton=0xdc151a0, pool=0xdc150c8)
at subversion/libsvn_delta/cancel.c:266
#7 0x00d21a1f in end_element (userdata=0xcde8408, state=222,
nspace=0xdc04f28 "svn:", elt_name=0xdc33de0 "add-directory")
at subversion/libsvn_ra_dav/fetch.c:2510
#8 0x001de6fd in end_element () from /opt/or-buildtools/lib/libneon.so.24
#9 0x00f4415d in xmlParseTextDecl () from /usr/lib/libxml2.so.2
#10 0x00fcf617 in xmlParseChunk () from /usr/lib/libxml2.so.2
#11 0x00000000 in ?? ()
Please let me know if you want any more information.
BTW - I have seen merges for large working copies consuming a lot of memory, so if r15233 is meant to fix just this case, that would still be nice. I don't know if the segfault is coming due to that fix or something else.
Thanks,
Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Suspected Memory Leak for merge
Posted by John Szakmeister <jo...@szakmeister.net>.
On Wednesday 13 July 2005 01:30, Brian Buesker wrote:
> Sorry, I forgot to attach the log. Here it is.
I have proposed a change for the 1.2 branch that clears up quite a bit of
the memory usage in merge operations (r15233). Would you care to try the
current Subversion trunk/?
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Suspected Memory Leak for merge
Posted by Brian Buesker <bb...@qualcomm.com>.
Sorry, I forgot to attach the log. Here it is.
Brian
Brian Buesker wrote:
>I have a script that uses the SVN perl bindings to do many (typically
>small) merges. Looking at the memory for the script in top shows the
>memory usage growing significantly. I created a simple perl script that
>just does the merges and nothing else, and captured the top output to a
>file. The script basically does something like this:
>
>my $client = new SVN::Client( auth => [SVN::Client::get_simple_provider(),
> SVN::Client::get_username_provider(),
> SVN::Client::get_ssl_server_trust_file_provider()]);
>
>$client->checkout (...);
>
>for my $i ( 1 .. $n ) {
> $client->merge (...);
>}
>
>Attached is the output of top for approximately 25 merges. In this log,
>the merges begin somewhere around the 20m mark. I wouldn't expect the
>memory usage to grow that quickly when doing merges. We have seen much
>worse growth than this where the RES value grows above 1 GB.
>
>Unfortunately I can't make our repository available for testing, and I
>haven't come up with an easy way to write a test script that will prep a
>repository with enough things that can be merged in order to reproduce
>this error. However, I am certainly willing to try any potential patches
>to see if the leak is fixed.
>
>Currently I have only tested this with 1.1.4, although I have not seen
>anything in the changelogs for 1.2.x that would indicate something like
>this has been fixed.
>
>The entire details of my environment are as follows:
>
>OS: Fedora Core 3
>
>SVN Version:
>% svn --version
>svn, version 1.1.4 (r13838)
> compiled Apr 5 2005, 15:49:54
>
>Copyright (C) 2000-2004 CollabNet.
>Subversion is open source software, see http://subversion.tigris.org/
>This product includes software developed by CollabNet
>(http://www.Collab.Net/).
>
>The following repository access (RA) modules are available:
>
>* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
>- handles 'http' schema
>- handles 'https' schema
>* ra_local : Module for accessing a repository on local disk.
>- handles 'file' schema
>* ra_svn : Module for accessing a repository using the svn network
>protocol.
>- handles 'svn' schema
>
>Compiler:
>$ gcc --version
>gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)
>Copyright (C) 2004 Free Software Foundation, Inc.
>This is free software; see the source for copying conditions. There is NO
>warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
>Perl:
>$ perl -v
>
>This is perl, v5.8.5 built for i386-linux-thread-multi
>
>Copyright 1987-2004, Larry Wall
>
>Perl may be copied only under the terms of either the Artistic License
>or the
>GNU General Public License, which may be found in the Perl 5 source kit.
>
>Complete documentation for Perl, including FAQ lists, should be found on
>this system using `man perl' or `perldoc perl'. If you have access to the
>Internet, point your browser at http://www.perl.com/, the Perl Home Page.
>
>Berkeley DB Version: 4.2.52
>
>OpenSSL Version:
>$ rpm -q openssl
>openssl-0.9.7a-40
>
>Please let me know if you need any more information.
>
>Thanks,
>Brian
>
>
>