You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "Ph. Marek" <ph...@bmlv.gv.at> on 2008/08/19 06:08:26 UTC
Memory leak in svn_ra_local__get_file() for symlinks
Hello everybody,
I found a (kind of) memory leak in svn_ra_get_file() for file://-urls, with
1.5.1 libraries, reproduceable with "svn cat", too.
It shows as an allocation of about 500kB for this call; for a svn+ssh:// url
it's only about 26k.
Reproduction recipe (with plain binaries, without symbols):
* a fsfs repository, accessible via file://, with a symlink in it
(in my tests in /trunk/subdir/symlink), with data of about 60 bytes.
* svn_ra_get_file(); for me into a svn_stringbuf_t, but a plain
"svn cat" (which does stream to disk, I think) shows that too.
* "gdb --args svn cat file:///repos/path/trunk/dir/symlink"
* Set a breakpoint on "svn_ra_get_file()"
* Start the svn process
* When the breakpoint is hit, look at the process' size
* Continue this function with "n"; gdb stops at the end
* Look at the size again - +500kB.
That's on debian, i686; if you need any more details just ask.
I "fixed" that by creating a new child pool, and destroying that directly
after the svn_ra_get_file() - but needing ~500kB for 80 byte data is a bit
much, isn't it?
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Memory leak in svn_ra_local__get_file() for symlinks
Posted by Karl Fogel <kf...@red-bean.com>.
"Ph. Marek" <ph...@bmlv.gv.at> writes:
> But I have to admit that here I can't reproduce the problem. This is a debian
> unstable i686, with 1.5.1; if the difference lies in the apr libraries or in
> the 64bit/32bit distinction I can't say ATM.
Dang. All I have is a Debian machine too (running 'testing' dist) :-(.
It doesn't reproduce for me either, using your script:
(gdb) r
Starting program: /usr/local/bin/svn cat file:///tmp/repos/bb
[Thread debugging using libthread_db enabled]
[New Thread 0xb79df6b0 (LWP 30977)]
[Switching to Thread 0xb79df6b0 (LWP 30977)]
Breakpoint 1, svn_ra_get_file (session=0x8eb5c40, path=0xb7f9a039 "",
revision=1, stream=0x0, fetched_rev=0x0, \
props=0xbfab9da8, pool=0x8eb5b98)
at subversion/libsvn_ra/ra_loader.c:785
785 SVN_ERR_ASSERT(*path != '/');
(gdb)
(gdb) shell cat /proc/`pidof svn`/status | grep Vm
VmPeak: 7708 kB
VmSize: 7708 kB
VmLck: 0 kB
VmHWM: 2444 kB
VmRSS: 2444 kB
VmData: 504 kB
VmStk: 84 kB
VmExe: 168 kB
VmLib: 5444 kB
VmPTE: 20 kB
(gdb) n
786 return session->vtable->get_file(session, path, revision, stream,
(gdb) shell cat /proc/`pidof svn`/status | grep Vm
VmPeak: 7708 kB
VmSize: 7708 kB
VmLck: 0 kB
VmHWM: 2464 kB
VmRSS: 2464 kB
VmData: 504 kB
VmStk: 84 kB
VmExe: 168 kB
VmLib: 5444 kB
VmPTE: 20 kB
(gdb) kill
Kill the program being debugged? (y or n) y
(gdb) q
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Memory leak in svn_ra_local__get_file() for symlinks
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
Hello Karl,
thank you for your answer.
On Dienstag, 19. August 2008, Karl Fogel wrote:
> "Ph. Marek" <ph...@bmlv.gv.at> writes:
> > I found a (kind of) memory leak in svn_ra_get_file() for file://-urls,
> > with 1.5.1 libraries, reproduceable with "svn cat", too.
> >
> > It shows as an allocation of about 500kB for this call; for a svn+ssh://
> > url it's only about 26k.
> >
> > Reproduction recipe (with plain binaries, without symbols):
> > * a fsfs repository, accessible via file://, with a symlink in it
> > (in my tests in /trunk/subdir/symlink), with data of about 60 bytes.
> > * svn_ra_get_file(); for me into a svn_stringbuf_t, but a plain
> > "svn cat" (which does stream to disk, I think) shows that too.
> > * "gdb --args svn cat file:///repos/path/trunk/dir/symlink"
> > * Set a breakpoint on "svn_ra_get_file()"
> > * Start the svn process
> > * When the breakpoint is hit, look at the process' size
> > * Continue this function with "n"; gdb stops at the end
> > * Look at the size again - +500kB.
> >
> > That's on debian, i686; if you need any more details just ask.
>
> If you cat multiple files in the same client session, does it grow by
> 500kb each time, or is it just a constant overhead to the first time?
I had a loop over the items in a directory; if they are symlinks, I fetch the
data.
One user had to kill the process after 1GB of RAM was eaten and swap was used
heavily (for /usr/lib and/or the X11 fonts, not sure - but many symlinks in a
single directory).
> > I "fixed" that by creating a new child pool, and destroying that directly
> > after the svn_ra_get_file() - but needing ~500kB for 80 byte data is a
> > bit much, isn't it?
>
> Can you show that patch?
The patch is in FSVS - apart from the subpool for each directory I made a new
pool for each call to svn_ra_get_file().
No fix for subversion yet.
> (And can you script your recipe? I know it takes time, but it saves N
> people time too... :-) ).
I'm on another machine currently; but something like this should work:
REPOS=test-repos
WC=test-wc
cd /tmp
test -d $REPOS && rm -fr $REPOS
svnadmin create $REPOS
test -d $WC && rm -fr $WC
svn co file:///tmp/$REPOS $WC
cd $WC
ln -s aaaaaaaaaaaaaaaaaaa/aaaaaaaaaaa/aaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaa bb
svn add bb
svn ci -m1
gdb --args svn cat file:///tmp/$REPOS/bb
And GDB needs this input:
b svn_ra_get_file
(acknowledge question about future symbol resolution with "Y")
r
shell cat /proc/`pidof svn`/status | grep Vm
n
shell cat /proc/`pidof svn`/status | grep Vm
kill
q
But I have to admit that here I can't reproduce the problem. This is a debian
unstable i686, with 1.5.1; if the difference lies in the apr libraries or in
the 64bit/32bit distinction I can't say ATM.
Here I have
libaprutil1 1.2.12+dfsg-7
libapr1 1.2.12-4
libapr0 2.0.55-4.2
subversion 1.5.1dfsg1-1
If somebody could reproduce that (or can't), we might be able to find the
misbehaving bit.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Memory leak in svn_ra_local__get_file() for symlinks
Posted by Karl Fogel <kf...@red-bean.com>.
"Ph. Marek" <ph...@bmlv.gv.at> writes:
> I found a (kind of) memory leak in svn_ra_get_file() for file://-urls, with
> 1.5.1 libraries, reproduceable with "svn cat", too.
>
> It shows as an allocation of about 500kB for this call; for a svn+ssh:// url
> it's only about 26k.
>
> Reproduction recipe (with plain binaries, without symbols):
> * a fsfs repository, accessible via file://, with a symlink in it
> (in my tests in /trunk/subdir/symlink), with data of about 60 bytes.
> * svn_ra_get_file(); for me into a svn_stringbuf_t, but a plain
> "svn cat" (which does stream to disk, I think) shows that too.
> * "gdb --args svn cat file:///repos/path/trunk/dir/symlink"
> * Set a breakpoint on "svn_ra_get_file()"
> * Start the svn process
> * When the breakpoint is hit, look at the process' size
> * Continue this function with "n"; gdb stops at the end
> * Look at the size again - +500kB.
>
> That's on debian, i686; if you need any more details just ask.
If you cat multiple files in the same client session, does it grow by
500kb each time, or is it just a constant overhead to the first time?
> I "fixed" that by creating a new child pool, and destroying that directly
> after the svn_ra_get_file() - but needing ~500kB for 80 byte data is a bit
> much, isn't it?
Can you show that patch?
(And can you script your recipe? I know it takes time, but it saves N
people time too... :-) ).
Thanks,
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org