You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Stefan Küng <to...@gmail.com> on 2010/10/19 15:51:25 UTC
massive memory leak
Hi,
I'm still using r1023755 from trunk, but I haven't seen a commit which I
think would fix this:
There's a massive memory leak somewhere. I can't check out even small
projects anymore since the memory consumption raises very fast and
reaches the limit of my available RAM (6GB) after about the first 100
files. After that, the system isn't usable anymore because the constant
swapping.
I then tried an update of the interrupted checkout (had to hard reset
the system because it wasn't reacting anymore), but the update too uses
up all memory in a few seconds.
Switching to neon instead of the default serf fixes the problem. So this
is limited to either serf itself or ra_serf.
I'm using serf 0.7.0
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
Re: massive memory leak
Posted by "Hyrum K. Wright" <hy...@mail.utexas.edu>.
On Tue, Oct 19, 2010 at 1:13 PM, Lieven Govaerts <sv...@mobsol.be> wrote:
> On Tue, Oct 19, 2010 at 6:58 PM, Stefan Küng <to...@gmail.com> wrote:
>> On 19.10.2010 18:56, C. Michael Pilato wrote:
>>>
>>> On 10/19/2010 12:53 PM, Stefan Küng wrote:
>>>>
>>>> On 19.10.2010 18:46, C. Michael Pilato wrote:
>>>>>
>>>>> On 10/19/2010 12:41 PM, Stefan Küng wrote:
>>>>>>
>>>>>> I tried to check out npp:
>>>>>> https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
>>>>>>
>>>>>> Nothing special about file sizes or number of files.
>>>>>
>>>>> Can you try it without SSL? (Is that possible?) I seem to recall Paul
>>>>> Burba looking into and solving an SSL-only massive ra_serf memory leak
>>>>> recently.
>>>>
>>>> Is there a public server without ssl available? I only have ssl-enabled
>>>> test servers set up at home.
>>>
>>> You can compare checkouts of
>>>
>>> https://svn.apache.org/repos/asf/subversion/trunk
>>>
>>> and
>>>
>>> http://svn.apache.org/repos/asf/subversion/trunk
>>
>> Ups, completely forgot that this was also accessible via http, not just
>> https :)
>>
>> Testing without ssl shows no memory leak at all. So this is related to ssl.
>>
>
> This has been fixed in serf trunk r1408 for a while, but hasn't shown
> up in a serf patch release yet.
Any word on when that might happen?
-Hyrum
Re: massive memory leak
Posted by Johan Corveleyn <jc...@gmail.com>.
On Tue, Oct 19, 2010 at 8:14 PM, Stefan Küng <to...@gmail.com> wrote:
> On 19.10.2010 20:13, Lieven Govaerts wrote:
>
>> This has been fixed in serf trunk r1408 for a while, but hasn't shown
>> up in a serf patch release yet.
>
> Sorry, I should have checked the serf commits first.
>
> Thanks for the update on this.
Also see:
http://subversion.tigris.org/issues/show_bug.cgi?id=3684 - ra_serf has
a memory usage problem
Where Paul Burba describes exactly this problem (memory leak with serf
0.7.0 over SSL, no leak with http), and confirms that serf@1408 fixes
it.
Cheers,
--
Johan
Re: massive memory leak
Posted by Stefan Küng <to...@gmail.com>.
On 19.10.2010 20:13, Lieven Govaerts wrote:
> This has been fixed in serf trunk r1408 for a while, but hasn't shown
> up in a serf patch release yet.
Sorry, I should have checked the serf commits first.
Thanks for the update on this.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
Re: massive memory leak
Posted by Lieven Govaerts <sv...@mobsol.be>.
On Tue, Oct 19, 2010 at 6:58 PM, Stefan Küng <to...@gmail.com> wrote:
> On 19.10.2010 18:56, C. Michael Pilato wrote:
>>
>> On 10/19/2010 12:53 PM, Stefan Küng wrote:
>>>
>>> On 19.10.2010 18:46, C. Michael Pilato wrote:
>>>>
>>>> On 10/19/2010 12:41 PM, Stefan Küng wrote:
>>>>>
>>>>> I tried to check out npp:
>>>>> https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
>>>>>
>>>>> Nothing special about file sizes or number of files.
>>>>
>>>> Can you try it without SSL? (Is that possible?) I seem to recall Paul
>>>> Burba looking into and solving an SSL-only massive ra_serf memory leak
>>>> recently.
>>>
>>> Is there a public server without ssl available? I only have ssl-enabled
>>> test servers set up at home.
>>
>> You can compare checkouts of
>>
>> https://svn.apache.org/repos/asf/subversion/trunk
>>
>> and
>>
>> http://svn.apache.org/repos/asf/subversion/trunk
>
> Ups, completely forgot that this was also accessible via http, not just
> https :)
>
> Testing without ssl shows no memory leak at all. So this is related to ssl.
>
This has been fixed in serf trunk r1408 for a while, but hasn't shown
up in a serf patch release yet.
Lieven
Re: massive memory leak
Posted by Stefan Küng <to...@gmail.com>.
On 19.10.2010 18:56, C. Michael Pilato wrote:
> On 10/19/2010 12:53 PM, Stefan Küng wrote:
>> On 19.10.2010 18:46, C. Michael Pilato wrote:
>>> On 10/19/2010 12:41 PM, Stefan Küng wrote:
>>>> I tried to check out npp:
>>>> https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
>>>>
>>>> Nothing special about file sizes or number of files.
>>>
>>> Can you try it without SSL? (Is that possible?) I seem to recall Paul
>>> Burba looking into and solving an SSL-only massive ra_serf memory leak
>>> recently.
>>
>> Is there a public server without ssl available? I only have ssl-enabled
>> test servers set up at home.
>
> You can compare checkouts of
>
> https://svn.apache.org/repos/asf/subversion/trunk
>
> and
>
> http://svn.apache.org/repos/asf/subversion/trunk
Ups, completely forgot that this was also accessible via http, not just
https :)
Testing without ssl shows no memory leak at all. So this is related to ssl.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
Re: massive memory leak
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 10/19/2010 12:53 PM, Stefan Küng wrote:
> On 19.10.2010 18:46, C. Michael Pilato wrote:
>> On 10/19/2010 12:41 PM, Stefan Küng wrote:
>>> I tried to check out npp:
>>> https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
>>>
>>> Nothing special about file sizes or number of files.
>>
>> Can you try it without SSL? (Is that possible?) I seem to recall Paul
>> Burba looking into and solving an SSL-only massive ra_serf memory leak
>> recently.
>
> Is there a public server without ssl available? I only have ssl-enabled
> test servers set up at home.
You can compare checkouts of
https://svn.apache.org/repos/asf/subversion/trunk
and
http://svn.apache.org/repos/asf/subversion/trunk
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: massive memory leak
Posted by Stefan Küng <to...@gmail.com>.
On 19.10.2010 18:46, C. Michael Pilato wrote:
> On 10/19/2010 12:41 PM, Stefan Küng wrote:
>> I tried to check out npp:
>> https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
>>
>> Nothing special about file sizes or number of files.
>
> Can you try it without SSL? (Is that possible?) I seem to recall Paul
> Burba looking into and solving an SSL-only massive ra_serf memory leak recently.
Is there a public server without ssl available? I only have ssl-enabled
test servers set up at home.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
Re: massive memory leak
Posted by Philip Martin <ph...@wandisco.com>.
Philip Martin <ph...@wandisco.com> writes:
> Using that SSL URL the memory is leaked on my Linux box using serf.
> When I replace https with http the checkout starts and appears not to
> leak memory, but it fails randomly, e.g. like this:
Checkouts work using neon for http and https.
--
Philip
Re: massive memory leak
Posted by Philip Martin <ph...@wandisco.com>.
"C. Michael Pilato" <cm...@collab.net> writes:
> On 10/19/2010 12:41 PM, Stefan Küng wrote:
>> I tried to check out npp:
>> https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
>>
>> Nothing special about file sizes or number of files.
>
> Can you try it without SSL? (Is that possible?) I seem to recall Paul
> Burba looking into and solving an SSL-only massive ra_serf memory leak recently.
Using that SSL URL the memory is leaked on my Linux box using serf.
When I replace https with http the checkout starts and appears not to
leak memory, but it fails randomly, e.g. like this:
../src/subversion/libsvn_wc/update_editor.c:4189: (apr_err=200014)
svn: Checksum mismatch for '/home/pm/sw/subversion/obj/x1/scintilla/cocoa/ScintillaView.mm':
expected: 10fdb67ead5e76cc5f9ac1a147144511
actual: 37ac596ba413089673fe32c6986c079f
or like this:
A x1/scintilla/cocoa/ScintillaView.h
../src/subversion/svn/checkout-cmd.c:172: (apr_err=20014)
../src/subversion/libsvn_client/checkout.c:246: (apr_err=20014)
../src/subversion/libsvn_client/update.c:295: (apr_err=20014)
../src/subversion/libsvn_client/update.c:295: (apr_err=20014)
../src/subversion/libsvn_client/update.c:218: (apr_err=20014)
../src/subversion/libsvn_ra_serf/update.c:2332: (apr_err=20014)
svn: Error retrieving REPORT (20014): Internal error
or like this:
A x1/scintilla/zipsrc.bat
../src/subversion/svn/checkout-cmd.c:172: (apr_err=175009)
../src/subversion/libsvn_client/checkout.c:246: (apr_err=175009)
../src/subversion/libsvn_client/update.c:295: (apr_err=175009)
../src/subversion/libsvn_client/update.c:295: (apr_err=175009)
../src/subversion/libsvn_client/update.c:218: (apr_err=175009)
../src/subversion/libsvn_ra_serf/update.c:2329: (apr_err=175009)
../src/subversion/libsvn_ra_serf/util.c:1591: (apr_err=175009)
../src/subversion/libsvn_ra_serf/util.c:1591: (apr_err=175009)
../src/subversion/libsvn_ra_serf/util.c:1298: (apr_err=175009)
../src/subversion/libsvn_ra_serf/util.c:1298: (apr_err=175009)
svn: XML parsing failed: (207 Multi-Status)
--
Philip
Re: massive memory leak
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 10/19/2010 12:41 PM, Stefan Küng wrote:
> I tried to check out npp:
> https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
>
> Nothing special about file sizes or number of files.
Can you try it without SSL? (Is that possible?) I seem to recall Paul
Burba looking into and solving an SSL-only massive ra_serf memory leak recently.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: massive memory leak
Posted by Stefan Küng <to...@gmail.com>.
On 19.10.2010 18:38, Philip Martin wrote:
> Stefan Küng<to...@gmail.com> writes:
>
>> I'm still using r1023755 from trunk, but I haven't seen a commit which
>> I think would fix this:
>>
>> There's a massive memory leak somewhere. I can't check out even small
>> projects anymore since the memory consumption raises very fast and
>> reaches the limit of my available RAM (6GB) after about the first 100
>> files. After that, the system isn't usable anymore because the
>> constant swapping.
>>
>> I then tried an update of the interrupted checkout (had to hard reset
>> the system because it wasn't reacting anymore), but the update too
>> uses up all memory in a few seconds.
>>
>> Switching to neon instead of the default serf fixes the problem. So
>> this is limited to either serf itself or ra_serf.
>> I'm using serf 0.7.0
>
> On my Linux machine I can checkout Subversion trunk using serf and the
> memory used by the process (as reported by top) is about 150MB. Neon
> uses slightly less memory, about 130MB. I'm using Subversion r1024287,
> serf 0.7.0, and apr 1.2.12.
>
> Does the problem occur with one big file? A hundred small files in
> one directory? A hundred nested directories with no files? With
> http-compression? Over https?
I tried to check out npp:
https://notepad-plus.svn.sourceforge.net/svnroot/notepad-plus/trunk
Nothing special about file sizes or number of files.
I'm using:
Windows 7 x64
sert 0.7.0, apr 1.3.8, apr-util 1.3.9
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
Re: massive memory leak
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Küng <to...@gmail.com> writes:
> I'm still using r1023755 from trunk, but I haven't seen a commit which
> I think would fix this:
>
> There's a massive memory leak somewhere. I can't check out even small
> projects anymore since the memory consumption raises very fast and
> reaches the limit of my available RAM (6GB) after about the first 100
> files. After that, the system isn't usable anymore because the
> constant swapping.
>
> I then tried an update of the interrupted checkout (had to hard reset
> the system because it wasn't reacting anymore), but the update too
> uses up all memory in a few seconds.
>
> Switching to neon instead of the default serf fixes the problem. So
> this is limited to either serf itself or ra_serf.
> I'm using serf 0.7.0
On my Linux machine I can checkout Subversion trunk using serf and the
memory used by the process (as reported by top) is about 150MB. Neon
uses slightly less memory, about 130MB. I'm using Subversion r1024287,
serf 0.7.0, and apr 1.2.12.
Does the problem occur with one big file? A hundred small files in
one directory? A hundred nested directories with no files? With
http-compression? Over https?
--
Philip