You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@subversion.apache.org by Stefan Küng <to...@gmail.com> on 2010/05/16 08:38:33 UTC

problems with serf

Hi,

During the last few days, I've changed TSVN to link against the svn 
trunk. The speed is much better now, so thanks for that. It's not as 
fast as the 1.6.x branch yet but it's usable.

About my problem:
serf is now the default DAV lib svn uses. But with serf I get tons of 
app crashes (serf calls abort() - something I won't discuss here anymore 
since you all should know my opinion about that).
For example: a simple checkout of the svn trunk crashed after about 5 
seconds. Subsequent updates did too - I had to run cleanup and restart 
the update 27 (!!) times until I had the svn trunk on my harddrive.

I'm using serf 0.6.1, but the same problem existed with 0.3.1. I updated 
to 0.6.1 yesterday hoping it would solve the problem, but apparently didn't.

Using neon instead of serf causes no problems at all. At least I haven't 
found any yet.

The abort() call is triggered because of a double free as far as I can see.

I've attached two stacktraces, one for a checkout and one for an update.

Maybe switching back to neon as the default would be a good idea? Serf 
clearly isn't usable in this state right now.

All this tested on Win7 x64, using an x64 build of svn/tsvn.

Since this problem is easily reproducable on my machine I'm wondering 
what might be so different that you guys never had this problem?

Stefan

-- 
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest Interface to (Sub)Version Control
    /_/   \_\     http://tortoisesvn.net

Re: problems with serf

Posted by masaru tsuchiyama <m....@gmail.com>.

The crash also happens on WinXP SP2.

I use svn.exe client
serf: trunk r1375
subversion: trunk r947299
apache: 2.2.15

The attached file is the callstack when doing
svn co http://svn.apache.org/repos/asf/subversion/trunk

Regards.

Masaru.

2010/5/17 Stefan Küng <to...@gmail.com>:
> On 16.05.2010 18:31, Bert Huijben wrote:
>>>
>>> -----Original Message-----
>>> From: Mark Phippard [mailto:markphip@gmail.com]
>>> Sent: zondag 16 mei 2010 18:19
>>> To: Greg Stein
>>> Cc: Stefan Küng; Subversion Development
>>> Subject: Re: problems with serf
>>>
>>> On Sun, May 16, 2010 at 11:29 AM, Greg Stein<gs...@gmail.com>  wrote:
>>>>
>>>> I'll take a look at your stack traces. Thanks!
>>>>
>>>> It is very strange that this happens for you. I've been using serf as
>>>> my only connection for the past couple years. I specifically disable
>>>> neon support. It's been working just fine for me, so there is
>>>> something on your system or about how TortoiseSVN interacts with the
>>>> libraries that triggers it.
>>>
>>> It is also possible it is Serf on Windows, and not specifically
>>> TortoiseSVN.  I do not know if Bert uses Serf regularly or not.  I
>>> seem to recall Paul mentioning he does not use it because of crashes.
>>> I do not know if any of us that use Windows are even using trunk
>>> regularly.  The performance difference on Windows was too great to use
>>> it for any real day to day work.
>>
>> The Slik buildbot runs the serf tests on Windows (The svn-slik-w2k3-x64-ra
>> bot checks svn:// and serf http://), but running the tests doesn't really
>> test actual network conditions.
>> (I can't checkout a normal Subversion trunk over HTTP using serf when the
>> TCG isa server is in its way (I get 404 errors). But I see no issues when
>> doing the same thing from home)
>
> Since the crash happens when memory is freed again (double free on the same
> memory), this could be an issue that only shows on Win7. Win XP doesn't
> really care if memory is freed more than once and crashes only if there's
> more 'wrong' than just the double-free. Win7 however has a much more strict
> memory management and crashes much more in such situations.
> But as I could see, serf has its own way to detect those double-frees for
> debug builds, so it should be possible to reproduce this on XP too.
>
> And maybe it's not even serf itself but ra_serf?
>
> Stefan
>
> --
>       ___
>  oo  // \\      "De Chelonian Mobile"
>  (_,\/ \_/ \     TortoiseSVN
>   \ \_/_\_/>    The coolest Interface to (Sub)Version Control
>   /_/   \_\     http://tortoisesvn.net
>

Re: problems with serf

Posted by Stefan Küng <to...@gmail.com>.

On 16.05.2010 18:31, Bert Huijben wrote:
>> -----Original Message-----
>> From: Mark Phippard [mailto:markphip@gmail.com]
>> Sent: zondag 16 mei 2010 18:19
>> To: Greg Stein
>> Cc: Stefan Küng; Subversion Development
>> Subject: Re: problems with serf
>>
>> On Sun, May 16, 2010 at 11:29 AM, Greg Stein<gs...@gmail.com>  wrote:
>>> I'll take a look at your stack traces. Thanks!
>>>
>>> It is very strange that this happens for you. I've been using serf as
>>> my only connection for the past couple years. I specifically disable
>>> neon support. It's been working just fine for me, so there is
>>> something on your system or about how TortoiseSVN interacts with the
>>> libraries that triggers it.
>>
>> It is also possible it is Serf on Windows, and not specifically
>> TortoiseSVN.  I do not know if Bert uses Serf regularly or not.  I
>> seem to recall Paul mentioning he does not use it because of crashes.
>> I do not know if any of us that use Windows are even using trunk
>> regularly.  The performance difference on Windows was too great to use
>> it for any real day to day work.
>
> The Slik buildbot runs the serf tests on Windows (The svn-slik-w2k3-x64-ra
> bot checks svn:// and serf http://), but running the tests doesn't really
> test actual network conditions.
> (I can't checkout a normal Subversion trunk over HTTP using serf when the
> TCG isa server is in its way (I get 404 errors). But I see no issues when
> doing the same thing from home)

Since the crash happens when memory is freed again (double free on the 
same memory), this could be an issue that only shows on Win7. Win XP 
doesn't really care if memory is freed more than once and crashes only 
if there's more 'wrong' than just the double-free. Win7 however has a 
much more strict memory management and crashes much more in such situations.
But as I could see, serf has its own way to detect those double-frees 
for debug builds, so it should be possible to reproduce this on XP too.

And maybe it's not even serf itself but ra_serf?

Stefan

-- 
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest Interface to (Sub)Version Control
    /_/   \_\     http://tortoisesvn.net

Re: problems with serf

Posted by Stefan Küng <to...@gmail.com>.

On 16.05.2010 18:31, Bert Huijben wrote:
>> -----Original Message-----
>> From: Mark Phippard [mailto:markphip@gmail.com]
>> Sent: zondag 16 mei 2010 18:19
>> To: Greg Stein
>> Cc: Stefan Küng; Subversion Development
>> Subject: Re: problems with serf
>>
>> On Sun, May 16, 2010 at 11:29 AM, Greg Stein<gs...@gmail.com>  wrote:
>>> I'll take a look at your stack traces. Thanks!
>>>
>>> It is very strange that this happens for you. I've been using serf as
>>> my only connection for the past couple years. I specifically disable
>>> neon support. It's been working just fine for me, so there is
>>> something on your system or about how TortoiseSVN interacts with the
>>> libraries that triggers it.
>>
>> It is also possible it is Serf on Windows, and not specifically
>> TortoiseSVN.  I do not know if Bert uses Serf regularly or not.  I
>> seem to recall Paul mentioning he does not use it because of crashes.
>> I do not know if any of us that use Windows are even using trunk
>> regularly.  The performance difference on Windows was too great to use
>> it for any real day to day work.
>
> The Slik buildbot runs the serf tests on Windows (The svn-slik-w2k3-x64-ra
> bot checks svn:// and serf http://), but running the tests doesn't really
> test actual network conditions.
> (I can't checkout a normal Subversion trunk over HTTP using serf when the
> TCG isa server is in its way (I get 404 errors). But I see no issues when
> doing the same thing from home)

Since the crash happens when memory is freed again (double free on the 
same memory), this could be an issue that only shows on Win7. Win XP 
doesn't really care if memory is freed more than once and crashes only 
if there's more 'wrong' than just the double-free. Win7 however has a 
much more strict memory management and crashes much more in such situations.
But as I could see, serf has its own way to detect those double-frees 
for debug builds, so it should be possible to reproduce this on XP too.

And maybe it's not even serf itself but ra_serf?

Stefan

-- 
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest Interface to (Sub)Version Control
    /_/   \_\     http://tortoisesvn.net

RE: problems with serf

Posted by Bert Huijben <be...@qqmail.nl>.

> -----Original Message-----
> From: Mark Phippard [mailto:markphip@gmail.com]
> Sent: zondag 16 mei 2010 18:19
> To: Greg Stein
> Cc: Stefan Küng; Subversion Development
> Subject: Re: problems with serf
> 
> On Sun, May 16, 2010 at 11:29 AM, Greg Stein <gs...@gmail.com> wrote:
> > I'll take a look at your stack traces. Thanks!
> >
> > It is very strange that this happens for you. I've been using serf as
> > my only connection for the past couple years. I specifically disable
> > neon support. It's been working just fine for me, so there is
> > something on your system or about how TortoiseSVN interacts with the
> > libraries that triggers it.
> 
> It is also possible it is Serf on Windows, and not specifically
> TortoiseSVN.  I do not know if Bert uses Serf regularly or not.  I
> seem to recall Paul mentioning he does not use it because of crashes.
> I do not know if any of us that use Windows are even using trunk
> regularly.  The performance difference on Windows was too great to use
> it for any real day to day work.

The Slik buildbot runs the serf tests on Windows (The svn-slik-w2k3-x64-ra
bot checks svn:// and serf http://), but running the tests doesn't really
test actual network conditions. 
(I can't checkout a normal Subversion trunk over HTTP using serf when the
TCG isa server is in its way (I get 404 errors). But I see no issues when
doing the same thing from home)

	Bert

RE: problems with serf

Posted by Bert Huijben <be...@qqmail.nl>.

> -----Original Message-----
> From: Mark Phippard [mailto:markphip@gmail.com]
> Sent: zondag 16 mei 2010 18:19
> To: Greg Stein
> Cc: Stefan Küng; Subversion Development
> Subject: Re: problems with serf
> 
> On Sun, May 16, 2010 at 11:29 AM, Greg Stein <gs...@gmail.com> wrote:
> > I'll take a look at your stack traces. Thanks!
> >
> > It is very strange that this happens for you. I've been using serf as
> > my only connection for the past couple years. I specifically disable
> > neon support. It's been working just fine for me, so there is
> > something on your system or about how TortoiseSVN interacts with the
> > libraries that triggers it.
> 
> It is also possible it is Serf on Windows, and not specifically
> TortoiseSVN.  I do not know if Bert uses Serf regularly or not.  I
> seem to recall Paul mentioning he does not use it because of crashes.
> I do not know if any of us that use Windows are even using trunk
> regularly.  The performance difference on Windows was too great to use
> it for any real day to day work.

The Slik buildbot runs the serf tests on Windows (The svn-slik-w2k3-x64-ra
bot checks svn:// and serf http://), but running the tests doesn't really
test actual network conditions. 
(I can't checkout a normal Subversion trunk over HTTP using serf when the
TCG isa server is in its way (I get 404 errors). But I see no issues when
doing the same thing from home)

	Bert

Re: problems with serf

Posted by Mark Phippard <ma...@gmail.com>.

On Sun, May 16, 2010 at 11:29 AM, Greg Stein <gs...@gmail.com> wrote:
> I'll take a look at your stack traces. Thanks!
>
> It is very strange that this happens for you. I've been using serf as
> my only connection for the past couple years. I specifically disable
> neon support. It's been working just fine for me, so there is
> something on your system or about how TortoiseSVN interacts with the
> libraries that triggers it.

It is also possible it is Serf on Windows, and not specifically
TortoiseSVN.  I do not know if Bert uses Serf regularly or not.  I
seem to recall Paul mentioning he does not use it because of crashes.
I do not know if any of us that use Windows are even using trunk
regularly.  The performance difference on Windows was too great to use
it for any real day to day work.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: problems with serf

Posted by Greg Stein <gs...@gmail.com>.

I'll take a look at your stack traces. Thanks!

It is very strange that this happens for you. I've been using serf as
my only connection for the past couple years. I specifically disable
neon support. It's been working just fine for me, so there is
something on your system or about how TortoiseSVN interacts with the
libraries that triggers it.

And no... serf will eventually stop calling abort(). Those are
generally in there to signal programmer error, not a proper runtime
condition. I agree with you in that respect.

Cheers,
-g

On Sun, May 16, 2010 at 04:38, Stefan Küng <to...@gmail.com> wrote:
> Hi,
>
> During the last few days, I've changed TSVN to link against the svn trunk.
> The speed is much better now, so thanks for that. It's not as fast as the
> 1.6.x branch yet but it's usable.
>
> About my problem:
> serf is now the default DAV lib svn uses. But with serf I get tons of app
> crashes (serf calls abort() - something I won't discuss here anymore since
> you all should know my opinion about that).
> For example: a simple checkout of the svn trunk crashed after about 5
> seconds. Subsequent updates did too - I had to run cleanup and restart the
> update 27 (!!) times until I had the svn trunk on my harddrive.
>
> I'm using serf 0.6.1, but the same problem existed with 0.3.1. I updated to
> 0.6.1 yesterday hoping it would solve the problem, but apparently didn't.
>
> Using neon instead of serf causes no problems at all. At least I haven't
> found any yet.
>
> The abort() call is triggered because of a double free as far as I can see.
>
> I've attached two stacktraces, one for a checkout and one for an update.
>
> Maybe switching back to neon as the default would be a good idea? Serf
> clearly isn't usable in this state right now.
>
> All this tested on Win7 x64, using an x64 build of svn/tsvn.
>
> Since this problem is easily reproducable on my machine I'm wondering what
> might be so different that you guys never had this problem?
>
> Stefan
>
> --
>       ___
>  oo  // \\      "De Chelonian Mobile"
>  (_,\/ \_/ \     TortoiseSVN
>   \ \_/_\_/>    The coolest Interface to (Sub)Version Control
>   /_/   \_\     http://tortoisesvn.net
>

Re: problems with serf

Posted by Stefan Küng <to...@gmail.com>.

On 27.05.2010 08:42, Lieven Govaerts wrote:
> Hi,
>
> On Mon, May 24, 2010 at 3:51 PM, Lieven Govaerts<sv...@mobsol.be>  wrote:
>> On Sun, May 16, 2010 at 10:38 AM, Stefan Küng<to...@gmail.com>  wrote:
>>> Hi,
>>>
>>> During the last few days, I've changed TSVN to link against the svn trunk.
>>> The speed is much better now, so thanks for that. It's not as fast as the
>>> 1.6.x branch yet but it's usable.
>>>
>>> About my problem:
>>> serf is now the default DAV lib svn uses. But with serf I get tons of app
>>> crashes (serf calls abort() - something I won't discuss here anymore since
>>> you all should know my opinion about that).
>>> For example: a simple checkout of the svn trunk crashed after about 5
>>> seconds. Subsequent updates did too - I had to run cleanup and restart the
>>> update 27 (!!) times until I had the svn trunk on my harddrive.
>>>
>>> I'm using serf 0.6.1, but the same problem existed with 0.3.1. I updated to
>>> 0.6.1 yesterday hoping it would solve the problem, but apparently didn't.
>>
>
> I have prepared serf 0.6.2 with this fix at:
> http://serf.googlecode.com/svn/branches/0.6.x
>
> Can anyone of you reporting this problems test if this makes them go away?

Tested with a fresh build of TSVN. The crashes are gone now, at least I 
didn't get one during my tests.

> Checkout takes plenty of memory with ra_serf, way more than when using
> ra_neon. I suspect these changes didn't make that worse, rather the
> choice of pools for file-related requests in ra_serf (not the serf
> library). I'll look into this.

Yes, there's a massive memory leak somewhere. A checkout of the TSVN 
trunk used up more than 2GB or RAM.
I had to stop a test checkout of the KDE trunk once it used up more than 
3.5 GB and basically slowed down my whole system.

To compare, I switched to neon and the memory use didn't grow over about 
60MB.

All this tested on Win7 x64.

Stefan

-- 
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest Interface to (Sub)Version Control
    /_/   \_\     http://tortoisesvn.net

Re: problems with serf

Posted by Stefan Küng <to...@gmail.com>.

On 27.05.2010 08:42, Lieven Govaerts wrote:
> Hi,
>
> On Mon, May 24, 2010 at 3:51 PM, Lieven Govaerts<sv...@mobsol.be>  wrote:
>> On Sun, May 16, 2010 at 10:38 AM, Stefan Küng<to...@gmail.com>  wrote:
>>> Hi,
>>>
>>> During the last few days, I've changed TSVN to link against the svn trunk.
>>> The speed is much better now, so thanks for that. It's not as fast as the
>>> 1.6.x branch yet but it's usable.
>>>
>>> About my problem:
>>> serf is now the default DAV lib svn uses. But with serf I get tons of app
>>> crashes (serf calls abort() - something I won't discuss here anymore since
>>> you all should know my opinion about that).
>>> For example: a simple checkout of the svn trunk crashed after about 5
>>> seconds. Subsequent updates did too - I had to run cleanup and restart the
>>> update 27 (!!) times until I had the svn trunk on my harddrive.
>>>
>>> I'm using serf 0.6.1, but the same problem existed with 0.3.1. I updated to
>>> 0.6.1 yesterday hoping it would solve the problem, but apparently didn't.
>>
>
> I have prepared serf 0.6.2 with this fix at:
> http://serf.googlecode.com/svn/branches/0.6.x
>
> Can anyone of you reporting this problems test if this makes them go away?

Tested with a fresh build of TSVN. The crashes are gone now, at least I 
didn't get one during my tests.

> Checkout takes plenty of memory with ra_serf, way more than when using
> ra_neon. I suspect these changes didn't make that worse, rather the
> choice of pools for file-related requests in ra_serf (not the serf
> library). I'll look into this.

Yes, there's a massive memory leak somewhere. A checkout of the TSVN 
trunk used up more than 2GB or RAM.
I had to stop a test checkout of the KDE trunk once it used up more than 
3.5 GB and basically slowed down my whole system.

To compare, I switched to neon and the memory use didn't grow over about 
60MB.

All this tested on Win7 x64.

Stefan

-- 
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest Interface to (Sub)Version Control
    /_/   \_\     http://tortoisesvn.net

Re: problems with serf

Posted by Paul Burba <pt...@gmail.com>.

On Wed, Sep 8, 2010 at 3:00 PM, Paul Burba <pt...@gmail.com> wrote:
> On Thu, May 27, 2010 at 2:42 AM, Lieven Govaerts <sv...@mobsol.be> wrote:
>> Hi,
>>
>> On Mon, May 24, 2010 at 3:51 PM, Lieven Govaerts <sv...@mobsol.be> wrote:
>>> On Sun, May 16, 2010 at 10:38 AM, Stefan Küng <to...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> During the last few days, I've changed TSVN to link against the svn trunk.
>>>> The speed is much better now, so thanks for that. It's not as fast as the
>>>> 1.6.x branch yet but it's usable.
>>>>
>>>> About my problem:
>>>> serf is now the default DAV lib svn uses. But with serf I get tons of app
>>>> crashes (serf calls abort() - something I won't discuss here anymore since
>>>> you all should know my opinion about that).
>>>> For example: a simple checkout of the svn trunk crashed after about 5
>>>> seconds. Subsequent updates did too - I had to run cleanup and restart the
>>>> update 27 (!!) times until I had the svn trunk on my harddrive.
>>>>
>>>> I'm using serf 0.6.1, but the same problem existed with 0.3.1. I updated to
>>>> 0.6.1 yesterday hoping it would solve the problem, but apparently didn't.
>>>
>>
>> I have prepared serf 0.6.2 with this fix at:
>> http://serf.googlecode.com/svn/branches/0.6.x
>>
>> Can anyone of you reporting this problems test if this makes them go away?
>>
>> Checkout takes plenty of memory with ra_serf, way more than when using
>> ra_neon. I suspect these changes didn't make that worse, rather the
>> choice of pools for file-related requests in ra_serf (not the serf
>> library). I'll look into this.
>
> Hi Lieven,
>
> Did you ever investigate checkout's memory usage with serf?  It seems
> to still be a problem with serf 0.7.0, see
> http://subversion.tigris.org/issues/show_bug.cgi?id=3684#desc4.
>
> I'm seeing some benefits with bumping
> serf/buckets/allocator.c:STANDARD_NODE_SIZE from 128 to 256, but the
> results vary...going to dump what I have found thus far into issue
> #3684.

> http://subversion.tigris.org/issues/show_bug.cgi?id=3684
>
> User lgo changed the following:
>
>                What    |Old value                 |New value
> ================================================================================
>                  Status|REOPENED                  |STARTED
> --------------------------------------------------------------------------------
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=463&dsMessageId=2656935
>
> To unsubscribe from this discussion, e-mail: [issues-unsubscribe@subversion.tigris.org].

Heh, nice timing.