server impact (was: 1.7.0-beta2 this afternoon)

[Greg Stein <gs...@gmail.com>]

[straying from original topic; changed subject]

On Wed, Jul 20, 2011 at 13:02,  <km...@rockwellcollins.com> wrote:
>...
>> Not saying it shouldn't be fixed... just saying that I'm unsure that
>> it is a *release-blocker*. Or more precisely, that it is an issue that
>> would cause serf to no longer be the default.
>
> I guess I would consider a large increase in memory use a release
> blocker/enough to make serf no longer the default.  In my experience,
> 10's of thousands of files is no longer a "large" test case.

Sure... and that's why I'm still unsure. It seems kind of an edge
case, but then again I don't know what we really want to call
"typical".

> Somewhat off-topic, but there was also previous concern that
> the multiple connections that serf uses might overly stress some
> larger servers.  Do we have any idea how many additional connections
> a typical server would see?  For example, if I see 1000 concurrent
> connections to a server with neon, will I need to support 10000 shorter
> connections with serf?  (The 10x I chose is purely arbitrary and
> not based upon any knowledge of the actual differences...)

In certain cases (eg. checkout/update) ra_serf tends to use 4
connections rather than 1, so there won't be a major connection
increase on the server. In many cases, ra_serf just sticks to a single
connection (ie. 'svn log') when it doesn't have a bunch of various
content to fetch.

Also, note that the multiple connections actually can *help* the
server. Mark noticed a much smoother load profile on the server since
multiple CPU cores can come into play, once you have multiple active
connections. In the Neon single-connection case, one cpu has to build
the entire response.

>...

Cheers,
-g

Re: server impact (was: 1.7.0-beta2 this afternoon)

[km...@rockwellcollins.com]

Greg Stein <gs...@gmail.com> wrote on 07/20/2011 12:32:42 PM:
> > Somewhat off-topic, but there was also previous concern that
> > the multiple connections that serf uses might overly stress some
> > larger servers.  Do we have any idea how many additional connections
> > a typical server would see?  For example, if I see 1000 concurrent
> > connections to a server with neon, will I need to support 10000 
shorter
> > connections with serf?  (The 10x I chose is purely arbitrary and
> > not based upon any knowledge of the actual differences...)
> 
> In certain cases (eg. checkout/update) ra_serf tends to use 4
> connections rather than 1, so there won't be a major connection
> increase on the server. In many cases, ra_serf just sticks to a single
> connection (ie. 'svn log') when it doesn't have a bunch of various
> content to fetch.

Good info.  Thanks for the details.  Exactly what I wanted!

> Also, note that the multiple connections actually can *help* the
> server. Mark noticed a much smoother load profile on the server since
> multiple CPU cores can come into play, once you have multiple active
> connections. In the Neon single-connection case, one cpu has to build
> the entire response.

Luckily(?) I normally see number of concurrent requests > number of cores,
but I can definitely see that things should improve for individual
users on lighter loaded systems due to the added concurrency.

Kevin R.