You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Mukund <mu...@tessna.com> on 2003/07/08 10:50:16 UTC
Performance with Sander's DB connection caching patch for mod_dav_svn
Hi all
Here is a quick benchmark of Sander's DB connection caching patch for
mod_dav_svn (I am also attaching a copy of the patch for 0.24.2).
We have discussed what this patch does before (if you read back on this
month's archives you will find a description of what this patch does).
Initially I had tried this on my Pentium3 laptop (Thinkpad R31 with 768 MB
of RAM) using Subversion 0.24.2 ra_dav with a Apache 2.0.46 server listening
locally. With both the client and server running on the same machine, the
performance improvements are easily between 100 - 300% and more. Since
this is not a typical real world test, I decided to repeat it with the
client and server on two different boxes and a network in between.
The client was my laptop. A 100 Mbps hub connected it to the server,
which was a dual Pentium3 workstation with 512 MB of RAM.
The results were not just as startling, but nevertheless significant.
The best part was that disk activity dropped, especially on
reads (checkouts, etc.). Anyone who has imported a large codebase into a
Subversion repository and checked it out knows how the disk goes mad.
Using a browser, when one repeatedly reloaded a file, there would be
no disk activity after the first load with the patched version (with a
browser which kept the connection open).
Results for import of CVS working copy of GIMP ~60 MB in size using svn
add + commit, and checking out the imported copy are as follows. Note
that for each test, I started with a blank repository created with
--bdb-txn-nosync.
1. Client and server on the laptop
Unpatched 0.24.2 svn commit - 31 minutes 33 seconds
Unpatched 0.24.2 svn co - 9 minutes 39 seconds
Patched 0.24.2 svn commit - 12 minutes 16 seconds
Patched 0.24.2 svn co - 3 minutes 46 seconds
2. Client and server on seperate machines on 100 Mbps LAN
Unpatched 0.24.2 svn commit - 15 minutes 24 seconds
Unpatched 0.24.2 svn co - 4 minutes 44 seconds
Patched 0.24.2 svn commit - 10 minutes 16 seconds
Patched 0.24.2 svn co - 3 minutes 33 seconds
Disk activity is much lower with the patch applied, especially during
checkouts. The reason for this has been discussed on this mailing list
earlier this month.
Thank you Sander :-)
Mukund
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Oops.. here is the patch
Posted by Paul L Lussier <pl...@lanminds.com>.
In a message dated: Tue, 08 Jul 2003 17:49:58 BST
Mukund said:
>
>On Tue, Jul 08, 2003 at 12:38:42PM -0400, Paul L Lussier wrote:
>| Please submit patches with the proper Subject line of [PATCH],
>| my procmail recipes don't work otherwise :)
>|
>
>I am sorry Paul. I was merely reposting the patch as a reference.
>The patch was posted by Sander to this list earlier this month.
Okay. I'm just trying to get a handle on this patch management as
well :)
--
Seeya,
Paul
--
Key fingerprint = 1660 FECC 5D21 D286 F853 E808 BB07 9239 53F1 28EE
It may look like I'm just sitting here doing nothing,
but I'm really actively waiting for all my problems to go away.
If you're not having fun, you're not doing it right!
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Oops.. here is the patch
Posted by Mukund <mu...@tessna.com>.
On Tue, Jul 08, 2003 at 12:38:42PM -0400, Paul L Lussier wrote:
| Please submit patches with the proper Subject line of [PATCH],
| my procmail recipes don't work otherwise :)
|
I am sorry Paul. I was merely reposting the patch as a reference.
The patch was posted by Sander to this list earlier this month.
Mukund
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Oops.. here is the patch
Posted by Paul L Lussier <pl...@lanminds.com>.
In a message dated: Tue, 08 Jul 2003 20:06:00 +0200
"Sander Striker" said:
>Paul, I'll be applying this patch myself. Don't worry
>about it ;).
Okay, thanks :)
--
Seeya,
Paul
--
Key fingerprint = 1660 FECC 5D21 D286 F853 E808 BB07 9239 53F1 28EE
It may look like I'm just sitting here doing nothing,
but I'm really actively waiting for all my problems to go away.
If you're not having fun, you're not doing it right!
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Oops.. here is the patch
Posted by Sander Striker <st...@apache.org>.
> From: Paul L Lussier [mailto:pll@lanminds.com]
> Sent: Tuesday, July 08, 2003 6:39 PM
> In a message dated: Tue, 08 Jul 2003 11:54:09 BST
> Mukund said:
>
>
> >Here is the patch for your reference, against 0.24.2.
>
> Please submit patches with the proper Subject line of [PATCH],
> my procmail recipes don't work otherwise :)
>
> Thanks kindly!
Paul, I'll be applying this patch myself. Don't worry
about it ;).
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Oops.. here is the patch
Posted by Paul L Lussier <pl...@lanminds.com>.
In a message dated: Tue, 08 Jul 2003 11:54:09 BST
Mukund said:
>Here is the patch for your reference, against 0.24.2.
Please submit patches with the proper Subject line of [PATCH],
my procmail recipes don't work otherwise :)
Thanks kindly!
--
Seeya,
Paul
--
Key fingerprint = 1660 FECC 5D21 D286 F853 E808 BB07 9239 53F1 28EE
It may look like I'm just sitting here doing nothing,
but I'm really actively waiting for all my problems to go away.
If you're not having fun, you're not doing it right!
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Oops.. here is the patch
Posted by Sander Striker <st...@apache.org>.
> From: Greg Hudson [mailto:ghudson@MIT.EDU]
> Sent: Tuesday, July 08, 2003 6:10 PM
[...]
> Incidentally, I feel a little weird about a performance improvement
> patch which *relies* on mod_dav not clearing the pool between requests.
> Or am I misunderstanding how this works?
Yes. You probably overlooked that we are using the _connection_ pool.
Everything in there is guaranteed to live as long as the connection.
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Oops.. here is the patch
Posted by Greg Hudson <gh...@MIT.EDU>.
On Tue, 2003-07-08 at 07:33, Greg Stein wrote:
> On Tue, Jul 08, 2003 at 11:54:09AM +0100, Mukund wrote:
> >...
> > + /* Cache open repository. Key it off by root_path, which should be more
> > + * unique than the fs_path, given that two Locations may point to the
> > + * same repository.
> > + */
> > + repos_key = apr_pstrcat(r->pool, "mod_dav_svn:", root_path, NULL);
>
> You know... I still don't understand this. You *want* to use the same key
> whenever possible. So why shoot for something "more unique" ?? If two
> Locations refer to the same repos, and requests come in over the connection
> for each of those Locations, then they ought to use the same FS!
Could virtual hosting cause the same fs_path to refer to two different
repositories? (If that's the case, then the code is more correct, but
the comment is severely confused.)
Incidentally, I feel a little weird about a performance improvement
patch which *relies* on mod_dav not clearing the pool between requests.
Or am I misunderstanding how this works?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Oops.. here is the patch
Posted by br...@xbc.nu.
Quoting Greg Stein <gs...@lyra.org>:
> Mind you, we don't want to use an FS simultaneously across threads, but
> that isn't the case here.
And why not? The BDB environment is supposed to be thread-safe, isn't it? We may
have to add another flag when we open the environment, and we definitely don't
want to serve the same request in more than one thread, but using the same
environment in different threads shouldn't be a problem.
Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Oops.. here is the patch
Posted by Sander Striker <st...@apache.org>.
> From: Greg Stein [mailto:gstein@lyra.org]
> Sent: Tuesday, July 08, 2003 1:34 PM
> On Tue, Jul 08, 2003 at 11:54:09AM +0100, Mukund wrote:
>>...
>> + /* Cache open repository. Key it off by root_path, which should be more
>> + * unique than the fs_path, given that two Locations may point to the
>> + * same repository.
>> + */
>> + repos_key = apr_pstrcat(r->pool, "mod_dav_svn:", root_path, NULL);
>
> You know... I still don't understand this. You *want* to use the same key
> whenever possible. So why shoot for something "more unique" ?? If two
> Locations refer to the same repos, and requests come in over the connection
> for each of those Locations, then they ought to use the same FS!
Dude, you're so right! ;). /me wakes up
Ok, I'll just key off on fs_path then.
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Oops.. here is the patch
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Jul 08, 2003 at 11:54:09AM +0100, Mukund wrote:
>...
> + /* Cache open repository. Key it off by root_path, which should be more
> + * unique than the fs_path, given that two Locations may point to the
> + * same repository.
> + */
> + repos_key = apr_pstrcat(r->pool, "mod_dav_svn:", root_path, NULL);
You know... I still don't understand this. You *want* to use the same key
whenever possible. So why shoot for something "more unique" ?? If two
Locations refer to the same repos, and requests come in over the connection
for each of those Locations, then they ought to use the same FS!
Mind you, we don't want to use an FS simultaneously across threads, but that
isn't the case here.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Oops.. here is the patch
Posted by Mukund <mu...@tessna.com>.
Here is the patch for your reference, against 0.24.2.
Mukund
Re: Performance with Sander's DB connection caching patch for mod_dav_svn
Posted by Mukund <mu...@tessna.com>.
On Mon, Jul 14, 2003 at 02:23:56PM -0400, Paul L Lussier wrote:
|
| Just wondering what the status of this is. Discussion seems to have
| died out on this early last week.
|
| Sander, are you still going to commit this patch personally, or
| should I file an issue for it?
|
It was committed as revision 6415.
Mukund
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Performance with Sander's DB connection caching patch for mod_dav_svn
Posted by Paul L Lussier <pl...@lanminds.com>.
Mukund> Here is a quick benchmark of Sander's DB connection caching
Mukund> patch for mod_dav_svn
Sander> Paul, I'll be applying this patch myself.
Hi guys,
Just wondering what the status of this is. Discussion seems to have
died out on this early last week.
Sander, are you still going to commit this patch personally, or
should I file an issue for it?
Thanks,
--
Seeya,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org