You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Chris Lieb <ch...@gmail.com> on 2009/05/13 16:31:40 UTC
Using memcached with FSFS repositories
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I noticed in the release notes for Subversion 1.6 that one of the new
features is caching for FSFS repositories using memcached. When I
install subversion-1.6.* on Gentoo, part of the post-install notes say
that, in order to use the caching, I need to install memcached,
configure and start it, then configure my repositories to use it.
I have installed memcached, configured it to have a cache of 256MB and
listen on 0.0.0.0:11211, and started it. Running memcached-tool locally
gives me an empty list of cached objects. Checking out some files using
http, svn+ssh, and file RAs does not seem to cause any cache to be
built, as memcached-tool still displays an empty cache pool.
What do I have to do to get memcached support enabled for my Subversion
repositories? I haven't been able to find a thing using Google other
than that the feature was added in Subverison 1.6.
Thanks,
Chris Lieb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iQEcBAEBAgAGBQJKCvXsAAoJEJWxx7fgsD+CWDAH/3n906F6/6wIoitUvw99D/cK
64Ydt6zAAqEbcpdA7sTDjiyLEx/wfKQtZDagxfA9U/uvcEM3W2OSM1zmTt98ldsX
1m+daZXF82RglT76ej41lWQPoFOfyCfvD+9W07kgZRMcLEiWzGgockdv5Zz6O8zp
FBhyujj3VkE2bf4NP00nUzQ5PV6NJKRKcPlMH6eT3T1I8LN3DpZ9YOieoWkWsUPh
beqVH9A5oNVs1A6vldBW+YM7aE36OOdgUz3D3FXja6QNb7a7xjEmZTOWS28JHapX
9DzOM04GUbwexKmxBTTqdVzewnFbNl19CGvVOepjPRcXNqmTbXLV+HzjXLCrj24=
=K3Xv
-----END PGP SIGNATURE-----
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2239916
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Using memcached with FSFS repositories
Posted by "Hyrum K. Wright" <hy...@hyrumwright.org>.
On May 13, 2009, at 11:56 AM, Chris Lieb wrote:
>
> On 5/13/2009 11:37 AM, Mark Phippard wrote:
>> On Wed, May 13, 2009 at 12:31 PM, Chris Lieb
>> <ch...@gmail.com> wrote:
>>
>>> I noticed in the release notes for Subversion 1.6 that one of the
>>> new
>>> features is caching for FSFS repositories using memcached. When I
>>> install subversion-1.6.* on Gentoo, part of the post-install notes
>>> say
>>> that, in order to use the caching, I need to install memcached,
>>> configure and start it, then configure my repositories to use it.
>>>
>>> I have installed memcached, configured it to have a cache of 256MB
>>> and
>>> listen on 0.0.0.0:11211, and started it. Running memcached-tool
>>> locally
>>> gives me an empty list of cached objects. Checking out some files
>>> using
>>> http, svn+ssh, and file RAs does not seem to cause any cache to be
>>> built, as memcached-tool still displays an empty cache pool.
>>>
>>> What do I have to do to get memcached support enabled for my
>>> Subversion
>>> repositories? I haven't been able to find a thing using Google
>>> other
>>> than that the feature was added in Subverison 1.6.
>>
>> When I create a new fsfs repository with SVN 1.6, inside the "db"
>> folder is a file named fsfs.conf. This contains the memcached
>> settings.
>>
> Thanks. That's exactly what I was looking for. Is this documented
> anywhere? I didn't know that that config file existed. I was busy
> looking in the config/ directory.
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.create.html#svn.reposadmin.create.fsfs
-Hyrum
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2240563
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Using memcached with FSFS repositories
Posted by Chris Lieb <ch...@gmail.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 5/13/2009 11:37 AM, Mark Phippard wrote:
> On Wed, May 13, 2009 at 12:31 PM, Chris Lieb
> <ch...@gmail.com> wrote:
>
>> I noticed in the release notes for Subversion 1.6 that one of the new
>> features is caching for FSFS repositories using memcached. When I
>> install subversion-1.6.* on Gentoo, part of the post-install notes say
>> that, in order to use the caching, I need to install memcached,
>> configure and start it, then configure my repositories to use it.
>>
>> I have installed memcached, configured it to have a cache of 256MB and
>> listen on 0.0.0.0:11211, and started it. Running memcached-tool locally
>> gives me an empty list of cached objects. Checking out some files using
>> http, svn+ssh, and file RAs does not seem to cause any cache to be
>> built, as memcached-tool still displays an empty cache pool.
>>
>> What do I have to do to get memcached support enabled for my Subversion
>> repositories? I haven't been able to find a thing using Google other
>> than that the feature was added in Subverison 1.6.
>
> When I create a new fsfs repository with SVN 1.6, inside the "db"
> folder is a file named fsfs.conf. This contains the memcached
> settings.
>
Thanks. That's exactly what I was looking for. Is this documented
anywhere? I didn't know that that config file existed. I was busy
looking in the config/ directory.
Chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iQEcBAEBAgAGBQJKCvu4AAoJEJWxx7fgsD+CDCwIAM1Vksi651MTFqbpa/hgGOJ2
7i+s08K0dh3B7C2GnE4K0P8WX9q5SSOK8G/KSokn7KbKGJ3PhcSD2UkG6GjIVxe1
SAf9d3Xu12S7GZt85fFR/X5doYk1DdMjynFq8g2T6FR+hxw7nGx9oJmfHjOm+5TP
1i2LhsYLArwIvQVlAwBI4ChfmTxVri4EiCU2TYcJ12Gd5pS2SP112UMjLdFA8jjs
y+R697Va+wVmD1RmSFMzl85eVwNND7ZtAWcoSceMb0EUc9Wh+RZ8JjzMFQyP3ab0
wJp6pAS6W3GuO7G3LlvM7SN98+HNh6sC2LyLn0M3tUX93t6nAz4dYH3kR56XkC8=
=IJmo
-----END PGP SIGNATURE-----
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2240169
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
RE: Using memcached with FSFS repositories
Posted by km...@rockwellcollins.com.
Johan Corveleyn <jo...@uz.kuleuven.ac.be> wrote on 05/27/2009
10:52:35 AM:
> > > If it's really a checkout, I think it's normal that you don't see an
> > > improvement (especially if it's with a windows client). In my
> > > experience, a checkout (or update) on Windows is mainly client-side
> > > cpu-bound (also heavy on IO, but mainly constrained by cpu). Could
> > > you verify this?
> >
> > The Windows client was a high end workstation (4 cores, 4G ram, 80+M
> > sustained I/O capability, Windows XP.) CPU and I/O were monitored
> > on the client and nowhere near max during most of the test.
>
> By maxed on cpu I mean just one core of course. On my pc (Windows
> XP, 2.8Ghz Pentium D (2 cores? or hyperthreading or something like
> that), 2G ram), the svn client (both TortoiseSVN and svn CLI client
> from SlikSVN) took one core, i.e. 50% cpu. This actually happened in
> bursts, every once in a while it would get a breather (during which
> one of the server cpus was maxed out (one httpd child)). Most of the
> time it was the client side that was maxed out though.
Yes, I was monitoring CPU usage per core (on both the client and server).
I have found the client CPU usage varies significantly based upon
the number of files in the working copy.
One big question I had, why does memcached only show 70% cache
hits when reperforming the same operation? That sounds like it
is checking the cache for some data, but not updating the cache
with the resulting data... The cache was not full - 600M/2G.
I'd also like to get it to "fail" gracefully if no memcached
servers are available.
I haven't dug into the code yet, but I was hopeful one of
the original developers of the memcached functionality would
know more.
I do believe the memcached performance would be a big win if
the repo was stored on a slow storage area. I haven't been
able to test this. (I'm complaining about storage that is
too fast. What is the world coming to... :)
Kevin R.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2355822
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
RE: Using memcached with FSFS repositories
Posted by Johan Corveleyn <jo...@uz.kuleuven.ac.be>.
> > If it's really a checkout, I think it's normal that you don't see an
> > improvement (especially if it's with a windows client). In my
> > experience, a checkout (or update) on Windows is mainly client-side
> > cpu-bound (also heavy on IO, but mainly constrained by cpu). Could
> > you verify this?
>
> The Windows client was a high end workstation (4 cores, 4G ram, 80+M
> sustained I/O capability, Windows XP.) CPU and I/O were monitored
> on the client and nowhere near max during most of the test.
By maxed on cpu I mean just one core of course. On my pc (Windows XP, 2.8Ghz Pentium D (2 cores? or hyperthreading or something like that), 2G ram), the svn client (both TortoiseSVN and svn CLI client from SlikSVN) took one core, i.e. 50% cpu. This actually happened in bursts, every once in a while it would get a breather (during which one of the server cpus was maxed out (one httpd child)). Most of the time it was the client side that was maxed out though.
Johan
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2355814
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
RE: Using memcached with FSFS repositories
Posted by km...@rockwellcollins.com.
Johan Corveleyn <jo...@uz.kuleuven.ac.be> wrote on 05/22/2009
10:31:03 AM:
> > Anyone do any performance analysis of using memcached? I'm not
> > seeing much (if any) benefit, but that could be due to a fairly
> > high performance SAN storage solution for storage.
> >
> > Here are my numbers:
> >
> > no cache: 10 min 33 sec
> > cold cache: 10 min 30 sec
> > hot cache: 10 min 26 sec
> > (The differences are statistically insignificant.)
> >
> > This is for a single repo with 600M of data in 30k files.
> >
> > Server cpu usage seemed consistent with and without
> > the cache. I (unfortunately) wasn't able to monitor disk I/O.
> >
> > I was using a 2G memcached, of which it used around 600M for the
> > cache. Oddly enough, monitoring memcached showed only a 70%
> > cache hit when repeatedly downloading the same repository.
> > (I would have expected a 100% hit ratio from the memcached side.)
> >
> > memcached was running on the same server as apache for the tests.
> > (A quad core 64-bit AMD server running solaris 10 x86. 4G of RAM
> > with storage on an enterprise SAN connected with fiber.)
> >
> > svn 1.6.2
> > httpd 2.2.11
> > apr 1.3.3
> > apr-util 1.3.4
> > neon 0.28.4
> > memcached 1.2.8
> >
> > Test repository was created and populated with svn 1.6.2.
> >
> > I also noticed if no memcached servers were running, but a repository
> > was configured to use it, a checkout would fail during the checkout
> > of the first directory. It probably should continue to work as
> > if no cache was configured instead of failing...
> >
> > Anyone else see some better results?
> >
>
> It's not really clear to me what you measured here, but deducing
> from your previous post, I'm guessing you timed a large checkout or
> something similar. What was your test setup (what kind of client,
> how did it access the repo (local/remote, svn(+ssh), http(s), file),
> ...)? And how did you measure it (e.g. running "time svn co ..." in
> a console, or via TortoiseSVN, or ...)?
Yes, it was a repeated clean checkout of 600M (30k files) over http.
The numbers are from TortoiseSVN, but verified the similar results on
RHEL4 (linux) and Solaris 10 x86. ("time svn co ..." on the unix
machines)
> If it's really a checkout, I think it's normal that you don't see an
> improvement (especially if it's with a windows client). In my
> experience, a checkout (or update) on Windows is mainly client-side
> cpu-bound (also heavy on IO, but mainly constrained by cpu). Could
> you verify this?
The Windows client was a high end workstation (4 cores, 4G ram, 80+M
sustained I/O capability, Windows XP.) CPU and I/O were monitored
on the client and nowhere near max during most of the test.
> I don't know that much about SVN (client) internals, but I'm hoping
> this will be improved in 1.7 by the new WC-NG (see http:
> //subversion.tigris.org/issues/show_bug.cgi?id=3357). Another issue
> that might be related (and will probably go away with WC-NG) is
> http://subversion.tigris.org/issues/show_bug.cgi?id=3369.
>
> It would be interesting to know which svn actions are significantly
> improved by memcached (log, blame, ...). Any chance you could test
> this with your setup?
Sure, I can test almost anything, but I was hopeful someone would
have an idea what operations would have the most benefit.
Kevin R.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2355812
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
RE: Using memcached with FSFS repositories
Posted by Johan Corveleyn <jo...@uz.kuleuven.ac.be>.
> Anyone do any performance analysis of using memcached? I'm not
> seeing much (if any) benefit, but that could be due to a fairly
> high performance SAN storage solution for storage.
>
> Here are my numbers:
>
> no cache: 10 min 33 sec
> cold cache: 10 min 30 sec
> hot cache: 10 min 26 sec
> (The differences are statistically insignificant.)
>
> This is for a single repo with 600M of data in 30k files.
>
> Server cpu usage seemed consistent with and without
> the cache. I (unfortunately) wasn't able to monitor disk I/O.
>
> I was using a 2G memcached, of which it used around 600M for the
> cache. Oddly enough, monitoring memcached showed only a 70%
> cache hit when repeatedly downloading the same repository.
> (I would have expected a 100% hit ratio from the memcached side.)
>
> memcached was running on the same server as apache for the tests.
> (A quad core 64-bit AMD server running solaris 10 x86. 4G of RAM
> with storage on an enterprise SAN connected with fiber.)
>
> svn 1.6.2
> httpd 2.2.11
> apr 1.3.3
> apr-util 1.3.4
> neon 0.28.4
> memcached 1.2.8
>
> Test repository was created and populated with svn 1.6.2.
>
> I also noticed if no memcached servers were running, but a repository
> was configured to use it, a checkout would fail during the checkout
> of the first directory. It probably should continue to work as
> if no cache was configured instead of failing...
>
> Anyone else see some better results?
>
It's not really clear to me what you measured here, but deducing from your previous post, I'm guessing you timed a large checkout or something similar. What was your test setup (what kind of client, how did it access the repo (local/remote, svn(+ssh), http(s), file), ...)? And how did you measure it (e.g. running "time svn co ..." in a console, or via TortoiseSVN, or ...)?
If it's really a checkout, I think it's normal that you don't see an improvement (especially if it's with a windows client). In my experience, a checkout (or update) on Windows is mainly client-side cpu-bound (also heavy on IO, but mainly constrained by cpu). Could you verify this?
I don't know that much about SVN (client) internals, but I'm hoping this will be improved in 1.7 by the new WC-NG (see http://subversion.tigris.org/issues/show_bug.cgi?id=3357). Another issue that might be related (and will probably go away with WC-NG) is http://subversion.tigris.org/issues/show_bug.cgi?id=3369.
It would be interesting to know which svn actions are significantly improved by memcached (log, blame, ...). Any chance you could test this with your setup?
Regards,
Johan
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2352921
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Using memcached with FSFS repositories
Posted by km...@rockwellcollins.com.
Blair Zajac <bl...@orcaware.com> wrote on 05/18/2009 03:52:17 PM:
> kmradke@rockwellcollins.com wrote:
> >
> > Mark Phippard <ma...@gmail.com> wrote on 05/13/2009 11:37:57 AM:
> > > On Wed, May 13, 2009 at 12:31 PM, Chris Lieb
> > > <ch...@gmail.com> wrote:
> > >
>
> > I also noticed if no memcached servers were running, but a repository
> > was configured to use it, a checkout would fail during the checkout
> > of the first directory. It probably should continue to work as
> > if no cache was configured instead of failing...
> >
> > Anyone else see some better results?
>
> Out of curiosity, did you see a decrease in the mount of IO to the
> disk that the
> repository is on?
I suspect yes, but I wasn't able to measure it. (I haven't been
able to figure out how to measure I/O to our SAN...)
I may repeat the tests on my linux machine where I'll have the
tools needed to measure the disk I/O changes...
Kevin R.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2303421
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Using memcached with FSFS repositories
Posted by Blair Zajac <bl...@orcaware.com>.
kmradke@rockwellcollins.com wrote:
>
> Mark Phippard <ma...@gmail.com> wrote on 05/13/2009 11:37:57 AM:
> > On Wed, May 13, 2009 at 12:31 PM, Chris Lieb
> > <ch...@gmail.com> wrote:
> >
> I also noticed if no memcached servers were running, but a repository
> was configured to use it, a checkout would fail during the checkout
> of the first directory. It probably should continue to work as
> if no cache was configured instead of failing...
>
> Anyone else see some better results?
Out of curiosity, did you see a decrease in the mount of IO to the disk that the
repository is on?
Blair
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2303375
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Using memcached with FSFS repositories
Posted by km...@rockwellcollins.com.
Mark Phippard <ma...@gmail.com> wrote on 05/13/2009 11:37:57 AM:
> On Wed, May 13, 2009 at 12:31 PM, Chris Lieb
> <ch...@gmail.com> wrote:
>
> > I noticed in the release notes for Subversion 1.6 that one of the new
> > features is caching for FSFS repositories using memcached. When I
> > install subversion-1.6.* on Gentoo, part of the post-install notes say
> > that, in order to use the caching, I need to install memcached,
> > configure and start it, then configure my repositories to use it.
> >
> > I have installed memcached, configured it to have a cache of 256MB and
> > listen on 0.0.0.0:11211, and started it. Running memcached-tool
locally
> > gives me an empty list of cached objects. Checking out some files
using
> > http, svn+ssh, and file RAs does not seem to cause any cache to be
> > built, as memcached-tool still displays an empty cache pool.
> >
> > What do I have to do to get memcached support enabled for my
Subversion
> > repositories? I haven't been able to find a thing using Google other
> > than that the feature was added in Subverison 1.6.
>
> When I create a new fsfs repository with SVN 1.6, inside the "db"
> folder is a file named fsfs.conf. This contains the memcached
> settings.
Anyone do any performance analysis of using memcached? I'm not
seeing much (if any) benefit, but that could be due to a fairly
high performance SAN storage solution for storage.
Here are my numbers:
no cache: 10 min 33 sec
cold cache: 10 min 30 sec
hot cache: 10 min 26 sec
(The differences are statistically insignificant.)
This is for a single repo with 600M of data in 30k files.
Server cpu usage seemed consistent with and without
the cache. I (unfortunately) wasn't able to monitor disk I/O.
I was using a 2G memcached, of which it used around 600M for the
cache. Oddly enough, monitoring memcached showed only a 70%
cache hit when repeatedly downloading the same repository.
(I would have expected a 100% hit ratio from the memcached side.)
memcached was running on the same server as apache for the tests.
(A quad core 64-bit AMD server running solaris 10 x86. 4G of RAM
with storage on an enterprise SAN connected with fiber.)
svn 1.6.2
httpd 2.2.11
apr 1.3.3
apr-util 1.3.4
neon 0.28.4
memcached 1.2.8
Test repository was created and populated with svn 1.6.2.
I also noticed if no memcached servers were running, but a repository
was configured to use it, a checkout would fail during the checkout
of the first directory. It probably should continue to work as
if no cache was configured instead of failing...
Anyone else see some better results?
Kevin R.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2302460
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Using memcached with FSFS repositories
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, May 13, 2009 at 12:31 PM, Chris Lieb
<ch...@gmail.com> wrote:
> I noticed in the release notes for Subversion 1.6 that one of the new
> features is caching for FSFS repositories using memcached. When I
> install subversion-1.6.* on Gentoo, part of the post-install notes say
> that, in order to use the caching, I need to install memcached,
> configure and start it, then configure my repositories to use it.
>
> I have installed memcached, configured it to have a cache of 256MB and
> listen on 0.0.0.0:11211, and started it. Running memcached-tool locally
> gives me an empty list of cached objects. Checking out some files using
> http, svn+ssh, and file RAs does not seem to cause any cache to be
> built, as memcached-tool still displays an empty cache pool.
>
> What do I have to do to get memcached support enabled for my Subversion
> repositories? I haven't been able to find a thing using Google other
> than that the feature was added in Subverison 1.6.
When I create a new fsfs repository with SVN 1.6, inside the "db"
folder is a file named fsfs.conf. This contains the memcached
settings.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2239956
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].