You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Dan Christian <dc...@google.com> on 2007/08/13 22:18:59 UTC
FSFS optimization
I want to work on improving the performance of FSFS on a NFS file
system. I'm looking for feedback on an approach and to find out if
others are already working on this.
The first observation is that there are a large number of small
open-read/write-close operations. Some files will be opened more than
once per file committed (see below).
The first thought is to work on caching some of these: "node.0.0" is
the node revision, and "props" contains the revision properties. The
"changes" file is always(?) appended to, so we might be able to keep
it open and keep appending to it. The tricky bit is that the httpd
will close at some point and an a fresh one will end up advancing the
transaction. This implies that the disk version must be kept up to
date.
An alternative/additional thought would be to move the transactions
directory to local disk so that the OS can cache more aggressively.
This would require network load balancers to always direct the same
client to the same server.
A completely separate idea is to work on caching revision files on
local disk (so the full text version doesn't have to be regenerated
repeatedly). The import case below does none of this, but a typical
commit would reconstruct the previous revision of every file. This
takes advantage of the fact that revisions never change after they are
written.
Here are the top open calls by name/flags for a 133 file import:
131 /svnrepo/db/transactions/39-18.txn/node._0.0.children", flag=10
133 /svnrepo/db/transactions/39-18.txn/next-ids", flag=129
133 /svnrepo/db/transactions/39-18.txn/next-ids", flag=18
133 /svnrepo/db/transactions/39-18.txn/rev", flag=130
133 /svnrepo/db/transactions/39-18.txn/rev-lock", flag=6
134 /svnrepo/db/transactions/39-18.txn/node._0.0", flag=129
296 /svnrepo/db/transactions/39-18.txn/changes", flag=142
306 /svnrepo/db/transactions/39-18.txn/props", flag=133
596 /svnrepo/db/transactions/39-18.txn/node.0.0", flag=129
Feedback is much appreciated.
Thanks,
-Dan Christian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS optimization
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 8/13/07, Eric Gillespie <ep...@pretzelnet.org> wrote:
> "Dan Christian" <dc...@google.com> writes:
>
> > A completely separate idea is to work on caching revision files on
> > local disk (so the full text version doesn't have to be regenerated
> > repeatedly). The import case below does none of this, but a typical
> > commit would reconstruct the previous revision of every file. This
> > takes advantage of the fact that revisions never change after they are
> > written.
>
> Not to mention read operations like update and merge. Here are
> the numbers of times the server opens various files when I svn
> merge a revision changing 5439 files. First, the diabolically
> interesting ones:
>
> revprops/4/4661 5967
> revs/0/2 20445
> revs/4/4661 330459
>
> r4661 is the revision I'm merging (svn merge -c 4661).
>
> The complete (almost; some small time passes between starting the
> svn merge and attaching strace to the httpd) report:
<snip>
Perhaps a "smart" cache could recognize cases like this. Track how
many times each file has been opened and keep the top N open at all
times? For read only files you even avoid the "multiple
threads/processes need to care" issues Dan was referring to in his
initial mail.
I seem to recall trying to build some sort of open file cache for fsfs
back in the day, and not getting too far, but it's certainly possible
that aided by some good profiling we could do better now.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS optimization
Posted by Eric Gillespie <ep...@pretzelnet.org>.
"Dan Christian" <dc...@google.com> writes:
> A completely separate idea is to work on caching revision files on
> local disk (so the full text version doesn't have to be regenerated
> repeatedly). The import case below does none of this, but a typical
> commit would reconstruct the previous revision of every file. This
> takes advantage of the fact that revisions never change after they are
> written.
Not to mention read operations like update and merge. Here are
the numbers of times the server opens various files when I svn
merge a revision changing 5439 files. First, the diabolically
interesting ones:
revprops/4/4661 5967
revs/0/2 20445
revs/4/4661 330459
r4661 is the revision I'm merging (svn merge -c 4661).
The complete (almost; some small time passes between starting the
svn merge and attaching strace to the httpd) report:
locks/00a/00a774e1684ba051a1a198f845b1c26e 1
locks/017/017253a4eecda1d50c2f346d41c3118c 1
locks/01b/01b2db1d3353d646984ae515612e10cd 1
locks/030/030bc2067cbd36e635cbe55f74fed6e6 1
locks/033/033a576e6bc61635903909451cdafb90 1
locks/04a/04adc821dc1e80fc28788fd11ceaf9eb 1
locks/0db/0db224a635c4b05fc8c8617cc90511f5 1
locks/15a/15ae43e0f9d1f13d3bf38c6dd2c84005 1
locks/173/1730a10bf7737a4034f3e0f716c02042 1
locks/1ae/1ae65bb90ea43d2b09071ab92934186c 1
locks/206/20602d1e060a28e941bf3bb5b068da55 1
locks/21d/21dcb87847176235f61d1f982f8b9aad 1
locks/22c/22c9504f479a3a94e740b74208ed71ab 1
locks/23e/23e0a5bf42c7ece3f596ab02359ba58f 1
locks/259/25991c561373a8e7cd8d5b75859f4ebd 1
locks/2b4/2b464393dc3fdddc0f4da6bda3fc826c 1
locks/2f6/2f629c8282bf86fc1a7c36f1ce4918da 1
locks/3b0/3b0d9982488b1836168669382516753e 1
locks/3d0/3d0cfd94713fdad86df9c742d6519892 1
locks/40c/40c32ff56e200c830baa47f672976f9d 1
locks/44d/44dc6d79a4cf30eb5440cd630439b4ae 1
locks/48e/48e9f7c12ccb9af525503022182c5f28 1
locks/4bc/4bc3477cb96bc3bbb7f3d10a0f6dd82a 1
locks/4dc/4dcd7f19746e5bccea67402f55b81143 1
locks/55b/55b84601e56f0723268c333304a678f9 1
locks/58f/58f939675e6e42214a49c5f1cb66928a 1
locks/5aa/5aa901026b2d089d2fbc192a7b39090c 1
locks/5b4/5b44035c3e8c20f7671cdc655d0dc4c1 1
locks/5cb/5cb67122e59b2ef5398e33de383af8cf 1
locks/704/7047bfa20bd4e4377515105d1a095209 1
locks/70a/70a67c60d97d3a9d85028e6966b5fa0d 1
locks/72b/72b5b1d823c60af80661aba6e0a2485f 1
locks/75f/75f249be79249576446bb5b815eb8335 1
locks/782/7824b1f181f621dc31a14577dffa0a4a 1
locks/797/797e97a72e167b56dbbda872dad56c00 1
locks/7a6/7a6d729ae3cc2a9eb88c5ce0bc8ccd34 1
locks/82e/82eff4106b8035b0e1e9eba9fabc088d 1
locks/85e/85e5289bcce05f818e36d83464588b0a 1
locks/86f/86f24d050157ae02d50918965cbc1e26 1
locks/872/8720e4a9670e3a09a30bd70db0884e95 1
locks/89e/89e0506efccd50d9f597f31021525f21 1
locks/8e6/8e628d527f542db244128ca4f0f2b747 1
locks/8f9/8f9d3cee72ee5b2a14811b9f92d18cd1 1
locks/917/917795ba30847fefb2a6d3ede02423f9 1
locks/940/9407568181295fbb517e491e9a913c38 1
locks/957/957e07b5ab346c7d35f059925cbe7811 1
locks/959/959dafea2ce99c7b54f01feef77f612e 1
locks/995/995cb749d7dfdf2a4949acf32e72d6e0 1
locks/996/9962f0e41d4c2e4554239ff077a206ea 1
locks/aa8/aa846b742ca04018ff6b9b0816851aa3 1
locks/b01/b01f60776f1611a2a8caeb6c7166d3a0 1
locks/b0e/b0e3e729e6081f7551c90d364f120b38 1
locks/b57/b57f141751c72ad8108a116bb401690c 1
locks/bbd/bbd4c9b811988c54c6a6b3c11529aacb 1
locks/be4/be46012d06047eaeaa96c281614c061a 1
locks/be6/be6e141b103283984d19501786b0f8da 1
locks/bf9/bf9153b8eee3bc1820ff612775f451bc 1
locks/c47/c478d47a9c24edbd95dcf0d28536e9f9 1
locks/c5a/c5a384422cc9f467f6ecfb120bbb9f2b 1
locks/c5b/c5bea8b8524d6ace3184ce96e767cc78 1
locks/c8e/c8e8a7cea6d5eecf7b60c1c50ecee590 1
locks/ca8/ca8a023282dab3376f2b2de02660fb55 1
locks/d2c/d2c6e6ebb1174f2c830c0d653f4f0e56 1
locks/d6a/d6a620c4f87c43e4f2100da220882228 1
locks/d87/d873f77dd2bb5669e04ae4855eb8fb22 1
locks/d8f/d8fd0726c4211cb373d1b11b2e98ef6f 1
locks/dbf/dbfe8482092721e3fc6116bbe1cf8da2 1
locks/ded/dedcb3732ceff6aa44e4fcb1df0e296e 1
locks/e36/e36d56890bb484defbc3610ec4371f3f 1
locks/eac/eacf2c26e3bcf4b7125b3d49448766d3 1
locks/eaf/eaf4007eda627542cf981d228c8484ae 1
locks/ebb/ebb75c58c0f87888a1a8babe5b9f4a8c 1
locks/ee0/ee01aee7a204a678e06d9b85bf329501 1
locks/f24/f243b0a6782e22d5eb06dc86ed3bdb58 1
locks/f4c/f4c0f8c9e2fe1b9871dac1894e34df30 1
locks/f5b/f5bba5dc1d5de32a4e09c297a64acc1f 1
locks/f71/f71a613b6bca5fd826021008f6d9165b 1
locks/fa2/fa2d0d8ea06bb1bb719826c1c45edb6d 1
revs/1/1143 1
revs/3/3474 1
revs/4/4511 1
revs/0/518 2
revs/0/759 2
revs/2/2927 2
revs/0/171 4
revs/0/182 4
revs/0/315 4
revs/0/383 4
revs/0/384 4
revs/0/420 4
revs/0/430 4
revs/0/483 4
revs/0/487 4
revs/0/499 4
revs/0/520 4
revs/0/543 4
revs/0/596 4
revs/0/627 4
revs/0/670 4
revs/0/674 4
revs/0/695 4
revs/0/777 4
revs/0/794 4
revs/0/827 4
revs/0/905 4
revs/1/1071 4
revs/1/1136 4
revs/1/1142 4
revs/1/1171 4
revs/1/1220 4
revs/1/1230 4
revs/1/1288 4
revs/1/1362 4
revs/1/1483 4
revs/1/1503 4
revs/1/1505 4
revs/1/1523 4
revs/1/1648 4
revs/1/1721 4
revs/1/1729 4
revs/1/1731 4
revs/1/1863 4
revs/1/1866 4
revs/1/1936 4
revs/1/1946 4
revs/1/1969 4
revs/2/2066 4
revs/2/2067 4
revs/2/2093 4
revs/2/2105 4
revs/2/2106 4
revs/2/2122 4
revs/2/2124 4
revs/2/2129 4
revs/2/2152 4
revs/2/2154 4
revs/2/2157 4
revs/2/2216 4
revs/2/2221 4
revs/2/2325 4
revs/2/2370 4
revs/2/2381 4
revs/2/2488 4
revs/2/2509 4
revs/2/2523 4
revs/2/2532 4
revs/2/2665 4
revs/2/2677 4
revs/2/2745 4
revs/2/2775 4
revs/2/2785 4
revs/2/2817 4
revs/2/2906 4
revs/2/2921 4
revs/3/3085 4
revs/3/3150 4
revs/3/3177 4
revs/3/3219 4
revs/3/3290 4
revs/3/3301 4
revs/3/3350 4
revs/3/3367 4
revs/3/3420 4
revs/3/3481 4
revs/3/3637 4
revs/3/3670 4
revs/3/3771 4
revs/3/3812 4
revs/3/3822 4
revs/3/3842 4
revs/3/3900 4
revs/3/3908 4
revs/3/3910 4
revs/3/3947 4
revs/3/3966 4
revs/3/3973 4
revs/4/4008 4
revs/4/4017 4
revs/4/4021 4
revs/4/4043 4
revs/4/4070 4
revs/4/4105 4
revs/4/4191 4
revs/4/4195 4
revs/4/4230 4
revs/4/4248 4
revs/4/4250 4
revs/4/4390 4
revs/4/4421 4
revs/4/4427 4
revs/4/4447 4
revs/4/4492 4
revs/4/4508 4
revs/4/4544 4
revs/4/4562 4
revs/4/4563 4
revs/4/4592 4
revs/4/4600 4
revs/4/4629 4
revs/1/1389 5
revs/1/1426 5
revs/1/1753 5
revs/1/1974 5
revs/2/2174 5
revs/3/3147 5
revs/3/3215 5
revs/3/3416 5
revs/3/3788 5
revs/4/4205 5
revs/4/4331 5
revs/4/4399 5
revs/4/4403 5
revs/4/4542 5
revs/4/4547 5
revs/4/4633 5
revs/0/584 6
revs/3/3639 6
revs/0/107 8
revs/0/128 8
revs/0/135 8
revs/0/224 8
revs/0/370 8
revs/0/443 8
revs/0/578 8
revs/0/609 8
revs/0/690 8
revs/0/843 8
revs/0/962 8
revs/0/987 8
revs/1/1595 8
revs/1/1763 8
revs/1/1852 8
revs/1/1870 8
revs/2/2137 8
revs/2/2148 8
revs/2/2149 8
revs/2/2186 8
revs/2/2258 8
revs/2/2594 8
revs/2/2596 8
revs/2/2734 8
revs/2/2772 8
revs/3/3138 8
revs/3/3319 8
revs/3/3329 8
revs/3/3382 8
revs/3/3480 8
revs/3/3524 8
revs/3/3634 8
revs/3/3668 8
revs/3/3755 8
revs/3/3942 8
revs/3/3970 8
revs/4/4011 8
revs/4/4094 8
revs/4/4193 8
revs/4/4231 8
revs/4/4299 8
revs/4/4360 8
revs/4/4371 8
revs/4/4435 8
revs/4/4442 8
revs/4/4466 8
revs/4/4516 8
revs/4/4589 8
revs/4/4594 8
revs/0/277 9
revs/0/710 9
revs/1/1138 9
revs/1/1992 9
revs/2/2159 9
revs/2/2363 9
revs/2/2616 9
revs/3/3247 9
revs/3/3383 9
revs/4/4100 9
revs/0/841 10
revs/2/2320 10
revs/2/2857 10
revs/2/2941 10
revs/3/3928 10
revs/4/4373 10
revs/0/527 12
revs/0/770 12
revs/0/857 12
revs/1/1175 12
revs/2/2101 12
revs/2/2187 12
revs/2/2849 12
revs/3/3141 12
revs/3/3316 12
revs/3/3349 12
revs/3/3689 12
revs/3/3795 12
revs/4/4091 12
revs/4/4228 12
revs/4/4268 12
revs/4/4500 12
revs/4/4536 12
revs/0/439 13
revs/1/1918 13
revs/2/2615 13
revs/4/4391 13
revs/4/4559 13
revs/1/1812 14
revs/4/4225 14
revs/3/3954 15
revs/0/328 16
revs/1/1006 16
revs/1/1853 16
revs/2/2097 16
revs/2/2579 16
revs/2/2640 16
revs/2/2762 16
revs/2/2871 16
revs/3/3234 16
revs/3/3410 16
revs/3/3636 16
revs/3/3766 16
revs/4/4027 16
revs/1/1187 17
revs/1/1331 17
revs/1/1926 17
revs/3/3127 17
revs/3/3376 17
revs/3/3990 17
revs/1/1783 18
revs/3/3682 18
revs/3/3750 18
revs/4/4452 18
revs/3/3501 19
revs/3/3690 19
revs/4/4392 19
revs/0/143 20
revs/0/547 20
revs/1/1860 20
revs/1/1978 20
revs/4/4454 20
revs/0/889 21
revs/2/2247 21
revs/3/3126 21
revs/3/3421 21
revs/4/4194 21
revs/4/4461 21
revs/1/1416 22
revs/3/3937 22
revs/4/4181 22
revs/3/3988 23
revs/4/4007 23
revs/4/4377 23
revs/4/4438 23
revs/4/4463 23
revs/1/1087 24
revs/1/1560 24
revs/3/3743 24
revs/4/4033 24
revs/4/4389 24
revs/4/4439 24
revs/4/4582 27
revs/0/260 28
revs/2/2528 28
revs/2/2650 28
revs/2/2908 28
revs/2/2933 28
revs/4/4412 29
revs/3/3436 30
revs/4/4227 30
revs/4/4537 30
revs/2/2435 31
revs/3/3930 32
revs/4/4211 33
revs/3/3886 35
revs/3/3227 36
revs/4/4560 36
revs/3/3288 38
revs/4/4644 38
revs/3/3905 40
revs/3/3109 41
revs/3/3496 41
revs/3/3737 42
revs/4/4479 44
revs/4/4615 44
revs/3/3769 45
revs/1/1568 46
revs/2/2572 46
revs/3/3424 47
revs/4/4441 51
revs/4/4455 53
revs/2/2753 54
revs/4/4066 56
revs/3/3710 59
revs/3/3977 60
revs/4/4062 60
revs/4/4555 61
revs/4/4650 64
revs/4/4630 66
revs/4/4402 69
revs/4/4406 81
revs/4/4624 81
revs/0/65 84
revs/4/4612 89
revs/4/4646 89
revs/4/4459 90
revs/4/4645 106
revs/3/3708 124
revs/2/2773 131
revs/4/4506 189
revs/2/2641 266
revs/0/556 311
revs/0/441 477
revs/4/4649 590
revs/1/1915 667
revs/2/2554 825
revs/3/3053 1014
revs/4/4180 3870
revprops/4/4661 5967
revs/0/2 20445
revs/4/4661 330459
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS optimization
Posted by Eric Gillespie <ep...@pretzelnet.org>.
"Dan Christian" <dc...@google.com> writes:
> UUID stands for Universally Unique Identifier. You've failed as soon
> as you have two different filesystems with the same UUID.
> The test suite would need to be modified to generate properly unique
> UUIDs. The current behavior is broken (IMHO).
+1
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS optimization
Posted by Dan Christian <dc...@google.com>.
On 9/21/07, Malcolm Rowe <ma...@farside.org.uk> wrote:
> On Wed, Sep 19, 2007 at 02:18:40PM +0100, Malcolm Rowe wrote:
> > > A completely separate idea is to work on caching revision files on
> > > local disk (so the full text version doesn't have to be regenerated
> > > repeatedly).
> >
> > +1. A server-side rep cache sounds like an excellent idea.
>
> Oh, something that makes any kind of caching hard to do in FSFS is that
> the following should work (or, rather, we haven't said it shouldn't, and
> haven't prohibited it in any meaningful way):
>
> $ svnserve -d -r .
> $ svnadmin create repo
> $ svnadmin load repo < dumpfile
>
> Option 1:
> $ svn co svn://localhost/repo wc
> $ touch wc/foo; svn add wc/foo; svn ci wc -m "log message"; rm -rf wc
> # now undo that commit:
> $ rm -rf repo; svnadmin create repo; svnadmin load repo < dumpfile
> # (how can svnserve [or any process] know to invalidate the cache?]
It doesn't have to. The files are still valid at certain revisions.
There is already a layer that knows whether that file "exists" at the
requested revision.
>
> Option 2:
> $ svnadmin hotcopy repo repocopy
> $ svn co svn://localhost/repo wc
> $ touch wc/foo; svn add wc/foo; svn ci wc -m "log message"; rm -rf wc
> $ svn ls svn://localhost/repocopy
> # oops! Two filesystems open with the same UUID. Hope we're not just
> # using UUID/path/rev as the cache key.
UUID stands for Universally Unique Identifier. You've failed as soon
as you have two different filesystems with the same UUID.
Yes, the svn test suite does this, and tests fail when caching is enabled.
I have an updated version of DannyB's memcached caching patch. I had
to add a hook to the test suite to restart memcached after every
repository creation step. This enables the test suite to pass with
caching enabled.
> I've been wondering whether to check for 'same UUID, different
> filesystem' at open time and disallow it (since we already have data
> structures that are keyed off the UUID).
This seems like the right thing to do. I don't know if there would be
special cases where you would have to skip the check.
> I'm concerned that that might
> cause more problems that it solves (like: I wouldn't be suprised if it
> breaks our test suite).
The test suite would need to be modified to generate properly unique
UUIDs. The current behavior is broken (IMHO).
-Dan C
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS optimization
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Wed, Sep 19, 2007 at 02:18:40PM +0100, Malcolm Rowe wrote:
> > A completely separate idea is to work on caching revision files on
> > local disk (so the full text version doesn't have to be regenerated
> > repeatedly).
>
> +1. A server-side rep cache sounds like an excellent idea.
Oh, something that makes any kind of caching hard to do in FSFS is that
the following should work (or, rather, we haven't said it shouldn't, and
haven't prohibited it in any meaningful way):
$ svnserve -d -r .
$ svnadmin create repo
$ svnadmin load repo < dumpfile
Option 1:
$ svn co svn://localhost/repo wc
$ touch wc/foo; svn add wc/foo; svn ci wc -m "log message"; rm -rf wc
# now undo that commit:
$ rm -rf repo; svnadmin create repo; svnadmin load repo < dumpfile
# (how can svnserve [or any process] know to invalidate the cache?]
Option 2:
$ svnadmin hotcopy repo repocopy
$ svn co svn://localhost/repo wc
$ touch wc/foo; svn add wc/foo; svn ci wc -m "log message"; rm -rf wc
$ svn ls svn://localhost/repocopy
# oops! Two filesystems open with the same UUID. Hope we're not just
# using UUID/path/rev as the cache key.
I've been wondering whether to check for 'same UUID, different
filesystem' at open time and disallow it (since we already have data
structures that are keyed off the UUID). I'm concerned that that might
cause more problems that it solves (like: I wouldn't be suprised if it
breaks our test suite).
Regards,
Malcolm
Re: FSFS optimization
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Wed, Sep 19, 2007 at 07:09:09AM -0700, Blair Zajac wrote:
> I need long lived transactions that are visible to multiple systems, so
> definitely don't want to make this the only place where transactions are
> built. I would like to see a configuration for fsfs that would specify
> where the transaction directory is.
>
This would absolutely need to be configurable, yes. If you want
mutliple front-end servers (against, say, an NFS backend), you need to
either use session-affinity or leave the transaction on NFS.
Regards,
Malcolm
Re: FSFS optimization
Posted by Blair Zajac <bl...@orcaware.com>.
On Sep 19, 2007, at 6:18 AM, Malcolm Rowe wrote:
> On Mon, Aug 13, 2007 at 03:18:59PM -0700, Dan Christian wrote:
>> The first thought is to work on caching some of these: "node.0.0" is
>> the node revision, and "props" contains the revision properties. The
>> "changes" file is always(?) appended to, so we might be able to keep
>> it open and keep appending to it. The tricky bit is that the httpd
>> will close at some point and an a fresh one will end up advancing the
>> transaction. This implies that the disk version must be kept up to
>> date.
>>
>
> This is the key, actually. NFS only guarantees close-to-open cache
> coherency, so we need to ensure a file written by one process/host is
> closed before it's opened by another one... and there unfortunately
> isn't any explicit FS call that says "I'm finished with this
> transaction
> on this process for the minute, feel free to flush state to disk now".
>
> And so we end up flushing to disk after _every_ operation, which
> sucks.
>
>> An alternative/additional thought would be to move the transactions
>> directory to local disk so that the OS can cache more aggressively.
>> This would require network load balancers to always direct the same
>> client to the same server.
>
> I think this is the way to go. I had a patch to do this, I think,
> though I don't think I had thought through everything, so there
> might be
> lurking issues. (Also, from a UI point-of-view, how do you switch it
> on?)
I need long lived transactions that are visible to multiple systems,
so definitely don't want to make this the only place where
transactions are built. I would like to see a configuration for fsfs
that would specify where the transaction directory is.
>
>> A completely separate idea is to work on caching revision files on
>> local disk (so the full text version doesn't have to be regenerated
>> repeatedly).
>
> +1. A server-side rep cache sounds like an excellent idea.
> (Again, I wonder about the configuration, though. Maybe FSFS needs
> some
> equivalent to the DB_CONFIG file... or maybe svnadmin should have an
> fsfstune command?)
A DB_CONFIG like file sounds easier to code, just to edit a text
file. But maybe you want to be able to edit that file in a live
repository, so you would need an svnadmin fsfstune that would allow
you to change values and move transactions in progress from one
directory to another.
Regards,
Blair
--
Blair Zajac, Ph.D.
<bl...@orcaware.com>
Subversion training, consulting and support
http://www.orcaware.com/training/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS optimization
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Mon, Aug 13, 2007 at 03:18:59PM -0700, Dan Christian wrote:
> The first thought is to work on caching some of these: "node.0.0" is
> the node revision, and "props" contains the revision properties. The
> "changes" file is always(?) appended to, so we might be able to keep
> it open and keep appending to it. The tricky bit is that the httpd
> will close at some point and an a fresh one will end up advancing the
> transaction. This implies that the disk version must be kept up to
> date.
>
This is the key, actually. NFS only guarantees close-to-open cache
coherency, so we need to ensure a file written by one process/host is
closed before it's opened by another one... and there unfortunately
isn't any explicit FS call that says "I'm finished with this transaction
on this process for the minute, feel free to flush state to disk now".
And so we end up flushing to disk after _every_ operation, which sucks.
> An alternative/additional thought would be to move the transactions
> directory to local disk so that the OS can cache more aggressively.
> This would require network load balancers to always direct the same
> client to the same server.
I think this is the way to go. I had a patch to do this, I think,
though I don't think I had thought through everything, so there might be
lurking issues. (Also, from a UI point-of-view, how do you switch it
on?)
> A completely separate idea is to work on caching revision files on
> local disk (so the full text version doesn't have to be regenerated
> repeatedly).
+1. A server-side rep cache sounds like an excellent idea.
(Again, I wonder about the configuration, though. Maybe FSFS needs some
equivalent to the DB_CONFIG file... or maybe svnadmin should have an
fsfstune command?)
Regards,
Malcolm