You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David Glasser <gl...@davidglasser.net> on 2007/12/24 23:36:39 UTC
proposed: move FSFS protorevs out of the transactions dir
The access patterns of the files in the FSFS repository fall into
three distinct patterns:
* The proto-revision files (transactions/<txnid>.txn/rev)
These are appended to during the course of the transaction, and
eventually renamed into place into the revs directory. These can be
arbitrarily long. I don't believe that any normal Subversion
operation even reads from the txn file (even though
open_and_seek_transaction exists, I don't think anything will
actually call it, though feel free to correct me on that).
* Everything else in the transaction directory
(transactions/<txnid>.txn/rev)
For example, txn noderevs, directory lists, properties, etc. These
are repeatedly read and written during the transaction, and are
generally relatively short.
* The revision data (revs/*)
This is read-only.
(Yeah, there are some other random things, but these are the biggies.)
Administrators may wish to use different forms of storage for these
different kinds of data. For example, the actual revision data (and
the current and txn-current files) may want to live on a network file
share; the transaction-specific data may want to live on local disk in
the server (which requires using a server setup that guarantees that
all operations on the same transaction use the same server).
This can easily be accomplished by symlinking "db/transactions/" to
local disk. (Well, at least, since r28578 fixed a bug in cross-device
copies.) This lets you keep the noderev/directory list/txnprops/etc
read/write cycles on local disk. On the other hand, it'll also keep
the (potentially large) protorev file, which is only appended to
during commit, on local disk, and then the move-into-place during the
write-locked part of the commit becomes a non-constant-time copy. And
the benefit of having the transaction data on local disk just doesn't
seem to apply to the append-only proto-rev.
Thus, it seems reasonable to allow administrators to use symlinks in
this way to move the non-protorev txn data to a different filesystem
without moving the protorevs. I suggest the attached patch, which
moves the protorev (and its lock) into db/txn-protorevs/. We're
already bumping the filesystem format in 1.5...
[[[
In already-bumped FSFS version 3, store the protorev and its lock
separately from the rest of the transaction metadata, to make it easy
for administrators to put the non-protorev metadata on a different
filesystem from the immutable filesystem.
* subversion/libsvn_fs_fs/fs.h
(PATH_TXN_PROTOS_DIR, PATH_EXT_REV, PATH_EXT_REV_LOCK): New.
(PATH_REV, PATH_REV_LOCK): Move later and document as legacy.
(SVN_FS_FS__MIN_PROTOREVS_DIR_FORMAT): New.
* subversion/libsvn_fs_fs/fs_fs.c
(path_txn_proto_rev, path_txn_proto_rev_lock): Return the new path
if the format is new enough.
(svn_fs_fs__hotcopy): Make an empty txn-protorevs directory if the
format supports it.
(svn_fs_fs__purge_txn): If the format is new enough, try to delete
the protorev file and its lock after removing the txn directory.
(svn_fs_fs__create): If the format is new enough, create
txn-protorevs directory.
* subversion/libsvn_fs_fs/structure
(Layout of the FS directory): Document txn-protorevs.
(Filesystem formats): Document difference between versions.
(Transaction layout): Document difference between versions.
]]]
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by Dan Christian <dc...@google.com>.
It's just my usual micro benchmark. I total over a series of
checkouts and commits. I doubt it affects checkouts. I could show
results at each step if that helps (it's a ton of data to sift
through).
The repo was on NFS, but the transactions directory was symlinked to a
directory on local disk.
-Dan C
On Jan 8, 2008 8:18 PM, David Glasser <gl...@davidglasser.net> wrote:
> I'll commit it in a day or so. I assume your testing was with NFS and
> local? (Checkouts and commits? Does it actually speed up checkups?)
>
> --dave
>
>
> On Jan 8, 2008 6:33 PM, Dan Christian <dc...@google.com> wrote:
> > This works for me under simple testing (just checkouts and commits).
> > It cuts the total time spent in file system operations roughly in half
> > (compared to everything on NFS). That should be a big win for
> > scalability!
> >
> > Can this go in?
> > -Dan C
> >
> >
> > On Dec 25, 2007 7:14 PM, David Glasser <gl...@davidglasser.net> wrote:
> > > On Dec 25, 2007 6:01 PM, Branko Čibej <br...@xbc.nu> wrote:
> > > > David Glasser wrote:
> > > > > In already-bumped FSFS version 3, store the protorev and its lock
> > > > > separately from the rest of the transaction metadata, to make it easy
> > > > > for administrators to put the non-protorev metadata on a different
> > > > > filesystem from the immutable filesystem.
> > > > >
> > > >
> > > > Wouldn't doing that change an atomic rename to a non-atomic cross-device
> > > > move?
> > >
> > > Actually, avoiding that is the point :-)
> > >
> > > It's already the case that the move-into-place code in FSFS checks for
> > > a cross-device error, and does a copy instead in that case. Note,
> > > though, that the "copy" is actually "copy to temporary file, then
> > > atomically rename".
> > >
> > > However, the point here is I want to make it possible to store the txn
> > > metadata that *doesn't* get atomically moved on a different device,
> > > and leave the proto-rev on the same device as the revs. The use case
> > > I'm imagining would be to make db/transactions/ a symlink to local
> > > disk, but leave db/txn-protorevs/ on the same device as db/revs/.
> > >
> > > --dave
> > >
> > >
> > > --
> > > David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
> > >
> >
>
>
>
> --
>
> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by Eric Gillespie <ep...@pretzelnet.org>.
"David Glasser" <gl...@davidglasser.net> writes:
> I'll commit it in a day or so. I assume your testing was with NFS and
> local? (Checkouts and commits? Does it actually speed up checkups?)
It shouldn't affect checkouts directly, as transactions are only
involved for commits. However, reducing NFS traffic from commits
should be a win on busy repositories.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by David Glasser <gl...@davidglasser.net>.
I'll commit it in a day or so. I assume your testing was with NFS and
local? (Checkouts and commits? Does it actually speed up checkups?)
--dave
On Jan 8, 2008 6:33 PM, Dan Christian <dc...@google.com> wrote:
> This works for me under simple testing (just checkouts and commits).
> It cuts the total time spent in file system operations roughly in half
> (compared to everything on NFS). That should be a big win for
> scalability!
>
> Can this go in?
> -Dan C
>
>
> On Dec 25, 2007 7:14 PM, David Glasser <gl...@davidglasser.net> wrote:
> > On Dec 25, 2007 6:01 PM, Branko Čibej <br...@xbc.nu> wrote:
> > > David Glasser wrote:
> > > > In already-bumped FSFS version 3, store the protorev and its lock
> > > > separately from the rest of the transaction metadata, to make it easy
> > > > for administrators to put the non-protorev metadata on a different
> > > > filesystem from the immutable filesystem.
> > > >
> > >
> > > Wouldn't doing that change an atomic rename to a non-atomic cross-device
> > > move?
> >
> > Actually, avoiding that is the point :-)
> >
> > It's already the case that the move-into-place code in FSFS checks for
> > a cross-device error, and does a copy instead in that case. Note,
> > though, that the "copy" is actually "copy to temporary file, then
> > atomically rename".
> >
> > However, the point here is I want to make it possible to store the txn
> > metadata that *doesn't* get atomically moved on a different device,
> > and leave the proto-rev on the same device as the revs. The use case
> > I'm imagining would be to make db/transactions/ a symlink to local
> > disk, but leave db/txn-protorevs/ on the same device as db/revs/.
> >
> > --dave
> >
> >
> > --
> > David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
> >
>
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by Dan Christian <dc...@google.com>.
This works for me under simple testing (just checkouts and commits).
It cuts the total time spent in file system operations roughly in half
(compared to everything on NFS). That should be a big win for
scalability!
Can this go in?
-Dan C
On Dec 25, 2007 7:14 PM, David Glasser <gl...@davidglasser.net> wrote:
> On Dec 25, 2007 6:01 PM, Branko Čibej <br...@xbc.nu> wrote:
> > David Glasser wrote:
> > > In already-bumped FSFS version 3, store the protorev and its lock
> > > separately from the rest of the transaction metadata, to make it easy
> > > for administrators to put the non-protorev metadata on a different
> > > filesystem from the immutable filesystem.
> > >
> >
> > Wouldn't doing that change an atomic rename to a non-atomic cross-device
> > move?
>
> Actually, avoiding that is the point :-)
>
> It's already the case that the move-into-place code in FSFS checks for
> a cross-device error, and does a copy instead in that case. Note,
> though, that the "copy" is actually "copy to temporary file, then
> atomically rename".
>
> However, the point here is I want to make it possible to store the txn
> metadata that *doesn't* get atomically moved on a different device,
> and leave the proto-rev on the same device as the revs. The use case
> I'm imagining would be to make db/transactions/ a symlink to local
> disk, but leave db/txn-protorevs/ on the same device as db/revs/.
>
> --dave
>
>
> --
> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by David Glasser <gl...@davidglasser.net>.
On Dec 25, 2007 6:01 PM, Branko Čibej <br...@xbc.nu> wrote:
> David Glasser wrote:
> > In already-bumped FSFS version 3, store the protorev and its lock
> > separately from the rest of the transaction metadata, to make it easy
> > for administrators to put the non-protorev metadata on a different
> > filesystem from the immutable filesystem.
> >
>
> Wouldn't doing that change an atomic rename to a non-atomic cross-device
> move?
Actually, avoiding that is the point :-)
It's already the case that the move-into-place code in FSFS checks for
a cross-device error, and does a copy instead in that case. Note,
though, that the "copy" is actually "copy to temporary file, then
atomically rename".
However, the point here is I want to make it possible to store the txn
metadata that *doesn't* get atomically moved on a different device,
and leave the proto-rev on the same device as the revs. The use case
I'm imagining would be to make db/transactions/ a symlink to local
disk, but leave db/txn-protorevs/ on the same device as db/revs/.
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by Branko Čibej <br...@xbc.nu>.
David Glasser wrote:
> In already-bumped FSFS version 3, store the protorev and its lock
> separately from the rest of the transaction metadata, to make it easy
> for administrators to put the non-protorev metadata on a different
> filesystem from the immutable filesystem.
>
Wouldn't doing that change an atomic rename to a non-atomic cross-device
move?
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by David Glasser <gl...@davidglasser.net>.
On Dec 24, 2007 4:00 PM, Arfrever Frehtes Taifersar Arahesis
<ar...@gmail.com> wrote:
> 2007-12-25 00:36:16 David Glasser napisał(a):
> > I suggest the attached patch
>
> So please attach it :).
OK!
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by Arfrever Frehtes Taifersar Arahesis <ar...@gmail.com>.
2007-12-25 00:36:16 David Glasser napisał(a):
> I suggest the attached patch
So please attach it :).
--
Arfrever Frehtes Taifersar Arahesis
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: proposed: move FSFS protorevs out of the transactions dir
Posted by David Glasser <gl...@davidglasser.net>.
Committed in r28829.
--dave
On Dec 24, 2007 6:36 PM, David Glasser <gl...@davidglasser.net> wrote:
> The access patterns of the files in the FSFS repository fall into
> three distinct patterns:
>
> * The proto-revision files (transactions/<txnid>.txn/rev)
>
> These are appended to during the course of the transaction, and
> eventually renamed into place into the revs directory. These can be
> arbitrarily long. I don't believe that any normal Subversion
> operation even reads from the txn file (even though
> open_and_seek_transaction exists, I don't think anything will
> actually call it, though feel free to correct me on that).
>
> * Everything else in the transaction directory
> (transactions/<txnid>.txn/rev)
>
> For example, txn noderevs, directory lists, properties, etc. These
> are repeatedly read and written during the transaction, and are
> generally relatively short.
>
> * The revision data (revs/*)
>
> This is read-only.
>
> (Yeah, there are some other random things, but these are the biggies.)
>
> Administrators may wish to use different forms of storage for these
> different kinds of data. For example, the actual revision data (and
> the current and txn-current files) may want to live on a network file
> share; the transaction-specific data may want to live on local disk in
> the server (which requires using a server setup that guarantees that
> all operations on the same transaction use the same server).
>
> This can easily be accomplished by symlinking "db/transactions/" to
> local disk. (Well, at least, since r28578 fixed a bug in cross-device
> copies.) This lets you keep the noderev/directory list/txnprops/etc
> read/write cycles on local disk. On the other hand, it'll also keep
> the (potentially large) protorev file, which is only appended to
> during commit, on local disk, and then the move-into-place during the
> write-locked part of the commit becomes a non-constant-time copy. And
> the benefit of having the transaction data on local disk just doesn't
> seem to apply to the append-only proto-rev.
>
> Thus, it seems reasonable to allow administrators to use symlinks in
> this way to move the non-protorev txn data to a different filesystem
> without moving the protorevs. I suggest the attached patch, which
> moves the protorev (and its lock) into db/txn-protorevs/. We're
> already bumping the filesystem format in 1.5...
>
> [[[
> In already-bumped FSFS version 3, store the protorev and its lock
> separately from the rest of the transaction metadata, to make it easy
> for administrators to put the non-protorev metadata on a different
> filesystem from the immutable filesystem.
>
> * subversion/libsvn_fs_fs/fs.h
> (PATH_TXN_PROTOS_DIR, PATH_EXT_REV, PATH_EXT_REV_LOCK): New.
> (PATH_REV, PATH_REV_LOCK): Move later and document as legacy.
> (SVN_FS_FS__MIN_PROTOREVS_DIR_FORMAT): New.
>
> * subversion/libsvn_fs_fs/fs_fs.c
> (path_txn_proto_rev, path_txn_proto_rev_lock): Return the new path
> if the format is new enough.
> (svn_fs_fs__hotcopy): Make an empty txn-protorevs directory if the
> format supports it.
> (svn_fs_fs__purge_txn): If the format is new enough, try to delete
> the protorev file and its lock after removing the txn directory.
> (svn_fs_fs__create): If the format is new enough, create
> txn-protorevs directory.
>
> * subversion/libsvn_fs_fs/structure
> (Layout of the FS directory): Document txn-protorevs.
> (Filesystem formats): Document difference between versions.
> (Transaction layout): Document difference between versions.
> ]]]
>
>
> --
> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org