Concurrency in filesystem's, root's and txn's

[Blair Zajac <bl...@orcaware.com>]

I'm writing a multithreaded server that uses Subversion as a backend database. 
The server exposes an object oriented view of repositories, filesystem roots, 
transactions', etc so clients can refer to them explicitly.

For each RPC I pull from an LRU cache an svn object and return it to the cache 
when it is completed, thereby saving IO and avoiding any concurrency issues 
since any svn object is in use by only one thread.

Since txn's are mutable, I am caching the fs root but opening a new transaction 
object anytime the transaction is modified.

I was looking through svnserve and it appears to open a new fs root for each 
operation, so there's no concurrency issues there.

Are there any problems with using revision fs roots across different threads at 
the same time?  Can you have a single svn_fs_t shared by multiple threads, each 
thread doing work with different revisions or tnx roots?

So what is safe and what isn't?

Thanks,
Blair

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Concurrency in filesystem's, root's and txn's

[David Glasser <gl...@davidglasser.net>]

Busy today, but quick answer is that I think you shouldn't share
svn_fs_t's themselves, but they internally share some state and locks
across threads if they're to the same FS.  Look at libsvn_fs_fs/fs.h
for more fun there.

Concurrent modifications of transactions is kinda broken.  There's
locking code to serialize writes to the proto-rev file, but no
serialization of other stuff (children, props, etc), which has always
struck me as a little problematic.

--dave

On Jan 7, 2008 1:51 PM, Blair Zajac <bl...@orcaware.com> wrote:
> I'm writing a multithreaded server that uses Subversion as a backend database.
> The server exposes an object oriented view of repositories, filesystem roots,
> transactions', etc so clients can refer to them explicitly.
>
> For each RPC I pull from an LRU cache an svn object and return it to the cache
> when it is completed, thereby saving IO and avoiding any concurrency issues
> since any svn object is in use by only one thread.
>
> Since txn's are mutable, I am caching the fs root but opening a new transaction
> object anytime the transaction is modified.
>
> I was looking through svnserve and it appears to open a new fs root for each
> operation, so there's no concurrency issues there.
>
> Are there any problems with using revision fs roots across different threads at
> the same time?  Can you have a single svn_fs_t shared by multiple threads, each
> thread doing work with different revisions or tnx roots?
>
> So what is safe and what isn't?
>
> Thanks,
> Blair
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>



-- 
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Concurrency in filesystem's, root's and txn's

[Garrett Rooney <ro...@electricjellyfish.net>]

On Jan 7, 2008 1:51 PM, Blair Zajac <bl...@orcaware.com> wrote:
> I'm writing a multithreaded server that uses Subversion as a backend database.
> The server exposes an object oriented view of repositories, filesystem roots,
> transactions', etc so clients can refer to them explicitly.
>
> For each RPC I pull from an LRU cache an svn object and return it to the cache
> when it is completed, thereby saving IO and avoiding any concurrency issues
> since any svn object is in use by only one thread.
>
> Since txn's are mutable, I am caching the fs root but opening a new transaction
> object anytime the transaction is modified.
>
> I was looking through svnserve and it appears to open a new fs root for each
> operation, so there's no concurrency issues there.
>
> Are there any problems with using revision fs roots across different threads at
> the same time?  Can you have a single svn_fs_t shared by multiple threads, each
> thread doing work with different revisions or tnx roots?
>
> So what is safe and what isn't?

AFAICT the internals of the revision root data structures do no
locking at all, and while the revision on disk is immutable the data
structures in memory certainly aren't (fsfs's copyfrom_cache for
example, it gets filled in fs_fs_paths_changed, which could
conceivably race with code in fs_copied_from that uses the cache,
since apr_hash_t is not thread safe internally).

In general I'd say that for basically any data structure in svn that
doesn't explicitly say in big bold letters "I am thread safe" it is
probably safe to assume is very much not thread safe.

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org