Two JVM accessing same TDB store

["Chaudhuri, Rajiv" <ra...@pearson.com>]

Hi

We are using fuseki server to store the triples and we want to have two JVM
instances (two tomcat server) to access the same tdb store using two
instances of fuseki server having same shared tdb file location. But it
does not work. We have found only one instance of fuseki can work at time.
If we want to have other instance of fuseki to work then we have to stop
the other one.

We would like to have both fuseki server works with the same shared tdb
file location so that load can be distributed and at the same time we can
achieve the horizontally scale up.
Please advice how can we get them work both at the same time using the same
tdb file location. Or is there any alternative architecture for this?

Rajiv

Re: Two JVM accessing same TDB store

["Chaudhuri, Rajiv" <ra...@pearson.com>]

Thank you, Andy!
On Dec 22, 2013 5:46 AM, "Andy Seaborne" <an...@apache.org> wrote:

> On 22/12/13 06:28, Chaudhuri, Rajiv wrote:
>
>> Hi
>>
>> We are using fuseki server to store the triples and we want to have two
>> JVM
>> instances (two tomcat server) to access the same tdb store using two
>> instances of fuseki server having same shared tdb file location. But it
>> does not work. We have found only one instance of fuseki can work at time.
>> If we want to have other instance of fuseki to work then we have to stop
>> the other one.
>>
>> We would like to have both fuseki server works with the same shared tdb
>> file location so that load can be distributed and at the same time we can
>> achieve the horizontally scale up.
>> Please advice how can we get them work both at the same time using the
>> same
>> tdb file location. Or is there any alternative architecture for this?
>>
>> Rajiv
>>
>>
> Rajiv,
>
> I'm guessing, but do you mean two different physical machines accessing
> shared files, the files being on some shared network drive?
>
> Only one TDB engine can work with one set of files at a time.  If any
> updates from both engines occur, the database will become corrupted; even
> if only from one, the other will give wrong answers or crash.
>
> TDB caches data in memory.  If you run two engines, they don't know about
> each other and there is no cache coordination.  Result - disaster.  Only in
> very carefully controlled read-only situations might anything useful be
> done but it is unsupported; if it's read-only you might as well have two
> separate databases.
>
> Fuseki provides the database server (think of it as equivalent to the
> MySQL daemon that provides JDBC).  It is the database layer in a 3-tier
> architecture.
>
> It should be run on a machine provisioned as a database server. Databases
> work better with more physical RAM dedicated to the task. What sort of
> machines are you running on?  How big is the database and how many triples?
>
> Guessing again but it sounds like you are running it on as if it were
> business logic layer and on the same machines as the rest of the
> application stack.  That may mean it's under provisioned.
>
> VMs also add complexity.
>
> Fuseki can support concurrent requests. This will given horizontal scale
> via more capacity in the database server.  The true concurrency level is
> one request per core.
>
> The next level of horizontal scale is to run replicated database servers
> (also good for fault tolerance reasons) but to do that you have to manage
> the replication of updates.  Fuseki/TDB on its own does not do that out of
> the box.  How that should be done depends very much on the architecture of
> your application.  We do it for systems we run.
>
>         Andy
>
>

Re: Two JVM accessing same TDB store

[Andy Seaborne <an...@apache.org>]

On 22/12/13 06:28, Chaudhuri, Rajiv wrote:
> Hi
>
> We are using fuseki server to store the triples and we want to have two JVM
> instances (two tomcat server) to access the same tdb store using two
> instances of fuseki server having same shared tdb file location. But it
> does not work. We have found only one instance of fuseki can work at time.
> If we want to have other instance of fuseki to work then we have to stop
> the other one.
>
> We would like to have both fuseki server works with the same shared tdb
> file location so that load can be distributed and at the same time we can
> achieve the horizontally scale up.
> Please advice how can we get them work both at the same time using the same
> tdb file location. Or is there any alternative architecture for this?
>
> Rajiv
>

Rajiv,

I'm guessing, but do you mean two different physical machines accessing 
shared files, the files being on some shared network drive?

Only one TDB engine can work with one set of files at a time.  If any 
updates from both engines occur, the database will become corrupted; 
even if only from one, the other will give wrong answers or crash.

TDB caches data in memory.  If you run two engines, they don't know 
about each other and there is no cache coordination.  Result - disaster. 
  Only in very carefully controlled read-only situations might anything 
useful be done but it is unsupported; if it's read-only you might as 
well have two separate databases.

Fuseki provides the database server (think of it as equivalent to the 
MySQL daemon that provides JDBC).  It is the database layer in a 3-tier 
architecture.

It should be run on a machine provisioned as a database server. 
Databases work better with more physical RAM dedicated to the task. 
What sort of machines are you running on?  How big is the database and 
how many triples?

Guessing again but it sounds like you are running it on as if it were 
business logic layer and on the same machines as the rest of the 
application stack.  That may mean it's under provisioned.

VMs also add complexity.

Fuseki can support concurrent requests. This will given horizontal scale 
via more capacity in the database server.  The true concurrency level is 
one request per core.

The next level of horizontal scale is to run replicated database servers 
(also good for fault tolerance reasons) but to do that you have to 
manage the replication of updates.  Fuseki/TDB on its own does not do 
that out of the box.  How that should be done depends very much on the 
architecture of your application.  We do it for systems we run.

	Andy