You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Andreas Grünwald <a....@gmail.com> on 2013/10/14 20:47:17 UTC
TDB and multiple Connections
According to
http://jena.apache.org/documentation/tdb/tdb_transactions.html#multi-threaded-use
only one DataSet connection can be used per Java thread. However, I need
several connections to (different) data models within a single application.
I noticed that if one tries to use several datasets per single-thread then
the application is blocked when trying to create the second dataset. I
tried to overcome the issue by creating a new thread per dataset (just as
recommended in the Jena documentation). However, the problem is still the
same: The second "TDB.createDataset(..)" statement causes the application
to be blocked.
This is my (runnable) test code:
JunitTest:
public static void main(String[] args) {
String dir
= "/Users/andreas_gruenwald/da-agr/resources/design/jena_tdb/itpm_jena";
Runnable tdb1 = new TDBThread(dir, null);
Runnable tdb2 = new TDBThread(dir, null);
tdb1.run();
tdb2.run();
System.out.println("Finished");
}
TDBThread.java:
public class TDBThread implements Runnable {
private Dataset tdbDataSet;
private String directory;
public TDBThread(String directory) {
tdbDataSet = null;
this.directory = directory;
}
@Override
public void run() {
System.out.println(String.format("Run (directory=%s)...", directory));
*tdbDataSet** = TDBFactory.createDataset(directory); //blocks here*
System.out.println("Tdbdataset is now " + tdbDataSet);
tdbDataSet.begin(ReadWrite.WRITE); *//causes block*
}
}
Any ideas what I (or Jena) is doing wrong?
--
Andreas Grünwald
Tel.: +43 650 77 82340
Re: TDB and multiple Connections
Posted by Andy Seaborne <an...@apache.org>.
On 15/10/13 10:53, Andreas Grünwald wrote:
> Hello Rob,
> thank you for the response. I think I got it. I changed the code within my
> application accordingly.
>
>
>
> On Tue, Oct 15, 2013 at 11:02 AM, Rob Vesse <rv...@dotnetrdf.org> wrote:
>
>> Also as a corollary to my previous comments you can have multiple datasets
>> against the same dataset but only if you respect the multiple reader
>> single writer concurrency, I.e. you can have one dataset with an open
>> write transaction but all others can only have read transactions or they
>> will block.
If you have multiple datasets against the same location, they are really
the same. Underneath, they meet up. A StoreConnection is the JVM-wide
controller to access a location and there is only one per location.
There is no clash possibilities - transactions on one are transaction on
the other.
Andy
>> Rob
>>
>> On 15/10/2013 09:25, "Rob Vesse" <rv...@dotnetrdf.org> wrote:
>>
>>> I'm afraid it's you that's done something wrong on this occasion:
>>>
>>> You are opening both datasets against the same underlying TDB dataset, as
>>> part of this open you start a Write transaction which then prevents the
>>> second thread from completing because TDB is single writer concurrency.
>>>
>>> This is entirely by design, if you want several connections to different
>>> datasets on the same thread this should work fine. However your example
>>> is trying to make several connections to the same dataset so by definition
>>> is going to block. I.e. you must pass in different directories for each
>>> thread for your example to work.
>>>
>>> Also you should be calling .close() on the datasets after you are done
>>> with them otherwise you risk corrupting them. You will also want to call
>>> either .commit() or .end()/.abort() on the dataset depending on whether
>>> you want to commit/rollback your transaction. If you don't call something
>>> explicitly before calling .close() it will assume an abort() call and any
>>> changes will be list.
>>>
>>> Rob
>>>
>>>
>>>
>>> On 14/10/2013 19:47, "Andreas Grünwald" <a....@gmail.com> wrote:
>>>
>>>> According to
>>>>
>> http://jena.apache.org/documentation/tdb/tdb_transactions.html#multi-thre
>>>> a
>>>> ded-use
>>>>
>>>> only one DataSet connection can be used per Java thread. However, I need
>>>> several connections to (different) data models within a single
>>>> application.
>>>> I noticed that if one tries to use several datasets per single-thread
>>>> then
>>>> the application is blocked when trying to create the second dataset. I
>>>> tried to overcome the issue by creating a new thread per dataset (just as
>>>> recommended in the Jena documentation). However, the problem is still the
>>>> same: The second "TDB.createDataset(..)" statement causes the application
>>>> to be blocked.
>>>>
>>>> This is my (runnable) test code:
>>>>
>>>> JunitTest:
>>>>
>>>> public static void main(String[] args) {
>>>> String dir
>>>> = "/Users/andreas_gruenwald/da-agr/resources/design/jena_tdb/itpm_jena";
>>>> Runnable tdb1 = new TDBThread(dir, null);
>>>> Runnable tdb2 = new TDBThread(dir, null);
>>>> tdb1.run();
>>>> tdb2.run();
>>>> System.out.println("Finished");
>>>> }
>>>>
>>>>
>>>> TDBThread.java:
>>>>
>>>> public class TDBThread implements Runnable {
>>>> private Dataset tdbDataSet;
>>>> private String directory;
>>>>
>>>> public TDBThread(String directory) {
>>>> tdbDataSet = null;
>>>> this.directory = directory;
>>>> }
>>>>
>>>> @Override
>>>> public void run() {
>>>> System.out.println(String.format("Run (directory=%s)...", directory));
>>>> *tdbDataSet** = TDBFactory.createDataset(directory); //blocks here*
>>>> System.out.println("Tdbdataset is now " + tdbDataSet);
>>>> tdbDataSet.begin(ReadWrite.WRITE); *//causes block*
>>>> }
>>>> }
>>>>
>>>> Any ideas what I (or Jena) is doing wrong?
>>>>
>>>> --
>>>> Andreas Grünwald
>>>> Tel.: +43 650 77 82340
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
>
Re: TDB and multiple Connections
Posted by Andreas Grünwald <a....@gmail.com>.
Hello Rob,
thank you for the response. I think I got it. I changed the code within my
application accordingly.
On Tue, Oct 15, 2013 at 11:02 AM, Rob Vesse <rv...@dotnetrdf.org> wrote:
> Also as a corollary to my previous comments you can have multiple datasets
> against the same dataset but only if you respect the multiple reader
> single writer concurrency, I.e. you can have one dataset with an open
> write transaction but all others can only have read transactions or they
> will block.
>
> Rob
>
> On 15/10/2013 09:25, "Rob Vesse" <rv...@dotnetrdf.org> wrote:
>
> >I'm afraid it's you that's done something wrong on this occasion:
> >
> >You are opening both datasets against the same underlying TDB dataset, as
> >part of this open you start a Write transaction which then prevents the
> >second thread from completing because TDB is single writer concurrency.
> >
> >This is entirely by design, if you want several connections to different
> >datasets on the same thread this should work fine. However your example
> >is trying to make several connections to the same dataset so by definition
> >is going to block. I.e. you must pass in different directories for each
> >thread for your example to work.
> >
> >Also you should be calling .close() on the datasets after you are done
> >with them otherwise you risk corrupting them. You will also want to call
> >either .commit() or .end()/.abort() on the dataset depending on whether
> >you want to commit/rollback your transaction. If you don't call something
> >explicitly before calling .close() it will assume an abort() call and any
> >changes will be list.
> >
> >Rob
> >
> >
> >
> >On 14/10/2013 19:47, "Andreas Grünwald" <a....@gmail.com> wrote:
> >
> >>According to
> >>
> http://jena.apache.org/documentation/tdb/tdb_transactions.html#multi-thre
> >>a
> >>ded-use
> >>
> >>only one DataSet connection can be used per Java thread. However, I need
> >>several connections to (different) data models within a single
> >>application.
> >>I noticed that if one tries to use several datasets per single-thread
> >>then
> >>the application is blocked when trying to create the second dataset. I
> >>tried to overcome the issue by creating a new thread per dataset (just as
> >>recommended in the Jena documentation). However, the problem is still the
> >>same: The second "TDB.createDataset(..)" statement causes the application
> >>to be blocked.
> >>
> >>This is my (runnable) test code:
> >>
> >>JunitTest:
> >>
> >>public static void main(String[] args) {
> >> String dir
> >>= "/Users/andreas_gruenwald/da-agr/resources/design/jena_tdb/itpm_jena";
> >> Runnable tdb1 = new TDBThread(dir, null);
> >> Runnable tdb2 = new TDBThread(dir, null);
> >> tdb1.run();
> >> tdb2.run();
> >> System.out.println("Finished");
> >>}
> >>
> >>
> >>TDBThread.java:
> >>
> >>public class TDBThread implements Runnable {
> >> private Dataset tdbDataSet;
> >> private String directory;
> >>
> >>public TDBThread(String directory) {
> >> tdbDataSet = null;
> >> this.directory = directory;
> >>}
> >>
> >>@Override
> >>public void run() {
> >> System.out.println(String.format("Run (directory=%s)...", directory));
> >> *tdbDataSet** = TDBFactory.createDataset(directory); //blocks here*
> >> System.out.println("Tdbdataset is now " + tdbDataSet);
> >> tdbDataSet.begin(ReadWrite.WRITE); *//causes block*
> >>}
> >>}
> >>
> >>Any ideas what I (or Jena) is doing wrong?
> >>
> >>--
> >>Andreas Grünwald
> >>Tel.: +43 650 77 82340
> >
> >
> >
> >
>
>
>
>
>
--
Andreas Grünwald
Tel.: +43 650 77 82340
Re: TDB and multiple Connections
Posted by Rob Vesse <rv...@dotnetrdf.org>.
Also as a corollary to my previous comments you can have multiple datasets
against the same dataset but only if you respect the multiple reader
single writer concurrency, I.e. you can have one dataset with an open
write transaction but all others can only have read transactions or they
will block.
Rob
On 15/10/2013 09:25, "Rob Vesse" <rv...@dotnetrdf.org> wrote:
>I'm afraid it's you that's done something wrong on this occasion:
>
>You are opening both datasets against the same underlying TDB dataset, as
>part of this open you start a Write transaction which then prevents the
>second thread from completing because TDB is single writer concurrency.
>
>This is entirely by design, if you want several connections to different
>datasets on the same thread this should work fine. However your example
>is trying to make several connections to the same dataset so by definition
>is going to block. I.e. you must pass in different directories for each
>thread for your example to work.
>
>Also you should be calling .close() on the datasets after you are done
>with them otherwise you risk corrupting them. You will also want to call
>either .commit() or .end()/.abort() on the dataset depending on whether
>you want to commit/rollback your transaction. If you don't call something
>explicitly before calling .close() it will assume an abort() call and any
>changes will be list.
>
>Rob
>
>
>
>On 14/10/2013 19:47, "Andreas Grünwald" <a....@gmail.com> wrote:
>
>>According to
>>http://jena.apache.org/documentation/tdb/tdb_transactions.html#multi-thre
>>a
>>ded-use
>>
>>only one DataSet connection can be used per Java thread. However, I need
>>several connections to (different) data models within a single
>>application.
>>I noticed that if one tries to use several datasets per single-thread
>>then
>>the application is blocked when trying to create the second dataset. I
>>tried to overcome the issue by creating a new thread per dataset (just as
>>recommended in the Jena documentation). However, the problem is still the
>>same: The second "TDB.createDataset(..)" statement causes the application
>>to be blocked.
>>
>>This is my (runnable) test code:
>>
>>JunitTest:
>>
>>public static void main(String[] args) {
>> String dir
>>= "/Users/andreas_gruenwald/da-agr/resources/design/jena_tdb/itpm_jena";
>> Runnable tdb1 = new TDBThread(dir, null);
>> Runnable tdb2 = new TDBThread(dir, null);
>> tdb1.run();
>> tdb2.run();
>> System.out.println("Finished");
>>}
>>
>>
>>TDBThread.java:
>>
>>public class TDBThread implements Runnable {
>> private Dataset tdbDataSet;
>> private String directory;
>>
>>public TDBThread(String directory) {
>> tdbDataSet = null;
>> this.directory = directory;
>>}
>>
>>@Override
>>public void run() {
>> System.out.println(String.format("Run (directory=%s)...", directory));
>> *tdbDataSet** = TDBFactory.createDataset(directory); //blocks here*
>> System.out.println("Tdbdataset is now " + tdbDataSet);
>> tdbDataSet.begin(ReadWrite.WRITE); *//causes block*
>>}
>>}
>>
>>Any ideas what I (or Jena) is doing wrong?
>>
>>--
>>Andreas Grünwald
>>Tel.: +43 650 77 82340
>
>
>
>
Re: TDB and multiple Connections
Posted by Rob Vesse <rv...@dotnetrdf.org>.
I'm afraid it's you that's done something wrong on this occasion:
You are opening both datasets against the same underlying TDB dataset, as
part of this open you start a Write transaction which then prevents the
second thread from completing because TDB is single writer concurrency.
This is entirely by design, if you want several connections to different
datasets on the same thread this should work fine. However your example
is trying to make several connections to the same dataset so by definition
is going to block. I.e. you must pass in different directories for each
thread for your example to work.
Also you should be calling .close() on the datasets after you are done
with them otherwise you risk corrupting them. You will also want to call
either .commit() or .end()/.abort() on the dataset depending on whether
you want to commit/rollback your transaction. If you don't call something
explicitly before calling .close() it will assume an abort() call and any
changes will be list.
Rob
On 14/10/2013 19:47, "Andreas Grünwald" <a....@gmail.com> wrote:
>According to
>http://jena.apache.org/documentation/tdb/tdb_transactions.html#multi-threa
>ded-use
>
>only one DataSet connection can be used per Java thread. However, I need
>several connections to (different) data models within a single
>application.
>I noticed that if one tries to use several datasets per single-thread then
>the application is blocked when trying to create the second dataset. I
>tried to overcome the issue by creating a new thread per dataset (just as
>recommended in the Jena documentation). However, the problem is still the
>same: The second "TDB.createDataset(..)" statement causes the application
>to be blocked.
>
>This is my (runnable) test code:
>
>JunitTest:
>
>public static void main(String[] args) {
> String dir
>= "/Users/andreas_gruenwald/da-agr/resources/design/jena_tdb/itpm_jena";
> Runnable tdb1 = new TDBThread(dir, null);
> Runnable tdb2 = new TDBThread(dir, null);
> tdb1.run();
> tdb2.run();
> System.out.println("Finished");
>}
>
>
>TDBThread.java:
>
>public class TDBThread implements Runnable {
> private Dataset tdbDataSet;
> private String directory;
>
>public TDBThread(String directory) {
> tdbDataSet = null;
> this.directory = directory;
>}
>
>@Override
>public void run() {
> System.out.println(String.format("Run (directory=%s)...", directory));
> *tdbDataSet** = TDBFactory.createDataset(directory); //blocks here*
> System.out.println("Tdbdataset is now " + tdbDataSet);
> tdbDataSet.begin(ReadWrite.WRITE); *//causes block*
>}
>}
>
>Any ideas what I (or Jena) is doing wrong?
>
>--
>Andreas Grünwald
>Tel.: +43 650 77 82340