You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Laurent Pellegrino <la...@gmail.com> on 2011/10/29 16:06:39 UTC
Transactional TDB
Hi all,
I have several questions about how to use the new transaction features
provided with the last version of TDB:
* Do we have to use the commit operation with a read transaction? Is
it compulsory (e.g. to release a lock), optional, or an error?
* It seems that operations on DatasetGraphTxn do not throw checked
exceptions. However, do these operations can throw an unchecked
exception?
* Does nested read transactions are allowed?
* When a StoreConnection is created with Location.mem() as location,
what is created in memory (dataset+journal, only dataset or only
journal)?
* Does the information on the following webpages are up to date?
- http://openjena.org/wiki/TDB/Transactions
- https://cwiki.apache.org/confluence/display/JENA/TxTDB-design
Kind Regards,
Laurent
Re: Transactional TDB
Posted by Andy Seaborne <an...@apache.org>.
On 01/11/11 14:17, Laurent Pellegrino wrote:
>> Adding a non-caching StoreConnection.createMem() is also doable.
>>
>> So I understand the use case, why do you want separate in-memory databases?
>
> When I am running some tests (instantiating several datastores), I
> prefer to use an in-memory datastore instead of a persistent
> datastore. It will improve the execution time (e.g. for tests which
> are I/O intensive) without affecting the result and without making any
> difference when the purpose of the test is not to measure the
> throughput of something. Moreover, if the test fail/crash, it avoids
> to remove manually the repositories associated to the tests which have
> crashed.
>
> Laurent
>
I've added:
StoreConnection.createMemUncached()
which should return a fresh StoreConnection each time. Please test it
and let me know how it goes.
A proper fix - named in-memory areas - would be better but it would take
some time to do. Patches welcome and it'll need testing as the code has
been designed with this in mind :-)
Andy
Re: Transactional TDB
Posted by Laurent Pellegrino <la...@gmail.com>.
> Adding a non-caching StoreConnection.createMem() is also doable.
>
> So I understand the use case, why do you want separate in-memory databases?
When I am running some tests (instantiating several datastores), I
prefer to use an in-memory datastore instead of a persistent
datastore. It will improve the execution time (e.g. for tests which
are I/O intensive) without affecting the result and without making any
difference when the purpose of the test is not to measure the
throughput of something. Moreover, if the test fail/crash, it avoids
to remove manually the repositories associated to the tests which have
crashed.
Laurent
>> On Sun, Oct 30, 2011 at 6:05 PM, Andy Seaborne<an...@apache.org> wrote:
>>>
>>> On 29/10/11 15:06, Laurent Pellegrino wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I have several questions about how to use the new transaction features
>>>> provided with the last version of TDB:
>>>>
>>>> * Do we have to use the commit operation with a read transaction? Is
>>>> it compulsory (e.g. to release a lock), optional, or an error?
>>>
>>> If you are using transaction at all, then you must use a read tranaction
>>> (it
>>> can end with commit, abort or close - it'll make no difference for read).
>>>
>>> Locking is not needed except that each transaction must be single
>>> threaded
>>> (basically, multiple-reader OR single-writer applies within a transaction
>>> but only one thread per transaction is tested; no guarantees multiple
>>> threads per transaction will everr be supported ; they just "should" work
>>> at
>>> the moment).
>>>
>>>> * It seems that operations on DatasetGraphTxn do not throw checked
>>>> exceptions. However, do these operations can throw an unchecked
>>>> exception?
>>>
>>> Yes.
>>>
>>> But any exception is a sign of internal problems (or breaching
>>> concurrency).
>>>
>>> No normal operation should throw an exception.
>>>
>>>> * Does nested read transactions are allowed?
>>>
>>> No. No nested trasnactions at all. But if you are in a read transaction
>>> you can make as many reads as you want.
>>>
>>>> * When a StoreConnection is created with Location.mem() as location,
>>>> what is created in memory (dataset+journal, only dataset or only
>>>> journal)?
>>>
>>> Everything is (should) be in-memory.
>>>
>>> It's for testing only.
>>>
>>>> * Does the information on the following webpages are up to date?
>>>> - http://openjena.org/wiki/TDB/Transactions
>>>
>>> Should be.
>>>
>>>
>>>> - https://cwiki.apache.org/confluence/display/JENA/TxTDB-design
>>>
>>> May be not.
>>> That was the design and no design survives first contact with
>>> implementation
>>> reality
>>>
>>> Andy
>>>
>>>>
>>>> Kind Regards,
>>>> Laurent
>>>
>>>
>
>
Re: Transactional TDB
Posted by Andy Seaborne <an...@apache.org>.
On 31/10/11 16:25, Laurent Pellegrino wrote:
> Thanks Andy for these information.
>
> Regarding the Location.mem() method I have noticed that it does not
> create internally a new "mem location object" each time the method is
> called. Thus, if I create several instances of StoreConnection in a
> JVM by using Location.mem() as parameter, all the StoreConnections
> will share the same location. Is it not possible to create a unique
> memory location each a call to Location.mem() is performed? This would
> be very useful to reproduce what is possible with new
> Location("/unique/path/").
>
> Kind Regards,
> Laurent
Currently, StoreConnection.make manages a cache based on location so
that the application gets the same StoreConnection (like JDBC, e.g. in a
web app asking for a connection).
The constructor for StoreConnection is private to force everythign to go
via the cache.
in-memory is a bit different - it's resaonable to want several separate
ones, and ideally Location.mem would itself take a name to identify
different Locations areas backed by RAM.
Adding a non-caching StoreConnection.createMem() is also doable.
So I understand the use case, why do you want separate in-memory databases?
Andy
>
> On Sun, Oct 30, 2011 at 6:05 PM, Andy Seaborne<an...@apache.org> wrote:
>> On 29/10/11 15:06, Laurent Pellegrino wrote:
>>>
>>> Hi all,
>>>
>>> I have several questions about how to use the new transaction features
>>> provided with the last version of TDB:
>>>
>>> * Do we have to use the commit operation with a read transaction? Is
>>> it compulsory (e.g. to release a lock), optional, or an error?
>>
>> If you are using transaction at all, then you must use a read tranaction (it
>> can end with commit, abort or close - it'll make no difference for read).
>>
>> Locking is not needed except that each transaction must be single threaded
>> (basically, multiple-reader OR single-writer applies within a transaction
>> but only one thread per transaction is tested; no guarantees multiple
>> threads per transaction will everr be supported ; they just "should" work at
>> the moment).
>>
>>> * It seems that operations on DatasetGraphTxn do not throw checked
>>> exceptions. However, do these operations can throw an unchecked
>>> exception?
>>
>> Yes.
>>
>> But any exception is a sign of internal problems (or breaching concurrency).
>>
>> No normal operation should throw an exception.
>>
>>> * Does nested read transactions are allowed?
>>
>> No. No nested trasnactions at all. But if you are in a read transaction
>> you can make as many reads as you want.
>>
>>> * When a StoreConnection is created with Location.mem() as location,
>>> what is created in memory (dataset+journal, only dataset or only
>>> journal)?
>>
>> Everything is (should) be in-memory.
>>
>> It's for testing only.
>>
>>> * Does the information on the following webpages are up to date?
>>> - http://openjena.org/wiki/TDB/Transactions
>>
>> Should be.
>>
>>
>>> - https://cwiki.apache.org/confluence/display/JENA/TxTDB-design
>>
>> May be not.
>> That was the design and no design survives first contact with implementation
>> reality
>>
>> Andy
>>
>>>
>>> Kind Regards,
>>> Laurent
>>
>>
Re: Transactional TDB
Posted by Laurent Pellegrino <la...@gmail.com>.
Thanks Andy for these information.
Regarding the Location.mem() method I have noticed that it does not
create internally a new "mem location object" each time the method is
called. Thus, if I create several instances of StoreConnection in a
JVM by using Location.mem() as parameter, all the StoreConnections
will share the same location. Is it not possible to create a unique
memory location each a call to Location.mem() is performed? This would
be very useful to reproduce what is possible with new
Location("/unique/path/").
Kind Regards,
Laurent
On Sun, Oct 30, 2011 at 6:05 PM, Andy Seaborne <an...@apache.org> wrote:
> On 29/10/11 15:06, Laurent Pellegrino wrote:
>>
>> Hi all,
>>
>> I have several questions about how to use the new transaction features
>> provided with the last version of TDB:
>>
>> * Do we have to use the commit operation with a read transaction? Is
>> it compulsory (e.g. to release a lock), optional, or an error?
>
> If you are using transaction at all, then you must use a read tranaction (it
> can end with commit, abort or close - it'll make no difference for read).
>
> Locking is not needed except that each transaction must be single threaded
> (basically, multiple-reader OR single-writer applies within a transaction
> but only one thread per transaction is tested; no guarantees multiple
> threads per transaction will everr be supported ; they just "should" work at
> the moment).
>
>> * It seems that operations on DatasetGraphTxn do not throw checked
>> exceptions. However, do these operations can throw an unchecked
>> exception?
>
> Yes.
>
> But any exception is a sign of internal problems (or breaching concurrency).
>
> No normal operation should throw an exception.
>
>> * Does nested read transactions are allowed?
>
> No. No nested trasnactions at all. But if you are in a read transaction
> you can make as many reads as you want.
>
>> * When a StoreConnection is created with Location.mem() as location,
>> what is created in memory (dataset+journal, only dataset or only
>> journal)?
>
> Everything is (should) be in-memory.
>
> It's for testing only.
>
>> * Does the information on the following webpages are up to date?
>> - http://openjena.org/wiki/TDB/Transactions
>
> Should be.
>
>
>> - https://cwiki.apache.org/confluence/display/JENA/TxTDB-design
>
> May be not.
> That was the design and no design survives first contact with implementation
> reality
>
> Andy
>
>>
>> Kind Regards,
>> Laurent
>
>
Re: Transactional TDB
Posted by Andy Seaborne <an...@apache.org>.
On 29/10/11 15:06, Laurent Pellegrino wrote:
> Hi all,
>
> I have several questions about how to use the new transaction features
> provided with the last version of TDB:
>
> * Do we have to use the commit operation with a read transaction? Is
> it compulsory (e.g. to release a lock), optional, or an error?
If you are using transaction at all, then you must use a read tranaction
(it can end with commit, abort or close - it'll make no difference for
read).
Locking is not needed except that each transaction must be single
threaded (basically, multiple-reader OR single-writer applies within a
transaction but only one thread per transaction is tested; no guarantees
multiple threads per transaction will everr be supported ; they just
"should" work at the moment).
> * It seems that operations on DatasetGraphTxn do not throw checked
> exceptions. However, do these operations can throw an unchecked
> exception?
Yes.
But any exception is a sign of internal problems (or breaching concurrency).
No normal operation should throw an exception.
> * Does nested read transactions are allowed?
No. No nested trasnactions at all. But if you are in a read
transaction you can make as many reads as you want.
> * When a StoreConnection is created with Location.mem() as location,
> what is created in memory (dataset+journal, only dataset or only
> journal)?
Everything is (should) be in-memory.
It's for testing only.
> * Does the information on the following webpages are up to date?
> - http://openjena.org/wiki/TDB/Transactions
Should be.
> - https://cwiki.apache.org/confluence/display/JENA/TxTDB-design
May be not.
That was the design and no design survives first contact with
implementation reality
Andy
>
> Kind Regards,
> Laurent