You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by George News <ge...@gmx.net> on 2017/12/27 10:27:54 UTC
Txn code not handling type of transaction
Hi,
As you know from other threads I'm having some issues with transactions.
Your suggestion is to use Txn instead of begin/end. Just for curiosity I
have checked the Txn code at [1] and it seems that inside you use begin/end.
However I have a doubt concerning how you handle the begin/end for READ
and WRITE. It seems that you open a transaction based on
txn.isInTransaction(), but how do you know if it is a READ or WRITE?
If you create something like:
Txn.executeRead(dataset, {
Txn.executeWrite(dataset, {
// Whatever
}
}
}
the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
leading to unexepected behaviours for the txn.commit().
could you give some hints on how this is handle internally? Before fully
modify the code I have, it might be easier to replicate the txn
behaviour ;) but I would like to know the above (if possible).
As always, thanks in advanced
Jorge
[1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
Re: Txn code not handling type of transaction
Posted by Andy Seaborne <an...@apache.org>.
Promotion - turning a READ into WRITE is work-in-progress. Missing is
the general exposure, testing and controlling modes. (Two versions with
and without read-committed).
The mechanism actually already works for TIM, TDB and TDB2.
If you want to play with the mechanism: enable with global settings:
// TDB
DatasetGraphTransaction.promotion = true;
DatasetGraphTransaction.readCommittedPromotion = false;
// TDB2
TransactionCoordinator.promotion = true;
TransactionCoordinator.readCommittedPromotion = false;
// TIM
DatasetGraphInMemory.promotion = true;
DatasetGraphInMemory.readCommittedPromotion = false;
and "add" etc inside a READ will cause the transaction to turn into a
WRITE (or, with readCommittedPromotion=false may be throw an exception).
Andy
On 27/12/17 16:18, ajs6f wrote:
>> Then nesting is not safe as you might have open initially a read transaction and then include a write. If the parent one is a write there
>> shouldn't be such an issue (I guess).
>
> Actual nesting is not supported right now, period. Transactions in Jena are currently thread-local, so what you describe above I would see as a mistaken use of the API. If a client needs to move from a READ to a WRITE transaction, it's usually appropriate to close the transaction and open a new one (transactions in TIM, for instance, are snapshot isolated, and I believe the same is true of TDB2). Transactional objects _should not_ be passed from thread to thread with open transactions, unless additional client machinery is in place to manage those transactions, such as Dick has described in another thread (using thread proxies).
>
> Promotion machinery does exist in Jena but I am not aware that any of the dataset implementations actually support it right now. I could be wrong about that, since I didn't write the promotion code. Jena isn't a SQL database and doesn't offer the same kinds of guarantees or tradeoffs. Correctly promoting transactions in the absence of information about data dependencies is non-trivial.
>
> That having been said, it should be possible to add a "transaction type" method to transactional objects, within the "thread-local" design. Some dataset implementations already have one, e.g. DatasetGraphInMemory::transactionType. You might want to start by adding it to the o.a.j.sparql.core.Transactional interface and then "catching up" the implementation code to build up a PR.
>
> Adam Soroka
>
>> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>>
>> On 2017-12-27 12:29, Claude Warren wrote:
>>> I recently wrote some code to try to handle a similar situation. In my
>>> case I knew I needed a transaction to be active at various points so I
>>> created a TransactionHolder. I create the holder and passing the object
>>> that has implements Transactional as well as the type of ReadWrite I want.
>>>
>>> If the transaction is active it does nothing (and I hope the proper
>>> transaction has been started) otherwise It starts the transaction.
>>> Ad the end I call commit or abort as appropriate. If I did not start the
>>> transaction the commit, abort or end is ignored.
>>
>> I see the same problem in your code as I pointed out before. A
>> transaction behaves differently if it is READ or WRITE and in your code
>> there is not such a thing. Actually the isInTransaction() doesn't give
>> you this information.
>>
>> Then nesting is not safe as you might have open initially a read
>> transaction and then include a write. If the parent one is a write there
>> shouldn't be such an issue (I guess).
>>
>> Just for you to know, your code is more or less integrated in [1], so
>> you can "update" your code.
>>
>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>
>>
>>> I think there may be an issue with abort in that it should probablyset up
>>> end() to throw an exception when I have not created the transaction so
>>> that the outer transaction will fail.
>>>
>>> import org.apache.jena.query.ReadWrite;
>>> import org.apache.jena.sparql.core.Transactional;
>>>
>>> public class TransactionHolder {
>>> private final Transactional txn;
>>> private final boolean started;
>>> private final ReadWrite rw;
>>>
>>> public TransactionHolder( Transactional txn, ReadWrite rw )
>>> {
>>> this.txn = txn;
>>> this.rw = rw;
>>> started = ! txn.isInTransaction();
>>> if (started)
>>> {
>>> txn.begin( rw );
>>> }
>>> }
>>>
>>> public boolean ownsTranaction() {
>>> return started;
>>> }
>>>
>>> public void commit() {
>>> if (started) {
>>> txn.commit();
>>> }
>>> }
>>>
>>> public void abort() {
>>> if (started)
>>> {
>>> txn.abort();
>>> }
>>> }
>>>
>>> public void end() {
>>> if (started) {
>>> txn.end();
>>> }
>>> }
>>>
>>> }
>>>
>>>
>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
>>>
>>>> You cannot nest transactions nor can you promote a read to a write.
>>>> You need to rewrite your code or use txn which correctly checks if a
>>>> transaction is available and if not will begin the correct one, either READ
>>>> or WRITE.
>>>>
>>>>
>>>> Dick
>>>> -------- Original message --------From: George News <ge...@gmx.net>
>>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>> users@jena.apache.org> Subject: Txn code not handling type of transaction
>>>> Hi,
>>>>
>>>> As you know from other threads I'm having some issues with transactions.
>>>> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
>>>> have checked the Txn code at [1] and it seems that inside you use
>>>> begin/end.
>>>>
>>>> However I have a doubt concerning how you handle the begin/end for READ
>>>> and WRITE. It seems that you open a transaction based on
>>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>>
>>>> If you create something like:
>>>>
>>>> Txn.executeRead(dataset, {
>>>> Txn.executeWrite(dataset, {
>>>> // Whatever
>>>> }
>>>> }
>>>> }
>>>>
>>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
>>>> leading to unexepected behaviours for the txn.commit().
>>>>
>>>> could you give some hints on how this is handle internally? Before fully
>>>> modify the code I have, it might be easier to replicate the txn
>>>> behaviour ;) but I would like to know the above (if possible).
>>>>
>>>> As always, thanks in advanced
>>>> Jorge
>>>>
>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>
>>>
>>>
>>>
>
Re: Txn code not handling type of transaction
Posted by ajs6f <aj...@apache.org>.
org.apache.jena.graph.compose.MultiUnion does not somehow automatically handle transactions.
If you read from a Multiunion, you need to manage transaction boundaries around that read, in that thread. If you don't, you will be working non-transactionally against the dataset. That is never a good idea and for TDB1, you _cannot_ work transactionally and non-transactionally against the dataset at the same time.
Adam Soroka
> On Dec 28, 2017, at 8:17 AM, George News <ge...@gmx.net> wrote:
>
> On 2017-12-27 19:32, Andy Seaborne wrote:
>>
>>
>> On 27/12/17 18:19, George News wrote:
>>> One think I have forgotten to mention is that I'm using a static
>>> variable to store the dataset reference. Is this important?
>>
>> No.
>>
>> The top layer of transaction code uses ThreadLocal variables.
>>
>> Internally, there are Transaction objects.
>>
>> Andy
>
> Another doubt on the issue with transactions. I'm generating Multiunions
> from the same dataset graphs. While in one thread I'm reading from a
> multiunion, on another thread I'm writing to one of the graphs included
> in the multiunion (standalone graph, not multiunion).
>
> I have modified my code to use Txn everywhere and still having issues.
> This is why I'm asking about how multiunions are handle concerning
> transactions.
>
>
>>>
>>>
>>> On 2017-12-27 19:13, George News wrote:
>>>> Just out of curiosity, just in case the problem is that everything is
>>>> done in the same thread. Do you know if Wildfly is handling every
>>>> request under the same thread? I guess not, it will be really strange.
>>
>>>>
>>>> The point is that I have one REST endpoint for writing and another for
>>>> reading. Writing is done almost per 1 to 10 seconds. If I execute a
>>>> reading that takes longer than that, I get the exception on alloc-write.
>>>>
>>>> Thanks a lot. I just don't know why it is happening. Now I think the
>>>> code is using Txn (I have one point where I copied the Txn.java
>>>> behaviour) and still got the error.
>>>>
>>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>>> Then nesting is not safe as you might have open initially a read
>>>>>> transaction and then include a write. If the parent one is a write
>>>>>> there
>>>>>> shouldn't be such an issue (I guess).
>>>>>
>>>>> Actual nesting is not supported right now, period. Transactions in
>>>>> Jena are currently thread-local, so what you describe above I would
>>>>> see as a mistaken use of the API. If a client needs to move from a
>>>>> READ to a WRITE transaction, it's usually appropriate to close the
>>>>> transaction and open a new one (transactions in TIM, for instance,
>>>>> are snapshot isolated, and I believe the same is true of TDB2).
>>>>> Transactional objects _should not_ be passed from thread to thread
>>>>> with open transactions, unless additional client machinery is in
>>>>> place to manage those transactions, such as Dick has described in
>>>>> another thread (using thread proxies).
>>>>>
>>>>> Promotion machinery does exist in Jena but I am not aware that any
>>>>> of the dataset implementations actually support it right now. I
>>>>> could be wrong about that, since I didn't write the promotion code.
>>>>> Jena isn't a SQL database and doesn't offer the same kinds of
>>>>> guarantees or tradeoffs. Correctly promoting transactions in the
>>>>> absence of information about data dependencies is non-trivial.
>>>>>
>>>>> That having been said, it should be possible to add a "transaction
>>>>> type" method to transactional objects, within the "thread-local"
>>>>> design. Some dataset implementations already have one, e.g.
>>>>> DatasetGraphInMemory::transactionType. You might want to start by
>>>>> adding it to the o.a.j.sparql.core.Transactional interface and then
>>>>> "catching up" the implementation code to build up a PR.
>>>>>
>>>>> Adam Soroka
>>>>>
>>>>>> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>>>>>>
>>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>>> I recently wrote some code to try to handle a similar situation.
>>>>>>> In my
>>>>>>> case I knew I needed a transaction to be active at various points
>>>>>>> so I
>>>>>>> created a TransactionHolder. I create the holder and passing the
>>>>>>> object
>>>>>>> that has implements Transactional as well as the type of ReadWrite
>>>>>>> I want.
>>>>>>>
>>>>>>> If the transaction is active it does nothing (and I hope the proper
>>>>>>> transaction has been started) otherwise It starts the transaction.
>>>>>>> Ad the end I call commit or abort as appropriate. If I did not
>>>>>>> start the
>>>>>>> transaction the commit, abort or end is ignored.
>>>>>>
>>>>>> I see the same problem in your code as I pointed out before. A
>>>>>> transaction behaves differently if it is READ or WRITE and in your
>>>>>> code
>>>>>> there is not such a thing. Actually the isInTransaction() doesn't give
>>>>>> you this information.
>>>>>>
>>>>>> Then nesting is not safe as you might have open initially a read
>>>>>> transaction and then include a write. If the parent one is a write
>>>>>> there
>>>>>> shouldn't be such an issue (I guess).
>>>>>>
>>>>>> Just for you to know, your code is more or less integrated in [1], so
>>>>>> you can "update" your code.
>>>>>>
>>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>
>>>>>>
>>>>>>> I think there may be an issue with abort in that it should
>>>>>>> probablyset up
>>>>>>> end() to throw an exception when I have not created the
>>>>>>> transaction so
>>>>>>> that the outer transaction will fail.
>>>>>>>
>>>>>>> import org.apache.jena.query.ReadWrite;
>>>>>>> import org.apache.jena.sparql.core.Transactional;
>>>>>>>
>>>>>>> public class TransactionHolder {
>>>>>>> private final Transactional txn;
>>>>>>> private final boolean started;
>>>>>>> private final ReadWrite rw;
>>>>>>>
>>>>>>> public TransactionHolder( Transactional txn, ReadWrite rw )
>>>>>>> {
>>>>>>> this.txn = txn;
>>>>>>> this.rw = rw;
>>>>>>> started = ! txn.isInTransaction();
>>>>>>> if (started)
>>>>>>> {
>>>>>>> txn.begin( rw );
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> public boolean ownsTranaction() {
>>>>>>> return started;
>>>>>>> }
>>>>>>>
>>>>>>> public void commit() {
>>>>>>> if (started) {
>>>>>>> txn.commit();
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> public void abort() {
>>>>>>> if (started)
>>>>>>> {
>>>>>>> txn.abort();
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> public void end() {
>>>>>>> if (started) {
>>>>>>> txn.end();
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> You cannot nest transactions nor can you promote a read to a write.
>>>>>>>> You need to rewrite your code or use txn which correctly checks if a
>>>>>>>> transaction is available and if not will begin the correct one,
>>>>>>>> either READ
>>>>>>>> or WRITE.
>>>>>>>>
>>>>>>>>
>>>>>>>> Dick
>>>>>>>> -------- Original message --------From: George News
>>>>>>>> <ge...@gmx.net>
>>>>>>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>>>>>> users@jena.apache.org> Subject: Txn code not handling type of
>>>>>>>> transaction
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> As you know from other threads I'm having some issues with
>>>>>>>> transactions.
>>>>>>>> Your suggestion is to use Txn instead of begin/end. Just for
>>>>>>>> curiosity I
>>>>>>>> have checked the Txn code at [1] and it seems that inside you use
>>>>>>>> begin/end.
>>>>>>>>
>>>>>>>> However I have a doubt concerning how you handle the begin/end
>>>>>>>> for READ
>>>>>>>> and WRITE. It seems that you open a transaction based on
>>>>>>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>>>>>>
>>>>>>>> If you create something like:
>>>>>>>>
>>>>>>>> Txn.executeRead(dataset, {
>>>>>>>> Txn.executeWrite(dataset, {
>>>>>>>> // Whatever
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it
>>>>>>>> might be
>>>>>>>> leading to unexepected behaviours for the txn.commit().
>>>>>>>>
>>>>>>>> could you give some hints on how this is handle internally?
>>>>>>>> Before fully
>>>>>>>> modify the code I have, it might be easier to replicate the txn
>>>>>>>> behaviour ;) but I would like to know the above (if possible).
>>>>>>>>
>>>>>>>> As always, thanks in advanced
>>>>>>>> Jorge
>>>>>>>>
>>>>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-29 14:36, ajs6f wrote:
> See comments in-line.
>
> ajs6f
>
>> On Dec 29, 2017, at 8:27 AM, George News <ge...@gmx.net>
>> wrote:
>>
>> I'm also thinking on creating a queue to handle writings and
>> reading. This way I will parallelize readings and stop them to
>> lauch writing based on scheduler.
>
> TDB already does something like this, and you are not likely to
> better it by adding more code on top. You will be better off figuring
> out why your code isn't working and correcting it.
>
>>> If you expect to scale this application very far, you will
>>> probably want to solve your problem by introducing Fuseki and
>>> having your application make calls to it, instead of trying to
>>> manage concurrency yourself. You are already using SPARQL
>>> according to your example. Then you can use SPARQL keywords to
>>> pick out the named graphs together over which you want to run
>>> your query:
>>
>> this is not the way. I'm not parsing the SPARQL sentences. The
>> think is that graphs are internally created periodically in order
>> to speed up response times. We have noticed that one single graph
>> handling all our data makes the system quite slow and unresponsive.
>> But dividing it into smaller subgraphs makes the system quicker.
>> Then we merge graphs depending on user requests (REST endpoint and
>> not FROM GRAPH).
>
> That doesn't in any way prevent you from using Fuseki or make it less
> attractive. If you are merging named graphs from a dataset in
> response to user requests, you are manually doing something that ARQ
> (and therefore Fuseki) can already do. Parsing SPARQL and adding the
> appropriate clauses might be a lot less work than what you are trying
> to do now.
>
> In both cases, you seem to be rewriting functionality that already
> exists in Jena.
>
From what I have been reading, I could get Fuseki and convert my service
in a proxy to Fuseki. The issue here is that when I took the code the
already implemented stuff was not using Fuseki and my knowledge was low.
Maybe if I don't get to solve this issue I will think on migrating
everything.
Re: Txn code not handling type of transaction
Posted by ajs6f <aj...@apache.org>.
See comments in-line.
ajs6f
> On Dec 29, 2017, at 8:27 AM, George News <ge...@gmx.net> wrote:
>
> I'm also thinking on creating a queue to handle writings and reading. This way I will parallelize readings and stop them to lauch writing
> based on scheduler.
TDB already does something like this, and you are not likely to better it by adding more code on top. You will be better off figuring out why your code isn't working and correcting it.
>> If you expect to scale this application very far, you will probably want to solve your problem by introducing Fuseki and having your application make calls to it, instead of trying to manage concurrency yourself. You are already using SPARQL according to your example. Then you can use SPARQL keywords to pick out the named graphs together over which you want to run your query:
>
> this is not the way. I'm not parsing the SPARQL sentences. The think is that graphs are internally created periodically in order to speed up response times. We have noticed that one single graph handling all our data makes the system quite slow and unresponsive. But dividing it into smaller subgraphs makes the system quicker. Then we merge graphs depending on user requests (REST endpoint and not FROM GRAPH).
That doesn't in any way prevent you from using Fuseki or make it less attractive. If you are merging named graphs from a dataset in response to user requests, you are manually doing something that ARQ (and therefore Fuseki) can already do. Parsing SPARQL and adding the appropriate clauses might be a lot less work than what you are trying to do now.
In both cases, you seem to be rewriting functionality that already exists in Jena.
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-29 12:59, ajs6f wrote:
> It's going to be a lot harder to solve this issue if other people
> cannot replicate it. You might want to write an integration test
> using something like Awaitility.
I will have a look at it. It is not that I don't want to share the code,
it is currently I can due to internal policies.
> One change you could make that probably won't solve your problem but
> will make your code easier to understand would be to stop using
> Graphs. That's part of the Jena SPI and while those types are stable,
> the intention is for you to use the API (Dataset and Model) and
> there's no reason here not to.
>
> Model union =
> dataset.getNamedModel("G1").union(dataset.getNamedModel("G2")).union(dataset.getNamedModel("G4"));
>
> etc.
I will try that to check if it works better. Keep you posted on it.
I'm also thinking on creating a queue to handle writings and reading.
This way I will parallelize readings and stop them to lauch writing
based on scheduler.
> If you expect to scale this application very far, you will probably
> want to solve your problem by introducing Fuseki and having your
> application make calls to it, instead of trying to manage concurrency
> yourself. You are already using SPARQL according to your example.
> Then you can use SPARQL keywords to pick out the named graphs
> together over which you want to run your query:
>
> https://jena.apache.org/tutorials/sparql_datasets.html
this is not the way. I'm not parsing the SPARQL sentences. The think is
that graphs are internally created periodically in order to speed up
response times. We have noticed that one single graph handling all our
data makes the system quite slow and unresponsive. But dividing it into
smaller subgraphs makes the system quicker. Then we merge graphs
depending on user requests (REST endpoint and not FROM GRAPH).
> Otherwise, your code shows you using qExec.execSelect(), but your
> stacktrace shows the exception arising after
> QueryExecutionBase.execAsk, so are you actually executing different
> code than what you are showing us?
Sorry for that. I launch a ASK in order to check if there are values in
the ResultSet or not. This was something I asked in another thread
(Problem with MAX when no result expected).
But anyway, suppose that ASK is a SELECT as the behaviour is similar if
I only enable select.
> ajs6f
>
>> On Dec 29, 2017, at 4:51 AM, George News <ge...@gmx.net>
>> wrote:
>>
>> On 2017-12-28 18:02, dandh988 wrote:
>>> Can you give a complete example? How are you calling MultiUnion?
>>> Are you calling txn read on the dataset then building the union,
>>> then calling txn write to update the graph?
>>
>> Let's describe a use case.
>>
>> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
>> 2.a) Someone initiates a sparql request and the internal procedure
>> followed is: return Txn.calculateRead(dataset, () -> { // Create
>> multiunion of 3 namegraphs MultiUnion union = new MultiUnion();
>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>> union.addGraph(dataset.getNamedModel("G4").getGraph());
>>
>> Model m = ModelFactory.createModelForGraph(union);
>>
>> // Launch Sparql query on it try (QueryExecution qExec =
>> QueryExecutionFactory.create(query, m)) { return
>> ResultSetFactory.copyResults(qExec.execSelect()) } });
>>
>> 2.b) Someone initiates a sparql request and the internal procedure
>> followed is: return Txn.calculateRead(dataset, () -> { // Create
>> multiunion of 2 namegraphs // This code in in a function MultiUnion
>> union = new MultiUnion();
>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>> union.addGraph(dataset.getNamedModel("G2").getGraph()); Model m1 =
>> ModelFactory.createModelForGraph(union);
>>
>> // Retrieve namegraph G3 // This code in in a function Model m2 =
>> dataset.getNamedModel("G3");
>>
>> Model m = ModelFactory.createUnion(m2, m1);
>>
>> // Launch Sparql query on it try (QueryExecution qExec =
>> QueryExecutionFactory.create(query, m)) { return
>> ResultSetFactory.copyResults(qExec.execSelect()) } });
>>
>>
>> 3) Someone initiaties a write in G2 // m stores the new entity
>> model Model m; Txn.executeWrite(dataset, () -> {
>> dataset.getNamedModel("G2").add(m); });
>>
>>
>>
>> If either version of 2), and 3) are not done in parallel there is
>> no problem and everything is executed correctly.
>>
>> The problem arise when 2.a) or 2.b) is run, and before ending
>> someone tries to perform 3). Then I get the FileException("In the
>> middle of an alloc-write").
>>
>> Do you have an idea on how to avoid this? How can I handle
>> transactions in this model?
>>
>>
>> I was thinking on creating a global mutex so if any action is being
>> performed over the dataset, then the rest would be blocked. The
>> problem here is that the code is part of a webservice and then if
>> the read/write operation lasts long, I will get a timeout that will
>> close the connection.
>>
>> The other option is to disable writing while someone is reading.
>> The main problem here is how to properly reschedule writings not to
>> have a big queue.
>>
>> Any help is more than welcome. I don't know what else to do to
>> solve this is issue, and the problem is making the service unusable
>> :(
>>
>> Thanks a lot for the great help you are all offering. Jorge
>>
>>>
>>>
>>> Dick -------- Original message --------From: George News
>>> <ge...@gmx.net> Date: 28/12/2017 13:17 (GMT+00:00) To:
>>> users@jena.apache.org Subject: Re: Txn code not handling type of
>>> transaction On 2017-12-27 19:32, Andy Seaborne wrote:
>>>>
>>>>
>>>> On 27/12/17 18:19, George News wrote:
>>>>> One think I have forgotten to mention is that I'm using a
>>>>> static variable to store the dataset reference. Is this
>>>>> important?
>>>>
>>>> No.
>>>>
>>>> The top layer of transaction code uses ThreadLocal variables.
>>>>
>>>> Internally, there are Transaction objects.
>>>>
>>>> Andy
>>>
>>> Another doubt on the issue with transactions. I'm generating
>>> Multiunions from the same dataset graphs. While in one thread
>>> I'm reading from a multiunion, on another thread I'm writing to
>>> one of the graphs included in the multiunion (standalone graph,
>>> not multiunion).
>>>
>>> I have modified my code to use Txn everywhere and still having
>>> issues. This is why I'm asking about how multiunions are handle
>>> concerning transactions.
>>>
>>>
>>>>>
>>>>>
>>>>> On 2017-12-27 19:13, George News wrote:
>>>>>> Just out of curiosity, just in case the problem is that
>>>>>> everything is done in the same thread. Do you know if
>>>>>> Wildfly is handling every request under the same thread? I
>>>>>> guess not, it will be really strange.
>>>>
>>>>>>
>>>>>> The point is that I have one REST endpoint for writing and
>>>>>> another for reading. Writing is done almost per 1 to 10
>>>>>> seconds. If I execute a reading that takes longer than
>>>>>> that, I get the exception on alloc-write.
>>>>>>
>>>>>> Thanks a lot. I just don't know why it is happening. Now I
>>>>>> think the code is using Txn (I have one point where I
>>>>>> copied the Txn.java behaviour) and still got the error.
>>>>>>
>>>>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>>>>> Then nesting is not safe as you might have open
>>>>>>>> initially a read transaction and then include a write.
>>>>>>>> If the parent one is a write there shouldn't be such an
>>>>>>>> issue (I guess).
>>>>>>>
>>>>>>> Actual nesting is not supported right now, period.
>>>>>>> Transactions in Jena are currently thread-local, so what
>>>>>>> you describe above I would see as a mistaken use of the
>>>>>>> API. If a client needs to move from a READ to a WRITE
>>>>>>> transaction, it's usually appropriate to close the
>>>>>>> transaction and open a new one (transactions in TIM, for
>>>>>>> instance, are snapshot isolated, and I believe the same
>>>>>>> is true of TDB2). Transactional objects _should not_ be
>>>>>>> passed from thread to thread with open transactions,
>>>>>>> unless additional client machinery is in place to manage
>>>>>>> those transactions, such as Dick has described in another
>>>>>>> thread (using thread proxies).
>>>>>>>
>>>>>>> Promotion machinery does exist in Jena but I am not
>>>>>>> aware that any of the dataset implementations actually
>>>>>>> support it right now. I could be wrong about that, since
>>>>>>> I didn't write the promotion code. Jena isn't a SQL
>>>>>>> database and doesn't offer the same kinds of guarantees
>>>>>>> or tradeoffs. Correctly promoting transactions in the
>>>>>>> absence of information about data dependencies is
>>>>>>> non-trivial.
>>>>>>>
>>>>>>> That having been said, it should be possible to add a
>>>>>>> "transaction type" method to transactional objects,
>>>>>>> within the "thread-local" design. Some dataset
>>>>>>> implementations already have one, e.g.
>>>>>>> DatasetGraphInMemory::transactionType. You might want to
>>>>>>> start by adding it to the o.a.j.sparql.core.Transactional
>>>>>>> interface and then "catching up" the implementation code
>>>>>>> to build up a PR.
>>>>>>>
>>>>>>> Adam Soroka
>>>>>>>
>>>>>>>> On Dec 27, 2017, at 6:53 AM, George News
>>>>>>>> <ge...@gmx.net> wrote:
>>>>>>>>
>>>>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>>>>> I recently wrote some code to try to handle a
>>>>>>>>> similar situation. In my case I knew I needed a
>>>>>>>>> transaction to be active at various points so I
>>>>>>>>> created a TransactionHolder. I create the holder and
>>>>>>>>> passing the object that has implements Transactional
>>>>>>>>> as well as the type of ReadWrite I want.
>>>>>>>>>
>>>>>>>>> If the transaction is active it does nothing (and I
>>>>>>>>> hope the proper transaction has been started)
>>>>>>>>> otherwise It starts the transaction. Ad the end I
>>>>>>>>> call commit or abort as appropriate. If I did not
>>>>>>>>> start the transaction the commit, abort or end is
>>>>>>>>> ignored.
>>>>>>>>
>>>>>>>> I see the same problem in your code as I pointed out
>>>>>>>> before. A transaction behaves differently if it is READ
>>>>>>>> or WRITE and in your code there is not such a thing.
>>>>>>>> Actually the isInTransaction() doesn't give you this
>>>>>>>> information.
>>>>>>>>
>>>>>>>> Then nesting is not safe as you might have open
>>>>>>>> initially a read transaction and then include a write.
>>>>>>>> If the parent one is a write there shouldn't be such an
>>>>>>>> issue (I guess).
>>>>>>>>
>>>>>>>> Just for you to know, your code is more or less
>>>>>>>> integrated in [1], so you can "update" your code.
>>>>>>>>
>>>>>>>> [1]:
>>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>
>>>>>>>>
I think there may be an issue with abort in that it should
>>>>>>>>> probablyset up end() to throw an exception when I
>>>>>>>>> have not created the transaction so that the outer
>>>>>>>>> transaction will fail.
>>>>>>>>>
>>>>>>>>> import org.apache.jena.query.ReadWrite; import
>>>>>>>>> org.apache.jena.sparql.core.Transactional;
>>>>>>>>>
>>>>>>>>> public class TransactionHolder { private final
>>>>>>>>> Transactional txn; private final boolean started;
>>>>>>>>> private final ReadWrite rw;
>>>>>>>>>
>>>>>>>>> public TransactionHolder( Transactional txn,
>>>>>>>>> ReadWrite rw ) { this.txn = txn; this.rw = rw;
>>>>>>>>> started = ! txn.isInTransaction(); if (started) {
>>>>>>>>> txn.begin( rw ); } }
>>>>>>>>>
>>>>>>>>> public boolean ownsTranaction() { return started; }
>>>>>>>>>
>>>>>>>>> public void commit() { if (started) { txn.commit(); }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> public void abort() { if (started) { txn.abort(); }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> public void end() { if (started) { txn.end(); } }
>>>>>>>>>
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988
>>>>>>>>> <da...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> You cannot nest transactions nor can you promote a
>>>>>>>>>> read to a write. You need to rewrite your code or
>>>>>>>>>> use txn which correctly checks if a transaction is
>>>>>>>>>> available and if not will begin the correct one,
>>>>>>>>>> either READ or WRITE.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dick -------- Original message --------From:
>>>>>>>>>> George News <ge...@gmx.net> Date: 27/12/2017
>>>>>>>>>> 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>>>>>>>> users@jena.apache.org> Subject: Txn code not
>>>>>>>>>> handling type of transaction Hi,
>>>>>>>>>>
>>>>>>>>>> As you know from other threads I'm having some
>>>>>>>>>> issues with transactions. Your suggestion is to use
>>>>>>>>>> Txn instead of begin/end. Just for curiosity I have
>>>>>>>>>> checked the Txn code at [1] and it seems that
>>>>>>>>>> inside you use begin/end.
>>>>>>>>>>
>>>>>>>>>> However I have a doubt concerning how you handle
>>>>>>>>>> the begin/end for READ and WRITE. It seems that you
>>>>>>>>>> open a transaction based on txn.isInTransaction(),
>>>>>>>>>> but how do you know if it is a READ or WRITE?
>>>>>>>>>>
>>>>>>>>>> If you create something like:
>>>>>>>>>>
>>>>>>>>>> Txn.executeRead(dataset, {
>>>>>>>>>> Txn.executeWrite(dataset, { // Whatever } } }
>>>>>>>>>>
>>>>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and
>>>>>>>>>> therefore it might be leading to unexepected
>>>>>>>>>> behaviours for the txn.commit().
>>>>>>>>>>
>>>>>>>>>> could you give some hints on how this is handle
>>>>>>>>>> internally? Before fully modify the code I have,
>>>>>>>>>> it might be easier to replicate the txn behaviour
>>>>>>>>>> ;) but I would like to know the above (if
>>>>>>>>>> possible).
>>>>>>>>>>
>>>>>>>>>> As always, thanks in advanced Jorge
>>>>>>>>>>
>>>>>>>>>> [1]:
>>>>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>
>>>>>>>>>>
>
Re: Txn code not handling type of transaction
Posted by ajs6f <aj...@apache.org>.
It's going to be a lot harder to solve this issue if other people cannot replicate it. You might want to write an integration test using something like Awaitility.
One change you could make that probably won't solve your problem but will make your code easier to understand would be to stop using Graphs. That's part of the Jena SPI and while those types are stable, the intention is for you to use the API (Dataset and Model) and there's no reason here not to.
Model union = dataset.getNamedModel("G1").union(dataset.getNamedModel("G2")).union(dataset.getNamedModel("G4"));
etc.
If you expect to scale this application very far, you will probably want to solve your problem by introducing Fuseki and having your application make calls to it, instead of trying to manage concurrency yourself. You are already using SPARQL according to your example. Then you can use SPARQL keywords to pick out the named graphs together over which you want to run your query:
https://jena.apache.org/tutorials/sparql_datasets.html
Otherwise, your code shows you using qExec.execSelect(), but your stacktrace shows the exception arising after QueryExecutionBase.execAsk, so are you actually executing different code than what you are showing us?
ajs6f
> On Dec 29, 2017, at 4:51 AM, George News <ge...@gmx.net> wrote:
>
> On 2017-12-28 18:02, dandh988 wrote:
>> Can you give a complete example? How are you calling MultiUnion? Are
>> you calling txn read on the dataset then building the union, then
>> calling txn write to update the graph?
>
> Let's describe a use case.
>
> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
> 2.a) Someone initiates a sparql request and the internal procedure followed is:
> return Txn.calculateRead(dataset, () -> {
> // Create multiunion of 3 namegraphs
> MultiUnion union = new MultiUnion();
> union.addGraph(dataset.getNamedModel("G1").getGraph());
> union.addGraph(dataset.getNamedModel("G2").getGraph());
> union.addGraph(dataset.getNamedModel("G4").getGraph());
>
> Model m = ModelFactory.createModelForGraph(union);
>
> // Launch Sparql query on it
> try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
> return ResultSetFactory.copyResults(qExec.execSelect())
> }
> });
>
> 2.b) Someone initiates a sparql request and the internal procedure followed is:
> return Txn.calculateRead(dataset, () -> {
> // Create multiunion of 2 namegraphs
> // This code in in a function
> MultiUnion union = new MultiUnion();
> union.addGraph(dataset.getNamedModel("G1").getGraph());
> union.addGraph(dataset.getNamedModel("G2").getGraph());
> Model m1 = ModelFactory.createModelForGraph(union);
>
> // Retrieve namegraph G3
> // This code in in a function
> Model m2 = dataset.getNamedModel("G3");
>
> Model m = ModelFactory.createUnion(m2, m1);
>
> // Launch Sparql query on it
> try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
> return ResultSetFactory.copyResults(qExec.execSelect())
> }
> });
>
>
> 3) Someone initiaties a write in G2
> // m stores the new entity model
> Model m;
> Txn.executeWrite(dataset, () -> {
> dataset.getNamedModel("G2").add(m);
> });
>
>
>
> If either version of 2), and 3) are not done in parallel there is no problem and everything is executed correctly.
>
> The problem arise when 2.a) or 2.b) is run, and before ending someone tries to perform 3). Then I get the FileException("In the middle of an alloc-write").
>
> Do you have an idea on how to avoid this? How can I handle transactions in this model?
>
>
> I was thinking on creating a global mutex so if any action is being performed over the dataset, then the rest would be blocked. The problem here is that the code is part of a webservice and then if the read/write operation lasts long, I will get a timeout that will close the connection.
>
> The other option is to disable writing while someone is reading. The main problem here is how to properly reschedule writings not to have a big queue.
>
> Any help is more than welcome. I don't know what else to do to solve this is issue, and the problem is making the service unusable :(
>
> Thanks a lot for the great help you are all offering.
> Jorge
>
>>
>>
>> Dick -------- Original message --------From: George News
>> <ge...@gmx.net> Date: 28/12/2017 13:17 (GMT+00:00) To:
>> users@jena.apache.org Subject: Re: Txn code not handling type of
>> transaction On 2017-12-27 19:32, Andy Seaborne wrote:
>>>
>>>
>>> On 27/12/17 18:19, George News wrote:
>>>> One think I have forgotten to mention is that I'm using a static
>>>> variable to store the dataset reference. Is this important?
>>>
>>> No.
>>>
>>> The top layer of transaction code uses ThreadLocal variables.
>>>
>>> Internally, there are Transaction objects.
>>>
>>> Andy
>>
>> Another doubt on the issue with transactions. I'm generating
>> Multiunions from the same dataset graphs. While in one thread I'm
>> reading from a multiunion, on another thread I'm writing to one of
>> the graphs included in the multiunion (standalone graph, not
>> multiunion).
>>
>> I have modified my code to use Txn everywhere and still having
>> issues. This is why I'm asking about how multiunions are handle
>> concerning transactions.
>>
>>
>>>>
>>>>
>>>> On 2017-12-27 19:13, George News wrote:
>>>>> Just out of curiosity, just in case the problem is that
>>>>> everything is done in the same thread. Do you know if Wildfly
>>>>> is handling every request under the same thread? I guess not,
>>>>> it will be really strange.
>>>
>>>>>
>>>>> The point is that I have one REST endpoint for writing and
>>>>> another for reading. Writing is done almost per 1 to 10
>>>>> seconds. If I execute a reading that takes longer than that, I
>>>>> get the exception on alloc-write.
>>>>>
>>>>> Thanks a lot. I just don't know why it is happening. Now I
>>>>> think the code is using Txn (I have one point where I copied
>>>>> the Txn.java behaviour) and still got the error.
>>>>>
>>>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>>>> Then nesting is not safe as you might have open initially a
>>>>>>> read transaction and then include a write. If the parent
>>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>>
>>>>>> Actual nesting is not supported right now, period.
>>>>>> Transactions in Jena are currently thread-local, so what you
>>>>>> describe above I would see as a mistaken use of the API. If a
>>>>>> client needs to move from a READ to a WRITE transaction, it's
>>>>>> usually appropriate to close the transaction and open a new
>>>>>> one (transactions in TIM, for instance, are snapshot
>>>>>> isolated, and I believe the same is true of TDB2).
>>>>>> Transactional objects _should not_ be passed from thread to
>>>>>> thread with open transactions, unless additional client
>>>>>> machinery is in place to manage those transactions, such as
>>>>>> Dick has described in another thread (using thread proxies).
>>>>>>
>>>>>> Promotion machinery does exist in Jena but I am not aware
>>>>>> that any of the dataset implementations actually support it
>>>>>> right now. I could be wrong about that, since I didn't write
>>>>>> the promotion code. Jena isn't a SQL database and doesn't
>>>>>> offer the same kinds of guarantees or tradeoffs. Correctly
>>>>>> promoting transactions in the absence of information about
>>>>>> data dependencies is non-trivial.
>>>>>>
>>>>>> That having been said, it should be possible to add a
>>>>>> "transaction type" method to transactional objects, within
>>>>>> the "thread-local" design. Some dataset implementations
>>>>>> already have one, e.g. DatasetGraphInMemory::transactionType.
>>>>>> You might want to start by adding it to the
>>>>>> o.a.j.sparql.core.Transactional interface and then "catching
>>>>>> up" the implementation code to build up a PR.
>>>>>>
>>>>>> Adam Soroka
>>>>>>
>>>>>>> On Dec 27, 2017, at 6:53 AM, George News
>>>>>>> <ge...@gmx.net> wrote:
>>>>>>>
>>>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>>>> I recently wrote some code to try to handle a similar
>>>>>>>> situation. In my case I knew I needed a transaction to be
>>>>>>>> active at various points so I created a
>>>>>>>> TransactionHolder. I create the holder and passing the
>>>>>>>> object that has implements Transactional as well as the
>>>>>>>> type of ReadWrite I want.
>>>>>>>>
>>>>>>>> If the transaction is active it does nothing (and I hope
>>>>>>>> the proper transaction has been started) otherwise It
>>>>>>>> starts the transaction. Ad the end I call commit or abort
>>>>>>>> as appropriate. If I did not start the transaction the
>>>>>>>> commit, abort or end is ignored.
>>>>>>>
>>>>>>> I see the same problem in your code as I pointed out
>>>>>>> before. A transaction behaves differently if it is READ or
>>>>>>> WRITE and in your code there is not such a thing. Actually
>>>>>>> the isInTransaction() doesn't give you this information.
>>>>>>>
>>>>>>> Then nesting is not safe as you might have open initially a
>>>>>>> read transaction and then include a write. If the parent
>>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>>>
>>>>>>> Just for you to know, your code is more or less integrated
>>>>>>> in [1], so you can "update" your code.
>>>>>>>
>>>>>>> [1]:
>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>
> I think there may be an issue with abort in that it should
>>>>>>>> probablyset up end() to throw an exception when I have
>>>>>>>> not created the transaction so that the outer
>>>>>>>> transaction will fail.
>>>>>>>>
>>>>>>>> import org.apache.jena.query.ReadWrite; import
>>>>>>>> org.apache.jena.sparql.core.Transactional;
>>>>>>>>
>>>>>>>> public class TransactionHolder { private final
>>>>>>>> Transactional txn; private final boolean started; private
>>>>>>>> final ReadWrite rw;
>>>>>>>>
>>>>>>>> public TransactionHolder( Transactional txn, ReadWrite rw
>>>>>>>> ) { this.txn = txn; this.rw = rw; started = !
>>>>>>>> txn.isInTransaction(); if (started) { txn.begin( rw ); }
>>>>>>>> }
>>>>>>>>
>>>>>>>> public boolean ownsTranaction() { return started; }
>>>>>>>>
>>>>>>>> public void commit() { if (started) { txn.commit(); } }
>>>>>>>>
>>>>>>>> public void abort() { if (started) { txn.abort(); } }
>>>>>>>>
>>>>>>>> public void end() { if (started) { txn.end(); } }
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988
>>>>>>>> <da...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> You cannot nest transactions nor can you promote a read
>>>>>>>>> to a write. You need to rewrite your code or use txn
>>>>>>>>> which correctly checks if a transaction is available
>>>>>>>>> and if not will begin the correct one, either READ or
>>>>>>>>> WRITE.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dick -------- Original message --------From: George
>>>>>>>>> News <ge...@gmx.net> Date: 27/12/2017 10:27
>>>>>>>>> (GMT+00:00) To: Jena User Mailing List <
>>>>>>>>> users@jena.apache.org> Subject: Txn code not handling
>>>>>>>>> type of transaction Hi,
>>>>>>>>>
>>>>>>>>> As you know from other threads I'm having some issues
>>>>>>>>> with transactions. Your suggestion is to use Txn
>>>>>>>>> instead of begin/end. Just for curiosity I have checked
>>>>>>>>> the Txn code at [1] and it seems that inside you use
>>>>>>>>> begin/end.
>>>>>>>>>
>>>>>>>>> However I have a doubt concerning how you handle the
>>>>>>>>> begin/end for READ and WRITE. It seems that you open a
>>>>>>>>> transaction based on txn.isInTransaction(), but how do
>>>>>>>>> you know if it is a READ or WRITE?
>>>>>>>>>
>>>>>>>>> If you create something like:
>>>>>>>>>
>>>>>>>>> Txn.executeRead(dataset, { Txn.executeWrite(dataset, {
>>>>>>>>> // Whatever } } }
>>>>>>>>>
>>>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and
>>>>>>>>> therefore it might be leading to unexepected behaviours
>>>>>>>>> for the txn.commit().
>>>>>>>>>
>>>>>>>>> could you give some hints on how this is handle
>>>>>>>>> internally? Before fully modify the code I have, it
>>>>>>>>> might be easier to replicate the txn behaviour ;) but I
>>>>>>>>> would like to know the above (if possible).
>>>>>>>>>
>>>>>>>>> As always, thanks in advanced Jorge
>>>>>>>>>
>>>>>>>>> [1]:
>>>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
Hi all,
thanks for your tips. I think I know understand what I need to do.
Although all the graphs comes from the same dataset, it is not the same
the merging of them as using the dataset itself.
Therefore, the lock to the dataset seems not to apply to a bunch of
merged graphs from the same dataset. I guess I have to start using the
FROM GRAPH statement in my SPARQL sentences. But, how is it done
internally? If I create a transaction on the dataset, and there are
several FROM GRAPH then internally you have to merge and execute the
query over the list of graphs.
Besides that, after exploring a bit Fuseki, I might be changing the
backend to Fuseki and modify my service to act as a proxy to it. Anyway
I have to go deeper in Fuseki to see if I can easily apply the updates,
etc on the dataset. In this sense:
- For SPARQLs, do you recomend creating a HTTP client and send the
request to the appropriate endpoint? Or is it better to use the
QueryExecutor based on service endpoint? Which one offers better control
over HTTP error?
- For Fuseki GSP endpoint and uploading data to a named graph I have to
use the "graph" query parameter as from
https://www.w3.org/TR/sparql11-http-rdf-update/, do I?
Thanks a lot for you help. It seems there is a lot to do for 2018.
Wish you all the best for the new year.
Regards,
Jorge
On 2017-12-29 18:29, Andy Seaborne wrote:
> I can't see that variable "dataset" is a TDB dataset, not a bunch of
> graphs pulled together at the start of the request. If it's a composite
> wrapper containing TDB graphs, and if it is different on request, that
> would be consistent with what has been presented. Not the only
> possibility though.
>
> Andy
>
> On 29/12/17 14:28, ajs6f wrote:
>> Just to add a bit of context to Andy's point:
>>
>>> It needs a dataset so it makes one and puts the model in as the
>>> default graph.
>>
>> This is (more or less) because SPARQL is defined as acting against
>> datasets, not individual graphs.
>>
>> https://www.w3.org/TR/sparql11-query/#rdfDataset:
>>
>> "A SPARQL query is executed against an RDF Dataset which represents a
>> collection of graphs."
>>
>> And if (as Andy pointed out is possible) the individual graph in
>> question is a view over several datasets, that doesn't work either.
>> There has to be a single dataset that is the basic context for SPARQL.
>> (Even while using federation/SERVICE, there is still a single dataset
>> against which the query is being executed.)
>>
>> ajs6f
>>
>>> On Dec 29, 2017, at 9:21 AM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>>
>>>
>>> On 29/12/17 13:58, George News wrote:
>>>> On 2017-12-29 14:46, Andy Seaborne wrote:
>>>>>
>>>>>
>>>>> On 29/12/17 09:51, George News wrote:
>>>>>> On 2017-12-28 18:02, dandh988 wrote:
>>>>>>> Can you give a complete example? How are you calling MultiUnion? Are
>>>>>>> you calling txn read on the dataset then building the union, then
>>>>>>> calling txn write to update the graph?
>>>>>>
>>>>>> Let's describe a use case.
>>>>>
>>>>> The stack trace does not agree with the details here (and details
>>>>> matter!) especially what dataset is because if that is a general
>>>>> dataset
>>>>> built out of TDB graphs, this whole thing will not work. A complete,
>>>>> minimal example is needed.
>>>>>
>>>>> A few general observations:
>>>>>
>>>>>>
>>>>>> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
>>>>>> 2.a) Someone initiates a sparql request and the internal procedure
>>>>>> followed is:
>>>>>> return Txn.calculateRead(dataset, () -> {
>>>>>> // Create multiunion of 3 namegraphs
>>>>>> MultiUnion union = new MultiUnion();
>>>>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>>>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>>>>> union.addGraph(dataset.getNamedModel("G4").getGraph());
>>>>>
>>>>> This can be done in SPARQL:
>>>>>
>>>>> either UNION or
>>>>>
>>>>> SELECT
>>>>> FROM <G1>
>>>>> FROM <G2>
>>>>> FROM <G4>
>>>>> WHERE { ... }
>>>>>
>>>>> (copy the query object and modify it).
>>>>>
>>>>> and TDB will handle it better.
>>>>>
>>>>>>
>>>>>> Model m = ModelFactory.createModelForGraph(union);
>>>>>
>>>>> You are not querying the TDB dataset if you pass a model to
>>>>> QueryExecutionFactory.create. It has to create a dataset (a general
>>>>> purpose one) for the query.
>>>> But if the model is created from graphs/models on a dataset, isn't it
>>>> the same? At the end the models are a reference to ones in the dataset.
>>>
>>> Not the same and the model is not a reference into a dataset.
>>>
>>> QueryExecutionFactory.create is given a model, here a composite one,
>>> could be a model without a datasets, could be a model over many
>>> datasets. It can't know what the structure is, model is an interface.
>>>
>>> It needs a dataset so it makes one and puts the model in as the
>>> default graph.
>>>
>>>>>> // Launch Sparql query on it
>>>>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>>>>> m)) {
>>>>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>>>>> }
>>>>>> });
>>>>>>
>>>>>> 2.b) Someone initiates a sparql request and the internal procedure
>>>>>> followed is:
>>>>>> return Txn.calculateRead(dataset, () -> {
>>>>>> // Create multiunion of 2 namegraphs
>>>>>> // This code in in a function
>>>>>> MultiUnion union = new MultiUnion();
>>>>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>>>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>>>>> Model m1 = ModelFactory.createModelForGraph(union);
>>>>>>
>>>>>> // Retrieve namegraph G3
>>>>>> // This code in in a function
>>>>>> Model m2 = dataset.getNamedModel("G3");
>>>>>>
>>>>>> Model m = ModelFactory.createUnion(m2, m1);
>>>>>>
>>>>>> // Launch Sparql query on it
>>>>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>>>>> m)) {
>>>>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>>>>> }
>>>>>> });
>>>>>>
>>>>>>
>>>>>> 3) Someone initiaties a write in G2
>>>>>> // m stores the new entity model
>>>>>> Model m;
>>>>>> Txn.executeWrite(dataset, () -> {
>>>>>> dataset.getNamedModel("G2").add(m);
>>>>>> });
>>>>>>
>>>>>>
>>>>>>
>>>>>> If either version of 2), and 3) are not done in parallel there is no
>>>>>> problem and everything is executed correctly.
>>>>>>
>>>>>> The problem arise when 2.a) or 2.b) is run, and before ending someone
>>>>>> tries to perform 3). Then I get the FileException("In the middle
>>>>>> of an
>>>>>> alloc-write").
>>>>>>
>>>>>> Do you have an idea on how to avoid this? How can I handle
>>>>>> transactions in this model?
>>>>>>
>>>>>>
>>>>>> I was thinking on creating a global mutex so if any action is being
>>>>>> performed over the dataset, then the rest would be blocked. The
>>>>>> problem here is that the code is part of a webservice and then if the
>>>>>> read/write operation lasts long, I will get a timeout that will close
>>>>>> the connection.
>>>>>>
>>>>>> The other option is to disable writing while someone is reading. The
>>>>>> main problem here is how to properly reschedule writings not to
>>>>>> have a
>>>>>> big queue.
>>>>>>
>>>>>> Any help is more than welcome. I don't know what else to do to solve
>>>>>> this is issue, and the problem is making the service unusable :(
>>>>>>
>>>>>> Thanks a lot for the great help you are all offering.
>>>>>> Jorge
>>>>>
>>
>
Re: Txn code not handling type of transaction
Posted by Andy Seaborne <an...@apache.org>.
I can't see that variable "dataset" is a TDB dataset, not a bunch of
graphs pulled together at the start of the request. If it's a composite
wrapper containing TDB graphs, and if it is different on request, that
would be consistent with what has been presented. Not the only
possibility though.
Andy
On 29/12/17 14:28, ajs6f wrote:
> Just to add a bit of context to Andy's point:
>
>> It needs a dataset so it makes one and puts the model in as the default graph.
>
> This is (more or less) because SPARQL is defined as acting against datasets, not individual graphs.
>
> https://www.w3.org/TR/sparql11-query/#rdfDataset:
>
> "A SPARQL query is executed against an RDF Dataset which represents a collection of graphs."
>
> And if (as Andy pointed out is possible) the individual graph in question is a view over several datasets, that doesn't work either. There has to be a single dataset that is the basic context for SPARQL. (Even while using federation/SERVICE, there is still a single dataset against which the query is being executed.)
>
> ajs6f
>
>> On Dec 29, 2017, at 9:21 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>>
>>
>> On 29/12/17 13:58, George News wrote:
>>> On 2017-12-29 14:46, Andy Seaborne wrote:
>>>>
>>>>
>>>> On 29/12/17 09:51, George News wrote:
>>>>> On 2017-12-28 18:02, dandh988 wrote:
>>>>>> Can you give a complete example? How are you calling MultiUnion? Are
>>>>>> you calling txn read on the dataset then building the union, then
>>>>>> calling txn write to update the graph?
>>>>>
>>>>> Let's describe a use case.
>>>>
>>>> The stack trace does not agree with the details here (and details
>>>> matter!) especially what dataset is because if that is a general dataset
>>>> built out of TDB graphs, this whole thing will not work. A complete,
>>>> minimal example is needed.
>>>>
>>>> A few general observations:
>>>>
>>>>>
>>>>> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
>>>>> 2.a) Someone initiates a sparql request and the internal procedure
>>>>> followed is:
>>>>> return Txn.calculateRead(dataset, () -> {
>>>>> // Create multiunion of 3 namegraphs
>>>>> MultiUnion union = new MultiUnion();
>>>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>>>> union.addGraph(dataset.getNamedModel("G4").getGraph());
>>>>
>>>> This can be done in SPARQL:
>>>>
>>>> either UNION or
>>>>
>>>> SELECT
>>>> FROM <G1>
>>>> FROM <G2>
>>>> FROM <G4>
>>>> WHERE { ... }
>>>>
>>>> (copy the query object and modify it).
>>>>
>>>> and TDB will handle it better.
>>>>
>>>>>
>>>>> Model m = ModelFactory.createModelForGraph(union);
>>>>
>>>> You are not querying the TDB dataset if you pass a model to
>>>> QueryExecutionFactory.create. It has to create a dataset (a general
>>>> purpose one) for the query.
>>> But if the model is created from graphs/models on a dataset, isn't it
>>> the same? At the end the models are a reference to ones in the dataset.
>>
>> Not the same and the model is not a reference into a dataset.
>>
>> QueryExecutionFactory.create is given a model, here a composite one, could be a model without a datasets, could be a model over many datasets. It can't know what the structure is, model is an interface.
>>
>> It needs a dataset so it makes one and puts the model in as the default graph.
>>
>>>>> // Launch Sparql query on it
>>>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>>>> m)) {
>>>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>>>> }
>>>>> });
>>>>>
>>>>> 2.b) Someone initiates a sparql request and the internal procedure
>>>>> followed is:
>>>>> return Txn.calculateRead(dataset, () -> {
>>>>> // Create multiunion of 2 namegraphs
>>>>> // This code in in a function
>>>>> MultiUnion union = new MultiUnion();
>>>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>>>> Model m1 = ModelFactory.createModelForGraph(union);
>>>>>
>>>>> // Retrieve namegraph G3
>>>>> // This code in in a function
>>>>> Model m2 = dataset.getNamedModel("G3");
>>>>>
>>>>> Model m = ModelFactory.createUnion(m2, m1);
>>>>>
>>>>> // Launch Sparql query on it
>>>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>>>> m)) {
>>>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>>>> }
>>>>> });
>>>>>
>>>>>
>>>>> 3) Someone initiaties a write in G2
>>>>> // m stores the new entity model
>>>>> Model m;
>>>>> Txn.executeWrite(dataset, () -> {
>>>>> dataset.getNamedModel("G2").add(m);
>>>>> });
>>>>>
>>>>>
>>>>>
>>>>> If either version of 2), and 3) are not done in parallel there is no
>>>>> problem and everything is executed correctly.
>>>>>
>>>>> The problem arise when 2.a) or 2.b) is run, and before ending someone
>>>>> tries to perform 3). Then I get the FileException("In the middle of an
>>>>> alloc-write").
>>>>>
>>>>> Do you have an idea on how to avoid this? How can I handle
>>>>> transactions in this model?
>>>>>
>>>>>
>>>>> I was thinking on creating a global mutex so if any action is being
>>>>> performed over the dataset, then the rest would be blocked. The
>>>>> problem here is that the code is part of a webservice and then if the
>>>>> read/write operation lasts long, I will get a timeout that will close
>>>>> the connection.
>>>>>
>>>>> The other option is to disable writing while someone is reading. The
>>>>> main problem here is how to properly reschedule writings not to have a
>>>>> big queue.
>>>>>
>>>>> Any help is more than welcome. I don't know what else to do to solve
>>>>> this is issue, and the problem is making the service unusable :(
>>>>>
>>>>> Thanks a lot for the great help you are all offering.
>>>>> Jorge
>>>>
>
Re: Txn code not handling type of transaction
Posted by ajs6f <aj...@apache.org>.
Just to add a bit of context to Andy's point:
> It needs a dataset so it makes one and puts the model in as the default graph.
This is (more or less) because SPARQL is defined as acting against datasets, not individual graphs.
https://www.w3.org/TR/sparql11-query/#rdfDataset:
"A SPARQL query is executed against an RDF Dataset which represents a collection of graphs."
And if (as Andy pointed out is possible) the individual graph in question is a view over several datasets, that doesn't work either. There has to be a single dataset that is the basic context for SPARQL. (Even while using federation/SERVICE, there is still a single dataset against which the query is being executed.)
ajs6f
> On Dec 29, 2017, at 9:21 AM, Andy Seaborne <an...@apache.org> wrote:
>
>
>
> On 29/12/17 13:58, George News wrote:
>> On 2017-12-29 14:46, Andy Seaborne wrote:
>>>
>>>
>>> On 29/12/17 09:51, George News wrote:
>>>> On 2017-12-28 18:02, dandh988 wrote:
>>>>> Can you give a complete example? How are you calling MultiUnion? Are
>>>>> you calling txn read on the dataset then building the union, then
>>>>> calling txn write to update the graph?
>>>>
>>>> Let's describe a use case.
>>>
>>> The stack trace does not agree with the details here (and details
>>> matter!) especially what dataset is because if that is a general dataset
>>> built out of TDB graphs, this whole thing will not work. A complete,
>>> minimal example is needed.
>>>
>>> A few general observations:
>>>
>>>>
>>>> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
>>>> 2.a) Someone initiates a sparql request and the internal procedure
>>>> followed is:
>>>> return Txn.calculateRead(dataset, () -> {
>>>> // Create multiunion of 3 namegraphs
>>>> MultiUnion union = new MultiUnion();
>>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>>> union.addGraph(dataset.getNamedModel("G4").getGraph());
>>>
>>> This can be done in SPARQL:
>>>
>>> either UNION or
>>>
>>> SELECT
>>> FROM <G1>
>>> FROM <G2>
>>> FROM <G4>
>>> WHERE { ... }
>>>
>>> (copy the query object and modify it).
>>>
>>> and TDB will handle it better.
>>>
>>>>
>>>> Model m = ModelFactory.createModelForGraph(union);
>>>
>>> You are not querying the TDB dataset if you pass a model to
>>> QueryExecutionFactory.create. It has to create a dataset (a general
>>> purpose one) for the query.
>> But if the model is created from graphs/models on a dataset, isn't it
>> the same? At the end the models are a reference to ones in the dataset.
>
> Not the same and the model is not a reference into a dataset.
>
> QueryExecutionFactory.create is given a model, here a composite one, could be a model without a datasets, could be a model over many datasets. It can't know what the structure is, model is an interface.
>
> It needs a dataset so it makes one and puts the model in as the default graph.
>
>>>> // Launch Sparql query on it
>>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>>> m)) {
>>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>>> }
>>>> });
>>>>
>>>> 2.b) Someone initiates a sparql request and the internal procedure
>>>> followed is:
>>>> return Txn.calculateRead(dataset, () -> {
>>>> // Create multiunion of 2 namegraphs
>>>> // This code in in a function
>>>> MultiUnion union = new MultiUnion();
>>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>>> Model m1 = ModelFactory.createModelForGraph(union);
>>>>
>>>> // Retrieve namegraph G3
>>>> // This code in in a function
>>>> Model m2 = dataset.getNamedModel("G3");
>>>>
>>>> Model m = ModelFactory.createUnion(m2, m1);
>>>>
>>>> // Launch Sparql query on it
>>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>>> m)) {
>>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>>> }
>>>> });
>>>>
>>>>
>>>> 3) Someone initiaties a write in G2
>>>> // m stores the new entity model
>>>> Model m;
>>>> Txn.executeWrite(dataset, () -> {
>>>> dataset.getNamedModel("G2").add(m);
>>>> });
>>>>
>>>>
>>>>
>>>> If either version of 2), and 3) are not done in parallel there is no
>>>> problem and everything is executed correctly.
>>>>
>>>> The problem arise when 2.a) or 2.b) is run, and before ending someone
>>>> tries to perform 3). Then I get the FileException("In the middle of an
>>>> alloc-write").
>>>>
>>>> Do you have an idea on how to avoid this? How can I handle
>>>> transactions in this model?
>>>>
>>>>
>>>> I was thinking on creating a global mutex so if any action is being
>>>> performed over the dataset, then the rest would be blocked. The
>>>> problem here is that the code is part of a webservice and then if the
>>>> read/write operation lasts long, I will get a timeout that will close
>>>> the connection.
>>>>
>>>> The other option is to disable writing while someone is reading. The
>>>> main problem here is how to properly reschedule writings not to have a
>>>> big queue.
>>>>
>>>> Any help is more than welcome. I don't know what else to do to solve
>>>> this is issue, and the problem is making the service unusable :(
>>>>
>>>> Thanks a lot for the great help you are all offering.
>>>> Jorge
>>>
Re: Txn code not handling type of transaction
Posted by Andy Seaborne <an...@apache.org>.
On 29/12/17 13:58, George News wrote:
> On 2017-12-29 14:46, Andy Seaborne wrote:
>>
>>
>> On 29/12/17 09:51, George News wrote:
>>> On 2017-12-28 18:02, dandh988 wrote:
>>>> Can you give a complete example? How are you calling MultiUnion? Are
>>>> you calling txn read on the dataset then building the union, then
>>>> calling txn write to update the graph?
>>>
>>> Let's describe a use case.
>>
>> The stack trace does not agree with the details here (and details
>> matter!) especially what dataset is because if that is a general dataset
>> built out of TDB graphs, this whole thing will not work. A complete,
>> minimal example is needed.
>>
>> A few general observations:
>>
>>>
>>> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
>>> 2.a) Someone initiates a sparql request and the internal procedure
>>> followed is:
>>> return Txn.calculateRead(dataset, () -> {
>>> // Create multiunion of 3 namegraphs
>>> MultiUnion union = new MultiUnion();
>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>> union.addGraph(dataset.getNamedModel("G4").getGraph());
>>
>> This can be done in SPARQL:
>>
>> either UNION or
>>
>> SELECT
>> FROM <G1>
>> FROM <G2>
>> FROM <G4>
>> WHERE { ... }
>>
>> (copy the query object and modify it).
>>
>> and TDB will handle it better.
>>
>>>
>>> Model m = ModelFactory.createModelForGraph(union);
>>
>> You are not querying the TDB dataset if you pass a model to
>> QueryExecutionFactory.create. It has to create a dataset (a general
>> purpose one) for the query.
>
> But if the model is created from graphs/models on a dataset, isn't it
> the same? At the end the models are a reference to ones in the dataset.
Not the same and the model is not a reference into a dataset.
QueryExecutionFactory.create is given a model, here a composite one,
could be a model without a datasets, could be a model over many
datasets. It can't know what the structure is, model is an interface.
It needs a dataset so it makes one and puts the model in as the default
graph.
>>> // Launch Sparql query on it
>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>> m)) {
>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>> }
>>> });
>>>
>>> 2.b) Someone initiates a sparql request and the internal procedure
>>> followed is:
>>> return Txn.calculateRead(dataset, () -> {
>>> // Create multiunion of 2 namegraphs
>>> // This code in in a function
>>> MultiUnion union = new MultiUnion();
>>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>>> Model m1 = ModelFactory.createModelForGraph(union);
>>>
>>> // Retrieve namegraph G3
>>> // This code in in a function
>>> Model m2 = dataset.getNamedModel("G3");
>>>
>>> Model m = ModelFactory.createUnion(m2, m1);
>>>
>>> // Launch Sparql query on it
>>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>>> m)) {
>>> return ResultSetFactory.copyResults(qExec.execSelect())
>>> }
>>> });
>>>
>>>
>>> 3) Someone initiaties a write in G2
>>> // m stores the new entity model
>>> Model m;
>>> Txn.executeWrite(dataset, () -> {
>>> dataset.getNamedModel("G2").add(m);
>>> });
>>>
>>>
>>>
>>> If either version of 2), and 3) are not done in parallel there is no
>>> problem and everything is executed correctly.
>>>
>>> The problem arise when 2.a) or 2.b) is run, and before ending someone
>>> tries to perform 3). Then I get the FileException("In the middle of an
>>> alloc-write").
>>>
>>> Do you have an idea on how to avoid this? How can I handle
>>> transactions in this model?
>>>
>>>
>>> I was thinking on creating a global mutex so if any action is being
>>> performed over the dataset, then the rest would be blocked. The
>>> problem here is that the code is part of a webservice and then if the
>>> read/write operation lasts long, I will get a timeout that will close
>>> the connection.
>>>
>>> The other option is to disable writing while someone is reading. The
>>> main problem here is how to properly reschedule writings not to have a
>>> big queue.
>>>
>>> Any help is more than welcome. I don't know what else to do to solve
>>> this is issue, and the problem is making the service unusable :(
>>>
>>> Thanks a lot for the great help you are all offering.
>>> Jorge
>>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-29 14:46, Andy Seaborne wrote:
>
>
> On 29/12/17 09:51, George News wrote:
>> On 2017-12-28 18:02, dandh988 wrote:
>>> Can you give a complete example? How are you calling MultiUnion? Are
>>> you calling txn read on the dataset then building the union, then
>>> calling txn write to update the graph?
>>
>> Let's describe a use case.
>
> The stack trace does not agree with the details here (and details
> matter!) especially what dataset is because if that is a general dataset
> built out of TDB graphs, this whole thing will not work. A complete,
> minimal example is needed.
>
> A few general observations:
>
>>
>> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
>> 2.a) Someone initiates a sparql request and the internal procedure
>> followed is:
>> return Txn.calculateRead(dataset, () -> {
>> // Create multiunion of 3 namegraphs
>> MultiUnion union = new MultiUnion();
>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>> union.addGraph(dataset.getNamedModel("G4").getGraph());
>
> This can be done in SPARQL:
>
> either UNION or
>
> SELECT
> FROM <G1>
> FROM <G2>
> FROM <G4>
> WHERE { ... }
>
> (copy the query object and modify it).
>
> and TDB will handle it better.
>
>>
>> Model m = ModelFactory.createModelForGraph(union);
>
> You are not querying the TDB dataset if you pass a model to
> QueryExecutionFactory.create. It has to create a dataset (a general
> purpose one) for the query.
But if the model is created from graphs/models on a dataset, isn't it
the same? At the end the models are a reference to ones in the dataset.
>> // Launch Sparql query on it
>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>> m)) {
>> return ResultSetFactory.copyResults(qExec.execSelect())
>> }
>> });
>>
>> 2.b) Someone initiates a sparql request and the internal procedure
>> followed is:
>> return Txn.calculateRead(dataset, () -> {
>> // Create multiunion of 2 namegraphs
>> // This code in in a function
>> MultiUnion union = new MultiUnion();
>> union.addGraph(dataset.getNamedModel("G1").getGraph());
>> union.addGraph(dataset.getNamedModel("G2").getGraph());
>> Model m1 = ModelFactory.createModelForGraph(union);
>>
>> // Retrieve namegraph G3
>> // This code in in a function
>> Model m2 = dataset.getNamedModel("G3");
>>
>> Model m = ModelFactory.createUnion(m2, m1);
>>
>> // Launch Sparql query on it
>> try (QueryExecution qExec = QueryExecutionFactory.create(query,
>> m)) {
>> return ResultSetFactory.copyResults(qExec.execSelect())
>> }
>> });
>>
>>
>> 3) Someone initiaties a write in G2
>> // m stores the new entity model
>> Model m;
>> Txn.executeWrite(dataset, () -> {
>> dataset.getNamedModel("G2").add(m);
>> });
>>
>>
>>
>> If either version of 2), and 3) are not done in parallel there is no
>> problem and everything is executed correctly.
>>
>> The problem arise when 2.a) or 2.b) is run, and before ending someone
>> tries to perform 3). Then I get the FileException("In the middle of an
>> alloc-write").
>>
>> Do you have an idea on how to avoid this? How can I handle
>> transactions in this model?
>>
>>
>> I was thinking on creating a global mutex so if any action is being
>> performed over the dataset, then the rest would be blocked. The
>> problem here is that the code is part of a webservice and then if the
>> read/write operation lasts long, I will get a timeout that will close
>> the connection.
>>
>> The other option is to disable writing while someone is reading. The
>> main problem here is how to properly reschedule writings not to have a
>> big queue.
>>
>> Any help is more than welcome. I don't know what else to do to solve
>> this is issue, and the problem is making the service unusable :(
>>
>> Thanks a lot for the great help you are all offering.
>> Jorge
>
Re: Txn code not handling type of transaction
Posted by Andy Seaborne <an...@apache.org>.
On 29/12/17 09:51, George News wrote:
> On 2017-12-28 18:02, dandh988 wrote:
>> Can you give a complete example? How are you calling MultiUnion? Are
>> you calling txn read on the dataset then building the union, then
>> calling txn write to update the graph?
>
> Let's describe a use case.
The stack trace does not agree with the details here (and details
matter!) especially what dataset is because if that is a general dataset
built out of TDB graphs, this whole thing will not work. A complete,
minimal example is needed.
A few general observations:
>
> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
> 2.a) Someone initiates a sparql request and the internal procedure followed is:
> return Txn.calculateRead(dataset, () -> {
> // Create multiunion of 3 namegraphs
> MultiUnion union = new MultiUnion();
> union.addGraph(dataset.getNamedModel("G1").getGraph());
> union.addGraph(dataset.getNamedModel("G2").getGraph());
> union.addGraph(dataset.getNamedModel("G4").getGraph());
This can be done in SPARQL:
either UNION or
SELECT
FROM <G1>
FROM <G2>
FROM <G4>
WHERE { ... }
(copy the query object and modify it).
and TDB will handle it better.
>
> Model m = ModelFactory.createModelForGraph(union);
You are not querying the TDB dataset if you pass a model to
QueryExecutionFactory.create. It has to create a dataset (a general
purpose one) for the query.
> // Launch Sparql query on it
> try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
> return ResultSetFactory.copyResults(qExec.execSelect())
> }
> });
>
> 2.b) Someone initiates a sparql request and the internal procedure followed is:
> return Txn.calculateRead(dataset, () -> {
> // Create multiunion of 2 namegraphs
> // This code in in a function
> MultiUnion union = new MultiUnion();
> union.addGraph(dataset.getNamedModel("G1").getGraph());
> union.addGraph(dataset.getNamedModel("G2").getGraph());
> Model m1 = ModelFactory.createModelForGraph(union);
>
> // Retrieve namegraph G3
> // This code in in a function
> Model m2 = dataset.getNamedModel("G3");
>
> Model m = ModelFactory.createUnion(m2, m1);
>
> // Launch Sparql query on it
> try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
> return ResultSetFactory.copyResults(qExec.execSelect())
> }
> });
>
>
> 3) Someone initiaties a write in G2
> // m stores the new entity model
> Model m;
> Txn.executeWrite(dataset, () -> {
> dataset.getNamedModel("G2").add(m);
> });
>
>
>
> If either version of 2), and 3) are not done in parallel there is no problem and everything is executed correctly.
>
> The problem arise when 2.a) or 2.b) is run, and before ending someone tries to perform 3). Then I get the FileException("In the middle of an alloc-write").
>
> Do you have an idea on how to avoid this? How can I handle transactions in this model?
>
>
> I was thinking on creating a global mutex so if any action is being performed over the dataset, then the rest would be blocked. The problem here is that the code is part of a webservice and then if the read/write operation lasts long, I will get a timeout that will close the connection.
>
> The other option is to disable writing while someone is reading. The main problem here is how to properly reschedule writings not to have a big queue.
>
> Any help is more than welcome. I don't know what else to do to solve this is issue, and the problem is making the service unusable :(
>
> Thanks a lot for the great help you are all offering.
> Jorge
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-29 11:05, dandh988 wrote:
> Not quite the complete working example I was expecting but...
I'm sorry. It is a rest service and don't have kind of "unit-test". For the moment I cannot disclose code.
> Can you post the exception stack please?
2017-12-29 10:14:30.101 [default task-58] ERROR com.myproject.rest.exceptions.UncaughtThrowableExceptionMapper - Uncaught throwable exception
org.apache.jena.tdb.base.file.FileException: In the middle of an alloc-write
at org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:311) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.store.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:186) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:111) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:70) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:128) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:82) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:50) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:129) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:123) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:59) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270) ~[jena-base-3.5.0.jar:3.5.0]
at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270) ~[jena-base-3.5.0.jar:3.5.0]
at org.apache.jena.atlas.iterator.Iter.next(Iter.java:875) ~[jena-base-3.5.0.jar:3.5.0]
at org.apache.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:94) ~[jena-core-3.5.0.jar:3.5.0]
at org.apache.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:94) ~[jena-core-3.5.0.jar:3.5.0]
at org.apache.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:56) ~[jena-core-3.5.0.jar:3.5.0]
at org.apache.jena.util.iterator.NiceIterator$1.hasNext(NiceIterator.java:105) ~[jena-core-3.5.0.jar:3.5.0]
at org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) ~[jena-core-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMapper.hasNextBinding(QueryIterTriplePattern.java:135) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:74) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:101) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:101) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:101) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:101) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:101) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:63) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.StageGeneratorGeneric.execute(StageGeneratorGeneric.java:61) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.StageGeneratorGeneric.execute(StageGeneratorGeneric.java:53) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.tdb.solver.StageGeneratorDirectTDB.execute(StageGeneratorDirectTDB.java:53) ~[jena-tdb-3.5.0.jar:3.5.0]
at org.apache.jena.tdb2.solver.StageGeneratorDirectTDB.execute(StageGeneratorDirectTDB.java:53) ~[jena-tdb2-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:128) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:58) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.algebra.op.OpBGP.visit(OpBGP.java:49) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:117) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:299) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:187) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.algebra.op.OpFilter.visit(OpFilter.java:132) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:117) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:228) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:130) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.algebra.op.OpSequence.visit(OpSequence.java:75) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:117) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:228) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:130) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.algebra.op.OpSequence.visit(OpSequence.java:75) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:117) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:405) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:315) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.algebra.op.OpGroup.visit(OpGroup.java:54) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:117) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:376) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:323) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.algebra.op.OpTopN.visit(OpTopN.java:50) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:117) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:88) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.QC.execute(QC.java:52) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.QueryEngineMain.eval(QueryEngineMain.java:55) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.QueryEngineBase.evaluate(QueryEngineBase.java:136) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.QueryEngineBase.createPlan(QueryEngineBase.java:106) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.QueryEngineBase.getPlan(QueryEngineBase.java:87) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.main.QueryEngineMain$QueryEngineMainFactory.create(QueryEngineMain.java:90) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.QueryExecutionBase.getPlan(QueryExecutionBase.java:524) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.QueryExecutionBase.startQueryIterator(QueryExecutionBase.java:472) ~[jena-arq-3.5.0.jar:3.5.0]
at org.apache.jena.sparql.engine.QueryExecutionBase.execAsk(QueryExecutionBase.java:331) ~[jena-arq-3.5.0.jar:3.5.0]
at com.myproject.storage.semantic.tdb.ReadSparqlExecutor.lambda$execute$0(ReadSparqlExecutor.java:67) ~[classes:?]
at org.apache.jena.system.Txn.calculateRead(Txn.java:56) ~[jena-arq-3.5.0.jar:3.5.0]
at com.myproject.storage.semantic.tdb.ReadSparqlExecutor.execute(ReadSparqlExecutor.java:51) ~[classes:?]
at com.myproject.rest.QueryRestService.askBeforeExecute(QueryRestService.java:492) ~[classes:?]
at com.myproject.rest.QueryRestService.executeQueryStream(QueryRestService.java:665) ~[classes:?]
at com.myproject.rest.QueryRestService.executeQuery(QueryRestService.java:453) ~[classes:?]
at com.myproject.rest.QueryRestService$Proxy$_$$_WeldClientProxy.executeQuery(Unknown Source) ~[classes:?]
at sun.reflect.GeneratedMethodAccessor365.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) ~[resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) ~[resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) ~[resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) ~[resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:395) [resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202) [resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) [resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) [resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) [resteasy-jaxrs-3.0.14.Final.jar!/:3.0.14.Final]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar!/:1.0.0.Final]
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71) [log4j-web-2.5.jar:2.5]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) [wildfly-undertow-10.0.0.Final.jar!/:10.0.0.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) [wildfly-undertow-10.0.0.Final.jar!/:10.0.0.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174) [undertow-servlet-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793) [undertow-core-1.3.15.Final.jar!/:1.3.15.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> Can you run the write on its own? I'm wondering if your TDB is broke from previous non transactional access.
Yes. As I mentioned, if read and write are not done simultaneously everything is working correctly. The problem is that at the beginning the service didn't have many subscribers for writting, but now we do have quite a lot and they are writing almost every 1 to 10 seconds. Considering that readings takes longer than that, Houston there is a problem ;)
Thanks a lot.
Jorge
>
> Dick
> -------- Original message --------From: George News <ge...@gmx.net> Date: 29/12/2017 09:51 (GMT+00:00) To: users@jena.apache.org Subject: Re: Txn code not handling type of transaction
> On 2017-12-28 18:02, dandh988 wrote:
>> Can you give a complete example? How are you calling MultiUnion? Are
>> you calling txn read on the dataset then building the union, then
>> calling txn write to update the graph?
>
> Let's describe a use case.
>
> 1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
> 2.a) Someone initiates a sparql request and the internal procedure followed is:
> return Txn.calculateRead(dataset, () -> {
> // Create multiunion of 3 namegraphs
> MultiUnion union = new MultiUnion();
> union.addGraph(dataset.getNamedModel("G1").getGraph());
> union.addGraph(dataset.getNamedModel("G2").getGraph());
> union.addGraph(dataset.getNamedModel("G4").getGraph());
>
> Model m = ModelFactory.createModelForGraph(union);
>
> // Launch Sparql query on it
> try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
> return ResultSetFactory.copyResults(qExec.execSelect())
> }
> });
>
> 2.b) Someone initiates a sparql request and the internal procedure followed is:
> return Txn.calculateRead(dataset, () -> {
> // Create multiunion of 2 namegraphs
> // This code in in a function
> MultiUnion union = new MultiUnion();
> union.addGraph(dataset.getNamedModel("G1").getGraph());
> union.addGraph(dataset.getNamedModel("G2").getGraph());
> Model m1 = ModelFactory.createModelForGraph(union);
>
> // Retrieve namegraph G3
> // This code in in a function
> Model m2 = dataset.getNamedModel("G3");
>
> Model m = ModelFactory.createUnion(m2, m1);
>
> // Launch Sparql query on it
> try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
> return ResultSetFactory.copyResults(qExec.execSelect())
> }
> });
>
>
> 3) Someone initiaties a write in G2
> // m stores the new entity model
> Model m;
> Txn.executeWrite(dataset, () -> {
> dataset.getNamedModel("G2").add(m);
> });
>
>
>
> If either version of 2), and 3) are not done in parallel there is no problem and everything is executed correctly.
>
> The problem arise when 2.a) or 2.b) is run, and before ending someone tries to perform 3). Then I get the FileException("In the middle of an alloc-write").
>
> Do you have an idea on how to avoid this? How can I handle transactions in this model?
>
>
> I was thinking on creating a global mutex so if any action is being performed over the dataset, then the rest would be blocked. The problem here is that the code is part of a webservice and then if the read/write operation lasts long, I will get a timeout that will close the connection.
>
> The other option is to disable writing while someone is reading. The main problem here is how to properly reschedule writings not to have a big queue.
>
> Any help is more than welcome. I don't know what else to do to solve this is issue, and the problem is making the service unusable :(
>
> Thanks a lot for the great help you are all offering.
> Jorge
>
>>
>>
>> Dick -------- Original message --------From: George News
>> <ge...@gmx.net> Date: 28/12/2017 13:17 (GMT+00:00) To:
>> users@jena.apache.org Subject: Re: Txn code not handling type of
>> transaction On 2017-12-27 19:32, Andy Seaborne wrote:
>>>
>>>
>>> On 27/12/17 18:19, George News wrote:
>>>> One think I have forgotten to mention is that I'm using a static
>>>> variable to store the dataset reference. Is this important?
>>>
>>> No.
>>>
>>> The top layer of transaction code uses ThreadLocal variables.
>>>
>>> Internally, there are Transaction objects.
>>>
>>> Andy
>>
>> Another doubt on the issue with transactions. I'm generating
>> Multiunions from the same dataset graphs. While in one thread I'm
>> reading from a multiunion, on another thread I'm writing to one of
>> the graphs included in the multiunion (standalone graph, not
>> multiunion).
>>
>> I have modified my code to use Txn everywhere and still having
>> issues. This is why I'm asking about how multiunions are handle
>> concerning transactions.
>>
>>
>>>>
>>>>
>>>> On 2017-12-27 19:13, George News wrote:
>>>>> Just out of curiosity, just in case the problem is that
>>>>> everything is done in the same thread. Do you know if Wildfly
>>>>> is handling every request under the same thread? I guess not,
>>>>> it will be really strange.
>>>
>>>>>
>>>>> The point is that I have one REST endpoint for writing and
>>>>> another for reading. Writing is done almost per 1 to 10
>>>>> seconds. If I execute a reading that takes longer than that, I
>>>>> get the exception on alloc-write.
>>>>>
>>>>> Thanks a lot. I just don't know why it is happening. Now I
>>>>> think the code is using Txn (I have one point where I copied
>>>>> the Txn.java behaviour) and still got the error.
>>>>>
>>>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>>>> Then nesting is not safe as you might have open initially a
>>>>>>> read transaction and then include a write. If the parent
>>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>>
>>>>>> Actual nesting is not supported right now, period.
>>>>>> Transactions in Jena are currently thread-local, so what you
>>>>>> describe above I would see as a mistaken use of the API. If a
>>>>>> client needs to move from a READ to a WRITE transaction, it's
>>>>>> usually appropriate to close the transaction and open a new
>>>>>> one (transactions in TIM, for instance, are snapshot
>>>>>> isolated, and I believe the same is true of TDB2).
>>>>>> Transactional objects _should not_ be passed from thread to
>>>>>> thread with open transactions, unless additional client
>>>>>> machinery is in place to manage those transactions, such as
>>>>>> Dick has described in another thread (using thread proxies).
>>>>>>
>>>>>> Promotion machinery does exist in Jena but I am not aware
>>>>>> that any of the dataset implementations actually support it
>>>>>> right now. I could be wrong about that, since I didn't write
>>>>>> the promotion code. Jena isn't a SQL database and doesn't
>>>>>> offer the same kinds of guarantees or tradeoffs. Correctly
>>>>>> promoting transactions in the absence of information about
>>>>>> data dependencies is non-trivial.
>>>>>>
>>>>>> That having been said, it should be possible to add a
>>>>>> "transaction type" method to transactional objects, within
>>>>>> the "thread-local" design. Some dataset implementations
>>>>>> already have one, e.g. DatasetGraphInMemory::transactionType.
>>>>>> You might want to start by adding it to the
>>>>>> o.a.j.sparql.core.Transactional interface and then "catching
>>>>>> up" the implementation code to build up a PR.
>>>>>>
>>>>>> Adam Soroka
>>>>>>
>>>>>>> On Dec 27, 2017, at 6:53 AM, George News
>>>>>>> <ge...@gmx.net> wrote:
>>>>>>>
>>>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>>>> I recently wrote some code to try to handle a similar
>>>>>>>> situation. In my case I knew I needed a transaction to be
>>>>>>>> active at various points so I created a
>>>>>>>> TransactionHolder. I create the holder and passing the
>>>>>>>> object that has implements Transactional as well as the
>>>>>>>> type of ReadWrite I want.
>>>>>>>>
>>>>>>>> If the transaction is active it does nothing (and I hope
>>>>>>>> the proper transaction has been started) otherwise It
>>>>>>>> starts the transaction. Ad the end I call commit or abort
>>>>>>>> as appropriate. If I did not start the transaction the
>>>>>>>> commit, abort or end is ignored.
>>>>>>>
>>>>>>> I see the same problem in your code as I pointed out
>>>>>>> before. A transaction behaves differently if it is READ or
>>>>>>> WRITE and in your code there is not such a thing. Actually
>>>>>>> the isInTransaction() doesn't give you this information.
>>>>>>>
>>>>>>> Then nesting is not safe as you might have open initially a
>>>>>>> read transaction and then include a write. If the parent
>>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>>>
>>>>>>> Just for you to know, your code is more or less integrated
>>>>>>> in [1], so you can "update" your code.
>>>>>>>
>>>>>>> [1]:
>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>
> I think there may be an issue with abort in that it should
>>>>>>>> probablyset up end() to throw an exception when I have
>>>>>>>> not created the transaction so that the outer
>>>>>>>> transaction will fail.
>>>>>>>>
>>>>>>>> import org.apache.jena.query.ReadWrite; import
>>>>>>>> org.apache.jena.sparql.core.Transactional;
>>>>>>>>
>>>>>>>> public class TransactionHolder { private final
>>>>>>>> Transactional txn; private final boolean started; private
>>>>>>>> final ReadWrite rw;
>>>>>>>>
>>>>>>>> public TransactionHolder( Transactional txn, ReadWrite rw
>>>>>>>> ) { this.txn = txn; this.rw = rw; started = !
>>>>>>>> txn.isInTransaction(); if (started) { txn.begin( rw ); }
>>>>>>>> }
>>>>>>>>
>>>>>>>> public boolean ownsTranaction() { return started; }
>>>>>>>>
>>>>>>>> public void commit() { if (started) { txn.commit(); } }
>>>>>>>>
>>>>>>>> public void abort() { if (started) { txn.abort(); } }
>>>>>>>>
>>>>>>>> public void end() { if (started) { txn.end(); } }
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988
>>>>>>>> <da...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> You cannot nest transactions nor can you promote a read
>>>>>>>>> to a write. You need to rewrite your code or use txn
>>>>>>>>> which correctly checks if a transaction is available
>>>>>>>>> and if not will begin the correct one, either READ or
>>>>>>>>> WRITE.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dick -------- Original message --------From: George
>>>>>>>>> News <ge...@gmx.net> Date: 27/12/2017 10:27
>>>>>>>>> (GMT+00:00) To: Jena User Mailing List <
>>>>>>>>> users@jena.apache.org> Subject: Txn code not handling
>>>>>>>>> type of transaction Hi,
>>>>>>>>>
>>>>>>>>> As you know from other threads I'm having some issues
>>>>>>>>> with transactions. Your suggestion is to use Txn
>>>>>>>>> instead of begin/end. Just for curiosity I have checked
>>>>>>>>> the Txn code at [1] and it seems that inside you use
>>>>>>>>> begin/end.
>>>>>>>>>
>>>>>>>>> However I have a doubt concerning how you handle the
>>>>>>>>> begin/end for READ and WRITE. It seems that you open a
>>>>>>>>> transaction based on txn.isInTransaction(), but how do
>>>>>>>>> you know if it is a READ or WRITE?
>>>>>>>>>
>>>>>>>>> If you create something like:
>>>>>>>>>
>>>>>>>>> Txn.executeRead(dataset, { Txn.executeWrite(dataset, {
>>>>>>>>> // Whatever } } }
>>>>>>>>>
>>>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and
>>>>>>>>> therefore it might be leading to unexepected behaviours
>>>>>>>>> for the txn.commit().
>>>>>>>>>
>>>>>>>>> could you give some hints on how this is handle
>>>>>>>>> internally? Before fully modify the code I have, it
>>>>>>>>> might be easier to replicate the txn behaviour ;) but I
>>>>>>>>> would like to know the above (if possible).
>>>>>>>>>
>>>>>>>>> As always, thanks in advanced Jorge
>>>>>>>>>
>>>>>>>>> [1]:
>>>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
Re: Txn code not handling type of transaction
Posted by dandh988 <da...@gmail.com>.
Not quite the complete working example I was expecting but...
Can you post the exception stack please?
Can you run the write on its own? I'm wondering if your TDB is broke from previous non transactional access.
Dick
-------- Original message --------From: George News <ge...@gmx.net> Date: 29/12/2017 09:51 (GMT+00:00) To: users@jena.apache.org Subject: Re: Txn code not handling type of transaction
On 2017-12-28 18:02, dandh988 wrote:
> Can you give a complete example? How are you calling MultiUnion? Are
> you calling txn read on the dataset then building the union, then
> calling txn write to update the graph?
Let's describe a use case.
1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
2.a) Someone initiates a sparql request and the internal procedure followed is:
return Txn.calculateRead(dataset, () -> {
// Create multiunion of 3 namegraphs
MultiUnion union = new MultiUnion();
union.addGraph(dataset.getNamedModel("G1").getGraph());
union.addGraph(dataset.getNamedModel("G2").getGraph());
union.addGraph(dataset.getNamedModel("G4").getGraph());
Model m = ModelFactory.createModelForGraph(union);
// Launch Sparql query on it
try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
return ResultSetFactory.copyResults(qExec.execSelect())
}
});
2.b) Someone initiates a sparql request and the internal procedure followed is:
return Txn.calculateRead(dataset, () -> {
// Create multiunion of 2 namegraphs
// This code in in a function
MultiUnion union = new MultiUnion();
union.addGraph(dataset.getNamedModel("G1").getGraph());
union.addGraph(dataset.getNamedModel("G2").getGraph());
Model m1 = ModelFactory.createModelForGraph(union);
// Retrieve namegraph G3
// This code in in a function
Model m2 = dataset.getNamedModel("G3");
Model m = ModelFactory.createUnion(m2, m1);
// Launch Sparql query on it
try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
return ResultSetFactory.copyResults(qExec.execSelect())
}
});
3) Someone initiaties a write in G2
// m stores the new entity model
Model m;
Txn.executeWrite(dataset, () -> {
dataset.getNamedModel("G2").add(m);
});
If either version of 2), and 3) are not done in parallel there is no problem and everything is executed correctly.
The problem arise when 2.a) or 2.b) is run, and before ending someone tries to perform 3). Then I get the FileException("In the middle of an alloc-write").
Do you have an idea on how to avoid this? How can I handle transactions in this model?
I was thinking on creating a global mutex so if any action is being performed over the dataset, then the rest would be blocked. The problem here is that the code is part of a webservice and then if the read/write operation lasts long, I will get a timeout that will close the connection.
The other option is to disable writing while someone is reading. The main problem here is how to properly reschedule writings not to have a big queue.
Any help is more than welcome. I don't know what else to do to solve this is issue, and the problem is making the service unusable :(
Thanks a lot for the great help you are all offering.
Jorge
>
>
> Dick -------- Original message --------From: George News
> <ge...@gmx.net> Date: 28/12/2017 13:17 (GMT+00:00) To:
> users@jena.apache.org Subject: Re: Txn code not handling type of
> transaction On 2017-12-27 19:32, Andy Seaborne wrote:
>>
>>
>> On 27/12/17 18:19, George News wrote:
>>> One think I have forgotten to mention is that I'm using a static
>>> variable to store the dataset reference. Is this important?
>>
>> No.
>>
>> The top layer of transaction code uses ThreadLocal variables.
>>
>> Internally, there are Transaction objects.
>>
>> Andy
>
> Another doubt on the issue with transactions. I'm generating
> Multiunions from the same dataset graphs. While in one thread I'm
> reading from a multiunion, on another thread I'm writing to one of
> the graphs included in the multiunion (standalone graph, not
> multiunion).
>
> I have modified my code to use Txn everywhere and still having
> issues. This is why I'm asking about how multiunions are handle
> concerning transactions.
>
>
>>>
>>>
>>> On 2017-12-27 19:13, George News wrote:
>>>> Just out of curiosity, just in case the problem is that
>>>> everything is done in the same thread. Do you know if Wildfly
>>>> is handling every request under the same thread? I guess not,
>>>> it will be really strange.
>>
>>>>
>>>> The point is that I have one REST endpoint for writing and
>>>> another for reading. Writing is done almost per 1 to 10
>>>> seconds. If I execute a reading that takes longer than that, I
>>>> get the exception on alloc-write.
>>>>
>>>> Thanks a lot. I just don't know why it is happening. Now I
>>>> think the code is using Txn (I have one point where I copied
>>>> the Txn.java behaviour) and still got the error.
>>>>
>>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>>> Then nesting is not safe as you might have open initially a
>>>>>> read transaction and then include a write. If the parent
>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>
>>>>> Actual nesting is not supported right now, period.
>>>>> Transactions in Jena are currently thread-local, so what you
>>>>> describe above I would see as a mistaken use of the API. If a
>>>>> client needs to move from a READ to a WRITE transaction, it's
>>>>> usually appropriate to close the transaction and open a new
>>>>> one (transactions in TIM, for instance, are snapshot
>>>>> isolated, and I believe the same is true of TDB2).
>>>>> Transactional objects _should not_ be passed from thread to
>>>>> thread with open transactions, unless additional client
>>>>> machinery is in place to manage those transactions, such as
>>>>> Dick has described in another thread (using thread proxies).
>>>>>
>>>>> Promotion machinery does exist in Jena but I am not aware
>>>>> that any of the dataset implementations actually support it
>>>>> right now. I could be wrong about that, since I didn't write
>>>>> the promotion code. Jena isn't a SQL database and doesn't
>>>>> offer the same kinds of guarantees or tradeoffs. Correctly
>>>>> promoting transactions in the absence of information about
>>>>> data dependencies is non-trivial.
>>>>>
>>>>> That having been said, it should be possible to add a
>>>>> "transaction type" method to transactional objects, within
>>>>> the "thread-local" design. Some dataset implementations
>>>>> already have one, e.g. DatasetGraphInMemory::transactionType.
>>>>> You might want to start by adding it to the
>>>>> o.a.j.sparql.core.Transactional interface and then "catching
>>>>> up" the implementation code to build up a PR.
>>>>>
>>>>> Adam Soroka
>>>>>
>>>>>> On Dec 27, 2017, at 6:53 AM, George News
>>>>>> <ge...@gmx.net> wrote:
>>>>>>
>>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>>> I recently wrote some code to try to handle a similar
>>>>>>> situation. In my case I knew I needed a transaction to be
>>>>>>> active at various points so I created a
>>>>>>> TransactionHolder. I create the holder and passing the
>>>>>>> object that has implements Transactional as well as the
>>>>>>> type of ReadWrite I want.
>>>>>>>
>>>>>>> If the transaction is active it does nothing (and I hope
>>>>>>> the proper transaction has been started) otherwise It
>>>>>>> starts the transaction. Ad the end I call commit or abort
>>>>>>> as appropriate. If I did not start the transaction the
>>>>>>> commit, abort or end is ignored.
>>>>>>
>>>>>> I see the same problem in your code as I pointed out
>>>>>> before. A transaction behaves differently if it is READ or
>>>>>> WRITE and in your code there is not such a thing. Actually
>>>>>> the isInTransaction() doesn't give you this information.
>>>>>>
>>>>>> Then nesting is not safe as you might have open initially a
>>>>>> read transaction and then include a write. If the parent
>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>>
>>>>>> Just for you to know, your code is more or less integrated
>>>>>> in [1], so you can "update" your code.
>>>>>>
>>>>>> [1]:
>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>
>>>>>>
>>>>>>>
>>>>>>
I think there may be an issue with abort in that it should
>>>>>>> probablyset up end() to throw an exception when I have
>>>>>>> not created the transaction so that the outer
>>>>>>> transaction will fail.
>>>>>>>
>>>>>>> import org.apache.jena.query.ReadWrite; import
>>>>>>> org.apache.jena.sparql.core.Transactional;
>>>>>>>
>>>>>>> public class TransactionHolder { private final
>>>>>>> Transactional txn; private final boolean started; private
>>>>>>> final ReadWrite rw;
>>>>>>>
>>>>>>> public TransactionHolder( Transactional txn, ReadWrite rw
>>>>>>> ) { this.txn = txn; this.rw = rw; started = !
>>>>>>> txn.isInTransaction(); if (started) { txn.begin( rw ); }
>>>>>>> }
>>>>>>>
>>>>>>> public boolean ownsTranaction() { return started; }
>>>>>>>
>>>>>>> public void commit() { if (started) { txn.commit(); } }
>>>>>>>
>>>>>>> public void abort() { if (started) { txn.abort(); } }
>>>>>>>
>>>>>>> public void end() { if (started) { txn.end(); } }
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988
>>>>>>> <da...@gmail.com> wrote:
>>>>>>>
>>>>>>>> You cannot nest transactions nor can you promote a read
>>>>>>>> to a write. You need to rewrite your code or use txn
>>>>>>>> which correctly checks if a transaction is available
>>>>>>>> and if not will begin the correct one, either READ or
>>>>>>>> WRITE.
>>>>>>>>
>>>>>>>>
>>>>>>>> Dick -------- Original message --------From: George
>>>>>>>> News <ge...@gmx.net> Date: 27/12/2017 10:27
>>>>>>>> (GMT+00:00) To: Jena User Mailing List <
>>>>>>>> users@jena.apache.org> Subject: Txn code not handling
>>>>>>>> type of transaction Hi,
>>>>>>>>
>>>>>>>> As you know from other threads I'm having some issues
>>>>>>>> with transactions. Your suggestion is to use Txn
>>>>>>>> instead of begin/end. Just for curiosity I have checked
>>>>>>>> the Txn code at [1] and it seems that inside you use
>>>>>>>> begin/end.
>>>>>>>>
>>>>>>>> However I have a doubt concerning how you handle the
>>>>>>>> begin/end for READ and WRITE. It seems that you open a
>>>>>>>> transaction based on txn.isInTransaction(), but how do
>>>>>>>> you know if it is a READ or WRITE?
>>>>>>>>
>>>>>>>> If you create something like:
>>>>>>>>
>>>>>>>> Txn.executeRead(dataset, { Txn.executeWrite(dataset, {
>>>>>>>> // Whatever } } }
>>>>>>>>
>>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and
>>>>>>>> therefore it might be leading to unexepected behaviours
>>>>>>>> for the txn.commit().
>>>>>>>>
>>>>>>>> could you give some hints on how this is handle
>>>>>>>> internally? Before fully modify the code I have, it
>>>>>>>> might be easier to replicate the txn behaviour ;) but I
>>>>>>>> would like to know the above (if possible).
>>>>>>>>
>>>>>>>> As always, thanks in advanced Jorge
>>>>>>>>
>>>>>>>> [1]:
>>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-28 18:02, dandh988 wrote:
> Can you give a complete example? How are you calling MultiUnion? Are
> you calling txn read on the dataset then building the union, then
> calling txn write to update the graph?
Let's describe a use case.
1) I have a dataset that includes 4 named graphs: G1, G2, G3, G4
2.a) Someone initiates a sparql request and the internal procedure followed is:
return Txn.calculateRead(dataset, () -> {
// Create multiunion of 3 namegraphs
MultiUnion union = new MultiUnion();
union.addGraph(dataset.getNamedModel("G1").getGraph());
union.addGraph(dataset.getNamedModel("G2").getGraph());
union.addGraph(dataset.getNamedModel("G4").getGraph());
Model m = ModelFactory.createModelForGraph(union);
// Launch Sparql query on it
try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
return ResultSetFactory.copyResults(qExec.execSelect())
}
});
2.b) Someone initiates a sparql request and the internal procedure followed is:
return Txn.calculateRead(dataset, () -> {
// Create multiunion of 2 namegraphs
// This code in in a function
MultiUnion union = new MultiUnion();
union.addGraph(dataset.getNamedModel("G1").getGraph());
union.addGraph(dataset.getNamedModel("G2").getGraph());
Model m1 = ModelFactory.createModelForGraph(union);
// Retrieve namegraph G3
// This code in in a function
Model m2 = dataset.getNamedModel("G3");
Model m = ModelFactory.createUnion(m2, m1);
// Launch Sparql query on it
try (QueryExecution qExec = QueryExecutionFactory.create(query, m)) {
return ResultSetFactory.copyResults(qExec.execSelect())
}
});
3) Someone initiaties a write in G2
// m stores the new entity model
Model m;
Txn.executeWrite(dataset, () -> {
dataset.getNamedModel("G2").add(m);
});
If either version of 2), and 3) are not done in parallel there is no problem and everything is executed correctly.
The problem arise when 2.a) or 2.b) is run, and before ending someone tries to perform 3). Then I get the FileException("In the middle of an alloc-write").
Do you have an idea on how to avoid this? How can I handle transactions in this model?
I was thinking on creating a global mutex so if any action is being performed over the dataset, then the rest would be blocked. The problem here is that the code is part of a webservice and then if the read/write operation lasts long, I will get a timeout that will close the connection.
The other option is to disable writing while someone is reading. The main problem here is how to properly reschedule writings not to have a big queue.
Any help is more than welcome. I don't know what else to do to solve this is issue, and the problem is making the service unusable :(
Thanks a lot for the great help you are all offering.
Jorge
>
>
> Dick -------- Original message --------From: George News
> <ge...@gmx.net> Date: 28/12/2017 13:17 (GMT+00:00) To:
> users@jena.apache.org Subject: Re: Txn code not handling type of
> transaction On 2017-12-27 19:32, Andy Seaborne wrote:
>>
>>
>> On 27/12/17 18:19, George News wrote:
>>> One think I have forgotten to mention is that I'm using a static
>>> variable to store the dataset reference. Is this important?
>>
>> No.
>>
>> The top layer of transaction code uses ThreadLocal variables.
>>
>> Internally, there are Transaction objects.
>>
>> Andy
>
> Another doubt on the issue with transactions. I'm generating
> Multiunions from the same dataset graphs. While in one thread I'm
> reading from a multiunion, on another thread I'm writing to one of
> the graphs included in the multiunion (standalone graph, not
> multiunion).
>
> I have modified my code to use Txn everywhere and still having
> issues. This is why I'm asking about how multiunions are handle
> concerning transactions.
>
>
>>>
>>>
>>> On 2017-12-27 19:13, George News wrote:
>>>> Just out of curiosity, just in case the problem is that
>>>> everything is done in the same thread. Do you know if Wildfly
>>>> is handling every request under the same thread? I guess not,
>>>> it will be really strange.
>>
>>>>
>>>> The point is that I have one REST endpoint for writing and
>>>> another for reading. Writing is done almost per 1 to 10
>>>> seconds. If I execute a reading that takes longer than that, I
>>>> get the exception on alloc-write.
>>>>
>>>> Thanks a lot. I just don't know why it is happening. Now I
>>>> think the code is using Txn (I have one point where I copied
>>>> the Txn.java behaviour) and still got the error.
>>>>
>>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>>> Then nesting is not safe as you might have open initially a
>>>>>> read transaction and then include a write. If the parent
>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>
>>>>> Actual nesting is not supported right now, period.
>>>>> Transactions in Jena are currently thread-local, so what you
>>>>> describe above I would see as a mistaken use of the API. If a
>>>>> client needs to move from a READ to a WRITE transaction, it's
>>>>> usually appropriate to close the transaction and open a new
>>>>> one (transactions in TIM, for instance, are snapshot
>>>>> isolated, and I believe the same is true of TDB2).
>>>>> Transactional objects _should not_ be passed from thread to
>>>>> thread with open transactions, unless additional client
>>>>> machinery is in place to manage those transactions, such as
>>>>> Dick has described in another thread (using thread proxies).
>>>>>
>>>>> Promotion machinery does exist in Jena but I am not aware
>>>>> that any of the dataset implementations actually support it
>>>>> right now. I could be wrong about that, since I didn't write
>>>>> the promotion code. Jena isn't a SQL database and doesn't
>>>>> offer the same kinds of guarantees or tradeoffs. Correctly
>>>>> promoting transactions in the absence of information about
>>>>> data dependencies is non-trivial.
>>>>>
>>>>> That having been said, it should be possible to add a
>>>>> "transaction type" method to transactional objects, within
>>>>> the "thread-local" design. Some dataset implementations
>>>>> already have one, e.g. DatasetGraphInMemory::transactionType.
>>>>> You might want to start by adding it to the
>>>>> o.a.j.sparql.core.Transactional interface and then "catching
>>>>> up" the implementation code to build up a PR.
>>>>>
>>>>> Adam Soroka
>>>>>
>>>>>> On Dec 27, 2017, at 6:53 AM, George News
>>>>>> <ge...@gmx.net> wrote:
>>>>>>
>>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>>> I recently wrote some code to try to handle a similar
>>>>>>> situation. In my case I knew I needed a transaction to be
>>>>>>> active at various points so I created a
>>>>>>> TransactionHolder. I create the holder and passing the
>>>>>>> object that has implements Transactional as well as the
>>>>>>> type of ReadWrite I want.
>>>>>>>
>>>>>>> If the transaction is active it does nothing (and I hope
>>>>>>> the proper transaction has been started) otherwise It
>>>>>>> starts the transaction. Ad the end I call commit or abort
>>>>>>> as appropriate. If I did not start the transaction the
>>>>>>> commit, abort or end is ignored.
>>>>>>
>>>>>> I see the same problem in your code as I pointed out
>>>>>> before. A transaction behaves differently if it is READ or
>>>>>> WRITE and in your code there is not such a thing. Actually
>>>>>> the isInTransaction() doesn't give you this information.
>>>>>>
>>>>>> Then nesting is not safe as you might have open initially a
>>>>>> read transaction and then include a write. If the parent
>>>>>> one is a write there shouldn't be such an issue (I guess).
>>>>>>
>>>>>> Just for you to know, your code is more or less integrated
>>>>>> in [1], so you can "update" your code.
>>>>>>
>>>>>> [1]:
>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>
>>>>>>
>>>>>>>
>>>>>>
I think there may be an issue with abort in that it should
>>>>>>> probablyset up end() to throw an exception when I have
>>>>>>> not created the transaction so that the outer
>>>>>>> transaction will fail.
>>>>>>>
>>>>>>> import org.apache.jena.query.ReadWrite; import
>>>>>>> org.apache.jena.sparql.core.Transactional;
>>>>>>>
>>>>>>> public class TransactionHolder { private final
>>>>>>> Transactional txn; private final boolean started; private
>>>>>>> final ReadWrite rw;
>>>>>>>
>>>>>>> public TransactionHolder( Transactional txn, ReadWrite rw
>>>>>>> ) { this.txn = txn; this.rw = rw; started = !
>>>>>>> txn.isInTransaction(); if (started) { txn.begin( rw ); }
>>>>>>> }
>>>>>>>
>>>>>>> public boolean ownsTranaction() { return started; }
>>>>>>>
>>>>>>> public void commit() { if (started) { txn.commit(); } }
>>>>>>>
>>>>>>> public void abort() { if (started) { txn.abort(); } }
>>>>>>>
>>>>>>> public void end() { if (started) { txn.end(); } }
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988
>>>>>>> <da...@gmail.com> wrote:
>>>>>>>
>>>>>>>> You cannot nest transactions nor can you promote a read
>>>>>>>> to a write. You need to rewrite your code or use txn
>>>>>>>> which correctly checks if a transaction is available
>>>>>>>> and if not will begin the correct one, either READ or
>>>>>>>> WRITE.
>>>>>>>>
>>>>>>>>
>>>>>>>> Dick -------- Original message --------From: George
>>>>>>>> News <ge...@gmx.net> Date: 27/12/2017 10:27
>>>>>>>> (GMT+00:00) To: Jena User Mailing List <
>>>>>>>> users@jena.apache.org> Subject: Txn code not handling
>>>>>>>> type of transaction Hi,
>>>>>>>>
>>>>>>>> As you know from other threads I'm having some issues
>>>>>>>> with transactions. Your suggestion is to use Txn
>>>>>>>> instead of begin/end. Just for curiosity I have checked
>>>>>>>> the Txn code at [1] and it seems that inside you use
>>>>>>>> begin/end.
>>>>>>>>
>>>>>>>> However I have a doubt concerning how you handle the
>>>>>>>> begin/end for READ and WRITE. It seems that you open a
>>>>>>>> transaction based on txn.isInTransaction(), but how do
>>>>>>>> you know if it is a READ or WRITE?
>>>>>>>>
>>>>>>>> If you create something like:
>>>>>>>>
>>>>>>>> Txn.executeRead(dataset, { Txn.executeWrite(dataset, {
>>>>>>>> // Whatever } } }
>>>>>>>>
>>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and
>>>>>>>> therefore it might be leading to unexepected behaviours
>>>>>>>> for the txn.commit().
>>>>>>>>
>>>>>>>> could you give some hints on how this is handle
>>>>>>>> internally? Before fully modify the code I have, it
>>>>>>>> might be easier to replicate the txn behaviour ;) but I
>>>>>>>> would like to know the above (if possible).
>>>>>>>>
>>>>>>>> As always, thanks in advanced Jorge
>>>>>>>>
>>>>>>>> [1]:
>>>>>>>> jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>
Re: Txn code not handling type of transaction
Posted by dandh988 <da...@gmail.com>.
Can you give a complete example? How are you calling MultiUnion? Are you calling txn read on the dataset then building the union, then calling txn write to update the graph?
Dick
-------- Original message --------From: George News <ge...@gmx.net> Date: 28/12/2017 13:17 (GMT+00:00) To: users@jena.apache.org Subject: Re: Txn code not handling type of transaction
On 2017-12-27 19:32, Andy Seaborne wrote:
>
>
> On 27/12/17 18:19, George News wrote:
>> One think I have forgotten to mention is that I'm using a static
>> variable to store the dataset reference. Is this important?
>
> No.
>
> The top layer of transaction code uses ThreadLocal variables.
>
> Internally, there are Transaction objects.
>
> Andy
Another doubt on the issue with transactions. I'm generating Multiunions
from the same dataset graphs. While in one thread I'm reading from a
multiunion, on another thread I'm writing to one of the graphs included
in the multiunion (standalone graph, not multiunion).
I have modified my code to use Txn everywhere and still having issues.
This is why I'm asking about how multiunions are handle concerning
transactions.
>>
>>
>> On 2017-12-27 19:13, George News wrote:
>>> Just out of curiosity, just in case the problem is that everything is
>>> done in the same thread. Do you know if Wildfly is handling every
>>> request under the same thread? I guess not, it will be really strange.
>
>>>
>>> The point is that I have one REST endpoint for writing and another for
>>> reading. Writing is done almost per 1 to 10 seconds. If I execute a
>>> reading that takes longer than that, I get the exception on alloc-write.
>>>
>>> Thanks a lot. I just don't know why it is happening. Now I think the
>>> code is using Txn (I have one point where I copied the Txn.java
>>> behaviour) and still got the error.
>>>
>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>> Then nesting is not safe as you might have open initially a read
>>>>> transaction and then include a write. If the parent one is a write
>>>>> there
>>>>> shouldn't be such an issue (I guess).
>>>>
>>>> Actual nesting is not supported right now, period. Transactions in
>>>> Jena are currently thread-local, so what you describe above I would
>>>> see as a mistaken use of the API. If a client needs to move from a
>>>> READ to a WRITE transaction, it's usually appropriate to close the
>>>> transaction and open a new one (transactions in TIM, for instance,
>>>> are snapshot isolated, and I believe the same is true of TDB2).
>>>> Transactional objects _should not_ be passed from thread to thread
>>>> with open transactions, unless additional client machinery is in
>>>> place to manage those transactions, such as Dick has described in
>>>> another thread (using thread proxies).
>>>>
>>>> Promotion machinery does exist in Jena but I am not aware that any
>>>> of the dataset implementations actually support it right now. I
>>>> could be wrong about that, since I didn't write the promotion code.
>>>> Jena isn't a SQL database and doesn't offer the same kinds of
>>>> guarantees or tradeoffs. Correctly promoting transactions in the
>>>> absence of information about data dependencies is non-trivial.
>>>>
>>>> That having been said, it should be possible to add a "transaction
>>>> type" method to transactional objects, within the "thread-local"
>>>> design. Some dataset implementations already have one, e.g.
>>>> DatasetGraphInMemory::transactionType. You might want to start by
>>>> adding it to the o.a.j.sparql.core.Transactional interface and then
>>>> "catching up" the implementation code to build up a PR.
>>>>
>>>> Adam Soroka
>>>>
>>>>> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>>>>>
>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>> I recently wrote some code to try to handle a similar situation.
>>>>>> In my
>>>>>> case I knew I needed a transaction to be active at various points
>>>>>> so I
>>>>>> created a TransactionHolder. I create the holder and passing the
>>>>>> object
>>>>>> that has implements Transactional as well as the type of ReadWrite
>>>>>> I want.
>>>>>>
>>>>>> If the transaction is active it does nothing (and I hope the proper
>>>>>> transaction has been started) otherwise It starts the transaction.
>>>>>> Ad the end I call commit or abort as appropriate. If I did not
>>>>>> start the
>>>>>> transaction the commit, abort or end is ignored.
>>>>>
>>>>> I see the same problem in your code as I pointed out before. A
>>>>> transaction behaves differently if it is READ or WRITE and in your
>>>>> code
>>>>> there is not such a thing. Actually the isInTransaction() doesn't give
>>>>> you this information.
>>>>>
>>>>> Then nesting is not safe as you might have open initially a read
>>>>> transaction and then include a write. If the parent one is a write
>>>>> there
>>>>> shouldn't be such an issue (I guess).
>>>>>
>>>>> Just for you to know, your code is more or less integrated in [1], so
>>>>> you can "update" your code.
>>>>>
>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>
>>>>>
>>>>>> I think there may be an issue with abort in that it should
>>>>>> probablyset up
>>>>>> end() to throw an exception when I have not created the
>>>>>> transaction so
>>>>>> that the outer transaction will fail.
>>>>>>
>>>>>> import org.apache.jena.query.ReadWrite;
>>>>>> import org.apache.jena.sparql.core.Transactional;
>>>>>>
>>>>>> public class TransactionHolder {
>>>>>> private final Transactional txn;
>>>>>> private final boolean started;
>>>>>> private final ReadWrite rw;
>>>>>>
>>>>>> public TransactionHolder( Transactional txn, ReadWrite rw )
>>>>>> {
>>>>>> this.txn = txn;
>>>>>> this.rw = rw;
>>>>>> started = ! txn.isInTransaction();
>>>>>> if (started)
>>>>>> {
>>>>>> txn.begin( rw );
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> public boolean ownsTranaction() {
>>>>>> return started;
>>>>>> }
>>>>>>
>>>>>> public void commit() {
>>>>>> if (started) {
>>>>>> txn.commit();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> public void abort() {
>>>>>> if (started)
>>>>>> {
>>>>>> txn.abort();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> public void end() {
>>>>>> if (started) {
>>>>>> txn.end();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> You cannot nest transactions nor can you promote a read to a write.
>>>>>>> You need to rewrite your code or use txn which correctly checks if a
>>>>>>> transaction is available and if not will begin the correct one,
>>>>>>> either READ
>>>>>>> or WRITE.
>>>>>>>
>>>>>>>
>>>>>>> Dick
>>>>>>> -------- Original message --------From: George News
>>>>>>> <ge...@gmx.net>
>>>>>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>>>>> users@jena.apache.org> Subject: Txn code not handling type of
>>>>>>> transaction
>>>>>>> Hi,
>>>>>>>
>>>>>>> As you know from other threads I'm having some issues with
>>>>>>> transactions.
>>>>>>> Your suggestion is to use Txn instead of begin/end. Just for
>>>>>>> curiosity I
>>>>>>> have checked the Txn code at [1] and it seems that inside you use
>>>>>>> begin/end.
>>>>>>>
>>>>>>> However I have a doubt concerning how you handle the begin/end
>>>>>>> for READ
>>>>>>> and WRITE. It seems that you open a transaction based on
>>>>>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>>>>>
>>>>>>> If you create something like:
>>>>>>>
>>>>>>> Txn.executeRead(dataset, {
>>>>>>> Txn.executeWrite(dataset, {
>>>>>>> // Whatever
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it
>>>>>>> might be
>>>>>>> leading to unexepected behaviours for the txn.commit().
>>>>>>>
>>>>>>> could you give some hints on how this is handle internally?
>>>>>>> Before fully
>>>>>>> modify the code I have, it might be easier to replicate the txn
>>>>>>> behaviour ;) but I would like to know the above (if possible).
>>>>>>>
>>>>>>> As always, thanks in advanced
>>>>>>> Jorge
>>>>>>>
>>>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-27 19:32, Andy Seaborne wrote:
>
>
> On 27/12/17 18:19, George News wrote:
>> One think I have forgotten to mention is that I'm using a static
>> variable to store the dataset reference. Is this important?
>
> No.
>
> The top layer of transaction code uses ThreadLocal variables.
>
> Internally, there are Transaction objects.
>
> Andy
Another doubt on the issue with transactions. I'm generating Multiunions
from the same dataset graphs. While in one thread I'm reading from a
multiunion, on another thread I'm writing to one of the graphs included
in the multiunion (standalone graph, not multiunion).
I have modified my code to use Txn everywhere and still having issues.
This is why I'm asking about how multiunions are handle concerning
transactions.
>>
>>
>> On 2017-12-27 19:13, George News wrote:
>>> Just out of curiosity, just in case the problem is that everything is
>>> done in the same thread. Do you know if Wildfly is handling every
>>> request under the same thread? I guess not, it will be really strange.
>
>>>
>>> The point is that I have one REST endpoint for writing and another for
>>> reading. Writing is done almost per 1 to 10 seconds. If I execute a
>>> reading that takes longer than that, I get the exception on alloc-write.
>>>
>>> Thanks a lot. I just don't know why it is happening. Now I think the
>>> code is using Txn (I have one point where I copied the Txn.java
>>> behaviour) and still got the error.
>>>
>>> On 2017-12-27 17:18, ajs6f wrote:
>>>>> Then nesting is not safe as you might have open initially a read
>>>>> transaction and then include a write. If the parent one is a write
>>>>> there
>>>>> shouldn't be such an issue (I guess).
>>>>
>>>> Actual nesting is not supported right now, period. Transactions in
>>>> Jena are currently thread-local, so what you describe above I would
>>>> see as a mistaken use of the API. If a client needs to move from a
>>>> READ to a WRITE transaction, it's usually appropriate to close the
>>>> transaction and open a new one (transactions in TIM, for instance,
>>>> are snapshot isolated, and I believe the same is true of TDB2).
>>>> Transactional objects _should not_ be passed from thread to thread
>>>> with open transactions, unless additional client machinery is in
>>>> place to manage those transactions, such as Dick has described in
>>>> another thread (using thread proxies).
>>>>
>>>> Promotion machinery does exist in Jena but I am not aware that any
>>>> of the dataset implementations actually support it right now. I
>>>> could be wrong about that, since I didn't write the promotion code.
>>>> Jena isn't a SQL database and doesn't offer the same kinds of
>>>> guarantees or tradeoffs. Correctly promoting transactions in the
>>>> absence of information about data dependencies is non-trivial.
>>>>
>>>> That having been said, it should be possible to add a "transaction
>>>> type" method to transactional objects, within the "thread-local"
>>>> design. Some dataset implementations already have one, e.g.
>>>> DatasetGraphInMemory::transactionType. You might want to start by
>>>> adding it to the o.a.j.sparql.core.Transactional interface and then
>>>> "catching up" the implementation code to build up a PR.
>>>>
>>>> Adam Soroka
>>>>
>>>>> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>>>>>
>>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>>> I recently wrote some code to try to handle a similar situation.
>>>>>> In my
>>>>>> case I knew I needed a transaction to be active at various points
>>>>>> so I
>>>>>> created a TransactionHolder. I create the holder and passing the
>>>>>> object
>>>>>> that has implements Transactional as well as the type of ReadWrite
>>>>>> I want.
>>>>>>
>>>>>> If the transaction is active it does nothing (and I hope the proper
>>>>>> transaction has been started) otherwise It starts the transaction.
>>>>>> Ad the end I call commit or abort as appropriate. If I did not
>>>>>> start the
>>>>>> transaction the commit, abort or end is ignored.
>>>>>
>>>>> I see the same problem in your code as I pointed out before. A
>>>>> transaction behaves differently if it is READ or WRITE and in your
>>>>> code
>>>>> there is not such a thing. Actually the isInTransaction() doesn't give
>>>>> you this information.
>>>>>
>>>>> Then nesting is not safe as you might have open initially a read
>>>>> transaction and then include a write. If the parent one is a write
>>>>> there
>>>>> shouldn't be such an issue (I guess).
>>>>>
>>>>> Just for you to know, your code is more or less integrated in [1], so
>>>>> you can "update" your code.
>>>>>
>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>
>>>>>
>>>>>> I think there may be an issue with abort in that it should
>>>>>> probablyset up
>>>>>> end() to throw an exception when I have not created the
>>>>>> transaction so
>>>>>> that the outer transaction will fail.
>>>>>>
>>>>>> import org.apache.jena.query.ReadWrite;
>>>>>> import org.apache.jena.sparql.core.Transactional;
>>>>>>
>>>>>> public class TransactionHolder {
>>>>>> private final Transactional txn;
>>>>>> private final boolean started;
>>>>>> private final ReadWrite rw;
>>>>>>
>>>>>> public TransactionHolder( Transactional txn, ReadWrite rw )
>>>>>> {
>>>>>> this.txn = txn;
>>>>>> this.rw = rw;
>>>>>> started = ! txn.isInTransaction();
>>>>>> if (started)
>>>>>> {
>>>>>> txn.begin( rw );
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> public boolean ownsTranaction() {
>>>>>> return started;
>>>>>> }
>>>>>>
>>>>>> public void commit() {
>>>>>> if (started) {
>>>>>> txn.commit();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> public void abort() {
>>>>>> if (started)
>>>>>> {
>>>>>> txn.abort();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> public void end() {
>>>>>> if (started) {
>>>>>> txn.end();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> You cannot nest transactions nor can you promote a read to a write.
>>>>>>> You need to rewrite your code or use txn which correctly checks if a
>>>>>>> transaction is available and if not will begin the correct one,
>>>>>>> either READ
>>>>>>> or WRITE.
>>>>>>>
>>>>>>>
>>>>>>> Dick
>>>>>>> -------- Original message --------From: George News
>>>>>>> <ge...@gmx.net>
>>>>>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>>>>> users@jena.apache.org> Subject: Txn code not handling type of
>>>>>>> transaction
>>>>>>> Hi,
>>>>>>>
>>>>>>> As you know from other threads I'm having some issues with
>>>>>>> transactions.
>>>>>>> Your suggestion is to use Txn instead of begin/end. Just for
>>>>>>> curiosity I
>>>>>>> have checked the Txn code at [1] and it seems that inside you use
>>>>>>> begin/end.
>>>>>>>
>>>>>>> However I have a doubt concerning how you handle the begin/end
>>>>>>> for READ
>>>>>>> and WRITE. It seems that you open a transaction based on
>>>>>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>>>>>
>>>>>>> If you create something like:
>>>>>>>
>>>>>>> Txn.executeRead(dataset, {
>>>>>>> Txn.executeWrite(dataset, {
>>>>>>> // Whatever
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it
>>>>>>> might be
>>>>>>> leading to unexepected behaviours for the txn.commit().
>>>>>>>
>>>>>>> could you give some hints on how this is handle internally?
>>>>>>> Before fully
>>>>>>> modify the code I have, it might be easier to replicate the txn
>>>>>>> behaviour ;) but I would like to know the above (if possible).
>>>>>>>
>>>>>>> As always, thanks in advanced
>>>>>>> Jorge
>>>>>>>
>>>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>
Re: Txn code not handling type of transaction
Posted by Andy Seaborne <an...@apache.org>.
On 27/12/17 18:19, George News wrote:
> One think I have forgotten to mention is that I'm using a static
> variable to store the dataset reference. Is this important?
No.
The top layer of transaction code uses ThreadLocal variables.
Internally, there are Transaction objects.
Andy
>
>
> On 2017-12-27 19:13, George News wrote:
>> Just out of curiosity, just in case the problem is that everything is
>> done in the same thread. Do you know if Wildfly is handling every
>> request under the same thread? I guess not, it will be really strange.
>>
>> The point is that I have one REST endpoint for writing and another for
>> reading. Writing is done almost per 1 to 10 seconds. If I execute a
>> reading that takes longer than that, I get the exception on alloc-write.
>>
>> Thanks a lot. I just don't know why it is happening. Now I think the
>> code is using Txn (I have one point where I copied the Txn.java
>> behaviour) and still got the error.
>>
>> On 2017-12-27 17:18, ajs6f wrote:
>>>> Then nesting is not safe as you might have open initially a read transaction and then include a write. If the parent one is a write there
>>>> shouldn't be such an issue (I guess).
>>>
>>> Actual nesting is not supported right now, period. Transactions in Jena are currently thread-local, so what you describe above I would see as a mistaken use of the API. If a client needs to move from a READ to a WRITE transaction, it's usually appropriate to close the transaction and open a new one (transactions in TIM, for instance, are snapshot isolated, and I believe the same is true of TDB2). Transactional objects _should not_ be passed from thread to thread with open transactions, unless additional client machinery is in place to manage those transactions, such as Dick has described in another thread (using thread proxies).
>>>
>>> Promotion machinery does exist in Jena but I am not aware that any of the dataset implementations actually support it right now. I could be wrong about that, since I didn't write the promotion code. Jena isn't a SQL database and doesn't offer the same kinds of guarantees or tradeoffs. Correctly promoting transactions in the absence of information about data dependencies is non-trivial.
>>>
>>> That having been said, it should be possible to add a "transaction type" method to transactional objects, within the "thread-local" design. Some dataset implementations already have one, e.g. DatasetGraphInMemory::transactionType. You might want to start by adding it to the o.a.j.sparql.core.Transactional interface and then "catching up" the implementation code to build up a PR.
>>>
>>> Adam Soroka
>>>
>>>> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>>>>
>>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>>> I recently wrote some code to try to handle a similar situation. In my
>>>>> case I knew I needed a transaction to be active at various points so I
>>>>> created a TransactionHolder. I create the holder and passing the object
>>>>> that has implements Transactional as well as the type of ReadWrite I want.
>>>>>
>>>>> If the transaction is active it does nothing (and I hope the proper
>>>>> transaction has been started) otherwise It starts the transaction.
>>>>> Ad the end I call commit or abort as appropriate. If I did not start the
>>>>> transaction the commit, abort or end is ignored.
>>>>
>>>> I see the same problem in your code as I pointed out before. A
>>>> transaction behaves differently if it is READ or WRITE and in your code
>>>> there is not such a thing. Actually the isInTransaction() doesn't give
>>>> you this information.
>>>>
>>>> Then nesting is not safe as you might have open initially a read
>>>> transaction and then include a write. If the parent one is a write there
>>>> shouldn't be such an issue (I guess).
>>>>
>>>> Just for you to know, your code is more or less integrated in [1], so
>>>> you can "update" your code.
>>>>
>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>
>>>>
>>>>> I think there may be an issue with abort in that it should probablyset up
>>>>> end() to throw an exception when I have not created the transaction so
>>>>> that the outer transaction will fail.
>>>>>
>>>>> import org.apache.jena.query.ReadWrite;
>>>>> import org.apache.jena.sparql.core.Transactional;
>>>>>
>>>>> public class TransactionHolder {
>>>>> private final Transactional txn;
>>>>> private final boolean started;
>>>>> private final ReadWrite rw;
>>>>>
>>>>> public TransactionHolder( Transactional txn, ReadWrite rw )
>>>>> {
>>>>> this.txn = txn;
>>>>> this.rw = rw;
>>>>> started = ! txn.isInTransaction();
>>>>> if (started)
>>>>> {
>>>>> txn.begin( rw );
>>>>> }
>>>>> }
>>>>>
>>>>> public boolean ownsTranaction() {
>>>>> return started;
>>>>> }
>>>>>
>>>>> public void commit() {
>>>>> if (started) {
>>>>> txn.commit();
>>>>> }
>>>>> }
>>>>>
>>>>> public void abort() {
>>>>> if (started)
>>>>> {
>>>>> txn.abort();
>>>>> }
>>>>> }
>>>>>
>>>>> public void end() {
>>>>> if (started) {
>>>>> txn.end();
>>>>> }
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
>>>>>
>>>>>> You cannot nest transactions nor can you promote a read to a write.
>>>>>> You need to rewrite your code or use txn which correctly checks if a
>>>>>> transaction is available and if not will begin the correct one, either READ
>>>>>> or WRITE.
>>>>>>
>>>>>>
>>>>>> Dick
>>>>>> -------- Original message --------From: George News <ge...@gmx.net>
>>>>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>>>> users@jena.apache.org> Subject: Txn code not handling type of transaction
>>>>>> Hi,
>>>>>>
>>>>>> As you know from other threads I'm having some issues with transactions.
>>>>>> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
>>>>>> have checked the Txn code at [1] and it seems that inside you use
>>>>>> begin/end.
>>>>>>
>>>>>> However I have a doubt concerning how you handle the begin/end for READ
>>>>>> and WRITE. It seems that you open a transaction based on
>>>>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>>>>
>>>>>> If you create something like:
>>>>>>
>>>>>> Txn.executeRead(dataset, {
>>>>>> Txn.executeWrite(dataset, {
>>>>>> // Whatever
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
>>>>>> leading to unexepected behaviours for the txn.commit().
>>>>>>
>>>>>> could you give some hints on how this is handle internally? Before fully
>>>>>> modify the code I have, it might be easier to replicate the txn
>>>>>> behaviour ;) but I would like to know the above (if possible).
>>>>>>
>>>>>> As always, thanks in advanced
>>>>>> Jorge
>>>>>>
>>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
One think I have forgotten to mention is that I'm using a static
variable to store the dataset reference. Is this important?
On 2017-12-27 19:13, George News wrote:
> Just out of curiosity, just in case the problem is that everything is
> done in the same thread. Do you know if Wildfly is handling every
> request under the same thread? I guess not, it will be really strange.
>
> The point is that I have one REST endpoint for writing and another for
> reading. Writing is done almost per 1 to 10 seconds. If I execute a
> reading that takes longer than that, I get the exception on alloc-write.
>
> Thanks a lot. I just don't know why it is happening. Now I think the
> code is using Txn (I have one point where I copied the Txn.java
> behaviour) and still got the error.
>
> On 2017-12-27 17:18, ajs6f wrote:
>>> Then nesting is not safe as you might have open initially a read transaction and then include a write. If the parent one is a write there
>>> shouldn't be such an issue (I guess).
>>
>> Actual nesting is not supported right now, period. Transactions in Jena are currently thread-local, so what you describe above I would see as a mistaken use of the API. If a client needs to move from a READ to a WRITE transaction, it's usually appropriate to close the transaction and open a new one (transactions in TIM, for instance, are snapshot isolated, and I believe the same is true of TDB2). Transactional objects _should not_ be passed from thread to thread with open transactions, unless additional client machinery is in place to manage those transactions, such as Dick has described in another thread (using thread proxies).
>>
>> Promotion machinery does exist in Jena but I am not aware that any of the dataset implementations actually support it right now. I could be wrong about that, since I didn't write the promotion code. Jena isn't a SQL database and doesn't offer the same kinds of guarantees or tradeoffs. Correctly promoting transactions in the absence of information about data dependencies is non-trivial.
>>
>> That having been said, it should be possible to add a "transaction type" method to transactional objects, within the "thread-local" design. Some dataset implementations already have one, e.g. DatasetGraphInMemory::transactionType. You might want to start by adding it to the o.a.j.sparql.core.Transactional interface and then "catching up" the implementation code to build up a PR.
>>
>> Adam Soroka
>>
>>> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>>>
>>> On 2017-12-27 12:29, Claude Warren wrote:
>>>> I recently wrote some code to try to handle a similar situation. In my
>>>> case I knew I needed a transaction to be active at various points so I
>>>> created a TransactionHolder. I create the holder and passing the object
>>>> that has implements Transactional as well as the type of ReadWrite I want.
>>>>
>>>> If the transaction is active it does nothing (and I hope the proper
>>>> transaction has been started) otherwise It starts the transaction.
>>>> Ad the end I call commit or abort as appropriate. If I did not start the
>>>> transaction the commit, abort or end is ignored.
>>>
>>> I see the same problem in your code as I pointed out before. A
>>> transaction behaves differently if it is READ or WRITE and in your code
>>> there is not such a thing. Actually the isInTransaction() doesn't give
>>> you this information.
>>>
>>> Then nesting is not safe as you might have open initially a read
>>> transaction and then include a write. If the parent one is a write there
>>> shouldn't be such an issue (I guess).
>>>
>>> Just for you to know, your code is more or less integrated in [1], so
>>> you can "update" your code.
>>>
>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>
>>>
>>>> I think there may be an issue with abort in that it should probablyset up
>>>> end() to throw an exception when I have not created the transaction so
>>>> that the outer transaction will fail.
>>>>
>>>> import org.apache.jena.query.ReadWrite;
>>>> import org.apache.jena.sparql.core.Transactional;
>>>>
>>>> public class TransactionHolder {
>>>> private final Transactional txn;
>>>> private final boolean started;
>>>> private final ReadWrite rw;
>>>>
>>>> public TransactionHolder( Transactional txn, ReadWrite rw )
>>>> {
>>>> this.txn = txn;
>>>> this.rw = rw;
>>>> started = ! txn.isInTransaction();
>>>> if (started)
>>>> {
>>>> txn.begin( rw );
>>>> }
>>>> }
>>>>
>>>> public boolean ownsTranaction() {
>>>> return started;
>>>> }
>>>>
>>>> public void commit() {
>>>> if (started) {
>>>> txn.commit();
>>>> }
>>>> }
>>>>
>>>> public void abort() {
>>>> if (started)
>>>> {
>>>> txn.abort();
>>>> }
>>>> }
>>>>
>>>> public void end() {
>>>> if (started) {
>>>> txn.end();
>>>> }
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
>>>>
>>>>> You cannot nest transactions nor can you promote a read to a write.
>>>>> You need to rewrite your code or use txn which correctly checks if a
>>>>> transaction is available and if not will begin the correct one, either READ
>>>>> or WRITE.
>>>>>
>>>>>
>>>>> Dick
>>>>> -------- Original message --------From: George News <ge...@gmx.net>
>>>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>>> users@jena.apache.org> Subject: Txn code not handling type of transaction
>>>>> Hi,
>>>>>
>>>>> As you know from other threads I'm having some issues with transactions.
>>>>> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
>>>>> have checked the Txn code at [1] and it seems that inside you use
>>>>> begin/end.
>>>>>
>>>>> However I have a doubt concerning how you handle the begin/end for READ
>>>>> and WRITE. It seems that you open a transaction based on
>>>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>>>
>>>>> If you create something like:
>>>>>
>>>>> Txn.executeRead(dataset, {
>>>>> Txn.executeWrite(dataset, {
>>>>> // Whatever
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
>>>>> leading to unexepected behaviours for the txn.commit().
>>>>>
>>>>> could you give some hints on how this is handle internally? Before fully
>>>>> modify the code I have, it might be easier to replicate the txn
>>>>> behaviour ;) but I would like to know the above (if possible).
>>>>>
>>>>> As always, thanks in advanced
>>>>> Jorge
>>>>>
>>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>>
>>>>
>>>>
>>>>
>>
>>
>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
Just out of curiosity, just in case the problem is that everything is
done in the same thread. Do you know if Wildfly is handling every
request under the same thread? I guess not, it will be really strange.
The point is that I have one REST endpoint for writing and another for
reading. Writing is done almost per 1 to 10 seconds. If I execute a
reading that takes longer than that, I get the exception on alloc-write.
Thanks a lot. I just don't know why it is happening. Now I think the
code is using Txn (I have one point where I copied the Txn.java
behaviour) and still got the error.
On 2017-12-27 17:18, ajs6f wrote:
>> Then nesting is not safe as you might have open initially a read transaction and then include a write. If the parent one is a write there
>> shouldn't be such an issue (I guess).
>
> Actual nesting is not supported right now, period. Transactions in Jena are currently thread-local, so what you describe above I would see as a mistaken use of the API. If a client needs to move from a READ to a WRITE transaction, it's usually appropriate to close the transaction and open a new one (transactions in TIM, for instance, are snapshot isolated, and I believe the same is true of TDB2). Transactional objects _should not_ be passed from thread to thread with open transactions, unless additional client machinery is in place to manage those transactions, such as Dick has described in another thread (using thread proxies).
>
> Promotion machinery does exist in Jena but I am not aware that any of the dataset implementations actually support it right now. I could be wrong about that, since I didn't write the promotion code. Jena isn't a SQL database and doesn't offer the same kinds of guarantees or tradeoffs. Correctly promoting transactions in the absence of information about data dependencies is non-trivial.
>
> That having been said, it should be possible to add a "transaction type" method to transactional objects, within the "thread-local" design. Some dataset implementations already have one, e.g. DatasetGraphInMemory::transactionType. You might want to start by adding it to the o.a.j.sparql.core.Transactional interface and then "catching up" the implementation code to build up a PR.
>
> Adam Soroka
>
>> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>>
>> On 2017-12-27 12:29, Claude Warren wrote:
>>> I recently wrote some code to try to handle a similar situation. In my
>>> case I knew I needed a transaction to be active at various points so I
>>> created a TransactionHolder. I create the holder and passing the object
>>> that has implements Transactional as well as the type of ReadWrite I want.
>>>
>>> If the transaction is active it does nothing (and I hope the proper
>>> transaction has been started) otherwise It starts the transaction.
>>> Ad the end I call commit or abort as appropriate. If I did not start the
>>> transaction the commit, abort or end is ignored.
>>
>> I see the same problem in your code as I pointed out before. A
>> transaction behaves differently if it is READ or WRITE and in your code
>> there is not such a thing. Actually the isInTransaction() doesn't give
>> you this information.
>>
>> Then nesting is not safe as you might have open initially a read
>> transaction and then include a write. If the parent one is a write there
>> shouldn't be such an issue (I guess).
>>
>> Just for you to know, your code is more or less integrated in [1], so
>> you can "update" your code.
>>
>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>
>>
>>> I think there may be an issue with abort in that it should probablyset up
>>> end() to throw an exception when I have not created the transaction so
>>> that the outer transaction will fail.
>>>
>>> import org.apache.jena.query.ReadWrite;
>>> import org.apache.jena.sparql.core.Transactional;
>>>
>>> public class TransactionHolder {
>>> private final Transactional txn;
>>> private final boolean started;
>>> private final ReadWrite rw;
>>>
>>> public TransactionHolder( Transactional txn, ReadWrite rw )
>>> {
>>> this.txn = txn;
>>> this.rw = rw;
>>> started = ! txn.isInTransaction();
>>> if (started)
>>> {
>>> txn.begin( rw );
>>> }
>>> }
>>>
>>> public boolean ownsTranaction() {
>>> return started;
>>> }
>>>
>>> public void commit() {
>>> if (started) {
>>> txn.commit();
>>> }
>>> }
>>>
>>> public void abort() {
>>> if (started)
>>> {
>>> txn.abort();
>>> }
>>> }
>>>
>>> public void end() {
>>> if (started) {
>>> txn.end();
>>> }
>>> }
>>>
>>> }
>>>
>>>
>>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
>>>
>>>> You cannot nest transactions nor can you promote a read to a write.
>>>> You need to rewrite your code or use txn which correctly checks if a
>>>> transaction is available and if not will begin the correct one, either READ
>>>> or WRITE.
>>>>
>>>>
>>>> Dick
>>>> -------- Original message --------From: George News <ge...@gmx.net>
>>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>>> users@jena.apache.org> Subject: Txn code not handling type of transaction
>>>> Hi,
>>>>
>>>> As you know from other threads I'm having some issues with transactions.
>>>> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
>>>> have checked the Txn code at [1] and it seems that inside you use
>>>> begin/end.
>>>>
>>>> However I have a doubt concerning how you handle the begin/end for READ
>>>> and WRITE. It seems that you open a transaction based on
>>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>>
>>>> If you create something like:
>>>>
>>>> Txn.executeRead(dataset, {
>>>> Txn.executeWrite(dataset, {
>>>> // Whatever
>>>> }
>>>> }
>>>> }
>>>>
>>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
>>>> leading to unexepected behaviours for the txn.commit().
>>>>
>>>> could you give some hints on how this is handle internally? Before fully
>>>> modify the code I have, it might be easier to replicate the txn
>>>> behaviour ;) but I would like to know the above (if possible).
>>>>
>>>> As always, thanks in advanced
>>>> Jorge
>>>>
>>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>>
>>>
>>>
>>>
>
>
Re: Txn code not handling type of transaction
Posted by ajs6f <aj...@apache.org>.
> Then nesting is not safe as you might have open initially a read transaction and then include a write. If the parent one is a write there
> shouldn't be such an issue (I guess).
Actual nesting is not supported right now, period. Transactions in Jena are currently thread-local, so what you describe above I would see as a mistaken use of the API. If a client needs to move from a READ to a WRITE transaction, it's usually appropriate to close the transaction and open a new one (transactions in TIM, for instance, are snapshot isolated, and I believe the same is true of TDB2). Transactional objects _should not_ be passed from thread to thread with open transactions, unless additional client machinery is in place to manage those transactions, such as Dick has described in another thread (using thread proxies).
Promotion machinery does exist in Jena but I am not aware that any of the dataset implementations actually support it right now. I could be wrong about that, since I didn't write the promotion code. Jena isn't a SQL database and doesn't offer the same kinds of guarantees or tradeoffs. Correctly promoting transactions in the absence of information about data dependencies is non-trivial.
That having been said, it should be possible to add a "transaction type" method to transactional objects, within the "thread-local" design. Some dataset implementations already have one, e.g. DatasetGraphInMemory::transactionType. You might want to start by adding it to the o.a.j.sparql.core.Transactional interface and then "catching up" the implementation code to build up a PR.
Adam Soroka
> On Dec 27, 2017, at 6:53 AM, George News <ge...@gmx.net> wrote:
>
> On 2017-12-27 12:29, Claude Warren wrote:
>> I recently wrote some code to try to handle a similar situation. In my
>> case I knew I needed a transaction to be active at various points so I
>> created a TransactionHolder. I create the holder and passing the object
>> that has implements Transactional as well as the type of ReadWrite I want.
>>
>> If the transaction is active it does nothing (and I hope the proper
>> transaction has been started) otherwise It starts the transaction.
>> Ad the end I call commit or abort as appropriate. If I did not start the
>> transaction the commit, abort or end is ignored.
>
> I see the same problem in your code as I pointed out before. A
> transaction behaves differently if it is READ or WRITE and in your code
> there is not such a thing. Actually the isInTransaction() doesn't give
> you this information.
>
> Then nesting is not safe as you might have open initially a read
> transaction and then include a write. If the parent one is a write there
> shouldn't be such an issue (I guess).
>
> Just for you to know, your code is more or less integrated in [1], so
> you can "update" your code.
>
> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>
>
>> I think there may be an issue with abort in that it should probablyset up
>> end() to throw an exception when I have not created the transaction so
>> that the outer transaction will fail.
>>
>> import org.apache.jena.query.ReadWrite;
>> import org.apache.jena.sparql.core.Transactional;
>>
>> public class TransactionHolder {
>> private final Transactional txn;
>> private final boolean started;
>> private final ReadWrite rw;
>>
>> public TransactionHolder( Transactional txn, ReadWrite rw )
>> {
>> this.txn = txn;
>> this.rw = rw;
>> started = ! txn.isInTransaction();
>> if (started)
>> {
>> txn.begin( rw );
>> }
>> }
>>
>> public boolean ownsTranaction() {
>> return started;
>> }
>>
>> public void commit() {
>> if (started) {
>> txn.commit();
>> }
>> }
>>
>> public void abort() {
>> if (started)
>> {
>> txn.abort();
>> }
>> }
>>
>> public void end() {
>> if (started) {
>> txn.end();
>> }
>> }
>>
>> }
>>
>>
>> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
>>
>>> You cannot nest transactions nor can you promote a read to a write.
>>> You need to rewrite your code or use txn which correctly checks if a
>>> transaction is available and if not will begin the correct one, either READ
>>> or WRITE.
>>>
>>>
>>> Dick
>>> -------- Original message --------From: George News <ge...@gmx.net>
>>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>>> users@jena.apache.org> Subject: Txn code not handling type of transaction
>>> Hi,
>>>
>>> As you know from other threads I'm having some issues with transactions.
>>> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
>>> have checked the Txn code at [1] and it seems that inside you use
>>> begin/end.
>>>
>>> However I have a doubt concerning how you handle the begin/end for READ
>>> and WRITE. It seems that you open a transaction based on
>>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>>
>>> If you create something like:
>>>
>>> Txn.executeRead(dataset, {
>>> Txn.executeWrite(dataset, {
>>> // Whatever
>>> }
>>> }
>>> }
>>>
>>> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
>>> leading to unexepected behaviours for the txn.commit().
>>>
>>> could you give some hints on how this is handle internally? Before fully
>>> modify the code I have, it might be easier to replicate the txn
>>> behaviour ;) but I would like to know the above (if possible).
>>>
>>> As always, thanks in advanced
>>> Jorge
>>>
>>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>>
>>
>>
>>
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-27 12:29, Claude Warren wrote:
> I recently wrote some code to try to handle a similar situation. In my
> case I knew I needed a transaction to be active at various points so I
> created a TransactionHolder. I create the holder and passing the object
> that has implements Transactional as well as the type of ReadWrite I want.
>
> If the transaction is active it does nothing (and I hope the proper
> transaction has been started) otherwise It starts the transaction.
> Ad the end I call commit or abort as appropriate. If I did not start the
> transaction the commit, abort or end is ignored.
I see the same problem in your code as I pointed out before. A
transaction behaves differently if it is READ or WRITE and in your code
there is not such a thing. Actually the isInTransaction() doesn't give
you this information.
Then nesting is not safe as you might have open initially a read
transaction and then include a write. If the parent one is a write there
shouldn't be such an issue (I guess).
Just for you to know, your code is more or less integrated in [1], so
you can "update" your code.
[1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
> I think there may be an issue with abort in that it should probablyset up
> end() to throw an exception when I have not created the transaction so
> that the outer transaction will fail.
>
> import org.apache.jena.query.ReadWrite;
> import org.apache.jena.sparql.core.Transactional;
>
> public class TransactionHolder {
> private final Transactional txn;
> private final boolean started;
> private final ReadWrite rw;
>
> public TransactionHolder( Transactional txn, ReadWrite rw )
> {
> this.txn = txn;
> this.rw = rw;
> started = ! txn.isInTransaction();
> if (started)
> {
> txn.begin( rw );
> }
> }
>
> public boolean ownsTranaction() {
> return started;
> }
>
> public void commit() {
> if (started) {
> txn.commit();
> }
> }
>
> public void abort() {
> if (started)
> {
> txn.abort();
> }
> }
>
> public void end() {
> if (started) {
> txn.end();
> }
> }
>
> }
>
>
> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
>
>> You cannot nest transactions nor can you promote a read to a write.
>> You need to rewrite your code or use txn which correctly checks if a
>> transaction is available and if not will begin the correct one, either READ
>> or WRITE.
>>
>>
>> Dick
>> -------- Original message --------From: George News <ge...@gmx.net>
>> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
>> users@jena.apache.org> Subject: Txn code not handling type of transaction
>> Hi,
>>
>> As you know from other threads I'm having some issues with transactions.
>> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
>> have checked the Txn code at [1] and it seems that inside you use
>> begin/end.
>>
>> However I have a doubt concerning how you handle the begin/end for READ
>> and WRITE. It seems that you open a transaction based on
>> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>>
>> If you create something like:
>>
>> Txn.executeRead(dataset, {
>> Txn.executeWrite(dataset, {
>> // Whatever
>> }
>> }
>> }
>>
>> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
>> leading to unexepected behaviours for the txn.commit().
>>
>> could you give some hints on how this is handle internally? Before fully
>> modify the code I have, it might be easier to replicate the txn
>> behaviour ;) but I would like to know the above (if possible).
>>
>> As always, thanks in advanced
>> Jorge
>>
>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>
>
>
>
Re: Txn code not handling type of transaction
Posted by Claude Warren <cl...@xenei.com>.
While my code does not solve the problem of lock promotion it handles the
other cases.
Since there is no lock promotion there is no way to do what you want unless
you convert all read locks to write locks or build some sort of
wrapper/intercepter that will do some sort of release the lock and start
again if necessary. Sounds fraught with problems.
Claude
On Wed, Dec 27, 2017 at 12:02 PM, George News <ge...@gmx.net> wrote:
> On 2017-12-27 12:52, dandh988 wrote:
> > LOL. Similar here, except I wrapped the Transactional with a
> > Reenterent lock because I needed to try the begin. A side effect
> > being I can promote the underlying Jena Transactional if I can
> > promote the Reenterent lock. With a few caveats... There's a volatile
> > write timestamp which I check after promoting to ensure I don't end
> > up reading a different write transaction.
>
> Do you have example code? Just to have a guide to migrate the idea.
>
> >
> > Dick -------- Original message --------From: Claude Warren
> > <cl...@xenei.com> Date: 27/12/2017 11:29 (GMT+00:00) To:
> > users@jena.apache.org Subject: Re: Txn code not handling type of
> > transaction I recently wrote some code to try to handle a similar
> > situation. In my case I knew I needed a transaction to be active at
> > various points so I created a TransactionHolder. I create the holder
> > and passing the object that has implements Transactional as well as
> > the type of ReadWrite I want.
> >
> > If the transaction is active it does nothing (and I hope the proper
> > transaction has been started) otherwise It starts the transaction. Ad
> > the end I call commit or abort as appropriate. If I did not start
> > the transaction the commit, abort or end is ignored.
> >
> > I think there may be an issue with abort in that it should
> > probablyset up end() to throw an exception when I have not created
> > the transaction so that the outer transaction will fail.
> >
> > import org.apache.jena.query.ReadWrite; import
> > org.apache.jena.sparql.core.Transactional;
> >
> > public class TransactionHolder { private final Transactional txn;
> > private final boolean started; private final ReadWrite rw;
> >
> > public TransactionHolder( Transactional txn, ReadWrite rw ) {
> > this.txn = txn; this.rw = rw; started = ! txn.isInTransaction(); if
> > (started) { txn.begin( rw ); } }
> >
> > public boolean ownsTranaction() { return started; }
> >
> > public void commit() { if (started) { txn.commit(); } }
> >
> > public void abort() { if (started) { txn.abort(); } }
> >
> > public void end() { if (started) { txn.end(); } }
> >
> > }
> >
> >
> > On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com>
> > wrote:
> >
> >> You cannot nest transactions nor can you promote a read to a
> >> write. You need to rewrite your code or use txn which correctly
> >> checks if a transaction is available and if not will begin the
> >> correct one, either READ or WRITE.
> >>
> >>
> >> Dick -------- Original message --------From: George News
> >> <ge...@gmx.net> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena
> >> User Mailing List < users@jena.apache.org> Subject: Txn code not
> >> handling type of transaction Hi,
> >>
> >> As you know from other threads I'm having some issues with
> >> transactions. Your suggestion is to use Txn instead of begin/end.
> >> Just for curiosity I have checked the Txn code at [1] and it seems
> >> that inside you use begin/end.
> >>
> >> However I have a doubt concerning how you handle the begin/end for
> >> READ and WRITE. It seems that you open a transaction based on
> >> txn.isInTransaction(), but how do you know if it is a READ or
> >> WRITE?
> >>
> >> If you create something like:
> >>
> >> Txn.executeRead(dataset, { Txn.executeWrite(dataset, { // Whatever
> >> } } }
> >>
> >> the txn.begin(ReadWrite.WRITE) is not called and therefore it might
> >> be leading to unexepected behaviours for the txn.commit().
> >>
> >> could you give some hints on how this is handle internally? Before
> >> fully modify the code I have, it might be easier to replicate the
> >> txn behaviour ;) but I would like to know the above (if possible).
> >>
> >> As always, thanks in advanced Jorge
> >>
> >> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
> >>
> >
> >
> >
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-27 12:52, dandh988 wrote:
> LOL. Similar here, except I wrapped the Transactional with a
> Reenterent lock because I needed to try the begin. A side effect
> being I can promote the underlying Jena Transactional if I can
> promote the Reenterent lock. With a few caveats... There's a volatile
> write timestamp which I check after promoting to ensure I don't end
> up reading a different write transaction.
Do you have example code? Just to have a guide to migrate the idea.
>
> Dick -------- Original message --------From: Claude Warren
> <cl...@xenei.com> Date: 27/12/2017 11:29 (GMT+00:00) To:
> users@jena.apache.org Subject: Re: Txn code not handling type of
> transaction I recently wrote some code to try to handle a similar
> situation. In my case I knew I needed a transaction to be active at
> various points so I created a TransactionHolder. I create the holder
> and passing the object that has implements Transactional as well as
> the type of ReadWrite I want.
>
> If the transaction is active it does nothing (and I hope the proper
> transaction has been started) otherwise It starts the transaction. Ad
> the end I call commit or abort as appropriate. If I did not start
> the transaction the commit, abort or end is ignored.
>
> I think there may be an issue with abort in that it should
> probablyset up end() to throw an exception when I have not created
> the transaction so that the outer transaction will fail.
>
> import org.apache.jena.query.ReadWrite; import
> org.apache.jena.sparql.core.Transactional;
>
> public class TransactionHolder { private final Transactional txn;
> private final boolean started; private final ReadWrite rw;
>
> public TransactionHolder( Transactional txn, ReadWrite rw ) {
> this.txn = txn; this.rw = rw; started = ! txn.isInTransaction(); if
> (started) { txn.begin( rw ); } }
>
> public boolean ownsTranaction() { return started; }
>
> public void commit() { if (started) { txn.commit(); } }
>
> public void abort() { if (started) { txn.abort(); } }
>
> public void end() { if (started) { txn.end(); } }
>
> }
>
>
> On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com>
> wrote:
>
>> You cannot nest transactions nor can you promote a read to a
>> write. You need to rewrite your code or use txn which correctly
>> checks if a transaction is available and if not will begin the
>> correct one, either READ or WRITE.
>>
>>
>> Dick -------- Original message --------From: George News
>> <ge...@gmx.net> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena
>> User Mailing List < users@jena.apache.org> Subject: Txn code not
>> handling type of transaction Hi,
>>
>> As you know from other threads I'm having some issues with
>> transactions. Your suggestion is to use Txn instead of begin/end.
>> Just for curiosity I have checked the Txn code at [1] and it seems
>> that inside you use begin/end.
>>
>> However I have a doubt concerning how you handle the begin/end for
>> READ and WRITE. It seems that you open a transaction based on
>> txn.isInTransaction(), but how do you know if it is a READ or
>> WRITE?
>>
>> If you create something like:
>>
>> Txn.executeRead(dataset, { Txn.executeWrite(dataset, { // Whatever
>> } } }
>>
>> the txn.begin(ReadWrite.WRITE) is not called and therefore it might
>> be leading to unexepected behaviours for the txn.commit().
>>
>> could you give some hints on how this is handle internally? Before
>> fully modify the code I have, it might be easier to replicate the
>> txn behaviour ;) but I would like to know the above (if possible).
>>
>> As always, thanks in advanced Jorge
>>
>> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>>
>
>
>
Re: Txn code not handling type of transaction
Posted by dandh988 <da...@gmail.com>.
LOL. Similar here, except I wrapped the Transactional with a Reenterent lock because I needed to try the begin. A side effect being I can promote the underlying Jena Transactional if I can promote the Reenterent lock. With a few caveats... There's a volatile write timestamp which I check after promoting to ensure I don't end up reading a different write transaction.
Dick
-------- Original message --------From: Claude Warren <cl...@xenei.com> Date: 27/12/2017 11:29 (GMT+00:00) To: users@jena.apache.org Subject: Re: Txn code not handling type of transaction
I recently wrote some code to try to handle a similar situation. In my
case I knew I needed a transaction to be active at various points so I
created a TransactionHolder. I create the holder and passing the object
that has implements Transactional as well as the type of ReadWrite I want.
If the transaction is active it does nothing (and I hope the proper
transaction has been started) otherwise It starts the transaction.
Ad the end I call commit or abort as appropriate. If I did not start the
transaction the commit, abort or end is ignored.
I think there may be an issue with abort in that it should probablyset up
end() to throw an exception when I have not created the transaction so
that the outer transaction will fail.
import org.apache.jena.query.ReadWrite;
import org.apache.jena.sparql.core.Transactional;
public class TransactionHolder {
private final Transactional txn;
private final boolean started;
private final ReadWrite rw;
public TransactionHolder( Transactional txn, ReadWrite rw )
{
this.txn = txn;
this.rw = rw;
started = ! txn.isInTransaction();
if (started)
{
txn.begin( rw );
}
}
public boolean ownsTranaction() {
return started;
}
public void commit() {
if (started) {
txn.commit();
}
}
public void abort() {
if (started)
{
txn.abort();
}
}
public void end() {
if (started) {
txn.end();
}
}
}
On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
> You cannot nest transactions nor can you promote a read to a write.
> You need to rewrite your code or use txn which correctly checks if a
> transaction is available and if not will begin the correct one, either READ
> or WRITE.
>
>
> Dick
> -------- Original message --------From: George News <ge...@gmx.net>
> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
> users@jena.apache.org> Subject: Txn code not handling type of transaction
> Hi,
>
> As you know from other threads I'm having some issues with transactions.
> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
> have checked the Txn code at [1] and it seems that inside you use
> begin/end.
>
> However I have a doubt concerning how you handle the begin/end for READ
> and WRITE. It seems that you open a transaction based on
> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>
> If you create something like:
>
> Txn.executeRead(dataset, {
> Txn.executeWrite(dataset, {
> // Whatever
> }
> }
> }
>
> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
> leading to unexepected behaviours for the txn.commit().
>
> could you give some hints on how this is handle internally? Before fully
> modify the code I have, it might be easier to replicate the txn
> behaviour ;) but I would like to know the above (if possible).
>
> As always, thanks in advanced
> Jorge
>
> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Txn code not handling type of transaction
Posted by Claude Warren <cl...@xenei.com>.
I recently wrote some code to try to handle a similar situation. In my
case I knew I needed a transaction to be active at various points so I
created a TransactionHolder. I create the holder and passing the object
that has implements Transactional as well as the type of ReadWrite I want.
If the transaction is active it does nothing (and I hope the proper
transaction has been started) otherwise It starts the transaction.
Ad the end I call commit or abort as appropriate. If I did not start the
transaction the commit, abort or end is ignored.
I think there may be an issue with abort in that it should probablyset up
end() to throw an exception when I have not created the transaction so
that the outer transaction will fail.
import org.apache.jena.query.ReadWrite;
import org.apache.jena.sparql.core.Transactional;
public class TransactionHolder {
private final Transactional txn;
private final boolean started;
private final ReadWrite rw;
public TransactionHolder( Transactional txn, ReadWrite rw )
{
this.txn = txn;
this.rw = rw;
started = ! txn.isInTransaction();
if (started)
{
txn.begin( rw );
}
}
public boolean ownsTranaction() {
return started;
}
public void commit() {
if (started) {
txn.commit();
}
}
public void abort() {
if (started)
{
txn.abort();
}
}
public void end() {
if (started) {
txn.end();
}
}
}
On Wed, Dec 27, 2017 at 11:03 AM, dandh988 <da...@gmail.com> wrote:
> You cannot nest transactions nor can you promote a read to a write.
> You need to rewrite your code or use txn which correctly checks if a
> transaction is available and if not will begin the correct one, either READ
> or WRITE.
>
>
> Dick
> -------- Original message --------From: George News <ge...@gmx.net>
> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <
> users@jena.apache.org> Subject: Txn code not handling type of transaction
> Hi,
>
> As you know from other threads I'm having some issues with transactions.
> Your suggestion is to use Txn instead of begin/end. Just for curiosity I
> have checked the Txn code at [1] and it seems that inside you use
> begin/end.
>
> However I have a doubt concerning how you handle the begin/end for READ
> and WRITE. It seems that you open a transaction based on
> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>
> If you create something like:
>
> Txn.executeRead(dataset, {
> Txn.executeWrite(dataset, {
> // Whatever
> }
> }
> }
>
> the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
> leading to unexepected behaviours for the txn.commit().
>
> could you give some hints on how this is handle internally? Before fully
> modify the code I have, it might be easier to replicate the txn
> behaviour ;) but I would like to know the above (if possible).
>
> As always, thanks in advanced
> Jorge
>
> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Txn code not handling type of transaction
Posted by George News <ge...@gmx.net>.
On 2017-12-27 12:03, dandh988 wrote:
> You cannot nest transactions nor can you promote a read to a write.
> You need to rewrite your code or use txn which correctly checks if a
> transaction is available and if not will begin the correct one,
> either READ or WRITE.
If you have a look at the TXN code, there is no way to know if it is a read or a write.
Just some snipsets.
public static <T extends Transactional, X> X calculateRead(T txn, Supplier<X> r) {
boolean b = txn.isInTransaction() ;
if ( !b )
txn.begin(ReadWrite.READ) ;
public static <T extends Transactional> void executeWrite(T txn, Runnable r) {
boolean b = txn.isInTransaction() ;
if ( !b )
txn.begin(ReadWrite.WRITE) ;
And then if you go to isInTransaction() there is no way to know if read or write. It is just a boolean.
>
> Dick -------- Original message --------From: George News
> <ge...@gmx.net> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena
> User Mailing List <us...@jena.apache.org> Subject: Txn code not
> handling type of transaction Hi,
>
> As you know from other threads I'm having some issues with
> transactions. Your suggestion is to use Txn instead of begin/end.
> Just for curiosity I have checked the Txn code at [1] and it seems
> that inside you use begin/end.
>
> However I have a doubt concerning how you handle the begin/end for
> READ and WRITE. It seems that you open a transaction based on
> txn.isInTransaction(), but how do you know if it is a READ or WRITE?
>
> If you create something like:
>
> Txn.executeRead(dataset, { Txn.executeWrite(dataset, { // Whatever }
> } }
>
> the txn.begin(ReadWrite.WRITE) is not called and therefore it might
> be leading to unexepected behaviours for the txn.commit().
>
> could you give some hints on how this is handle internally? Before
> fully modify the code I have, it might be easier to replicate the
> txn behaviour ;) but I would like to know the above (if possible).
>
> As always, thanks in advanced Jorge
>
> [1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java
>
Re: Txn code not handling type of transaction
Posted by dandh988 <da...@gmail.com>.
You cannot nest transactions nor can you promote a read to a write.
You need to rewrite your code or use txn which correctly checks if a transaction is available and if not will begin the correct one, either READ or WRITE.
Dick
-------- Original message --------From: George News <ge...@gmx.net> Date: 27/12/2017 10:27 (GMT+00:00) To: Jena User Mailing List <us...@jena.apache.org> Subject: Txn code not handling type of transaction
Hi,
As you know from other threads I'm having some issues with transactions.
Your suggestion is to use Txn instead of begin/end. Just for curiosity I
have checked the Txn code at [1] and it seems that inside you use begin/end.
However I have a doubt concerning how you handle the begin/end for READ
and WRITE. It seems that you open a transaction based on
txn.isInTransaction(), but how do you know if it is a READ or WRITE?
If you create something like:
Txn.executeRead(dataset, {
Txn.executeWrite(dataset, {
// Whatever
}
}
}
the txn.begin(ReadWrite.WRITE) is not called and therefore it might be
leading to unexepected behaviours for the txn.commit().
could you give some hints on how this is handle internally? Before fully
modify the code I have, it might be easier to replicate the txn
behaviour ;) but I would like to know the above (if possible).
As always, thanks in advanced
Jorge
[1]: jena/jena-arq/src/main/java/org/apache/jena/system/Txn.java