You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "Simon Helsen (Created) (JIRA)" <ji...@apache.org> on 2011/10/20 16:18:10 UTC
[jira] [Created] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
-----------------------------------------------------------------------------------
Key: JENA-143
URL: https://issues.apache.org/jira/browse/JENA-143
Project: Jena
Issue Type: Bug
Components: TDB
Environment: tdb-0.9.0-20111010.121635
Reporter: Simon Helsen
The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
and then shortly after:
15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
DatasetGraphTxn dsGraph = null;
try {
dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
Dataset ds = dsGraph.toDataset();
...
QueryExecution qe = null;
...
try {
results = qe.execSelect();
...
} finally {
if (qe != null) {
qe.close();
}
}
} catch (QueryCancelledException e) {
if (dsGraph != null) {
dsGraph.abort();
}
} finally {
if (dsGraph != null) {
dsGraph.close();
}
}
A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Paolo Castagna (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paolo Castagna updated JENA-143:
--------------------------------
Summary: QueryExecution.abort seems to corrupt TDB when interrupting transactional queries (was: QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries)
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Simon Helsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154243#comment-13154243 ]
Simon Helsen commented on JENA-143:
-----------------------------------
Andy, while you are probably right about the test-patch I provided, I am not entirely sure if the original problem I uncovered is actually the same. That problem I have only been able to see with my own internal tests. Now, it seems that you are finding something in JENA-163, so I will wait for a fix there and then redo the whole thing to see if I still see the problem as provided it in the description
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Assignee: Andy Seaborne
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Andy Seaborne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132551#comment-13132551 ]
Andy Seaborne commented on JENA-143:
------------------------------------
(from JENA-131Transaction)
Please can we have a test case, not a fragment? From JENA-131, you've talked about heavy read/write concurrent access. If this the same? Is it always or sometimes?
It's really hard to track this down when the environment is unclear.
Abort should only set flags, and the clearup is done on the query thread itself.
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Simon Helsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150764#comment-13150764 ]
Simon Helsen commented on JENA-143:
-----------------------------------
In my attempt to adjust T_TransSystem, I was able to produce a problem by aborting queries, but I am not 100% positive if this is the same problem as I found in my own tests. Either way, I will attach a patch which should allow you to reproduce a problem. The console in my last run was the following (where I print CD to indicate an aborted describe query and CS to indicate an aborted select query):
START (disk[direct], 1000 iterations)
000: . CD CS CD . CS CD CS CS CS .. CD ... CD CD . CS CS CS CS . CD . CS .. CD . CD .. CS .. CD . CS .. CS .. CS CS CS . CD CD . CD CS CD . CS . CS .. CS ... CS . CD CS . CD CS CD CD ... CS CD . CD CS ... CD .. CD . CD CS . CS CS ..... CS . CD .. CS CS CD CD CS . CS .. CS .. CS CS CS CS ... CS CS ... CD CS ... CS CD CS . CS . CS CS . CS . CD CD ... CS CS . CS ... CD . CD CS . CD . CS CS . CS .. CD ... CS .. CS .. CD .. CS CD . CS . CS .. CD CS .. CS . CD CS .
CD CD CS CS CS 100: ... CD . CS CS .. CS CD .. CS CD ... CS CS .. CS CS ..... CS . CD CS . CD CS . CS . CD CS .. CS . CS . CD CD CS .... CD CS CD . CD CD . CD CS ..... CD .. CD . CS CS CS . CD . CS .. CS . CS CS . CS . CD .... CS CS .. CS . CD ... CD CS . CD .... CS . CS CS . CD .. CD ..... CS CS CS CS . CS CD . CS .. CS . CS .. CD CS . CS . CS CS CS CS .. CS CS CD .. CS .com.hp.hpl.jena.tdb.TDBException: Different ids for S: allocated: expected [000000000000000E], got [0000000000000000]
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.append(NodeTableTrans.java:178)
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.writeNodeJournal(NodeTableTrans.java:210)
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.commitPrepare(NodeTableTrans.java:190)
at com.hp.hpl.jena.tdb.transaction.Transaction.prepare(Transaction.java:108)
at com.hp.hpl.jena.tdb.transaction.Transaction.commit(Transaction.java:92)
at com.hp.hpl.jena.tdb.DatasetGraphTxn.commit(DatasetGraphTxn.java:38)
at com.hp.hpl.jena.tdb.transaction.T_TransSystem$Writer.call(T_TransSystem.java:278)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
at java.util.concurrent.FutureTask.run(FutureTask.java:149)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
at java.lang.Thread.run(Thread.java:736)
although the exact time when it happens differs between runs. Also, sometimes when running the patched test, I seem to hit the deadlock in JENA-161. Because I am having trouble to force a stack dump on my windows 64 bit machine, I am not able to do much with the deadlock. I just cancel and restart the test in those cases
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Paolo Castagna (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paolo Castagna updated JENA-143:
--------------------------------
Fix Version/s: (was: Jena 2.6.5)
TDB 0.9.0
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Fix For: TDB 0.9.0
>
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Simon Helsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132671#comment-13132671 ]
Simon Helsen commented on JENA-143:
-----------------------------------
Andy, I don't have a test case right now. When I see a bug I report it. If it was easy for me to produce a test case, I would do so. As far as I can see, there is no relation with JENA-131.
I think it is troublesome that you dismiss defect reporting if it does not contain a test case. I am giving you as much information as possible to reproduce the problem. So far, the bugs I have reported for TxTDB have always been "real" bugs, which eventually bite you. So it is in everybody's interest to investigate them asap. I do what I can to help, but for us Jena is a library deeply embedded in a framework and I am always very careful to see that we are not violating any invariants.
The reason I know that the QueryOperation.abort is negatively affecting what is going on is because I can avoid corruptions as reported in this defect when I uncomment the line where we call QueryExecution.abort. If I turn it on, the corruptions happen.
The environment is very clear and I am asking in return if you have a junit which tests the behavior of transactions while aborting queries?
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Paolo Castagna (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paolo Castagna updated JENA-143:
--------------------------------
Fix Version/s: (was: TDB 0.9.0)
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Andy Seaborne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153840#comment-13153840 ]
Andy Seaborne commented on JENA-143:
------------------------------------
I don't think this is connected to abort. I get the same error with or without an abort query included.
It looks like the same as JENA-163.
I only ever get the error for 000E / 0000.
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Paolo Castagna (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paolo Castagna updated JENA-143:
--------------------------------
Fix Version/s: 2.6.5
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Fix For: 2.6.5
>
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Andy Seaborne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132685#comment-13132685 ]
Andy Seaborne commented on JENA-143:
------------------------------------
The environment is not clear! I have spent an hour trying to recreate this but I am guessing about your setup.
When is the corruption detected? Later? When later? Different transaction?
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Simon Helsen (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Helsen updated JENA-143:
------------------------------
Attachment: T_TransSystem_patchedForJena143.txt
as discussed above
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Paolo Castagna (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152086#comment-13152086 ]
Paolo Castagna commented on JENA-143:
-------------------------------------
Hi Simon, I experience same issue, however I still do not understand what's causing it.
START (disk[direct], 1000 iterations)
000: . CS CS CS CD CD CD . CS CS CD CD CD CD CD CD CD CD ......... CS .. CS CD CS .. CS . CS .... CS CS . CS . CD .. CS .. CS .. CS CS .. CS .. [...]
800: ........ CS . CS .. CS .. CS .... CS ................... CS .. CS ... CS . CS ........... CS . CS ........ CS CS ...... CS com.hp.hpl.jena.tdb.TDBException: Different ids for S: allocated: expected [000000000000000E], got [0000000000000000]
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.append(NodeTableTrans.java:178)
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.writeNodeJournal(NodeTableTrans.java:210)
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.commitPrepare(NodeTableTrans.java:190)
at com.hp.hpl.jena.tdb.transaction.Transaction.prepare(Transaction.java:108)
at com.hp.hpl.jena.tdb.transaction.Transaction.commit(Transaction.java:92)
at com.hp.hpl.jena.tdb.DatasetGraphTxn.commit(DatasetGraphTxn.java:41)
at com.hp.hpl.jena.tdb.transaction.T_TransSystem$Writer.call(T_TransSystem.java:301)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Simon Helsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148036#comment-13148036 ]
Simon Helsen commented on JENA-143:
-----------------------------------
After the fix for JENA-131, there still seems to be an issue, but it is not 100% identical in the sense that I don't see the warnings of the type
WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
anymore. but I did get
16:39:00,060 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8023)[83229][83229]: Impossibly large object : 591743342 bytes
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:384)
at org.openjena.atlas.iterator.Iter$4.next(Iter.java:293)
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.append(NodeTableTrans.java:171)
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.writeNodeJournal(NodeTableTrans.java:210)
at com.hp.hpl.jena.tdb.transaction.NodeTableTrans.commitPrepare(NodeTableTrans.java:190)
at com.hp.hpl.jena.tdb.transaction.Transaction.prepare(Transaction.java:109)
at com.hp.hpl.jena.tdb.transaction.Transaction.commit(Transaction.java:93)
at com.hp.hpl.jena.tdb.DatasetGraphTxn.commit(DatasetGraphTxn.java:38)
at com.ibm.team.jfs.rdf.internal.jena.tdb.JenaTdbTxProvider.storeOperation(JenaTdbTxProvider.java:321)
<snip>
and
16:39:00,064 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - com.hp.hpl.jena.tdb.transaction.TDBTransactionException: Transaction has already committed or aborted
interesting enough the 2nd message (produced around the same time as the previous error) is identical to what I found when running a different test to verify JENA-131.
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Fix For: TDB 0.9.0
>
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Andy Seaborne (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Seaborne resolved JENA-143.
--------------------------------
Resolution: Duplicate
The query abort code is just changing the timing. Putting in non-aborting queries also exhibits the same symptoms.
Therefore this is the same as JENA-163.
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Assignee: Andy Seaborne
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Simon Helsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148038#comment-13148038 ]
Simon Helsen commented on JENA-143:
-----------------------------------
as we discussed before, it needs a test case and perhaps it can help unearth the secondary observation in JENA-131. At this point, I don't think JENA-161 is related to this issue, but I may be wrong of course
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Fix For: TDB 0.9.0
>
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Paolo Castagna (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paolo Castagna updated JENA-143:
--------------------------------
Description:
The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
and then shortly after:
15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
DatasetGraphTxn dsGraph = null;
try {
dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
Dataset ds = dsGraph.toDataset();
...
QueryExecution qe = null;
...
try {
results = qe.execSelect();
...
} finally {
if (qe != null) {
qe.close();
}
}
} catch (QueryCancelledException e) {
if (dsGraph != null) {
dsGraph.abort();
}
} finally {
if (dsGraph != null) {
dsGraph.close();
}
}
A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
this.queryExecution.abort();
was:
The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
and then shortly after:
15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
DatasetGraphTxn dsGraph = null;
try {
dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
Dataset ds = dsGraph.toDataset();
...
QueryExecution qe = null;
...
try {
results = qe.execSelect();
...
} finally {
if (qe != null) {
qe.close();
}
}
} catch (QueryCancelledException e) {
if (dsGraph != null) {
dsGraph.abort();
}
} finally {
if (dsGraph != null) {
dsGraph.close();
}
}
A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
this.queryExecution.abort();
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TxTDB when interrupting transactional queries
Posted by "Simon Helsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132720#comment-13132720 ]
Simon Helsen commented on JENA-143:
-----------------------------------
Andy, I am sorry I am not able to produce a laboratory-clean setup, but as I have explained many times before, TDB is deeply embedded in a complex framework. We have about 1200 unit tests which do all sorts of stuff and they run entirely green if I comment out QueryExecution.abort in the code which manages cancellations. The only difference is of course that in that case, the test suite which tests cancellation does not actually cancel the queries.
With regard to when the corruption shows up. It happens relatively soon after the abort is initiated, but I have no clear information on when it happens. It always starts with a few of the following warnings
WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
and shortly after that the corruption spreads with the impossible large object message and I have to stop the unit tests. I'll try to isolate the behavior more, but having a unit tests which fires a few threads that query and one other threads which aborts them in mid-execution may reproduce the problem. Interesting enough, I don't think any write transaction is necessary to reproduce, but I'll try to verify that
> QueryExecution.abort seems to corrupt TxTDB when interrupting transactional queries
> -----------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
>
> The interaction between queryExecution.abort() and TxTDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Andy Seaborne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154264#comment-13154264 ]
Andy Seaborne commented on JENA-143:
------------------------------------
I'm getting failures mostly now -- I'm running something quite different from the T_TransSystem in terms of the ordering. Also, using the style of the test case for JENA-161 has made the problem occur fo rme. When I originally applied your patch, I didn't see errors.
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Assignee: Andy Seaborne
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (JENA-143) QueryExecution.abort seems to corrupt
TDB when interrupting transactional queries
Posted by "Andy Seaborne (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Seaborne reassigned JENA-143:
----------------------------------
Assignee: Andy Seaborne
> QueryExecution.abort seems to corrupt TDB when interrupting transactional queries
> ---------------------------------------------------------------------------------
>
> Key: JENA-143
> URL: https://issues.apache.org/jira/browse/JENA-143
> Project: Jena
> Issue Type: Bug
> Components: TDB
> Environment: tdb-0.9.0-20111010.121635
> Reporter: Simon Helsen
> Assignee: Andy Seaborne
> Attachments: T_TransSystem_patchedForJena143.txt
>
>
> The interaction between queryExecution.abort() and TDB transactions seems to suffer from a problem. When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing
> 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159
> and then shortly after:
> 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception:
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:294)
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next(ObjectFileStorage.java:409)
> This is the used coding pattern. The main thread just sets up the transaction, so, something like this:
> DatasetGraphTxn dsGraph = null;
> try {
> dsGraph = StoreConnection.make(this.location).begin(ReadWrite.READ);
> Dataset ds = dsGraph.toDataset();
>
> ...
> QueryExecution qe = null;
> ...
> try {
> results = qe.execSelect();
> ...
> } finally {
> if (qe != null) {
> qe.close();
> }
> }
> } catch (QueryCancelledException e) {
> if (dsGraph != null) {
> dsGraph.abort();
> }
> } finally {
> if (dsGraph != null) {
> dsGraph.close();
> }
> }
> A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call
> this.queryExecution.abort();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira