You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Claude Warren <cl...@xenei.com> on 2017/12/28 23:49:59 UTC
TDB Graph usage question
Would the following code snippit work
Graph g = // a TDB graph instance
if (g.getTransactionHandler().transactionsSupported()) {
Graph initial = graphWith( GraphFactory.createGraphMem(),
"initial hasValue 42; also hasURI hello");
Graph extra = graphWith(GraphFactory.createGraphMem(),
"extra hasValue 17; also hasURI world");
GraphUtil.addInto(g, initial);
g.getTransactionHandler().begin();
GraphUtil.addInto(g, extra);
g.getTransactionHandler().abort();
assertIsomorphic(initial, g);
Notes:
graphWith populates the graph with the triples defined in the string.
GraphUtil.addInto() performs no transaction handling and copies the
contents of the second param into the first param.
I suspect that this will/should fail when the begin() is called as the
transaction as we are switching from not using transactions to using them.
Claude
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: TDB Graph usage question
Posted by Claude Warren <cl...@xenei.com>.
I am just looking at the contract test code that has been around for
awhile. Old test I suppose, or difference between TDB and other graph
implementations. Hence my argument for contract tests, it helps identify
where the implementations differ and allows us to document exactly what the
expectations are.
In this case, based on my understanding of the above conversation, if I
make one of the graphs a memory graph the test would pass.
Thx for the assistance,
Claude
On Sat, Dec 30, 2017 at 10:54 AM, Andy Seaborne <an...@apache.org> wrote:
> Have you tried TDB2 and TIM yet?
>
> On 30/12/17 00:09, Claude Warren wrote:
>
>> When I tried to add a transaction on dest there was an error with a
>> transaction already being active.
>>
>
> This is what I mean by details - you are now saying that graphs are from
> the same dataset. That's an additional feature imposed by this use of the
> test. I'm sure if source and est were the same graph, nasty things will
> happen. (Elsewhere, GraphUtil works carefully for the cases when the graphs
> are the same).
>
> As a general Java comment on iterators over the same data structure:
>
> Don't modify the datastructure while iterating.
>
> You'll get either an exception (best case) or inconsistent results and
> possibly differently inconsistent from run to run.
>
> Does the concurrent modification error indicate that the model was somehow
>> changed outside of an iterator? That is where I am used to seeing it.
>>
>
> Graphs from dataset are stateless views of datasets. See the code in
> GraphView.
>
> (IIRC -- they are in TDB2, and I've been slowly working on improving the
> situation for TDB1 without making a breaking change. TIM - I think it's OK.)
>
> The iterator call is in a recursive call to extractInfo().
>>
>> Both graphs are in the same dataset.
>>
>> all txnX() methods are of the form:
>>
>> public static void txnBegin(Graph g) {
>> if (g.getTransactionHandler().transactionsSupported()) {
>> g.getTransactionHandler().begin();
>> }
>> }
>>
>
> ----
> Unrelated to the iterator: The test code could be shorted if:
>
> public static void txn(Graph g, Runnable action) {
> if ( g.getTransactionHandler().transactionsSupported() )
> g.getTransactionHandler().execute(action);
> else
> action.run();
> }
>
> then
> txn(source, ()->
> GraphExtract e = ....
> e.extractInto(dest, node("a"), source)
> );
>
> The try-catch stuff is handled by TransactionHandler.execute.
>
> BTW You are lucky - you are passing graphs across transactions which was a
> no-no in TDB1 until quite recently.
>
> Graph transactions and dataset transactions used to be unconnected. Graph
> transactions were written for the pre-dataset world.
>
> Andy
>
>
> Stepping through the extract code there is this method:
>>
>> public void extractInto( Node root )
>> {
>> active.add( root );
>> Iterator<Triple> it = extractFrom.find( root, Node.ANY,
>> Node.ANY );
>> while (it.hasNext())
>> {
>> Triple t = it.next();
>> Node subRoot = t.getObject();
>> toUpdate.add( t );
>> if (! (active.contains( subRoot ) || b.stopAt( t )))
>> extractInto( subRoot );
>> }
>> }
>> }
>>
>> When the toUpdate.add(t) is called the
>> DatasetControlMRSW$IteratorCheckNotConcurrent.eCount
>> is incremented.
>>
>> I see in the code comments for IteratorCheckNotConcurrent that it expects
>> writer to be on a different thread than reader. Is there a way around
>> that
>> restriction? I ask because all other graphs that I have tested work fine
>> when reading and writing on the same thread.
>>
>> Thoughts?
>>
>> Claude
>>
>>
>>
>>
>> On Fri, Dec 29, 2017 at 10:28 PM, Andy Seaborne <an...@apache.org> wrote:
>>
>> The exception says there is incorrect MRSW use of iterators in the test.
>>>
>>> Iterator checking is very old code and I believe it works reliably - it
>>> is
>>> not guaranteed to always catch errors though. That was never the claim.
>>>
>>> Too many details missing to say much else but I don't see any txn on
>>> dest.
>>>
>>> Andy
>>>
>>>
>>> On 29/12/17 13:10, Claude Warren wrote:
>>>
>>> fair enough.
>>>>
>>>> I created a ContractTest for GraphTxnTDB and discovered a lot of issues
>>>> with transactions in the contract tests. I cleaned most of them up but
>>>> am
>>>> left with the following:
>>>>
>>>> /**
>>>> * This test exposed that the update-existing-graph functionality
>>>> was
>>>> broken
>>>> * if the target graph already contained any statements with a
>>>> subject S
>>>> * appearing as subject in the source graph - no further Spo
>>>> statements
>>>> were
>>>> * added.
>>>> */
>>>> @ContractTest
>>>> public void testPartialUpdate() {
>>>> Graph source = graphWith(producer.newInstance(), "a R b; b S
>>>> e");
>>>> Graph dest = graphWith(producer.newInstance(), "b R d");
>>>> txnBegin(source);
>>>> try {
>>>> GraphExtract e = new GraphExtract(TripleBoundary.st
>>>> opNowhere);
>>>> e.extractInto(dest, node("a"), source);
>>>> txnCommit(source);
>>>> }
>>>> catch (RuntimeException e)
>>>> {
>>>> txnRollback(source);
>>>> fail( e.getMessage() );
>>>>
>>>> }
>>>> txnBegin(source);
>>>> assertIsomorphic(
>>>> graphWith( "a R b; b S e; b R d"),
>>>> dest);
>>>> txnRollback(source);
>>>>
>>>> }
>>>>
>>>>
>>>> Notes:
>>>> graphWith populates the graph with the triples defined in the
>>>> string
>>>> using a transaction.
>>>> txnRollback performs an abort if transactions are supported.
>>>>
>>>> The problem is that an exception is thrown in the extractInfo() call.
>>>>
>>>> java.util.ConcurrentModificationException: Iterator: started at 6, now
>>>> 7
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(Datas
>>>> etControlMRSW.java:147)
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(Datase
>>>> tControlMRSW.java:32)
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>>>> oncurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103)
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>>>> oncurrent.hasNext(DatasetControlMRSW.java:110)
>>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>>>> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886)
>>>> at
>>>> org.apache.jena.util.iterator.WrappedIterator.hasNext(Wrappe
>>>> dIterator.java:90)
>>>> at
>>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>>>> GraphExtract.java:80)
>>>> at
>>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>>>> GraphExtract.java:85)
>>>> at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract.
>>>> java:52)
>>>> at
>>>> org.apache.jena.graph.GraphContractTest.testPartialUpdate(Gr
>>>> aphContractTest.java:1564)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>>> ssorImpl.java:62)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>>> thodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at
>>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>>>> FrameworkMethod.java:45)
>>>> at
>>>> org.junit.internal.runners.model.ReflectiveCallable.run(Refl
>>>> ectiveCallable.java:15)
>>>> at
>>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
>>>> ameworkMethod.java:42)
>>>> at
>>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(
>>>> InvokeMethod.java:20)
>>>> at
>>>> org.junit.internal.runners.statements.RunAfters.evaluate(Run
>>>> Afters.java:30)
>>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>>>> at
>>>> org.xenei.junit.contract.ContractTestRunner.runChild(Contrac
>>>> tTestRunner.java:138)
>>>> at
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
>>>> 4ClassRunner.java:47)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:
>>>> 60)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.
>>>> java:229)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:
>>>> 50)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:
>>>> 222)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>>> at
>>>> org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369)
>>>> at org.xenei.junit.contract.ContractSuite.runChild(ContractSuit
>>>> e.java:1)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:
>>>> 60)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.
>>>> java:229)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:
>>>> 50)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:
>>>> 222)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>>> at
>>>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.
>>>> run(JUnit4TestReference.java:86)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.TestExecution.run(
>>>> TestExecution.java:38)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>>>> sts(RemoteTestRunner.java:459)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>>>> sts(RemoteTestRunner.java:678)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(
>>>> RemoteTestRunner.java:382)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(
>>>> RemoteTestRunner.java:192)
>>>>
>>>> Is this a bug, or a known limitation of GraphTxnTDB?
>>>>
>>>> Claude
>>>>
>>>> On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <an...@apache.org>
>>>> wrote:
>>>>
>>>> Try it :-)
>>>>
>>>>>
>>>>> Then try running it twice in a JVM.
>>>>>
>>>>> And then try TDB2 vs TDB1 vs TIM.
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> On 28/12/17 23:49, Claude Warren wrote:
>>>>>
>>>>> Would the following code snippit work
>>>>>
>>>>>>
>>>>>> Graph g = // a TDB graph instance
>>>>>> if (g.getTransactionHandler().transactionsSupported()) {
>>>>>> Graph initial = graphWith(
>>>>>> GraphFactory.createGraphMem(),
>>>>>> "initial hasValue 42; also hasURI hello");
>>>>>> Graph extra = graphWith(GraphFactory.createGraphMem(),
>>>>>> "extra hasValue 17; also hasURI world");
>>>>>> GraphUtil.addInto(g, initial);
>>>>>> g.getTransactionHandler().begin();
>>>>>> GraphUtil.addInto(g, extra);
>>>>>> g.getTransactionHandler().abort();
>>>>>> assertIsomorphic(initial, g);
>>>>>>
>>>>>>
>>>>>> Notes:
>>>>>> graphWith populates the graph with the triples defined in the
>>>>>> string.
>>>>>> GraphUtil.addInto() performs no transaction handling and copies
>>>>>> the
>>>>>> contents of the second param into the first param.
>>>>>>
>>>>>> I suspect that this will/should fail when the begin() is called as the
>>>>>> transaction as we are switching from not using transactions to using
>>>>>> them.
>>>>>>
>>>>>> Claude
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: TDB Graph usage question
Posted by ajs6f <aj...@apache.org>.
TIM uses a subclass of GraphView called GraphInMemory which is indeed also stateless.
ajs6f
> On Dec 30, 2017, at 5:54 AM, Andy Seaborne <an...@apache.org> wrote:
>
> Have you tried TDB2 and TIM yet?
>
> On 30/12/17 00:09, Claude Warren wrote:
>> When I tried to add a transaction on dest there was an error with a
>> transaction already being active.
>
> This is what I mean by details - you are now saying that graphs are from the same dataset. That's an additional feature imposed by this use of the test. I'm sure if source and est were the same graph, nasty things will happen. (Elsewhere, GraphUtil works carefully for the cases when the graphs are the same).
>
> As a general Java comment on iterators over the same data structure:
>
> Don't modify the datastructure while iterating.
>
> You'll get either an exception (best case) or inconsistent results and possibly differently inconsistent from run to run.
>
>> Does the concurrent modification error indicate that the model was somehow
>> changed outside of an iterator? That is where I am used to seeing it.
>
> Graphs from dataset are stateless views of datasets. See the code in GraphView.
>
> (IIRC -- they are in TDB2, and I've been slowly working on improving the situation for TDB1 without making a breaking change. TIM - I think it's OK.)
>
>> The iterator call is in a recursive call to extractInfo().
>> Both graphs are in the same dataset.
>> all txnX() methods are of the form:
>> public static void txnBegin(Graph g) {
>> if (g.getTransactionHandler().transactionsSupported()) {
>> g.getTransactionHandler().begin();
>> }
>> }
>
> ----
> Unrelated to the iterator: The test code could be shorted if:
>
> public static void txn(Graph g, Runnable action) {
> if ( g.getTransactionHandler().transactionsSupported() )
> g.getTransactionHandler().execute(action);
> else
> action.run();
> }
>
> then
> txn(source, ()->
> GraphExtract e = ....
> e.extractInto(dest, node("a"), source)
> );
>
> The try-catch stuff is handled by TransactionHandler.execute.
>
> BTW You are lucky - you are passing graphs across transactions which was a no-no in TDB1 until quite recently.
>
> Graph transactions and dataset transactions used to be unconnected. Graph transactions were written for the pre-dataset world.
>
> Andy
>
>> Stepping through the extract code there is this method:
>> public void extractInto( Node root )
>> {
>> active.add( root );
>> Iterator<Triple> it = extractFrom.find( root, Node.ANY,
>> Node.ANY );
>> while (it.hasNext())
>> {
>> Triple t = it.next();
>> Node subRoot = t.getObject();
>> toUpdate.add( t );
>> if (! (active.contains( subRoot ) || b.stopAt( t )))
>> extractInto( subRoot );
>> }
>> }
>> }
>> When the toUpdate.add(t) is called the
>> DatasetControlMRSW$IteratorCheckNotConcurrent.eCount
>> is incremented.
>> I see in the code comments for IteratorCheckNotConcurrent that it expects
>> writer to be on a different thread than reader. Is there a way around that
>> restriction? I ask because all other graphs that I have tested work fine
>> when reading and writing on the same thread.
>> Thoughts?
>> Claude
>> On Fri, Dec 29, 2017 at 10:28 PM, Andy Seaborne <an...@apache.org> wrote:
>>> The exception says there is incorrect MRSW use of iterators in the test.
>>>
>>> Iterator checking is very old code and I believe it works reliably - it is
>>> not guaranteed to always catch errors though. That was never the claim.
>>>
>>> Too many details missing to say much else but I don't see any txn on dest.
>>>
>>> Andy
>>>
>>>
>>> On 29/12/17 13:10, Claude Warren wrote:
>>>
>>>> fair enough.
>>>>
>>>> I created a ContractTest for GraphTxnTDB and discovered a lot of issues
>>>> with transactions in the contract tests. I cleaned most of them up but am
>>>> left with the following:
>>>>
>>>> /**
>>>> * This test exposed that the update-existing-graph functionality was
>>>> broken
>>>> * if the target graph already contained any statements with a
>>>> subject S
>>>> * appearing as subject in the source graph - no further Spo
>>>> statements
>>>> were
>>>> * added.
>>>> */
>>>> @ContractTest
>>>> public void testPartialUpdate() {
>>>> Graph source = graphWith(producer.newInstance(), "a R b; b S
>>>> e");
>>>> Graph dest = graphWith(producer.newInstance(), "b R d");
>>>> txnBegin(source);
>>>> try {
>>>> GraphExtract e = new GraphExtract(TripleBoundary.st
>>>> opNowhere);
>>>> e.extractInto(dest, node("a"), source);
>>>> txnCommit(source);
>>>> }
>>>> catch (RuntimeException e)
>>>> {
>>>> txnRollback(source);
>>>> fail( e.getMessage() );
>>>>
>>>> }
>>>> txnBegin(source);
>>>> assertIsomorphic(
>>>> graphWith( "a R b; b S e; b R d"),
>>>> dest);
>>>> txnRollback(source);
>>>>
>>>> }
>>>>
>>>>
>>>> Notes:
>>>> graphWith populates the graph with the triples defined in the string
>>>> using a transaction.
>>>> txnRollback performs an abort if transactions are supported.
>>>>
>>>> The problem is that an exception is thrown in the extractInfo() call.
>>>>
>>>> java.util.ConcurrentModificationException: Iterator: started at 6, now 7
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(Datas
>>>> etControlMRSW.java:147)
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(Datase
>>>> tControlMRSW.java:32)
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>>>> oncurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103)
>>>> at
>>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>>>> oncurrent.hasNext(DatasetControlMRSW.java:110)
>>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>>>> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886)
>>>> at
>>>> org.apache.jena.util.iterator.WrappedIterator.hasNext(Wrappe
>>>> dIterator.java:90)
>>>> at
>>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>>>> GraphExtract.java:80)
>>>> at
>>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>>>> GraphExtract.java:85)
>>>> at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract.
>>>> java:52)
>>>> at
>>>> org.apache.jena.graph.GraphContractTest.testPartialUpdate(Gr
>>>> aphContractTest.java:1564)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>>> ssorImpl.java:62)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>>> thodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at
>>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>>>> FrameworkMethod.java:45)
>>>> at
>>>> org.junit.internal.runners.model.ReflectiveCallable.run(Refl
>>>> ectiveCallable.java:15)
>>>> at
>>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
>>>> ameworkMethod.java:42)
>>>> at
>>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(
>>>> InvokeMethod.java:20)
>>>> at
>>>> org.junit.internal.runners.statements.RunAfters.evaluate(Run
>>>> Afters.java:30)
>>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>>>> at
>>>> org.xenei.junit.contract.ContractTestRunner.runChild(Contrac
>>>> tTestRunner.java:138)
>>>> at
>>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
>>>> 4ClassRunner.java:47)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>>> at
>>>> org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369)
>>>> at org.xenei.junit.contract.ContractSuite.runChild(ContractSuit
>>>> e.java:1)
>>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>>> at
>>>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.
>>>> run(JUnit4TestReference.java:86)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.TestExecution.run(
>>>> TestExecution.java:38)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>>>> sts(RemoteTestRunner.java:459)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>>>> sts(RemoteTestRunner.java:678)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(
>>>> RemoteTestRunner.java:382)
>>>> at
>>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(
>>>> RemoteTestRunner.java:192)
>>>>
>>>> Is this a bug, or a known limitation of GraphTxnTDB?
>>>>
>>>> Claude
>>>>
>>>> On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> Try it :-)
>>>>>
>>>>> Then try running it twice in a JVM.
>>>>>
>>>>> And then try TDB2 vs TDB1 vs TIM.
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> On 28/12/17 23:49, Claude Warren wrote:
>>>>>
>>>>> Would the following code snippit work
>>>>>>
>>>>>> Graph g = // a TDB graph instance
>>>>>> if (g.getTransactionHandler().transactionsSupported()) {
>>>>>> Graph initial = graphWith( GraphFactory.createGraphMem(),
>>>>>> "initial hasValue 42; also hasURI hello");
>>>>>> Graph extra = graphWith(GraphFactory.createGraphMem(),
>>>>>> "extra hasValue 17; also hasURI world");
>>>>>> GraphUtil.addInto(g, initial);
>>>>>> g.getTransactionHandler().begin();
>>>>>> GraphUtil.addInto(g, extra);
>>>>>> g.getTransactionHandler().abort();
>>>>>> assertIsomorphic(initial, g);
>>>>>>
>>>>>>
>>>>>> Notes:
>>>>>> graphWith populates the graph with the triples defined in the
>>>>>> string.
>>>>>> GraphUtil.addInto() performs no transaction handling and copies
>>>>>> the
>>>>>> contents of the second param into the first param.
>>>>>>
>>>>>> I suspect that this will/should fail when the begin() is called as the
>>>>>> transaction as we are switching from not using transactions to using
>>>>>> them.
>>>>>>
>>>>>> Claude
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
Re: TDB Graph usage question
Posted by Andy Seaborne <an...@apache.org>.
Have you tried TDB2 and TIM yet?
On 30/12/17 00:09, Claude Warren wrote:
> When I tried to add a transaction on dest there was an error with a
> transaction already being active.
This is what I mean by details - you are now saying that graphs are from
the same dataset. That's an additional feature imposed by this use of
the test. I'm sure if source and est were the same graph, nasty things
will happen. (Elsewhere, GraphUtil works carefully for the cases when
the graphs are the same).
As a general Java comment on iterators over the same data structure:
Don't modify the datastructure while iterating.
You'll get either an exception (best case) or inconsistent results and
possibly differently inconsistent from run to run.
> Does the concurrent modification error indicate that the model was somehow
> changed outside of an iterator? That is where I am used to seeing it.
Graphs from dataset are stateless views of datasets. See the code in
GraphView.
(IIRC -- they are in TDB2, and I've been slowly working on improving the
situation for TDB1 without making a breaking change. TIM - I think it's OK.)
> The iterator call is in a recursive call to extractInfo().
>
> Both graphs are in the same dataset.
>
> all txnX() methods are of the form:
>
> public static void txnBegin(Graph g) {
> if (g.getTransactionHandler().transactionsSupported()) {
> g.getTransactionHandler().begin();
> }
> }
----
Unrelated to the iterator: The test code could be shorted if:
public static void txn(Graph g, Runnable action) {
if ( g.getTransactionHandler().transactionsSupported() )
g.getTransactionHandler().execute(action);
else
action.run();
}
then
txn(source, ()->
GraphExtract e = ....
e.extractInto(dest, node("a"), source)
);
The try-catch stuff is handled by TransactionHandler.execute.
BTW You are lucky - you are passing graphs across transactions which was
a no-no in TDB1 until quite recently.
Graph transactions and dataset transactions used to be unconnected.
Graph transactions were written for the pre-dataset world.
Andy
> Stepping through the extract code there is this method:
>
> public void extractInto( Node root )
> {
> active.add( root );
> Iterator<Triple> it = extractFrom.find( root, Node.ANY,
> Node.ANY );
> while (it.hasNext())
> {
> Triple t = it.next();
> Node subRoot = t.getObject();
> toUpdate.add( t );
> if (! (active.contains( subRoot ) || b.stopAt( t )))
> extractInto( subRoot );
> }
> }
> }
>
> When the toUpdate.add(t) is called the
> DatasetControlMRSW$IteratorCheckNotConcurrent.eCount
> is incremented.
>
> I see in the code comments for IteratorCheckNotConcurrent that it expects
> writer to be on a different thread than reader. Is there a way around that
> restriction? I ask because all other graphs that I have tested work fine
> when reading and writing on the same thread.
>
> Thoughts?
>
> Claude
>
>
>
>
> On Fri, Dec 29, 2017 at 10:28 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> The exception says there is incorrect MRSW use of iterators in the test.
>>
>> Iterator checking is very old code and I believe it works reliably - it is
>> not guaranteed to always catch errors though. That was never the claim.
>>
>> Too many details missing to say much else but I don't see any txn on dest.
>>
>> Andy
>>
>>
>> On 29/12/17 13:10, Claude Warren wrote:
>>
>>> fair enough.
>>>
>>> I created a ContractTest for GraphTxnTDB and discovered a lot of issues
>>> with transactions in the contract tests. I cleaned most of them up but am
>>> left with the following:
>>>
>>> /**
>>> * This test exposed that the update-existing-graph functionality was
>>> broken
>>> * if the target graph already contained any statements with a
>>> subject S
>>> * appearing as subject in the source graph - no further Spo
>>> statements
>>> were
>>> * added.
>>> */
>>> @ContractTest
>>> public void testPartialUpdate() {
>>> Graph source = graphWith(producer.newInstance(), "a R b; b S
>>> e");
>>> Graph dest = graphWith(producer.newInstance(), "b R d");
>>> txnBegin(source);
>>> try {
>>> GraphExtract e = new GraphExtract(TripleBoundary.st
>>> opNowhere);
>>> e.extractInto(dest, node("a"), source);
>>> txnCommit(source);
>>> }
>>> catch (RuntimeException e)
>>> {
>>> txnRollback(source);
>>> fail( e.getMessage() );
>>>
>>> }
>>> txnBegin(source);
>>> assertIsomorphic(
>>> graphWith( "a R b; b S e; b R d"),
>>> dest);
>>> txnRollback(source);
>>>
>>> }
>>>
>>>
>>> Notes:
>>> graphWith populates the graph with the triples defined in the string
>>> using a transaction.
>>> txnRollback performs an abort if transactions are supported.
>>>
>>> The problem is that an exception is thrown in the extractInfo() call.
>>>
>>> java.util.ConcurrentModificationException: Iterator: started at 6, now 7
>>> at
>>> org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(Datas
>>> etControlMRSW.java:147)
>>> at
>>> org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(Datase
>>> tControlMRSW.java:32)
>>> at
>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>>> oncurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103)
>>> at
>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>>> oncurrent.hasNext(DatasetControlMRSW.java:110)
>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>>> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886)
>>> at
>>> org.apache.jena.util.iterator.WrappedIterator.hasNext(Wrappe
>>> dIterator.java:90)
>>> at
>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>>> GraphExtract.java:80)
>>> at
>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>>> GraphExtract.java:85)
>>> at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract.
>>> java:52)
>>> at
>>> org.apache.jena.graph.GraphContractTest.testPartialUpdate(Gr
>>> aphContractTest.java:1564)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>> ssorImpl.java:62)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>> thodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>>> FrameworkMethod.java:45)
>>> at
>>> org.junit.internal.runners.model.ReflectiveCallable.run(Refl
>>> ectiveCallable.java:15)
>>> at
>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
>>> ameworkMethod.java:42)
>>> at
>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(
>>> InvokeMethod.java:20)
>>> at
>>> org.junit.internal.runners.statements.RunAfters.evaluate(Run
>>> Afters.java:30)
>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>>> at
>>> org.xenei.junit.contract.ContractTestRunner.runChild(Contrac
>>> tTestRunner.java:138)
>>> at
>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
>>> 4ClassRunner.java:47)
>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>> at
>>> org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369)
>>> at org.xenei.junit.contract.ContractSuite.runChild(ContractSuit
>>> e.java:1)
>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>> at
>>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.
>>> run(JUnit4TestReference.java:86)
>>> at
>>> org.eclipse.jdt.internal.junit.runner.TestExecution.run(
>>> TestExecution.java:38)
>>> at
>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>>> sts(RemoteTestRunner.java:459)
>>> at
>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>>> sts(RemoteTestRunner.java:678)
>>> at
>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(
>>> RemoteTestRunner.java:382)
>>> at
>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(
>>> RemoteTestRunner.java:192)
>>>
>>> Is this a bug, or a known limitation of GraphTxnTDB?
>>>
>>> Claude
>>>
>>> On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> Try it :-)
>>>>
>>>> Then try running it twice in a JVM.
>>>>
>>>> And then try TDB2 vs TDB1 vs TIM.
>>>>
>>>> Andy
>>>>
>>>>
>>>> On 28/12/17 23:49, Claude Warren wrote:
>>>>
>>>> Would the following code snippit work
>>>>>
>>>>> Graph g = // a TDB graph instance
>>>>> if (g.getTransactionHandler().transactionsSupported()) {
>>>>> Graph initial = graphWith( GraphFactory.createGraphMem(),
>>>>> "initial hasValue 42; also hasURI hello");
>>>>> Graph extra = graphWith(GraphFactory.createGraphMem(),
>>>>> "extra hasValue 17; also hasURI world");
>>>>> GraphUtil.addInto(g, initial);
>>>>> g.getTransactionHandler().begin();
>>>>> GraphUtil.addInto(g, extra);
>>>>> g.getTransactionHandler().abort();
>>>>> assertIsomorphic(initial, g);
>>>>>
>>>>>
>>>>> Notes:
>>>>> graphWith populates the graph with the triples defined in the
>>>>> string.
>>>>> GraphUtil.addInto() performs no transaction handling and copies
>>>>> the
>>>>> contents of the second param into the first param.
>>>>>
>>>>> I suspect that this will/should fail when the begin() is called as the
>>>>> transaction as we are switching from not using transactions to using
>>>>> them.
>>>>>
>>>>> Claude
>>>>>
>>>>>
>>>>>
>>>
>>>
>
>
Re: TDB Graph usage question
Posted by Claude Warren <cl...@xenei.com>.
When I tried to add a transaction on dest there was an error with a
transaction already being active.
Does the concurrent modification error indicate that the model was somehow
changed outside of an iterator? That is where I am used to seeing it.
The iterator call is in a recursive call to extractInfo().
Both graphs are in the same dataset.
all txnX() methods are of the form:
public static void txnBegin(Graph g) {
if (g.getTransactionHandler().transactionsSupported()) {
g.getTransactionHandler().begin();
}
}
Stepping through the extract code there is this method:
public void extractInto( Node root )
{
active.add( root );
Iterator<Triple> it = extractFrom.find( root, Node.ANY,
Node.ANY );
while (it.hasNext())
{
Triple t = it.next();
Node subRoot = t.getObject();
toUpdate.add( t );
if (! (active.contains( subRoot ) || b.stopAt( t )))
extractInto( subRoot );
}
}
}
When the toUpdate.add(t) is called the
DatasetControlMRSW$IteratorCheckNotConcurrent.eCount
is incremented.
I see in the code comments for IteratorCheckNotConcurrent that it expects
writer to be on a different thread than reader. Is there a way around that
restriction? I ask because all other graphs that I have tested work fine
when reading and writing on the same thread.
Thoughts?
Claude
On Fri, Dec 29, 2017 at 10:28 PM, Andy Seaborne <an...@apache.org> wrote:
> The exception says there is incorrect MRSW use of iterators in the test.
>
> Iterator checking is very old code and I believe it works reliably - it is
> not guaranteed to always catch errors though. That was never the claim.
>
> Too many details missing to say much else but I don't see any txn on dest.
>
> Andy
>
>
> On 29/12/17 13:10, Claude Warren wrote:
>
>> fair enough.
>>
>> I created a ContractTest for GraphTxnTDB and discovered a lot of issues
>> with transactions in the contract tests. I cleaned most of them up but am
>> left with the following:
>>
>> /**
>> * This test exposed that the update-existing-graph functionality was
>> broken
>> * if the target graph already contained any statements with a
>> subject S
>> * appearing as subject in the source graph - no further Spo
>> statements
>> were
>> * added.
>> */
>> @ContractTest
>> public void testPartialUpdate() {
>> Graph source = graphWith(producer.newInstance(), "a R b; b S
>> e");
>> Graph dest = graphWith(producer.newInstance(), "b R d");
>> txnBegin(source);
>> try {
>> GraphExtract e = new GraphExtract(TripleBoundary.st
>> opNowhere);
>> e.extractInto(dest, node("a"), source);
>> txnCommit(source);
>> }
>> catch (RuntimeException e)
>> {
>> txnRollback(source);
>> fail( e.getMessage() );
>>
>> }
>> txnBegin(source);
>> assertIsomorphic(
>> graphWith( "a R b; b S e; b R d"),
>> dest);
>> txnRollback(source);
>>
>> }
>>
>>
>> Notes:
>> graphWith populates the graph with the triples defined in the string
>> using a transaction.
>> txnRollback performs an abort if transactions are supported.
>>
>> The problem is that an exception is thrown in the extractInfo() call.
>>
>> java.util.ConcurrentModificationException: Iterator: started at 6, now 7
>> at
>> org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(Datas
>> etControlMRSW.java:147)
>> at
>> org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(Datase
>> tControlMRSW.java:32)
>> at
>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>> oncurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103)
>> at
>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
>> oncurrent.hasNext(DatasetControlMRSW.java:110)
>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
>> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886)
>> at
>> org.apache.jena.util.iterator.WrappedIterator.hasNext(Wrappe
>> dIterator.java:90)
>> at
>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>> GraphExtract.java:80)
>> at
>> org.apache.jena.graph.GraphExtract$Extraction.extractInto(
>> GraphExtract.java:85)
>> at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract.
>> java:52)
>> at
>> org.apache.jena.graph.GraphContractTest.testPartialUpdate(Gr
>> aphContractTest.java:1564)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>> FrameworkMethod.java:45)
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(Refl
>> ectiveCallable.java:15)
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
>> ameworkMethod.java:42)
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(
>> InvokeMethod.java:20)
>> at
>> org.junit.internal.runners.statements.RunAfters.evaluate(Run
>> Afters.java:30)
>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>> at
>> org.xenei.junit.contract.ContractTestRunner.runChild(Contrac
>> tTestRunner.java:138)
>> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
>> 4ClassRunner.java:47)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>> at
>> org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369)
>> at org.xenei.junit.contract.ContractSuite.runChild(ContractSuit
>> e.java:1)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>> at
>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.
>> run(JUnit4TestReference.java:86)
>> at
>> org.eclipse.jdt.internal.junit.runner.TestExecution.run(
>> TestExecution.java:38)
>> at
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>> sts(RemoteTestRunner.java:459)
>> at
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
>> sts(RemoteTestRunner.java:678)
>> at
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(
>> RemoteTestRunner.java:382)
>> at
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(
>> RemoteTestRunner.java:192)
>>
>> Is this a bug, or a known limitation of GraphTxnTDB?
>>
>> Claude
>>
>> On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> Try it :-)
>>>
>>> Then try running it twice in a JVM.
>>>
>>> And then try TDB2 vs TDB1 vs TIM.
>>>
>>> Andy
>>>
>>>
>>> On 28/12/17 23:49, Claude Warren wrote:
>>>
>>> Would the following code snippit work
>>>>
>>>> Graph g = // a TDB graph instance
>>>> if (g.getTransactionHandler().transactionsSupported()) {
>>>> Graph initial = graphWith( GraphFactory.createGraphMem(),
>>>> "initial hasValue 42; also hasURI hello");
>>>> Graph extra = graphWith(GraphFactory.createGraphMem(),
>>>> "extra hasValue 17; also hasURI world");
>>>> GraphUtil.addInto(g, initial);
>>>> g.getTransactionHandler().begin();
>>>> GraphUtil.addInto(g, extra);
>>>> g.getTransactionHandler().abort();
>>>> assertIsomorphic(initial, g);
>>>>
>>>>
>>>> Notes:
>>>> graphWith populates the graph with the triples defined in the
>>>> string.
>>>> GraphUtil.addInto() performs no transaction handling and copies
>>>> the
>>>> contents of the second param into the first param.
>>>>
>>>> I suspect that this will/should fail when the begin() is called as the
>>>> transaction as we are switching from not using transactions to using
>>>> them.
>>>>
>>>> Claude
>>>>
>>>>
>>>>
>>
>>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: TDB Graph usage question
Posted by Andy Seaborne <an...@apache.org>.
The exception says there is incorrect MRSW use of iterators in the test.
Iterator checking is very old code and I believe it works reliably - it
is not guaranteed to always catch errors though. That was never the claim.
Too many details missing to say much else but I don't see any txn on dest.
Andy
On 29/12/17 13:10, Claude Warren wrote:
> fair enough.
>
> I created a ContractTest for GraphTxnTDB and discovered a lot of issues
> with transactions in the contract tests. I cleaned most of them up but am
> left with the following:
>
> /**
> * This test exposed that the update-existing-graph functionality was
> broken
> * if the target graph already contained any statements with a subject S
> * appearing as subject in the source graph - no further Spo statements
> were
> * added.
> */
> @ContractTest
> public void testPartialUpdate() {
> Graph source = graphWith(producer.newInstance(), "a R b; b S e");
> Graph dest = graphWith(producer.newInstance(), "b R d");
> txnBegin(source);
> try {
> GraphExtract e = new GraphExtract(TripleBoundary.stopNowhere);
> e.extractInto(dest, node("a"), source);
> txnCommit(source);
> }
> catch (RuntimeException e)
> {
> txnRollback(source);
> fail( e.getMessage() );
>
> }
> txnBegin(source);
> assertIsomorphic(
> graphWith( "a R b; b S e; b R d"),
> dest);
> txnRollback(source);
>
> }
>
>
> Notes:
> graphWith populates the graph with the triples defined in the string
> using a transaction.
> txnRollback performs an abort if transactions are supported.
>
> The problem is that an exception is thrown in the extractInfo() call.
>
> java.util.ConcurrentModificationException: Iterator: started at 6, now 7
> at
> org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(DatasetControlMRSW.java:147)
> at
> org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(DatasetControlMRSW.java:32)
> at
> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103)
> at
> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.hasNext(DatasetControlMRSW.java:110)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886)
> at
> org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
> at
> org.apache.jena.graph.GraphExtract$Extraction.extractInto(GraphExtract.java:80)
> at
> org.apache.jena.graph.GraphExtract$Extraction.extractInto(GraphExtract.java:85)
> at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract.java:52)
> at
> org.apache.jena.graph.GraphContractTest.testPartialUpdate(GraphContractTest.java:1564)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.xenei.junit.contract.ContractTestRunner.runChild(ContractTestRunner.java:138)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369)
> at org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:1)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
>
> Is this a bug, or a known limitation of GraphTxnTDB?
>
> Claude
>
> On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> Try it :-)
>>
>> Then try running it twice in a JVM.
>>
>> And then try TDB2 vs TDB1 vs TIM.
>>
>> Andy
>>
>>
>> On 28/12/17 23:49, Claude Warren wrote:
>>
>>> Would the following code snippit work
>>>
>>> Graph g = // a TDB graph instance
>>> if (g.getTransactionHandler().transactionsSupported()) {
>>> Graph initial = graphWith( GraphFactory.createGraphMem(),
>>> "initial hasValue 42; also hasURI hello");
>>> Graph extra = graphWith(GraphFactory.createGraphMem(),
>>> "extra hasValue 17; also hasURI world");
>>> GraphUtil.addInto(g, initial);
>>> g.getTransactionHandler().begin();
>>> GraphUtil.addInto(g, extra);
>>> g.getTransactionHandler().abort();
>>> assertIsomorphic(initial, g);
>>>
>>>
>>> Notes:
>>> graphWith populates the graph with the triples defined in the string.
>>> GraphUtil.addInto() performs no transaction handling and copies the
>>> contents of the second param into the first param.
>>>
>>> I suspect that this will/should fail when the begin() is called as the
>>> transaction as we are switching from not using transactions to using them.
>>>
>>> Claude
>>>
>>>
>
>
Re: TDB Graph usage question
Posted by Claude Warren <cl...@xenei.com>.
fair enough.
I created a ContractTest for GraphTxnTDB and discovered a lot of issues
with transactions in the contract tests. I cleaned most of them up but am
left with the following:
/**
* This test exposed that the update-existing-graph functionality was
broken
* if the target graph already contained any statements with a subject S
* appearing as subject in the source graph - no further Spo statements
were
* added.
*/
@ContractTest
public void testPartialUpdate() {
Graph source = graphWith(producer.newInstance(), "a R b; b S e");
Graph dest = graphWith(producer.newInstance(), "b R d");
txnBegin(source);
try {
GraphExtract e = new GraphExtract(TripleBoundary.stopNowhere);
e.extractInto(dest, node("a"), source);
txnCommit(source);
}
catch (RuntimeException e)
{
txnRollback(source);
fail( e.getMessage() );
}
txnBegin(source);
assertIsomorphic(
graphWith( "a R b; b S e; b R d"),
dest);
txnRollback(source);
}
Notes:
graphWith populates the graph with the triples defined in the string
using a transaction.
txnRollback performs an abort if transactions are supported.
The problem is that an exception is thrown in the extractInfo() call.
java.util.ConcurrentModificationException: Iterator: started at 6, now 7
at
org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(DatasetControlMRSW.java:147)
at
org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(DatasetControlMRSW.java:32)
at
org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103)
at
org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.hasNext(DatasetControlMRSW.java:110)
at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886)
at
org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
at
org.apache.jena.graph.GraphExtract$Extraction.extractInto(GraphExtract.java:80)
at
org.apache.jena.graph.GraphExtract$Extraction.extractInto(GraphExtract.java:85)
at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract.java:52)
at
org.apache.jena.graph.GraphContractTest.testPartialUpdate(GraphContractTest.java:1564)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.xenei.junit.contract.ContractTestRunner.runChild(ContractTestRunner.java:138)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369)
at org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:1)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Is this a bug, or a known limitation of GraphTxnTDB?
Claude
On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <an...@apache.org> wrote:
> Try it :-)
>
> Then try running it twice in a JVM.
>
> And then try TDB2 vs TDB1 vs TIM.
>
> Andy
>
>
> On 28/12/17 23:49, Claude Warren wrote:
>
>> Would the following code snippit work
>>
>> Graph g = // a TDB graph instance
>> if (g.getTransactionHandler().transactionsSupported()) {
>> Graph initial = graphWith( GraphFactory.createGraphMem(),
>> "initial hasValue 42; also hasURI hello");
>> Graph extra = graphWith(GraphFactory.createGraphMem(),
>> "extra hasValue 17; also hasURI world");
>> GraphUtil.addInto(g, initial);
>> g.getTransactionHandler().begin();
>> GraphUtil.addInto(g, extra);
>> g.getTransactionHandler().abort();
>> assertIsomorphic(initial, g);
>>
>>
>> Notes:
>> graphWith populates the graph with the triples defined in the string.
>> GraphUtil.addInto() performs no transaction handling and copies the
>> contents of the second param into the first param.
>>
>> I suspect that this will/should fail when the begin() is called as the
>> transaction as we are switching from not using transactions to using them.
>>
>> Claude
>>
>>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: TDB Graph usage question
Posted by Andy Seaborne <an...@apache.org>.
Try it :-)
Then try running it twice in a JVM.
And then try TDB2 vs TDB1 vs TIM.
Andy
On 28/12/17 23:49, Claude Warren wrote:
> Would the following code snippit work
>
> Graph g = // a TDB graph instance
> if (g.getTransactionHandler().transactionsSupported()) {
> Graph initial = graphWith( GraphFactory.createGraphMem(),
> "initial hasValue 42; also hasURI hello");
> Graph extra = graphWith(GraphFactory.createGraphMem(),
> "extra hasValue 17; also hasURI world");
> GraphUtil.addInto(g, initial);
> g.getTransactionHandler().begin();
> GraphUtil.addInto(g, extra);
> g.getTransactionHandler().abort();
> assertIsomorphic(initial, g);
>
>
> Notes:
> graphWith populates the graph with the triples defined in the string.
> GraphUtil.addInto() performs no transaction handling and copies the
> contents of the second param into the first param.
>
> I suspect that this will/should fail when the begin() is called as the
> transaction as we are switching from not using transactions to using them.
>
> Claude
>