You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Jean-Marc Vanel <je...@gmail.com> on 2018/02/06 10:32:46 UTC
SPARQL: binding to a graph URI results in NullPointerException
Hi
I consistently get a NullPointerException from this query on my test TDB
on laptop.
SELECT DISTINCT ?thing
WHERE {
graph ?thing {
[] ?p ?O .
}
} LIMIT 200
Looking into code : BindingBase.java line 203 , it appears that a binding's
node is null for one SELECT result.
This binding stands for a named graph.
So the database is probably broken. Of course, this is a work and test
database whose content has no value, but I try to understand and prevent
this to happen on production database.
Is a there a bug somewhere ?
What is the likely cause ? During tests and debugs, transactions may have
not been rolled back , application was killed, etc .
How could the database be repaired in such cases ? By doing a tdbdump and
recreate the TDB ?
Running on Jena 3.6.0 and Ubuntu 2017.10 and java version "1.8.0_121" .
java.lang.NullPointerException
at
org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:203)
at
org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:183)
at java.util.HashMap.hash(HashMap.java:338)
at java.util.HashMap.containsKey(HashMap.java:595)
at java.util.HashSet.contains(HashSet.java:203)
at
org.apache.jena.sparql.engine.iterator.QueryIterDistinct.getInputNextUnseen(QueryIterDistinct.java:106)
at
org.apache.jena.sparql.engine.iterator.QueryIterDistinct.hasNextBinding(QueryIterDistinct.java:70)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:74)
* at
org.apache.jena.sparql.engine.ResultSetCheckCondition.hasNext(ResultSetCheckCondition.java:55)*
at
scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
at scala.collection.Iterator$class.toStream(Iterator.scala:1320)
at scala.collection.AbstractIterator.toStream(Iterator.scala:1334)
at
scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
at
scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
at
scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
at
scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
at scala.collection.generic.Growable$class.loop$1(Growable.scala:54)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:57)
at
scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:183)
at
scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45)
at scala.collection.TraversableLike$class.to(TraversableLike.scala:590)
at scala.collection.AbstractTraversable.to(Traversable.scala:104)
at
deductions.runtime.sparql_cache.SPARQLHelpers$$anonfun$sparqlSelectQueryVariablesNT$1.apply(SPARQLHelpers.scala:426)
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: SPARQL: binding to a graph URI results in NullPointerException
Posted by Andy Seaborne <an...@apache.org>.
On 06/02/18 14:24, Andy Seaborne wrote:
> We can put in null checks to catch it earlier (that would be a good idea
> anyway)
https://github.com/apache/jena/pull/354
Andy
Re: SPARQL: binding to a graph URI results in NullPointerException
Posted by Jean-Marc Vanel <je...@gmail.com>.
2018-02-06 18:40 GMT+01:00 Andy Seaborne <an...@apache.org>:
> An oddity:
>
> The only way I can see to set with a null binding (and not get a warning)
> is on the path where the dataset is not a TDB one, but with a graph in a
> general dataset.
>
> Is that the setup here?
>
The TDB is created and opened always by the same application , by
TDBFactory.createDataset(Paths.get(database_location).toString())
I forgot to say that sometimes during the tuning of my app. , there were
uncaught exception that caused end() being called without commit or
rollback .
> (that said, tracing down and eliminating all other possible routes back
> isn't simple so I might have missed another route)
>
> Andy
>
>
> On 06/02/18 14:24, Andy Seaborne wrote:
>
>> Can you dump the database?
>>
>>
>> On 06/02/18 10:32, Jean-Marc Vanel wrote:
>>
>>> Hi
>>>
>>> I consistently get a NullPointerException from this query on my test TDB
>>> on laptop.
>>>
>>> SELECT DISTINCT ?thing
>>> WHERE {
>>> graph ?thing {
>>> [] ?p ?O .
>>> }
>>> } LIMIT 200
>>>
>>> Looking into code : BindingBase.java line 203 , it appears that a
>>> binding's
>>> node is null for one SELECT result.
>>>
>>
>> In which case there is a binding and the value (Node) is null.
>>
>> We can put in null checks to catch it earlier (that would be a good idea
>> anyway) but still not at the point where the problem starts because there
>> isn't one such point - it depends on cache flush order.
>>
>> This binding stands for a named graph.
>>>
>>
>> Just a though - do your named graph have URI names? or are any blank
>> nodes?
>>
>> So the database is probably broken. Of course, this is a work and test
>>> database whose content has no value, but I try to understand and prevent
>>> this to happen on production database.
>>> Is a there a bug somewhere ?
>>>
>>
>> Yes :-) Where? now that is the question!
>>
>> What is the likely cause ? During tests and debugs, transactions may have
>>> not been rolled back , application was killed, etc .
>>> How could the database be repaired in such cases ? By doing a tdbdump and
>>> recreate the TDB ?
>>>
>>
>> Non-transactional TDB needs to explicitly flush modification to disk
>> (TDB.sync). Now due to caching something are being written all the time
>> but not competently and consistently.
>>
>> If an index gets to disk, with new Nodes (not already used) nodes, then
>> the NodeTable also needs to be written. If it is not, a later lookup of
>> the NodeId can't find the node -> null.
>>
>> If it is transactional, and is always used transactionally, the problem
>> does not arise.
>>
>> You might try TDB2 - it does not allow non-transactional use.
>>
>> In the future, it might get an "autocommit" feature ... but then users@
>> will get questions about why it is so slow. Full SQL compatibility!
>>
>> Andy
>>
>>
>>> Running on Jena 3.6.0 and Ubuntu 2017.10 and java version "1.8.0_121" .
>>>
>>> java.lang.NullPointerException
>>> at
>>> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:203)
>>>
>>> at
>>> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:183)
>>>
>>> at java.util.HashMap.hash(HashMap.java:338)
>>> at java.util.HashMap.containsKey(HashMap.java:595)
>>> at java.util.HashSet.contains(HashSet.java:203)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.get
>>> InputNextUnseen(QueryIterDistinct.java:106)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.has
>>> NextBinding(QueryIterDistinct.java:70)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.has
>>> Next(QueryIteratorBase.java:114)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNex
>>> tBinding(QueryIterSlice.java:76)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.has
>>> Next(QueryIteratorBase.java:114)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.
>>> hasNextBinding(QueryIteratorWrapper.java:39)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.has
>>> Next(QueryIteratorBase.java:114)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.
>>> hasNextBinding(QueryIteratorWrapper.java:39)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.has
>>> Next(QueryIteratorBase.java:114)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.
>>> hasNextBinding(QueryIteratorWrapper.java:39)
>>> at
>>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.has
>>> Next(QueryIteratorBase.java:114)
>>> at
>>> org.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:74)
>>>
>>> * at
>>> org.apache.jena.sparql.engine.ResultSetCheckCondition.hasNex
>>> t(ResultSetCheckCondition.java:55)*
>>> at
>>> scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
>>>
>>> at scala.collection.Iterator$class.toStream(Iterator.scala:1320)
>>> at scala.collection.AbstractIterator.toStream(Iterator.scala:1334)
>>> at
>>> scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
>>> at
>>> scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
>>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
>>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
>>> at
>>> scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
>>> at
>>> scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
>>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
>>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
>>> at scala.collection.generic.Growable$class.loop$1(Growable.
>>> scala:54)
>>> at
>>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:57)
>>> at
>>> scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:183)
>>> at
>>> scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45)
>>> at scala.collection.TraversableLike$class.to(TraversableLike.
>>> scala:590)
>>> at scala.collection.AbstractTraversable.to(Traversable.scala:104)
>>> at
>>> deductions.runtime.sparql_cache.SPARQLHelpers$$anonfun$sparq
>>> lSelectQueryVariablesNT$1.apply(SPARQLHelpers.scala:426)
>>>
>>>
>>>
>>>
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: SPARQL: binding to a graph URI results in NullPointerException
Posted by Andy Seaborne <an...@apache.org>.
An oddity:
The only way I can see to set with a null binding (and not get a
warning) is on the path where the dataset is not a TDB one, but with a
graph in a general dataset.
Is that the setup here?
(that said, tracing down and eliminating all other possible routes back
isn't simple so I might have missed another route)
Andy
On 06/02/18 14:24, Andy Seaborne wrote:
> Can you dump the database?
>
>
> On 06/02/18 10:32, Jean-Marc Vanel wrote:
>> Hi
>>
>> I consistently get a NullPointerException from this query on my test TDB
>> on laptop.
>>
>> SELECT DISTINCT ?thing
>> WHERE {
>> graph ?thing {
>> [] ?p ?O .
>> }
>> } LIMIT 200
>>
>> Looking into code : BindingBase.java line 203 , it appears that a
>> binding's
>> node is null for one SELECT result.
>
> In which case there is a binding and the value (Node) is null.
>
> We can put in null checks to catch it earlier (that would be a good idea
> anyway) but still not at the point where the problem starts because
> there isn't one such point - it depends on cache flush order.
>
>> This binding stands for a named graph.
>
> Just a though - do your named graph have URI names? or are any blank nodes?
>
>> So the database is probably broken. Of course, this is a work and test
>> database whose content has no value, but I try to understand and prevent
>> this to happen on production database.
>> Is a there a bug somewhere ?
>
> Yes :-) Where? now that is the question!
>
>> What is the likely cause ? During tests and debugs, transactions may have
>> not been rolled back , application was killed, etc .
>> How could the database be repaired in such cases ? By doing a tdbdump and
>> recreate the TDB ?
>
> Non-transactional TDB needs to explicitly flush modification to disk
> (TDB.sync). Now due to caching something are being written all the time
> but not competently and consistently.
>
> If an index gets to disk, with new Nodes (not already used) nodes, then
> the NodeTable also needs to be written. If it is not, a later lookup of
> the NodeId can't find the node -> null.
>
> If it is transactional, and is always used transactionally, the problem
> does not arise.
>
> You might try TDB2 - it does not allow non-transactional use.
>
> In the future, it might get an "autocommit" feature ... but then users@
> will get questions about why it is so slow. Full SQL compatibility!
>
> Andy
>
>>
>> Running on Jena 3.6.0 and Ubuntu 2017.10 and java version "1.8.0_121" .
>>
>> java.lang.NullPointerException
>> at
>> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:203)
>>
>> at
>> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:183)
>>
>> at java.util.HashMap.hash(HashMap.java:338)
>> at java.util.HashMap.containsKey(HashMap.java:595)
>> at java.util.HashSet.contains(HashSet.java:203)
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.getInputNextUnseen(QueryIterDistinct.java:106)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.hasNextBinding(QueryIterDistinct.java:70)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
>>
>> at
>> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
>>
>> at
>> org.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:74)
>>
>> * at
>> org.apache.jena.sparql.engine.ResultSetCheckCondition.hasNext(ResultSetCheckCondition.java:55)*
>>
>> at
>> scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
>>
>> at scala.collection.Iterator$class.toStream(Iterator.scala:1320)
>> at scala.collection.AbstractIterator.toStream(Iterator.scala:1334)
>> at
>> scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
>> at
>> scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
>> at
>> scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
>> at
>> scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
>> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
>> at scala.collection.generic.Growable$class.loop$1(Growable.scala:54)
>> at
>> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:57)
>> at
>> scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:183)
>> at
>> scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45)
>> at
>> scala.collection.TraversableLike$class.to(TraversableLike.scala:590)
>> at scala.collection.AbstractTraversable.to(Traversable.scala:104)
>> at
>> deductions.runtime.sparql_cache.SPARQLHelpers$$anonfun$sparqlSelectQueryVariablesNT$1.apply(SPARQLHelpers.scala:426)
>>
>>
>>
>>
Re: SPARQL: binding to a graph URI results in NullPointerException
Posted by Jean-Marc Vanel <je...@gmail.com>.
Thanks Andy for looking at the DB,
I'll see if the bug creating literal RDF subjects is still there.
Although in theory this is just a development database,
I think it is several month old , which speaks well for Jena's robustness !
I start a fresh database, but I keep the damaged one;
tell me if you want binary TDB files .
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: SPARQL: binding to a graph URI results in NullPointerException
Posted by Andy Seaborne <an...@apache.org>.
The dump also contains (around 280):
"https://dr.jones.dk/me/#me" <urn:displayLabel>
"\"https://dr.jones.dk/me/#me\""
<urn:/semforms/labelsGraphUri/fr> .
and more literals-as-subjects
"XMLLiteral" <urn:displayLabel> "\"XMLLiteral\""
<urn:/semforms/labelsGraphUri/fr> .
...
Try:
grep '^"' dump2.nq | wc -l
-> 691
On 06/02/18 20:16, Jean-Marc Vanel wrote:
> Sorry Andy
>
> I did one or 2 more additions to the database,
> and after that I had those very bad messages "Impossibly large object" ,
> during the dump :
The DB is damaged.
When did you last rebuild the database from a complete clean state?
The point t which it got corrupted can be a long time in the past.
Andy
>
> ERROR TDB
> ObjectFileStorage.read[nodes](14903272)[filesize=14989232][file.size()=14989232]:
> Impossibly large object : 1936684402 bytes > filesize-(loc+SizeOfInt)=85956
> org.apache.jena.tdb.base.file.FileException:
> ObjectFileStorage.read[nodes](14903272)[filesize=14989232][file.size()=14989232]:
> Impossibly large object : 1936684402 bytes > filesize-(loc+SizeOfInt)=85956
> at
> org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:348)
> at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)
> at
> org.apache.jena.tdb.store.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:186)
> at
> org.apache.jena.tdb.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:111)
> at
> org.apache.jena.tdb.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:70)
> at
> org.apache.jena.tdb.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:128)
> at
> org.apache.jena.tdb.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:82)
> at
> org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:50)
> at
> org.apache.jena.tdb.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
> at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:129)
> at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:123)
> at
> org.apache.jena.tdb.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:59)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.riot.system.StreamOps.sendQuadsToStream(StreamOps.java:140)
> at org.apache.jena.riot.writer.NQuadsWriter.write$(NQuadsWriter.java:62)
> at org.apache.jena.riot.writer.NQuadsWriter.write(NQuadsWriter.java:45)
> at org.apache.jena.riot.writer.NQuadsWriter.write(NQuadsWriter.java:91)
> at org.apache.jena.riot.RDFWriter.write$(RDFWriter.java:208)
> at org.apache.jena.riot.RDFWriter.output(RDFWriter.java:165)
> at org.apache.jena.riot.RDFWriter.output(RDFWriter.java:112)
> at
> org.apache.jena.riot.RDFWriterBuilder.output(RDFWriterBuilder.java:149)
> at org.apache.jena.riot.RDFDataMgr.write$(RDFDataMgr.java:1269)
> at org.apache.jena.riot.RDFDataMgr.write(RDFDataMgr.java:1162)
> at tdb.tdbdump.exec(tdbdump.java:67)
> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> at tdb.tdbdump.main(tdbdump.java:36)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sbt.Run.invokeMain(Run.scala:67)
> at sbt.Run.run0(Run.scala:61)
> at sbt.Run.sbt$Run$$execute$1(Run.scala:51)
> at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:55)
> at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
> at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
> at sbt.Logger$$anon$4.apply(Logger.scala:84)
> at sbt.TrapExit$App.run(TrapExit.scala:248)
> at java.lang.Thread.run(Thread.java:745)
>
> Exception: sbt.TrapExitSecurityException thrown from the
> UncaughtExceptionHandler in thread "run-main-0"
> -rw-r--r-- 1 jmv jmv 64649634 févr. 6 19:39 dump.nq
>
>
> But, unexpectedly, after restarting once or twice , everything is well
> again .
>
> Answers interleaved below.
>
>
> 2018-02-06 15:24 GMT+01:00 Andy Seaborne <an...@apache.org>:
>
>> Can you dump the database?
>>
>
> I put it here:
> http://jmvanel.free.fr/tmp/dump2.nq.zip
> I has been checked by rapper .
>
>
>> On 06/02/18 10:32, Jean-Marc Vanel wrote:
>>
>>> Hi
>>>
>>> I consistently get a NullPointerException from this query on my test TDB
>>> on laptop.
>>> .....
>>
>>
>>> Just a though - do your named graph have URI names? or are any blank
>> nodes?
>>
>
> No , not any graph named by a blank node; I don't do that, and moreover I
> checked by SPARQL .
> However, there a subject "Celtis pumila" that is not a resource.
>
>
>> So the database is probably broken. Of course, this is a work and test
>>> database whose content has no value, but I try to understand and prevent
>>> this to happen on production database.
>>> Is a there a bug somewhere ?
>>>
>>
>> Yes :-) Where? now that is the question!
>>
>> What is the likely cause ? During tests and debugs, transactions may have
>>> not been rolled back , application was killed, etc .
>>> How could the database be repaired in such cases ? By doing a tdbdump and
>>> recreate the TDB ?
>>>
>>
>> Non-transactional TDB needs to explicitly flush modification to disk
>> (TDB.sync). Now due to caching something are being written all the time
>> but not competently and consistently.
>>
>> If an index gets to disk, with new Nodes (not already used) nodes, then
>> the NodeTable also needs to be written. If it is not, a later lookup of
>> the NodeId can't find the node -> null.
>>
>> If it is transactional, and is always used transactionally, the problem
>> does not arise.
>>
>
> Indeed , that is the case: always used transactionally ( anyway, if I
> forget a transaction in my code , at runtime I get an exception "Note in a
> Transaction" ).
>
>
>>
>> You might try TDB2 - it does not allow non-transactional use.
>>
>
> I did, but got into problems that I didn't have time to investigate .
>
>
>>
>> In the future, it might get an "autocommit" feature ... but then users@
>> will get questions about why it is so slow. Full SQL compatibility!
>>
>
Re: SPARQL: binding to a graph URI results in NullPointerException
Posted by Jean-Marc Vanel <je...@gmail.com>.
Sorry Andy
I did one or 2 more additions to the database,
and after that I had those very bad messages "Impossibly large object" ,
during the dump :
ERROR TDB
ObjectFileStorage.read[nodes](14903272)[filesize=14989232][file.size()=14989232]:
Impossibly large object : 1936684402 bytes > filesize-(loc+SizeOfInt)=85956
org.apache.jena.tdb.base.file.FileException:
ObjectFileStorage.read[nodes](14903272)[filesize=14989232][file.size()=14989232]:
Impossibly large object : 1936684402 bytes > filesize-(loc+SizeOfInt)=85956
at
org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:348)
at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)
at
org.apache.jena.tdb.store.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:186)
at
org.apache.jena.tdb.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:111)
at
org.apache.jena.tdb.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:70)
at
org.apache.jena.tdb.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:128)
at
org.apache.jena.tdb.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:82)
at
org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:50)
at
org.apache.jena.tdb.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:129)
at org.apache.jena.tdb.lib.TupleLib.quad(TupleLib.java:123)
at
org.apache.jena.tdb.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:59)
at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
at
org.apache.jena.riot.system.StreamOps.sendQuadsToStream(StreamOps.java:140)
at org.apache.jena.riot.writer.NQuadsWriter.write$(NQuadsWriter.java:62)
at org.apache.jena.riot.writer.NQuadsWriter.write(NQuadsWriter.java:45)
at org.apache.jena.riot.writer.NQuadsWriter.write(NQuadsWriter.java:91)
at org.apache.jena.riot.RDFWriter.write$(RDFWriter.java:208)
at org.apache.jena.riot.RDFWriter.output(RDFWriter.java:165)
at org.apache.jena.riot.RDFWriter.output(RDFWriter.java:112)
at
org.apache.jena.riot.RDFWriterBuilder.output(RDFWriterBuilder.java:149)
at org.apache.jena.riot.RDFDataMgr.write$(RDFDataMgr.java:1269)
at org.apache.jena.riot.RDFDataMgr.write(RDFDataMgr.java:1162)
at tdb.tdbdump.exec(tdbdump.java:67)
at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
at tdb.tdbdump.main(tdbdump.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sbt.Run.invokeMain(Run.scala:67)
at sbt.Run.run0(Run.scala:61)
at sbt.Run.sbt$Run$$execute$1(Run.scala:51)
at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:55)
at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
at sbt.Logger$$anon$4.apply(Logger.scala:84)
at sbt.TrapExit$App.run(TrapExit.scala:248)
at java.lang.Thread.run(Thread.java:745)
Exception: sbt.TrapExitSecurityException thrown from the
UncaughtExceptionHandler in thread "run-main-0"
-rw-r--r-- 1 jmv jmv 64649634 févr. 6 19:39 dump.nq
But, unexpectedly, after restarting once or twice , everything is well
again .
Answers interleaved below.
2018-02-06 15:24 GMT+01:00 Andy Seaborne <an...@apache.org>:
> Can you dump the database?
>
I put it here:
http://jmvanel.free.fr/tmp/dump2.nq.zip
I has been checked by rapper .
> On 06/02/18 10:32, Jean-Marc Vanel wrote:
>
>> Hi
>>
>> I consistently get a NullPointerException from this query on my test TDB
>> on laptop.
>> .....
>
>
>> Just a though - do your named graph have URI names? or are any blank
> nodes?
>
No , not any graph named by a blank node; I don't do that, and moreover I
checked by SPARQL .
However, there a subject "Celtis pumila" that is not a resource.
> So the database is probably broken. Of course, this is a work and test
>> database whose content has no value, but I try to understand and prevent
>> this to happen on production database.
>> Is a there a bug somewhere ?
>>
>
> Yes :-) Where? now that is the question!
>
> What is the likely cause ? During tests and debugs, transactions may have
>> not been rolled back , application was killed, etc .
>> How could the database be repaired in such cases ? By doing a tdbdump and
>> recreate the TDB ?
>>
>
> Non-transactional TDB needs to explicitly flush modification to disk
> (TDB.sync). Now due to caching something are being written all the time
> but not competently and consistently.
>
> If an index gets to disk, with new Nodes (not already used) nodes, then
> the NodeTable also needs to be written. If it is not, a later lookup of
> the NodeId can't find the node -> null.
>
> If it is transactional, and is always used transactionally, the problem
> does not arise.
>
Indeed , that is the case: always used transactionally ( anyway, if I
forget a transaction in my code , at runtime I get an exception "Note in a
Transaction" ).
>
> You might try TDB2 - it does not allow non-transactional use.
>
I did, but got into problems that I didn't have time to investigate .
>
> In the future, it might get an "autocommit" feature ... but then users@
> will get questions about why it is so slow. Full SQL compatibility!
>
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Re: SPARQL: binding to a graph URI results in NullPointerException
Posted by Andy Seaborne <an...@apache.org>.
Can you dump the database?
On 06/02/18 10:32, Jean-Marc Vanel wrote:
> Hi
>
> I consistently get a NullPointerException from this query on my test TDB
> on laptop.
>
> SELECT DISTINCT ?thing
> WHERE {
> graph ?thing {
> [] ?p ?O .
> }
> } LIMIT 200
>
> Looking into code : BindingBase.java line 203 , it appears that a binding's
> node is null for one SELECT result.
In which case there is a binding and the value (Node) is null.
We can put in null checks to catch it earlier (that would be a good idea
anyway) but still not at the point where the problem starts because
there isn't one such point - it depends on cache flush order.
> This binding stands for a named graph.
Just a though - do your named graph have URI names? or are any blank nodes?
> So the database is probably broken. Of course, this is a work and test
> database whose content has no value, but I try to understand and prevent
> this to happen on production database.
> Is a there a bug somewhere ?
Yes :-) Where? now that is the question!
> What is the likely cause ? During tests and debugs, transactions may have
> not been rolled back , application was killed, etc .
> How could the database be repaired in such cases ? By doing a tdbdump and
> recreate the TDB ?
Non-transactional TDB needs to explicitly flush modification to disk
(TDB.sync). Now due to caching something are being written all the time
but not competently and consistently.
If an index gets to disk, with new Nodes (not already used) nodes, then
the NodeTable also needs to be written. If it is not, a later lookup of
the NodeId can't find the node -> null.
If it is transactional, and is always used transactionally, the problem
does not arise.
You might try TDB2 - it does not allow non-transactional use.
In the future, it might get an "autocommit" feature ... but then users@
will get questions about why it is so slow. Full SQL compatibility!
Andy
>
> Running on Jena 3.6.0 and Ubuntu 2017.10 and java version "1.8.0_121" .
>
> java.lang.NullPointerException
> at
> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:203)
> at
> org.apache.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:183)
> at java.util.HashMap.hash(HashMap.java:338)
> at java.util.HashMap.containsKey(HashMap.java:595)
> at java.util.HashSet.contains(HashSet.java:203)
> at
> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.getInputNextUnseen(QueryIterDistinct.java:106)
> at
> org.apache.jena.sparql.engine.iterator.QueryIterDistinct.hasNextBinding(QueryIterDistinct.java:70)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
> at
> org.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
> at
> org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
> at
> org.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:74)
> * at
> org.apache.jena.sparql.engine.ResultSetCheckCondition.hasNext(ResultSetCheckCondition.java:55)*
> at
> scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
> at scala.collection.Iterator$class.toStream(Iterator.scala:1320)
> at scala.collection.AbstractIterator.toStream(Iterator.scala:1334)
> at
> scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
> at
> scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1320)
> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
> at
> scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
> at
> scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)
> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)
> at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)
> at scala.collection.generic.Growable$class.loop$1(Growable.scala:54)
> at
> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:57)
> at
> scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:183)
> at
> scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45)
> at scala.collection.TraversableLike$class.to(TraversableLike.scala:590)
> at scala.collection.AbstractTraversable.to(Traversable.scala:104)
> at
> deductions.runtime.sparql_cache.SPARQLHelpers$$anonfun$sparqlSelectQueryVariablesNT$1.apply(SPARQLHelpers.scala:426)
>
>
>