You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Osma Suominen <os...@helsinki.fi> on 2018/04/16 11:31:45 UTC
Corrupted TDB2 database
Hi,
We're setting up a new dev server using Fuseki2 3.7.0 and TDB2. We have
been loading some SKOS vocabularies (e.g. LCSH) into the store via
Fuseki (s-put). We created the database with a then current
3.7.0-SNAPSHOT on 2018-03-28 and switched to the final 3.7.0 soon after
it was released.
Today the database somehow got corrupted. SPARQL queries throw an
exception. We tried to use tdb2.tdbcompact, but it gave the same
exception. Traceback:
12:04:11 ERROR NodeTableTRDF :: Bad encoding: NodeId = [0x
1C1E78F3]
org.apache.jena.riot.thrift.RiotThriftException: No conversion to a
Node: <RDF_Term >
at
org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:282)
at
org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:209)
at
org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:81)
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
at
org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
at
org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:114)
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110)
at
org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
at
org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
at
org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
at
java.util.Iterator.forEachRemaining(java.base@9-internal/Iterator.java:120)
at org.apache.jena.tdb2.sys.CopyDSG.lambda$null$0(CopyDSG.java:38)
at org.apache.jena.system.Txn.exec(Txn.java:81)
at org.apache.jena.system.Txn.executeWrite(Txn.java:133)
at org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$1(CopyDSG.java:36)
at org.apache.jena.system.Txn.exec(Txn.java:81)
at org.apache.jena.system.Txn.executeRead(Txn.java:123)
at org.apache.jena.tdb2.sys.CopyDSG.copy(CopyDSG.java:35)
at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:235)
at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:201)
at tdb2.tdbcompact.exec(tdbcompact.java:44)
at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
at tdb2.tdbcompact.main(tdbcompact.java:28)
This is not (at least not yet) an important data store for us so at the
moment we just put the corrupdated database aside and created a new one.
But I thought I'd report the problem here in case someone else has seen
the same.
-Osma
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Corrupted TDB2 database
Posted by Andy Seaborne <an...@apache.org>.
On 19/04/18 14:19, Osma Suominen wrote:
> Hi Andy!
>
> Okay, I finally found the time to package these for you. I will send you
> URL links in a separate message.
-----
12:04:11 ERROR NodeTableTRDF ::
Bad encoding: NodeId = [0x 1C1E78F3]
org.apache.jena.riot.thrift.RiotThriftException:
No conversion to a Node: <RDF_Term >
-----
Some investigation:
[0x 1C1E78F3] is not a valid NodeId - it does not point to decodable
bytes. The number if a file offset. The file is 511,121,653 bytes,
0x1E7718F5, so it is pointing into the middle of the file.
Starting for the start, the last valid nodes is 0x1BFEA0FD. At that
point, something yucky happens. The next bytes can't be decoded. This
may or may not be related.
[0x 1BFEA14C] ** Bad read ** don't know what type: 14
so we have:
[0x 1BFEA0AF] <http://data.europa.eu/esco/isco>
[0x 1BFEA0D3] <http://data.europa.eu/esco/isco/C2631>
[0x 1BFEA0FD]
<http://data.europa.eu/esco/occupation/99492920-e5a5-4dba-9e5a-93193147198c>
[0x 1BFEA14C] ** Bad read ** don't know what type: 14
The file realigns after some junk bytes.
This could be something partial from a crash but it could be relate to
why 0x1C1E78F3 is not valid.
Was anything happening at or after the time that data was loaded?
(I haven't worked up when it was loaded)
Other findings:
I see several "Failed to to LOAD" in the logs connected with some data
transfer issues:
e.g. logs/fuseki.log-20180413
[3] POST http://localhost:3030/finto/update
[line: 453450, col: 14] Bad input stream [java.io.EOFException:
Unexpected end of ZLIB input stream]
[3] 500 Failed to LOAD
'http://dev.finto.fi/download/paikat/paikka-skos.ttl' (26.277 s)
which looks like a truncated HTTP request using compression.
Andy
Re: Corrupted TDB2 database
Posted by Osma Suominen <os...@helsinki.fi>.
Hi Andy!
Okay, I finally found the time to package these for you. I will send you
URL links in a separate message.
I can't really blame you if you decide not to investigate further. We
were surprised by this problem and hadn't made proper preparations, so
we don't have the full history of the database, unfortunately. But the
basic idea is very simple: the data set consists of graphs, each graph
contains one SKOS vocabulary (downloaded from the web), and we have all
of those in files that we uploaded using s-put - some may have been PUT
multiple times. Some of the graphs (STW and EuroVoc) were extended
(using s-post) with mappings to other vocabularies too. No other updates
to the database have been performed apart from those s-put and s-post
operations, until the final tdb2.tdbcompact which failed with the
traceback I gave.
We have now started building anew dataset from scratch and are trying to
be more diligent in logging all the operations, so that if this happens
again, we have a better idea of what led to it.
vg0-root is not shared in any way. For all I know the VMware environment
just provides a normal block device to the VM kernel.
-Osma
Andy Seaborne kirjoitti 17.04.2018 klo 19:34:
> If you could wrap up a database, and data+sequcne of s-puts that would
> be great. I don't have a VMWare environment to try it on but I can try
> to replicate it. I don't know what else to try.
>
> I don't see why a VM would make difference but elsewhere they seem to,
> maybe because some file process runs on the real hardware (docker), or
> may be file locking can be interfered with.
>
> Is vg0-root shared in anyway?
>
> Andy
>
> On 16/04/18 15:21, Osma Suominen wrote:
>> Hi Andy!
>>
>> Forgot to answer the VM part - yes, this is a VM running on VMWare.
>> This is what mount shows:
>>
>> /dev/mapper/vg0-root on / type ext4
>> (rw,relatime,errors=remount-ro,data=ordered)
>>
>> I.e. a normal ext4 filesystem on LVM.
>>
>> We have plenty of VMs pretty much exactly like this and haven't
>> experienced any filesystem related problems that I know of.
>>
>> -Osma
>>
>> PS. I've been trying to reproduce this with smaller data, but so far
>> to no avail. But by digging the logs I noticed that the same problem
>> appeared also in another TDB2 database that's configured within the
>> same Fuseki instance on the same server. Also in that case the errors
>> appeared soon after loading some new triples into specific graphs,
>> overwriting their previous content.
>>
>>>> One other question - is any of this docker or a VM, if so, what is
>>>> the filesystem setup?
>>>
>>> No, this is not Docker. Ubuntu 16.04 LTS amd64 with Java 9:
>>>
>>> openjdk version "9-internal"
>>> OpenJDK Runtime Environment (build
>>> 9-internal+0-2016-04-14-195246.buildd.src)
>>> OpenJDK 64-Bit Server VM (build
>>> 9-internal+0-2016-04-14-195246.buildd.src, mixed mode)
>>>
>>>
>>> Just tell me what you really need (considering the size of the files)
>>> and I will send them to you.
>>>
>>> -Osma
>>>
>>>
>>
>>
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Corrupted TDB2 database
Posted by Andy Seaborne <an...@apache.org>.
If you could wrap up a database, and data+sequcne of s-puts that would
be great. I don't have a VMWare environment to try it on but I can try
to replicate it. I don't know what else to try.
I don't see why a VM would make difference but elsewhere they seem to,
maybe because some file process runs on the real hardware (docker), or
may be file locking can be interfered with.
Is vg0-root shared in anyway?
Andy
On 16/04/18 15:21, Osma Suominen wrote:
> Hi Andy!
>
> Forgot to answer the VM part - yes, this is a VM running on VMWare. This
> is what mount shows:
>
> /dev/mapper/vg0-root on / type ext4
> (rw,relatime,errors=remount-ro,data=ordered)
>
> I.e. a normal ext4 filesystem on LVM.
>
> We have plenty of VMs pretty much exactly like this and haven't
> experienced any filesystem related problems that I know of.
>
> -Osma
>
> PS. I've been trying to reproduce this with smaller data, but so far to
> no avail. But by digging the logs I noticed that the same problem
> appeared also in another TDB2 database that's configured within the same
> Fuseki instance on the same server. Also in that case the errors
> appeared soon after loading some new triples into specific graphs,
> overwriting their previous content.
>
>>> One other question - is any of this docker or a VM, if so, what is
>>> the filesystem setup?
>>
>> No, this is not Docker. Ubuntu 16.04 LTS amd64 with Java 9:
>>
>> openjdk version "9-internal"
>> OpenJDK Runtime Environment (build
>> 9-internal+0-2016-04-14-195246.buildd.src)
>> OpenJDK 64-Bit Server VM (build
>> 9-internal+0-2016-04-14-195246.buildd.src, mixed mode)
>>
>>
>> Just tell me what you really need (considering the size of the files)
>> and I will send them to you.
>>
>> -Osma
>>
>>
>
>
Re: Corrupted TDB2 database
Posted by Osma Suominen <os...@helsinki.fi>.
Hi Andy!
Forgot to answer the VM part - yes, this is a VM running on VMWare. This
is what mount shows:
/dev/mapper/vg0-root on / type ext4
(rw,relatime,errors=remount-ro,data=ordered)
I.e. a normal ext4 filesystem on LVM.
We have plenty of VMs pretty much exactly like this and haven't
experienced any filesystem related problems that I know of.
-Osma
PS. I've been trying to reproduce this with smaller data, but so far to
no avail. But by digging the logs I noticed that the same problem
appeared also in another TDB2 database that's configured within the same
Fuseki instance on the same server. Also in that case the errors
appeared soon after loading some new triples into specific graphs,
overwriting their previous content.
>> One other question - is any of this docker or a VM, if so, what is the
>> filesystem setup?
>
> No, this is not Docker. Ubuntu 16.04 LTS amd64 with Java 9:
>
> openjdk version "9-internal"
> OpenJDK Runtime Environment (build
> 9-internal+0-2016-04-14-195246.buildd.src)
> OpenJDK 64-Bit Server VM (build
> 9-internal+0-2016-04-14-195246.buildd.src, mixed mode)
>
>
> Just tell me what you really need (considering the size of the files)
> and I will send them to you.
>
> -Osma
>
>
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Corrupted TDB2 database
Posted by Osma Suominen <os...@helsinki.fi>.
Hi Andy!
Andy Seaborne kirjoitti 16.04.2018 klo 15:43:
> Could you send me the database? (zip the location, which should have the
> pre-compacted database in it).
I can, if you really want, but it's pretty huge - now 32GB after the
failed compaction. I think it was around 24GB before trying that.
> What changed just before queries started failing?
Looking at the logs...the first failed request with traceback happened
at 10:01 this morning and it looked like this:
2018-04-16 10:01:20,866 INFO Fuseki :: [772] GET
http://localhost:3030/skosmos-demo/sparql?query=PREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0ASELECT+DISTINCT+%28ucase%28str%28substr%28%3Flabel
%2C+1%2C+1%29%29%29+as+%3Fl%29+FROM+%3Chttp%3A%2F%2Feurovoc.europa.eu%2F%3E+WHERE+%7B%0A++%3Fc+skos%3AprefLabel+%3Flabel+.%0A++%3Fc+a+%3Ftype%0A++FILTER%28langMatches%28lang%28%3Flabel%29%2C+%27en%27%29%29%0A++VALUES+%28%3Ftype%29+%7B+%2
8%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23Concept%3E%29+%7D%0A%7D
2018-04-16 10:01:20,866 INFO Fuseki :: [772] Query =
PREFIX skos: <http://www.w3.org/2004/02/skos/core#> SELECT DISTINCT
(ucase(str(substr(?label, 1, 1))) as ?l) FROM
<http://eurovoc.europa.eu/> WHERE { ?c skos:prefLabel
?label . ?c a ?type FILTER(langMatches(lang(?label), 'en')) VALUES
(?type) { (<http://www.w3.org/2004/02/skos/core#Concept>) } }
2018-04-16 10:01:47,247 ERRORNodeTableTRDF :: Bad encoding:
NodeId = [0x 1BFEE810]
2018-04-16 10:01:47,247 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:07:01:47 +0000] "GET /skosmos-demo/sparql" 200 - ""
"EasyRdf HTTP Client"
2018-04-16 10:01:47,248 WARN Fuseki :: [772] RC = 500 : No
conversion to a Node: <RDF_Term >
org.apache.jena.riot.thrift.RiotThriftException: No conversion to a
Node: <RDF_Term >
at
org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:282)
at
org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:209)
at
org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:81)
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
at
org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
at
org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
at
org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
at
org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:117)
at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110)
at
org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
at
org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
at
org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
at org.apache.jena.atlas.iterator.Iter.next(Iter.java:891)
at
org.apache.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:94)
at
org.apache.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMapper.hasNextBinding(QueryIterTriplePattern.java:137)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:74)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:63)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterProcessBinding.hasNextBinding(QueryIterProcessBinding.java:66)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:101)
at
org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:63)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterProcessBinding.hasNextBinding(QueryIterProcessBinding.java:66)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:58)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIterDistinct.getInputNextUnseen(QueryIterDistinct.java:104)
at
org.apache.jena.sparql.engine.iterator.QueryIterDistinct.hasNextBinding(QueryIterDistinct.java:70)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:39)
at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:114)
at
org.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:74)
at
org.apache.jena.sparql.engine.ResultSetCheckCondition.hasNext(ResultSetCheckCondition.java:55)
at
org.apache.jena.sparql.resultset.ResultSetApply.apply(ResultSetApply.java:38)
at
org.apache.jena.riot.resultset.rw.ResultSetWriterJSON.write(ResultSetWriterJSON.java:82)
at
org.apache.jena.riot.resultset.rw.ResultsWriter.write(ResultsWriter.java:127)
at
org.apache.jena.riot.resultset.rw.ResultsWriter.write(ResultsWriter.java:97)
at
org.apache.jena.fuseki.servlets.ResponseResultSet.lambda$generalOutput$1(ResponseResultSet.java:209)
at
org.apache.jena.fuseki.servlets.ResponseResultSet.output(ResponseResultSet.java:225)
at
org.apache.jena.fuseki.servlets.ResponseResultSet.generalOutput(ResponseResultSet.java:215)
at
org.apache.jena.fuseki.servlets.ResponseResultSet.doResponseResultSet$(ResponseResultSet.java:176)
at
org.apache.jena.fuseki.servlets.ResponseResultSet.doResponseResultSet(ResponseResultSet.java:85)
at
org.apache.jena.fuseki.servlets.SPARQL_Query.sendResults(SPARQL_Query.java:393)
at
org.apache.jena.fuseki.servlets.SPARQL_Query.execute(SPARQL_Query.java:272)
at
org.apache.jena.fuseki.servlets.SPARQL_Query.executeWithParameter(SPARQL_Query.java:224)
at
org.apache.jena.fuseki.servlets.SPARQL_Query.perform(SPARQL_Query.java:199)
at
org.apache.jena.fuseki.servlets.ActionService.executeLifecycle(ActionService.java:192)
at
org.apache.jena.fuseki.servlets.ActionService.execCommonWorker(ActionService.java:106)
at
org.apache.jena.fuseki.servlets.ActionBase.doCommon(ActionBase.java:73)
at
org.apache.jena.fuseki.servlets.FusekiFilter.doFilter(FusekiFilter.java:73)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
at
org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
at
org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at
org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at
org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at
org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at
org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at
org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at
org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at
org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at
org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at
org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at
org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
at
org.apache.jena.fuseki.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:285)
at
org.apache.jena.fuseki.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:248)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:561)
at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:334)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:104)
at
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:243)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:679)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:597)
at java.lang.Thread.run(java.base@9-internal/Thread.java:804)
2018-04-16 10:01:47,270 INFO Fuseki :: [772] 500 No
conversion to a Node: <RDF_Term > (26.403 s)
This was a SELECT query issued by Skosmos.
Before that there were around 100 other queries, mostly SELECT and some
CONSTRUCT (ie all read-only). The previous write operations were at
9:22, when a lot of mappings were loaded into the EuroVoc graph using s-put:
2018-04-16 09:02:34,681 INFO Fuseki :: [608] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Fzbw.eu%2Fstw%2F
2018-04-16 09:05:28,753 INFO Fuseki :: [608] Body:
Content-Length=4998676, Content-Type=text/turtle, Charset=utf-8 =>
Turtle : Count=79002 Triples=79002 Quads=0
2018-04-16 09:05:29,497 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:05:29 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:05:29,497 INFO Fuseki :: [608] 200 OK
(174.816 s)
2018-04-16 09:05:29,688 INFO Fuseki :: [609] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Fzbw.eu%2Fstw%2F
2018-04-16 09:05:31,112 INFO Fuseki :: [609] Body:
Content-Length=530242, Content-Type=text/turtle, Charset=utf-8 => Turtle
: Count=7981 Triples=7981 Quads=0
2018-04-16 09:05:31,383 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:05:31 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:05:31,383 INFO Fuseki :: [609] 200 OK (1.694 s)
2018-04-16 09:05:31,539 INFO Fuseki :: [610] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Fzbw.eu%2Fstw%2F
2018-04-16 09:05:31,600 INFO Fuseki :: [610] Body:
Content-Length=28429, Content-Type=text/turtle, Charset=utf-8 => Turtle
: Count=411 Triples=411 Quads=0
2018-04-16 09:05:31,732 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:05:31 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:05:31,732 INFO Fuseki :: [610] 200 OK (193 ms)
2018-04-16 09:05:31,889 INFO Fuseki :: [611] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Fzbw.eu%2Fstw%2F
2018-04-16 09:05:43,161 INFO Fuseki :: [611] Body:
Content-Length=2412964, Content-Type=text/turtle, Charset=utf-8 =>
Turtle : Count=39213 Triples=39213 Quads=0
2018-04-16 09:05:43,835 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:05:43 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:05:43,835 INFO Fuseki :: [611] 200 OK (11.945 s)
2018-04-16 09:05:44,091 INFO Fuseki :: [612] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Fzbw.eu%2Fstw%2F
2018-04-16 09:05:44,354 INFO Fuseki :: [612] Body:
Content-Length=139246, Content-Type=text/turtle, Charset=utf-8 => Turtle
: Count=2064 Triples=2064 Quads=0
2018-04-16 09:05:44,551 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:05:44 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:05:44,551 INFO Fuseki :: [612] 200 OK (459 ms)
2018-04-16 09:19:14,421 INFO Fuseki :: [613] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:05,356 INFO Fuseki :: [613] Body:
Content-Length=12775105, Content-Type=application/rdf+xml, Charset=null
=> RDF/XML : Count=190759 Triples=190759 Quads=0
2018-04-16 09:21:06,855 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:06 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:06,856 INFO Fuseki :: [613] 200 OK
(112.433 s)
2018-04-16 09:21:07,117 INFO Fuseki :: [614] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:35,849 WARN Fuseki :: [line: 57133, col:
170] {W131} String not in Unicode Normal Form C: "Σύμβαση διεθνούς
εμπορίας ειδών της άγριας πανίδας και χλωρίδας που κινδυνεύουν να
εξαφανιστούν; Σύμβαση της Ουάσιν
γκτον"
2018-04-16 09:21:36,231 INFO Fuseki :: [614] Body:
Content-Length=3459059, Content-Type=application/rdf+xml, Charset=null
=> RDF/XML : Count=36493 Triples=36493 Quads=0
2018-04-16 09:21:37,966 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:37 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:37,967 INFO Fuseki :: [614] 200 OK (30.849 s)
2018-04-16 09:21:38,117 INFO Fuseki :: [615] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:38,487 INFO Fuseki :: [615] Body:
Content-Length=138114, Content-Type=application/rdf+xml, Charset=null =>
RDF/XML : Count=758 Triples=758 Quads=0
2018-04-16 09:21:38,625 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:38 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:38,625 INFO Fuseki :: [615] 200 OK (508 ms)
2018-04-16 09:21:38,803 INFO Fuseki :: [616] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:38,810 INFO Fuseki :: [616] Body:
Content-Length=1643, Content-Type=application/rdf+xml, Charset=null =>
RDF/XML : Count=8 Triples=8 Quads=0
2018-04-16 09:21:38,874 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:38 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:38,874 INFO Fuseki :: [616] 200 OK (70 ms)
2018-04-16 09:21:39,043 INFO Fuseki :: [617] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:42,865 ERRORFuseki :: [line: 12863, col:
167] {E202} Expecting XML start or end element(s). String data "
Pristojnost - označba vseh zadev, ki jih opravlja določen organ, tudi
pravica in dolžnost določenega organa, da opravlja določene zadeve.
Pristojnost določajo zakoni." not allowed. Maybe a striping error.
2018-04-16 09:21:42,908 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:42 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:42,909 INFO Fuseki :: [617] 400 Parse
error: [line: 12863, col: 167] {E202} Expecting XML start or end
element(s). String data "
Pristojnost - označba vseh zadev, ki jih opravlja določen organ, tudi
pravica in dolžnost določenega organa, da opravlja določene zadeve.
Pristojnost določajo zakoni." not allowed. Maybe a striping error. (3.865 s)
2018-04-16 09:21:43,079 INFO Fuseki :: [618] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:43,287 INFO Fuseki :: [618] Body:
Content-Length=73708, Content-Type=application/rdf+xml, Charset=null =>
RDF/XML : Count=423 Triples=423 Quads=0
2018-04-16 09:21:43,781 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:43 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:43,781 INFO Fuseki :: [618] 200 OK (701 ms)
2018-04-16 09:21:43,935 INFO Fuseki :: [619] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:44,659 ERRORFuseki :: Bad character in
IRI (space): <http://id.loc.gov/authorities/subjects/sh85067082[space]...>
2018-04-16 09:21:44,692 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:44 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:44,693 INFO Fuseki :: [619] 400 Parse
error: Bad character in IRI (space):
<http://id.loc.gov/authorities/subjects/sh85067082[space]...> (757 ms)
2018-04-16 09:21:44,849 INFO Fuseki :: [620] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:44,898 ERRORFuseki :: Bad character in
IRI (space): <http://catalogue.bnf.fr/ark:/12148/cb11976077b[space]...>
2018-04-16 09:21:44,935 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:44 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:44,939 INFO Fuseki :: [620] 400 Parse
error: Bad character in IRI (space):
<http://catalogue.bnf.fr/ark:/12148/cb11976077b[space]...> (89 ms)
2018-04-16 09:21:45,145 INFO Fuseki :: [621] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:49,455 INFO Fuseki :: [621] Body:
Content-Length=2159676, Content-Type=application/rdf+xml, Charset=null
=> RDF/XML : Count=14080 Triples=14080 Quads=0
2018-04-16 09:21:49,880 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:21:49 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:21:49,880 INFO Fuseki :: [621] 200 OK (4.735 s)
2018-04-16 09:21:50,117 INFO Fuseki :: [622] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:21:58,650 INFO Fuseki :: [622] Body:
Content-Length=5868983, Content-Type=application/rdf+xml, Charset=null
=> RDF/XML : Count=24671 Triples=24671 Quads=0
2018-04-16 09:22:00,221 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:22:00 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:22:00,222 INFO Fuseki :: [622] 200 OK (10.104 s)
2018-04-16 09:22:00,416 INFO Fuseki :: [623] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:22:09,311 INFO Fuseki :: [623] Body:
Content-Length=1948278, Content-Type=application/rdf+xml, Charset=null
=> RDF/XML : Count=18999 Triples=18999 Quads=0
2018-04-16 09:22:10,990 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:22:10 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:22:10,991 INFO Fuseki :: [623] 200 OK (10.574 s)
2018-04-16 09:22:11,235 INFO Fuseki :: [624] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:22:11,241 INFO Fuseki :: [624] Body:
Content-Length=1457, Content-Type=application/rdf+xml, Charset=null =>
RDF/XML : Count=7 Triples=7 Quads=0
2018-04-16 09:22:11,305 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:22:11 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:22:11,305 INFO Fuseki :: [624] 200 OK (70 ms)
2018-04-16 09:22:11,484 INFO Fuseki :: [625] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:22:11,558 ERRORFuseki :: Bad character in
IRI (space): <http://d-nb.info/gnd/4161343-0[space]...>
2018-04-16 09:22:11,595 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:22:11 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:22:11,599 INFO Fuseki :: [625] 400 Parse
error: Bad character in IRI (space):
<http://d-nb.info/gnd/4161343-0[space]...> (114 ms)
2018-04-16 09:22:11,780 INFO Fuseki :: [626] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:22:11,785 INFO Fuseki :: [626] Body:
Content-Length=2916, Content-Type=application/rdf+xml, Charset=null =>
RDF/XML : Count=16 Triples=16 Quads=0
2018-04-16 09:22:11,889 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:22:11 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:22:11,889 INFO Fuseki :: [626] 200 OK (108 ms)
2018-04-16 09:22:12,124 INFO Fuseki :: [627] POST
http://localhost:3030/skosmos-demo?graph=http%3A%2F%2Feurovoc.europa.eu%2F
2018-04-16 09:22:12,137 INFO Fuseki :: [627] Body:
Content-Length=3146, Content-Type=application/rdf+xml, Charset=null =>
RDF/XML : Count=18 Triples=18 Quads=0
2018-04-16 09:22:12,192 INFO Request :: 0:0:0:0:0:0:0:1 - -
[16/Apr/2018:06:22:12 +0000] "POST /skosmos-demo" 200 - "" "SOH/Fuseki
1.0.0"
2018-04-16 09:22:12,192 INFO Fuseki :: [627] 200 OK (67 ms)
I can see now that there were some parse errors, URIs in spaces etc.
(This was done by my colleague and he's away now, so I can't ask for
further details - but apparently he didn't notice anything amiss)
> Do you happen to have the data used to build the database and the Fuseki
> logs?
I do. The data is about 4.5GB unzipped.
I have Fuseki logs going back 2 weeks - not quite enough all the way to
the day the database was created. Just today's log is 4.8GB uncompressed
(probably due to the large number of tracebacks after 10:01), though the
rest are much smaller.
If you really want I can send these to you e.g. via the Funet Filesender
service, which is OK for moving around large files. You will get a
download link by e-mail.
There are no secrets here, as this is intended to be a demo site with
Skosmos (replacing the current skosmos.dev.finto.fi) displaying a number
of SKOS vocabularies that are available for download from their
publishers. We have just collected them and loaded them into Fuseki.
> One other question - is any of this docker or a VM, if so, what is the filesystem setup?
No, this is not Docker. Ubuntu 16.04 LTS amd64 with Java 9:
openjdk version "9-internal"
OpenJDK Runtime Environment (build
9-internal+0-2016-04-14-195246.buildd.src)
OpenJDK 64-Bit Server VM (build
9-internal+0-2016-04-14-195246.buildd.src, mixed mode)
Just tell me what you really need (considering the size of the files)
and I will send them to you.
-Osma
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Corrupted TDB2 database
Posted by Andy Seaborne <an...@apache.org>.
One other question - is any of this docker or a VM, if so, what is the
filesystem setup?
Andy
On 16/04/18 13:43, Andy Seaborne wrote:
> Hi Osma,
>
> Could you send me the database? (zip the location, which should have the
> pre-compacted database in it).
>
> What changed just before queries started failing?
>
> Do you happen to have the data used to build the database and the Fuseki
> logs?
>
> Andy
>
> On 16/04/18 12:31, Osma Suominen wrote:
>> Hi,
>>
>> We're setting up a new dev server using Fuseki2 3.7.0 and TDB2. We
>> have been loading some SKOS vocabularies (e.g. LCSH) into the store
>> via Fuseki (s-put). We created the database with a then current
>> 3.7.0-SNAPSHOT on 2018-03-28 and switched to the final 3.7.0 soon
>> after it was released.
>>
>> Today the database somehow got corrupted. SPARQL queries throw an
>> exception. We tried to use tdb2.tdbcompact, but it gave the same
>> exception. Traceback:
>>
>> 12:04:11 ERROR NodeTableTRDF :: Bad encoding: NodeId = [0x
>> 1C1E78F3]
>> org.apache.jena.riot.thrift.RiotThriftException: No conversion to a
>> Node: <RDF_Term >
>> at
>> org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:282)
>> at
>> org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:209)
>> at
>> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:81)
>>
>> at
>> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
>>
>> at
>> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
>>
>> at
>> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
>>
>> at
>> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
>>
>> at
>> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
>>
>> at
>> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
>>
>> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:114)
>> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110)
>> at
>> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
>>
>> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
>> at
>> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
>>
>> at
>> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
>>
>> at
>> java.util.Iterator.forEachRemaining(java.base@9-internal/Iterator.java:120)
>>
>> at org.apache.jena.tdb2.sys.CopyDSG.lambda$null$0(CopyDSG.java:38)
>> at org.apache.jena.system.Txn.exec(Txn.java:81)
>> at org.apache.jena.system.Txn.executeWrite(Txn.java:133)
>> at org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$1(CopyDSG.java:36)
>> at org.apache.jena.system.Txn.exec(Txn.java:81)
>> at org.apache.jena.system.Txn.executeRead(Txn.java:123)
>> at org.apache.jena.tdb2.sys.CopyDSG.copy(CopyDSG.java:35)
>> at
>> org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:235)
>> at
>> org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:201)
>> at tdb2.tdbcompact.exec(tdbcompact.java:44)
>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
>> at tdb2.tdbcompact.main(tdbcompact.java:28)
>>
>> This is not (at least not yet) an important data store for us so at
>> the moment we just put the corrupdated database aside and created a
>> new one. But I thought I'd report the problem here in case someone
>> else has seen the same.
>>
>> -Osma
>>
>>
Re: Corrupted TDB2 database
Posted by Andy Seaborne <an...@apache.org>.
Hi Osma,
Could you send me the database? (zip the location, which should have the
pre-compacted database in it).
What changed just before queries started failing?
Do you happen to have the data used to build the database and the Fuseki
logs?
Andy
On 16/04/18 12:31, Osma Suominen wrote:
> Hi,
>
> We're setting up a new dev server using Fuseki2 3.7.0 and TDB2. We have
> been loading some SKOS vocabularies (e.g. LCSH) into the store via
> Fuseki (s-put). We created the database with a then current
> 3.7.0-SNAPSHOT on 2018-03-28 and switched to the final 3.7.0 soon after
> it was released.
>
> Today the database somehow got corrupted. SPARQL queries throw an
> exception. We tried to use tdb2.tdbcompact, but it gave the same
> exception. Traceback:
>
> 12:04:11 ERROR NodeTableTRDF :: Bad encoding: NodeId = [0x 1C1E78F3]
> org.apache.jena.riot.thrift.RiotThriftException: No conversion to a
> Node: <RDF_Term >
> at
> org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:282)
> at
> org.apache.jena.riot.thrift.ThriftConvert.convert(ThriftConvert.java:209)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:81)
>
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
>
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
>
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
>
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
>
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
>
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
>
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:114)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:110)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
>
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
>
> at
> java.util.Iterator.forEachRemaining(java.base@9-internal/Iterator.java:120)
> at org.apache.jena.tdb2.sys.CopyDSG.lambda$null$0(CopyDSG.java:38)
> at org.apache.jena.system.Txn.exec(Txn.java:81)
> at org.apache.jena.system.Txn.executeWrite(Txn.java:133)
> at org.apache.jena.tdb2.sys.CopyDSG.lambda$copy$1(CopyDSG.java:36)
> at org.apache.jena.system.Txn.exec(Txn.java:81)
> at org.apache.jena.system.Txn.executeRead(Txn.java:123)
> at org.apache.jena.tdb2.sys.CopyDSG.copy(CopyDSG.java:35)
> at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:235)
> at org.apache.jena.tdb2.sys.DatabaseOps.compact(DatabaseOps.java:201)
> at tdb2.tdbcompact.exec(tdbcompact.java:44)
> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> at tdb2.tdbcompact.main(tdbcompact.java:28)
>
> This is not (at least not yet) an important data store for us so at the
> moment we just put the corrupdated database aside and created a new one.
> But I thought I'd report the problem here in case someone else has seen
> the same.
>
> -Osma
>
>