You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Bill Roberts <bi...@swirrl.com> on 2012/03/29 14:39:12 UTC
'impossibly large object' from TDB
Hi there
Just experimenting with ways of loading data into Fuseki (now 0.2.1 distribution version from 16 March).
Have been loading some data with the HTTP graph protocol. Then stopped Fuseki and loaded another graph using tdbloader (0.9.0). Then started fuseki. Trying to access the data I loaded via tdbloader gave an exception:
Extract of stacktrace:
13:23:24 ERROR TDB :: ObjectFileStorage.read[nodes.dat](314300772)[filesize=317885149][file.size()=317885149]: Impossibly large object : 825376256 bytes > filesize-(loc+SizeOfInt)=3584373
13:23:24 ERROR BindingTDB :: get1(?ds)
com.hp.hpl.jena.tdb.base.file.FileException: ObjectFileStorage.read[nodes.dat](314300772)[filesize=317885149][file.size()=317885149]: Impossibly large object : 825376256 bytes > filesize-(loc+SizeOfInt)=3584373
at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:319)
at com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:72)
at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:176)
at com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:100)
at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:71)
at com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:103)
at com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:74)
at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55)
at com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55)
at com.hp.hpl.jena.tdb.solver.BindingTDB.get1(BindingTDB.java:123)
Am I doing something wrong here? Is there something unsafe about mixing tdbloader and graph protocol loading?
Thanks
Bill
Re: 'impossibly large object' from TDB
Posted by Bill Roberts <bi...@swirrl.com>.
Andy
Thanks once again for your help and advice.
Yes, this question relates to the same data as my previous question. I had already deleted the faulty DB and reverted to a clean backup copy, so can't send you the nodes.dat file, but if it happens again, I'll keep a copy.
The problems have occurred when I have been loading the same .ttl file. I'll email you a copy of that in case it helps. I think everything in it is ASCII, but not impossible that somewhere in it there is 2-or-more char Unicode.
tdbrecovery sounds useful - I had assumed that stopping fuseki (via /etc/init.d) would 'clear up' but sounds like not necessarily. Can't see any docs on tdbrecovery. How should i use that? With a --loc={path to tdb dir} ?
Will set up some newer versions. Since my first and second posts to the list, I upgraded to:
fuseki:
Jena: VERSION: 2.7.0-incubating
Jena: BUILD_DATE: 2011-12-14T14:54:09+0000
ARQ: VERSION: 2.9.0-incubating
ARQ: BUILD_DATE: 2011-12-14T15:04:27+0000
TDB: VERSION: 0.9.0-incubating
TDB: BUILD_DATE: 2012-02-29T19:39:52+0000
Fuseki: VERSION: 0.2.1-incubating
Fuseki: BUILD_DATE: 2012-03-16T15:08:53+0000
and tdbloader:
Jena: VERSION: 2.7.0-incubating
Jena: BUILD_DATE: 2011-12-14T14:54:09+0000
ARQ: VERSION: 2.9.0-incubating
ARQ: BUILD_DATE: 2011-12-14T15:04:27+0000
TDB: VERSION: 0.9.0-incubating
TDB: BUILD_DATE: 2012-02-29T19:39:52+0000
Thanks
Bill
On 29 Mar 2012, at 16:54, Andy Seaborne wrote:
> On 29/03/12 13:39, Bill Roberts wrote:
>> Hi there
>>
>> Just experimenting with ways of loading data into Fuseki (now 0.2.1 distribution version from 16 March).
>>
>> Have been loading some data with the HTTP graph protocol. Then stopped Fuseki and loaded another graph using tdbloader (0.9.0). Then started fuseki. Trying to access the data I loaded via tdbloader gave an exception:
>>
>> Extract of stacktrace:
>>
>> 13:23:24 ERROR TDB :: ObjectFileStorage.read[nodes.dat](314300772)[filesize=317885149][file.size()=317885149]: Impossibly large object : 825376256 bytes> filesize-(loc+SizeOfInt)=3584373
>> 13:23:24 ERROR BindingTDB :: get1(?ds)
>> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFileStorage.read[nodes.dat](314300772)[filesize=317885149][file.size()=317885149]: Impossibly large object : 825376256 bytes> filesize-(loc+SizeOfInt)=3584373
>> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:319)
>> at com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:72)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:176)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:100)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:71)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:103)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:74)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55)
>> at com.hp.hpl.jena.tdb.solver.BindingTDB.get1(BindingTDB.java:123)
>>
>> Am I doing something wrong here? Is there something unsafe about mixing tdbloader and graph protocol loading?
>
> No - it is safe to mix usages providing you let Fuseki stop at a quiet point. There may be outstanding committed transactions that would integrated on next Fuseki start-up.
>
> (or run tdbrecovery -- that flushes the transaction log to the main database).
>
> The bulk loader on an existing database does not do anything special but you must let it finish cleanly.
>
> Is this related to the data from your last message?
>
> Andy
Re: 'impossibly large object' from TDB
Posted by Andy Seaborne <an...@apache.org>.
On 29/03/12 13:39, Bill Roberts wrote:
> Hi there
>
> Just experimenting with ways of loading data into Fuseki (now 0.2.1 distribution version from 16 March).
>
> Have been loading some data with the HTTP graph protocol. Then stopped Fuseki and loaded another graph using tdbloader (0.9.0). Then started fuseki. Trying to access the data I loaded via tdbloader gave an exception:
>
> Extract of stacktrace:
>
> 13:23:24 ERROR TDB :: ObjectFileStorage.read[nodes.dat](314300772)[filesize=317885149][file.size()=317885149]: Impossibly large object : 825376256 bytes> filesize-(loc+SizeOfInt)=3584373
> 13:23:24 ERROR BindingTDB :: get1(?ds)
> com.hp.hpl.jena.tdb.base.file.FileException: ObjectFileStorage.read[nodes.dat](314300772)[filesize=317885149][file.size()=317885149]: Impossibly large object : 825376256 bytes> filesize-(loc+SizeOfInt)=3584373
> at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:319)
> at com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:72)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:176)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:100)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:71)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:103)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:74)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
> at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55)
> at com.hp.hpl.jena.tdb.solver.BindingTDB.get1(BindingTDB.java:123)
>
> Am I doing something wrong here? Is there something unsafe about mixing tdbloader and graph protocol loading?
No - it is safe to mix usages providing you let Fuseki stop at a quiet
point. There may be outstanding committed transactions that would
integrated on next Fuseki start-up.
(or run tdbrecovery -- that flushes the transaction log to the main
database).
The bulk loader on an existing database does not do anything special but
you must let it finish cleanly.
Is this related to the data from your last message?
Andy