You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Wang Dongsheng <ds...@gmail.com> on 2014/09/01 03:53:26 UTC
Re: How to solve the max limit of jena query
Hi Vesse,
Thank you for your answer and sorry to reply so late due to a long time
absent.
But I think I use transaction perfectly for each functions.
And the error does not happen intermittently, it must happen when I do not
put a "limit" keyword.
I don't know whether the two " Impossibly large object" errors mean the
same one, and I still could not find out the solutions. :(
On Thu, Aug 21, 2014 at 12:49 PM, Rob Vesse <rv...@dotnetrdf.org> wrote:
> Please see the TDB FAQs, more specifically "What is the Impossibly Large
> Object exception?"
>
> https://jena.apache.org/documentation/tdb/faqs.html#impossibly-large-object
>
> Rob
>
> On 20/08/2014 18:25, "Wang Dongsheng" <ds...@gmail.com> wrote:
>
> >Hi, all
> >
> >If I put a "limit 200" at the end of the query it works, but If I do not
> >set a limit keyword the error happened as the following.
> >
> >-----------error code location----------------
> >
> >ResultSet rs = qExec.execSelect();
> >while(rs.hasNext()){ // here is error of
> >(DiseaseTDBConnection.java:745)
> > .....
> >}
> >---------------------------
> >
> >com.hp.hpl.jena.tdb.base.file.FileException:
> >ObjectFileStorage.read[nodes.dat](47756900)[filesize=295558078][file.size(
> >)=295558078]:
> >Impossibly large object : 1630433138 bytes >
> >filesize-(loc+SizeOfInt)=247801174
> >
> com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStor
> >age.java:346)
> > com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTable
> >Native.java:178)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeT
> >ableNative.java:103)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableN
> >ative.java:74)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTa
> >bleCache.java:103)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCa
> >che.java:74)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTable
> >Wrapper.java:55)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableI
> >nline.java:67)
> >
> com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTable
> >Wrapper.java:55)
> > com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:159)
> > com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:153)
> > com.hp.hpl.jena.tdb.lib.TupleLib.access$100(TupleLib.java:45)
> > com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:87)
> > com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:83)
> > org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:299)
> >
> com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTD
> >BBase.java:199)
> >
> com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTD
> >BBase.java:187)
> >
> com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:9
> >4)
> >
> org.apache.jena.atlas.iterator.IteratorConcat.next(IteratorConcat.java:81
> >)
> > org.apache.jena.atlas.iterator.Iter$3.hasNext(Iter.java:184)
> >
> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.jav
> >a:90)
> >
> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.jav
> >a:90)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMappe
> >r.hasNextBinding(QueryIterTriplePattern.java:151)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >g(QueryIterRepeatApply.java:81)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >(QueryIterRepeatApply.java:108)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >g(QueryIterRepeatApply.java:67)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >(QueryIterRepeatApply.java:108)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >g(QueryIterRepeatApply.java:67)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >(QueryIterRepeatApply.java:108)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >g(QueryIterRepeatApply.java:67)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >(QueryIterRepeatApply.java:108)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >g(QueryIterRepeatApply.java:67)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >(QueryIterRepeatApply.java:108)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >g(QueryIterRepeatApply.java:67)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >(QueryIterRepeatApply.java:108)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >g(QueryIterRepeatApply.java:67)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBindi
> >ng(QueryIterBlockTriples.java:64)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> org.apache.jena.atlas.data.AbstractDataBag.addAll(AbstractDataBag.java:74
> >)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterSort$SortedBindingIterato
> >r.initializeIterator(QueryIterSort.java:89)
> >
> org.apache.jena.atlas.iterator.IteratorDelayedInitialization.init(Iterato
> >rDelayedInitialization.java:40)
> >
> org.apache.jena.atlas.iterator.IteratorDelayedInitialization.hasNext(Iter
> >atorDelayedInitialization.java:50)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterPlainWrapper.hasNextBindi
> >ng(QueryIterPlainWrapper.java:54)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(Qu
> >eryIterConvert.java:59)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
> >g(QueryIteratorWrapper.java:40)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
> >g(QueryIteratorWrapper.java:40)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
> >g(QueryIteratorWrapper.java:40)
> >
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >ratorBase.java:112)
> >
> com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.jav
> >a:75)
> >
> com.wds.util.DiseaseTDBConnection.htmlQuery(DiseaseTDBConnection.java:745
> >)
> > com.wds.servlet.relateness.ListReg2Dis.service(ListReg2Dis.java:86)
> > javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
> >
> >
> >Thanks in advance if anyone can give some advise..
>
>
>
>
>
Re: How to solve the max limit of jena query
Posted by Wang Dongsheng <ds...@gmail.com>.
Hi Vesse,
Thank you , I solved the problem by reconstructing the TDB from scratch and
from original ttl files.
You are right, seems like the construction process itself should be
strictly protected by transaction.
Thank you again, :)
On Mon, Sep 1, 2014 at 3:18 PM, Rob Vesse <rv...@dotnetrdf.org> wrote:
> Hi
>
> Well you haven't shown a minimal complete code sample so we can't tell if
> you "use transaction perfectly". Even if you are using transactions
> correctly now the corruption may have happened at some point in the past
> when you weren't doing so (or if you've had this database around for
> several years from a time before TDB supported transactions)
>
> The fact that the error only happens some of the time means it is an
> intermittent error. That it happens when you don't include a LIMIT is
> down the fact that without a LIMIT your queries are more likely to touch
> the entire database and therefore more likely to try and read the
> corrupted portion of the database.
>
> The solution is to rebuild a fresh database from the original source data,
> there is no way to repair the corruption
>
> Rob
>
> On 01/09/2014 02:53, "Wang Dongsheng" <ds...@gmail.com> wrote:
>
> >Hi Vesse,
> >
> >Thank you for your answer and sorry to reply so late due to a long time
> >absent.
> >
> >But I think I use transaction perfectly for each functions.
> >
> >And the error does not happen intermittently, it must happen when I do not
> >put a "limit" keyword.
> >
> >I don't know whether the two " Impossibly large object" errors mean the
> >same one, and I still could not find out the solutions. :(
> >
> >
> >
> >On Thu, Aug 21, 2014 at 12:49 PM, Rob Vesse <rv...@dotnetrdf.org> wrote:
> >
> >> Please see the TDB FAQs, more specifically "What is the Impossibly Large
> >> Object exception?"
> >>
> >>
> >>
> https://jena.apache.org/documentation/tdb/faqs.html#impossibly-large-obje
> >>ct
> >>
> >> Rob
> >>
> >> On 20/08/2014 18:25, "Wang Dongsheng" <ds...@gmail.com> wrote:
> >>
> >> >Hi, all
> >> >
> >> >If I put a "limit 200" at the end of the query it works, but If I do
> >>not
> >> >set a limit keyword the error happened as the following.
> >> >
> >> >-----------error code location----------------
> >> >
> >> >ResultSet rs = qExec.execSelect();
> >> >while(rs.hasNext()){ // here is error of
> >> >(DiseaseTDBConnection.java:745)
> >> > .....
> >> >}
> >> >---------------------------
> >> >
> >> >com.hp.hpl.jena.tdb.base.file.FileException:
> >>
> >>>ObjectFileStorage.read[nodes.dat](47756900)[filesize=295558078][file.siz
> >>>e(
> >> >)=295558078]:
> >> >Impossibly large object : 1630433138 bytes >
> >> >filesize-(loc+SizeOfInt)=247801174
> >> >
> >>
> >>com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStor
> >> >age.java:346)
> >> > com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTable
> >> >Native.java:178)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeT
> >> >ableNative.java:103)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableN
> >> >ative.java:74)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTa
> >> >bleCache.java:103)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCa
> >> >che.java:74)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTable
> >> >Wrapper.java:55)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableI
> >> >nline.java:67)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTable
> >> >Wrapper.java:55)
> >> > com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:159)
> >> > com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:153)
> >> > com.hp.hpl.jena.tdb.lib.TupleLib.access$100(TupleLib.java:45)
> >> > com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:87)
> >> > com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:83)
> >> > org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:299)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTD
> >> >BBase.java:199)
> >> >
> >>
> >>com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTD
> >> >BBase.java:187)
> >> >
> >>
> >>com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:9
> >> >4)
> >> >
> >>
> >>org.apache.jena.atlas.iterator.IteratorConcat.next(IteratorConcat.java:81
> >> >)
> >> > org.apache.jena.atlas.iterator.Iter$3.hasNext(Iter.java:184)
> >> >
> >>
> >>com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.jav
> >> >a:90)
> >> >
> >>
> >>com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.jav
> >> >a:90)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMappe
> >> >r.hasNextBinding(QueryIterTriplePattern.java:151)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >> >g(QueryIterRepeatApply.java:81)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >> >(QueryIterRepeatApply.java:108)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >> >g(QueryIterRepeatApply.java:67)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >> >(QueryIterRepeatApply.java:108)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >> >g(QueryIterRepeatApply.java:67)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >> >(QueryIterRepeatApply.java:108)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >> >g(QueryIterRepeatApply.java:67)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >> >(QueryIterRepeatApply.java:108)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >> >g(QueryIterRepeatApply.java:67)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >> >(QueryIterRepeatApply.java:108)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >> >g(QueryIterRepeatApply.java:67)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
> >> >(QueryIterRepeatApply.java:108)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
> >> >g(QueryIterRepeatApply.java:67)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBindi
> >> >ng(QueryIterBlockTriples.java:64)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>org.apache.jena.atlas.data.AbstractDataBag.addAll(AbstractDataBag.java:74
> >> >)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterSort$SortedBindingIterato
> >> >r.initializeIterator(QueryIterSort.java:89)
> >> >
> >>
> >>org.apache.jena.atlas.iterator.IteratorDelayedInitialization.init(Iterato
> >> >rDelayedInitialization.java:40)
> >> >
> >>
> >>org.apache.jena.atlas.iterator.IteratorDelayedInitialization.hasNext(Iter
> >> >atorDelayedInitialization.java:50)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterPlainWrapper.hasNextBindi
> >> >ng(QueryIterPlainWrapper.java:54)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(Qu
> >> >eryIterConvert.java:59)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
> >> >g(QueryIteratorWrapper.java:40)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
> >> >g(QueryIteratorWrapper.java:40)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
> >> >g(QueryIteratorWrapper.java:40)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
> >> >ratorBase.java:112)
> >> >
> >>
> >>com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.jav
> >> >a:75)
> >> >
> >>
> >>com.wds.util.DiseaseTDBConnection.htmlQuery(DiseaseTDBConnection.java:745
> >> >)
> >> >
> >>com.wds.servlet.relateness.ListReg2Dis.service(ListReg2Dis.java:86)
> >> > javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
> >> >
> >> >
> >> >Thanks in advance if anyone can give some advise..
> >>
> >>
> >>
> >>
> >>
>
>
>
>
>
Re: How to solve the max limit of jena query
Posted by Rob Vesse <rv...@dotnetrdf.org>.
Hi
Well you haven't shown a minimal complete code sample so we can't tell if
you "use transaction perfectly". Even if you are using transactions
correctly now the corruption may have happened at some point in the past
when you weren't doing so (or if you've had this database around for
several years from a time before TDB supported transactions)
The fact that the error only happens some of the time means it is an
intermittent error. That it happens when you don't include a LIMIT is
down the fact that without a LIMIT your queries are more likely to touch
the entire database and therefore more likely to try and read the
corrupted portion of the database.
The solution is to rebuild a fresh database from the original source data,
there is no way to repair the corruption
Rob
On 01/09/2014 02:53, "Wang Dongsheng" <ds...@gmail.com> wrote:
>Hi Vesse,
>
>Thank you for your answer and sorry to reply so late due to a long time
>absent.
>
>But I think I use transaction perfectly for each functions.
>
>And the error does not happen intermittently, it must happen when I do not
>put a "limit" keyword.
>
>I don't know whether the two " Impossibly large object" errors mean the
>same one, and I still could not find out the solutions. :(
>
>
>
>On Thu, Aug 21, 2014 at 12:49 PM, Rob Vesse <rv...@dotnetrdf.org> wrote:
>
>> Please see the TDB FAQs, more specifically "What is the Impossibly Large
>> Object exception?"
>>
>>
>>https://jena.apache.org/documentation/tdb/faqs.html#impossibly-large-obje
>>ct
>>
>> Rob
>>
>> On 20/08/2014 18:25, "Wang Dongsheng" <ds...@gmail.com> wrote:
>>
>> >Hi, all
>> >
>> >If I put a "limit 200" at the end of the query it works, but If I do
>>not
>> >set a limit keyword the error happened as the following.
>> >
>> >-----------error code location----------------
>> >
>> >ResultSet rs = qExec.execSelect();
>> >while(rs.hasNext()){ // here is error of
>> >(DiseaseTDBConnection.java:745)
>> > .....
>> >}
>> >---------------------------
>> >
>> >com.hp.hpl.jena.tdb.base.file.FileException:
>>
>>>ObjectFileStorage.read[nodes.dat](47756900)[filesize=295558078][file.siz
>>>e(
>> >)=295558078]:
>> >Impossibly large object : 1630433138 bytes >
>> >filesize-(loc+SizeOfInt)=247801174
>> >
>>
>>com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStor
>> >age.java:346)
>> > com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTable
>> >Native.java:178)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeT
>> >ableNative.java:103)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableN
>> >ative.java:74)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTa
>> >bleCache.java:103)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCa
>> >che.java:74)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTable
>> >Wrapper.java:55)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableI
>> >nline.java:67)
>> >
>>
>>com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTable
>> >Wrapper.java:55)
>> > com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:159)
>> > com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:153)
>> > com.hp.hpl.jena.tdb.lib.TupleLib.access$100(TupleLib.java:45)
>> > com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:87)
>> > com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:83)
>> > org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:299)
>> >
>>
>>com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTD
>> >BBase.java:199)
>> >
>>
>>com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.next(GraphTD
>> >BBase.java:187)
>> >
>>
>>com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:9
>> >4)
>> >
>>
>>org.apache.jena.atlas.iterator.IteratorConcat.next(IteratorConcat.java:81
>> >)
>> > org.apache.jena.atlas.iterator.Iter$3.hasNext(Iter.java:184)
>> >
>>
>>com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.jav
>> >a:90)
>> >
>>
>>com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.jav
>> >a:90)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMappe
>> >r.hasNextBinding(QueryIterTriplePattern.java:151)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
>> >g(QueryIterRepeatApply.java:81)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
>> >(QueryIterRepeatApply.java:108)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
>> >g(QueryIterRepeatApply.java:67)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
>> >(QueryIterRepeatApply.java:108)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
>> >g(QueryIterRepeatApply.java:67)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
>> >(QueryIterRepeatApply.java:108)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
>> >g(QueryIterRepeatApply.java:67)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
>> >(QueryIterRepeatApply.java:108)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
>> >g(QueryIterRepeatApply.java:67)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
>> >(QueryIterRepeatApply.java:108)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
>> >g(QueryIterRepeatApply.java:67)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage
>> >(QueryIterRepeatApply.java:108)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBindin
>> >g(QueryIterRepeatApply.java:67)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBindi
>> >ng(QueryIterBlockTriples.java:64)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>org.apache.jena.atlas.data.AbstractDataBag.addAll(AbstractDataBag.java:74
>> >)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterSort$SortedBindingIterato
>> >r.initializeIterator(QueryIterSort.java:89)
>> >
>>
>>org.apache.jena.atlas.iterator.IteratorDelayedInitialization.init(Iterato
>> >rDelayedInitialization.java:40)
>> >
>>
>>org.apache.jena.atlas.iterator.IteratorDelayedInitialization.hasNext(Iter
>> >atorDelayedInitialization.java:50)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterPlainWrapper.hasNextBindi
>> >ng(QueryIterPlainWrapper.java:54)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(Qu
>> >eryIterConvert.java:59)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
>> >g(QueryIteratorWrapper.java:40)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
>> >g(QueryIteratorWrapper.java:40)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBindin
>> >g(QueryIteratorWrapper.java:40)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIte
>> >ratorBase.java:112)
>> >
>>
>>com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.jav
>> >a:75)
>> >
>>
>>com.wds.util.DiseaseTDBConnection.htmlQuery(DiseaseTDBConnection.java:745
>> >)
>> >
>>com.wds.servlet.relateness.ListReg2Dis.service(ListReg2Dis.java:86)
>> > javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
>> >
>> >
>> >Thanks in advance if anyone can give some advise..
>>
>>
>>
>>
>>