You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Andy Seaborne <an...@apache.org> on 2014/02/18 17:12:10 UTC
Re: BlockException: No such block
Ping?
On 31/01/14 09:41, Andy Seaborne wrote:
> Steve,
>
> I have tried more runs, both on disk and using an in-memory (hopefully
> faster so the problem arises earlier). No luck, no more failures seen.
>
> 1/ Could help me understand the test structure some more?
>
> ReaderWriterTest does some setup to fill the two reader graphs. The
> threads are kicked off:
>
> * the readers (2 of them), doing listStatements on their respective graphs
>
> * the writer (1 of) doing two WRITE transactions, one to full it's graph
> and one to delete all it's data.
>
> Are the numbers of readers/writers critical?
>
> 2/ At the crash point is it always:
>
> fromRec=0000000000000000000000000000000000000000000000000000000000000000
> toRec=0000000000000001000000000000000000000000000000000000000000000000
>
> ?
>
> This is always the listStatements of the first reader (reader 2). It's
> the NodeId=0 allocated in ReaderWriterTest setup on the first triple
> added - it will be the nodeid for the graph name.
>
> 3/ Could you run with the following changes:
>
> 3a/ Set SystemTDB.NullOut = true very early e.g.
>
> public static void main(String[] args) {
> SystemTDB.NullOut = true ;
>
> 3b/ model.listStatements() , not using the SimpleSelector
>
>
> Sorry I haven't a test setup - all my trying to find possible better
> test cases hasn't got anywhere.
>
> Do you have other test cases that illustrate the issue?
>
> Andy
>
>
> On 29/01/14 12:26, Steve Groeger wrote:
>> Andy,
>>
>> Adding the code as mentioned in your earlier post I get the following:
>>
>> RecordRangeIterator.iterator -- No such block (pageId=117440534,
>>
>> fromRec=0000000000000000000000000000000000000000000000000000000000000000,
>>
>> toRec=0000000000000001000000000000000000000000000000000000000000000000)
>>
>>
>> Thanks
>> Steve Groeger
>>
>>
>>
>>
>> From: Andy Seaborne <an...@apache.org>
>> To: users@jena.apache.org,
>> Date: 26/01/2014 19:57
>> Subject: Re: BlockException: No such block
>>
>>
>>
>> Steve,
>>
>> I'm not getting systematic failures, even after running multiple times
>> for well over 4 hours. I saw one failure but had several runs with no
>> failure at all. Looks like my setup does not have the right timing.
>>
>> I've changed the internal exception point to print (stderr) some details
>> and exit the VM:
>>
>> if ( ! pageMgr.valid(pageId) ) {
>> String msg =
>> "RecordRangeIterator.iterator -- No such block (pageId="+
>> pageId+
>> ", fromRec="+fromRec+", toRec="+toRec+ ")" ;
>> System.err.println(msg) ;
>> System.exit(0) ;
>> throw new BlockException(msg) ;
>> }
>>
>> Do you have some test servers you could run your test code with the
>> development Jena? Ideally, running multiple tests so we build up a
>> number of outcomes to see if there is a pattern.
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
>>
>>
>>
>> apache-jena/2.11.2-SNAPSHOT for a binary build or use maven directly on
>> apache-jena-libs.
>>
>> The other thing that may well be needed is a more sensitive test case.
>> Running in the debugger is too slow to be practical.
>>
>> What other patterns of transactions have you tried?
>>
>> Andy
>>
>>
>> On 24/01/14 16:00, Steve Groeger wrote:
>>> Andy,
>>>
>>> I am currently running with the TDB being written to my harddrive (of my
>>> Linux RHEL 64bit machine). This failure can sometimes take up to 3-4
>> hours
>>> before it fails.
>>> I have tried using a RAM disk but it fills up before the failure occurs
>> as
>>> I can only give it 10Gb of space and due the test the TDB does get very
>>> large due to all the creates & deletes.
>>> I am running on RHEL 6.4 with the following JVM:
>>> java version "1.6.0"
>>> Java(TM) SE Runtime Environment (build
>>> pxa6460sr15-20131017_01(SR15))
>>> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64
>>> jvmxa6460sr15-20131016_170922 (JIT enabled, AOT enabled)
>>> J9VM - 20131016_170922
>>> JIT - r9_20130920_46510ifx2
>>> GC - GA24_Java6_SR15_20131016_1337_B170922)
>>> JCL - 20131015_01
>>>
>>> Thanks
>>> Steve Groeger
>>> IBM WebSphere Service Registry Development
>>> MP 211, Hursley
>>> Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129
>>> Fax (44) 1962 816800
>>> Lotus Notes: Steve Groeger/UK/IBM
>>> Internet: groeges@uk.ibm.com
>>>
>>>
>>>
>>> From: Andy Seaborne <an...@apache.org>
>>> To: users@jena.apache.org,
>>> Date: 24/01/2014 13:57
>>> Subject: Re: BlockException: No such block
>>>
>>>
>>>
>>> On 24/01/14 07:18, Steve Groeger wrote:
>>>> It took 16690 iterations of the writer thread to obtain this error but
>>> as
>>>> you can see the reader thread gets the exception when trying to execute
>>>> this line:
>>>
>>> How long in time did that take? As it may be a timing issue, do you
>>> have an SSD?
>>>
>>> Running
>>>
>>> java version "1.7.0_51"
>>> OpenJDK Runtime Environment (IcedTea 2.4.4)
>> (7u51-2.4.4-0ubuntu0.13.10.1)
>>> OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
>>>
>>> Andy
>>>
>>>
>>>
>>> Unless stated otherwise above:
>>> IBM United Kingdom Limited - Registered in England and Wales with number
>>> 741598.
>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>> 3AU
>>>
>>
>>
>>
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with number
>> 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>> 3AU
>>
>
Re: BlockException: No such block
Posted by Steve Groeger <GR...@uk.ibm.com>.
Hi Andy,
Apologies, I got tied up with other issues (not Jena related) and also a
few days vacation so I haven't got around to testing with your changes.
1) The description of our test structure is correct. I don't think the
number of readers has any impact, other than the test crashing with Out of
Memory in the JVM as the data cannot be written to the TDB due to there
always being a read with a transaction active.
2) Offg the top of my head I think it is always the toRec and fromRec that
you mentioned but will,need to rerun the tests to confirm.
3) I will pickup one of the latest SNAPHOTs make the changes you have
mentioned and see if I can get the test to fail using that code.
We do have other tests but they are along the same lines as the test I
gave you earliuer so I dont tjhink they will make any difference in your
testing.
Will respond again shortly when I have rerun the tests.
Thanks
Steve Groeger
IBM WebSphere Service Registry Development
MP 211, Hursley
Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129
Fax (44) 1962 816800
Lotus Notes: Steve Groeger/UK/IBM
Internet: groeges@uk.ibm.com
From: Andy Seaborne <an...@apache.org>
To: "users@jena.apache.org" <us...@jena.apache.org>,
Cc: Steve Groeger/UK/IBM@IBMGB
Date: 18/02/2014 16:12
Subject: Re: BlockException: No such block
Ping?
On 31/01/14 09:41, Andy Seaborne wrote:
> Steve,
>
> I have tried more runs, both on disk and using an in-memory (hopefully
> faster so the problem arises earlier). No luck, no more failures seen.
>
> 1/ Could help me understand the test structure some more?
>
> ReaderWriterTest does some setup to fill the two reader graphs. The
> threads are kicked off:
>
> * the readers (2 of them), doing listStatements on their respective
graphs
>
> * the writer (1 of) doing two WRITE transactions, one to full it's graph
> and one to delete all it's data.
>
> Are the numbers of readers/writers critical?
>
> 2/ At the crash point is it always:
>
> fromRec=0000000000000000000000000000000000000000000000000000000000000000
> toRec=0000000000000001000000000000000000000000000000000000000000000000
>
> ?
>
> This is always the listStatements of the first reader (reader 2). It's
> the NodeId=0 allocated in ReaderWriterTest setup on the first triple
> added - it will be the nodeid for the graph name.
>
> 3/ Could you run with the following changes:
>
> 3a/ Set SystemTDB.NullOut = true very early e.g.
>
> public static void main(String[] args) {
> SystemTDB.NullOut = true ;
>
> 3b/ model.listStatements() , not using the SimpleSelector
>
>
> Sorry I haven't a test setup - all my trying to find possible better
> test cases hasn't got anywhere.
>
> Do you have other test cases that illustrate the issue?
>
> Andy
>
>
> On 29/01/14 12:26, Steve Groeger wrote:
>> Andy,
>>
>> Adding the code as mentioned in your earlier post I get the following:
>>
>> RecordRangeIterator.iterator -- No such block (pageId=117440534,
>>
>>
fromRec=0000000000000000000000000000000000000000000000000000000000000000,
>>
>> toRec=0000000000000001000000000000000000000000000000000000000000000000)
>>
>>
>> Thanks
>> Steve Groeger
>>
>>
>>
>>
>> From: Andy Seaborne <an...@apache.org>
>> To: users@jena.apache.org,
>> Date: 26/01/2014 19:57
>> Subject: Re: BlockException: No such block
>>
>>
>>
>> Steve,
>>
>> I'm not getting systematic failures, even after running multiple times
>> for well over 4 hours. I saw one failure but had several runs with no
>> failure at all. Looks like my setup does not have the right timing.
>>
>> I've changed the internal exception point to print (stderr) some
details
>> and exit the VM:
>>
>> if ( ! pageMgr.valid(pageId) ) {
>> String msg =
>> "RecordRangeIterator.iterator -- No such block (pageId="+
>> pageId+
>> ", fromRec="+fromRec+", toRec="+toRec+ ")" ;
>> System.err.println(msg) ;
>> System.exit(0) ;
>> throw new BlockException(msg) ;
>> }
>>
>> Do you have some test servers you could run your test code with the
>> development Jena? Ideally, running multiple tests so we build up a
>> number of outcomes to see if there is a pattern.
>>
>>
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
>>
>>
>>
>> apache-jena/2.11.2-SNAPSHOT for a binary build or use maven directly on
>> apache-jena-libs.
>>
>> The other thing that may well be needed is a more sensitive test case.
>> Running in the debugger is too slow to be practical.
>>
>> What other patterns of transactions have you tried?
>>
>> Andy
>>
>>
>> On 24/01/14 16:00, Steve Groeger wrote:
>>> Andy,
>>>
>>> I am currently running with the TDB being written to my harddrive (of
my
>>> Linux RHEL 64bit machine). This failure can sometimes take up to 3-4
>> hours
>>> before it fails.
>>> I have tried using a RAM disk but it fills up before the failure
occurs
>> as
>>> I can only give it 10Gb of space and due the test the TDB does get
very
>>> large due to all the creates & deletes.
>>> I am running on RHEL 6.4 with the following JVM:
>>> java version "1.6.0"
>>> Java(TM) SE Runtime Environment (build
>>> pxa6460sr15-20131017_01(SR15))
>>> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64
>>> jvmxa6460sr15-20131016_170922 (JIT enabled, AOT enabled)
>>> J9VM - 20131016_170922
>>> JIT - r9_20130920_46510ifx2
>>> GC - GA24_Java6_SR15_20131016_1337_B170922)
>>> JCL - 20131015_01
>>>
>>> Thanks
>>> Steve Groeger
>>> IBM WebSphere Service Registry Development
>>> MP 211, Hursley
>>> Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129
>>> Fax (44) 1962 816800
>>> Lotus Notes: Steve Groeger/UK/IBM
>>> Internet: groeges@uk.ibm.com
>>>
>>>
>>>
>>> From: Andy Seaborne <an...@apache.org>
>>> To: users@jena.apache.org,
>>> Date: 24/01/2014 13:57
>>> Subject: Re: BlockException: No such block
>>>
>>>
>>>
>>> On 24/01/14 07:18, Steve Groeger wrote:
>>>> It took 16690 iterations of the writer thread to obtain this error
but
>>> as
>>>> you can see the reader thread gets the exception when trying to
execute
>>>> this line:
>>>
>>> How long in time did that take? As it may be a timing issue, do you
>>> have an SSD?
>>>
>>> Running
>>>
>>> java version "1.7.0_51"
>>> OpenJDK Runtime Environment (IcedTea 2.4.4)
>> (7u51-2.4.4-0ubuntu0.13.10.1)
>>> OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
>>>
>>> Andy
>>>
>>>
>>>
>>> Unless stated otherwise above:
>>> IBM United Kingdom Limited - Registered in England and Wales with
number
>>> 741598.
>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>> 3AU
>>>
>>
>>
>>
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with
number
>> 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>> 3AU
>>
>
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU