You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Dave Latham (JIRA)" <ji...@apache.org> on 2010/06/27 05:44:50 UTC
[jira] Created: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Another NPE in ReadWriteConsistencyControl
------------------------------------------
Key: HBASE-2797
URL: https://issues.apache.org/jira/browse/HBASE-2797
Project: HBase
Issue Type: Bug
Affects Versions: 0.20.5
Reporter: Dave Latham
Priority: Blocker
Fix For: 0.20.6
This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
This exception appears in the .out file:
Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
at java.util.PriorityQueue.poll(PriorityQueue.java:523)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886028#action_12886028 ]
Jean-Daniel Cryans commented on HBASE-2797:
-------------------------------------------
Pranav,
Ryan posted a patch for review here http://review.hbase.org/r/241/diff/
Can you try your test with that patch on? If it works, can you +1 the patch?
> Another NPE in ReadWriteConsistencyControl
> ------------------------------------------
>
> Key: HBASE-2797
> URL: https://issues.apache.org/jira/browse/HBASE-2797
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.5
> Reporter: Dave Latham
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.6
>
> Attachments: testDebugNPE.diff
>
>
> This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
> This exception appears in the .out file:
> Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
> at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
> at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
> at java.util.PriorityQueue.poll(PriorityQueue.java:523)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Posted by "Pranav Khaitan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pranav Khaitan updated HBASE-2797:
----------------------------------
Attachment: testDebugNPE.diff
Got a similar NPE error while running a test for HBase-2265. Wrote a simple function which always replicates the NPE error so that it is easy to detect the cause.
java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:45)
at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:559)
at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:73)
at org.apache.hadoop.hbase.regionserver.TestStore.testDebugNPE(TestStore.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:46)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Another NPE in ReadWriteConsistencyControl
> ------------------------------------------
>
> Key: HBASE-2797
> URL: https://issues.apache.org/jira/browse/HBASE-2797
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.5
> Reporter: Dave Latham
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.6
>
> Attachments: testDebugNPE.diff
>
>
> This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
> This exception appears in the .out file:
> Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
> at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
> at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
> at java.util.PriorityQueue.poll(PriorityQueue.java:523)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Posted by "Dave Latham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883548#action_12883548 ]
Dave Latham commented on HBASE-2797:
------------------------------------
Also getting them with the similar stack trace:
Exception in thread "regionserver/192.168.41.19:60020.leaseChecker" java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
at java.util.PriorityQueue.poll(PriorityQueue.java:523)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
at org.apache.hadoop.hbase.regionserver.HRegionServer$ScannerListener.leaseExpired(HRegionServer.java:1962)
at org.apache.hadoop.hbase.Leases.run(Leases.java:98)
> Another NPE in ReadWriteConsistencyControl
> ------------------------------------------
>
> Key: HBASE-2797
> URL: https://issues.apache.org/jira/browse/HBASE-2797
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.5
> Reporter: Dave Latham
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.6
>
>
> This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
> This exception appears in the .out file:
> Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
> at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
> at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
> at java.util.PriorityQueue.poll(PriorityQueue.java:523)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886142#action_12886142 ]
HBase Review Board commented on HBASE-2797:
-------------------------------------------
Message from: "Ryan Rawson" <ry...@gmail.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/241/
-----------------------------------------------------------
(Updated 2010-07-07 15:49:39.980588)
Review request for hbase.
Summary
-------
HBASE-2797 another NPE in ReadWriteConsistencyControl
This addresses bug HBASE-2797.
http://issues.apache.org/jira/browse/HBASE-2797
Diffs
-----
src/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java 737d6af
src/test/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java 1a89d65
Diff: http://review.hbase.org/r/241/diff
Testing
-------
Thanks,
Ryan
> Another NPE in ReadWriteConsistencyControl
> ------------------------------------------
>
> Key: HBASE-2797
> URL: https://issues.apache.org/jira/browse/HBASE-2797
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.5
> Reporter: Dave Latham
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.6
>
> Attachments: testDebugNPE.diff
>
>
> This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
> This exception appears in the .out file:
> Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
> at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
> at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
> at java.util.PriorityQueue.poll(PriorityQueue.java:523)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Posted by "Pranav Khaitan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886149#action_12886149 ]
Pranav Khaitan commented on HBASE-2797:
---------------------------------------
This patch doesn't fix the error I was getting. I see that the fix was only in the peek function. However, the bug I am getting doesnt even touch the peek function. Just instantiating StoreScanner() gives that error. It is possible that the instantiation is not done in the right context but even then I would not expect this kind of error. I have attached a diff file which contains the function I wrote for this bug (its just 4 lines)
> Another NPE in ReadWriteConsistencyControl
> ------------------------------------------
>
> Key: HBASE-2797
> URL: https://issues.apache.org/jira/browse/HBASE-2797
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.5
> Reporter: Dave Latham
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.6
>
> Attachments: testDebugNPE.diff
>
>
> This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
> This exception appears in the .out file:
> Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
> at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
> at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
> at java.util.PriorityQueue.poll(PriorityQueue.java:523)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson resolved HBASE-2797.
--------------------------------
Fix Version/s: 0.90.0
Resolution: Fixed
committed to trunk and branch
> Another NPE in ReadWriteConsistencyControl
> ------------------------------------------
>
> Key: HBASE-2797
> URL: https://issues.apache.org/jira/browse/HBASE-2797
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.5
> Reporter: Dave Latham
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.6, 0.90.0
>
> Attachments: testDebugNPE.diff
>
>
> This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
> This exception appears in the .out file:
> Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
> at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
> at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
> at java.util.PriorityQueue.poll(PriorityQueue.java:523)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2797) Another NPE in
ReadWriteConsistencyControl
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2797:
-------------------------
Assignee: ryan rawson
Ryan, can you take a look at this one boss?
> Another NPE in ReadWriteConsistencyControl
> ------------------------------------------
>
> Key: HBASE-2797
> URL: https://issues.apache.org/jira/browse/HBASE-2797
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.5
> Reporter: Dave Latham
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.6
>
>
> This occurred on a cluster with 46 slaves, running a couple MR jobs. One doing heavy writes copying everything from one table to a new table with a different schema. After one regionserver went down, about 40 of them died within an hour before it was caught and the jobs stopped. Let me know if any other piece of context would be particularly helpful.
> This exception appears in the .out file:
> Exception in thread "regionserver/192.168.41.2:60020" java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532)
> at org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117)
> at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:644)
> at java.util.PriorityQueue.siftDown(PriorityQueue.java:612)
> at java.util.PriorityQueue.poll(PriorityQueue.java:523)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.closeAllRegions(HRegionServer.java:1610)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:621)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.