You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Liang Xie (JIRA)" <ji...@apache.org> on 2014/12/29 06:39:13 UTC
[jira] [Updated] (HBASE-12767) Fix a StoreFileScanner NPE in
reverse scan flow
[ https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liang Xie updated HBASE-12767:
------------------------------
Description:
we observed a NPE in our internal 0.94 branch prd env:
ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
at org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:154)
at org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.<init>(ReversedStoreScanner.java:49)
at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:3984)
at org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.<init>(ReversedRegionScannerImpl.java:43)
at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
at org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
After tracing, it turned out a boundary bug at HalfStoreFileReader. If we have a reverse scan request at "bottom" part, once the start row >= the last row at bottom part, the NPE will be encourtered, since we will invoke "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when we want to do "seekBefore" or other sth in StoreFileScanner, StoreFileScanner.cur will call:
{code}
public Cell getKeyValue() {
if (atEnd) return null;
{code}
The fix is simple, we need to reset the "atEnd" flag once seekBefore op is successful. You can undo my HalfStoreFileReader change and run the new added case to see NPE.
ps: I also found some cases in TestHRegion missing the "@Test" annotation, so fix them in the same patch as well.
was:
we observed a NPE in our internal 0.94 branch prd env:
ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
at org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:154)
at org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.<init>(ReversedStoreScanner.java:49)
at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:3984)
at org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.<init>(ReversedRegionScannerImpl.java:43)
at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
at org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
After tracing, it turned out a boundary bug at HalfStoreFileReader. If we have a reverse scan request at "bottom" part, once the start row >= the last row at bottom part, the NPE will be encourtered, since we will invoke "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when we want to do "seekBefore" or other sth in StoreFileScanner, StoreFileScanner.cur will call:
{code}
public Cell getKeyValue() {
if (atEnd) return null;
{code}
The fix is simple, we need to reset the "atEnd" flag once seekBefore op is successful. You can undo my HalfStoreFileReader change and run the next case to see NPE.
ps: I also found some cases in TestHRegion missing the "@Test" annotation, so fix them in the same patch as well.
> Fix a StoreFileScanner NPE in reverse scan flow
> -----------------------------------------------
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
> Affects Versions: 1.0.0, 2.0.0, 0.98.9
> Reporter: Liang Xie
> Assignee: Liang Xie
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:154)
> at org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.<init>(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:3984)
> at org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.<init>(ReversedRegionScannerImpl.java:43)
> at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we have a reverse scan request at "bottom" part, once the start row >= the last row at bottom part, the NPE will be encourtered, since we will invoke "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when we want to do "seekBefore" or other sth in StoreFileScanner, StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is successful. You can undo my HalfStoreFileReader change and run the new added case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)