You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "terry zhang (Created) (JIRA)" <ji...@apache.org> on 2012/03/06 09:58:58 UTC
[jira] [Created] (HBASE-5532) get NPE during MajorCompactionChecker
get NPE during MajorCompactionChecker
--------------------------------------
Key: HBASE-5532
URL: https://issues.apache.org/jira/browse/HBASE-5532
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: terry zhang
We found error log (NullPointerException) below on our online cluster:
2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during
MajorCompactionChecker
Posted by "terry zhang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224955#comment-13224955 ]
terry zhang commented on HBASE-5532:
------------------------------------
yes,Nicolas. This is a race condition and we can use lock to avoid this issue. And now in the function toDetermines if Store should be split also didn't use lock to protect.
{code:title=Store.java|borderStyle=solid}
/**
* Determines if Store should be split
* @return byte[] if store should be split, null otherwise.
*/
public byte[] getSplitPoint() {
this.lock.readLock().lock();
try {
// sanity checks
if (this.storefiles.isEmpty()) {
return null;
}
StoreFile.Reader r = sf.getReader(); * <= check if the reader is null*
if (r == null) {
LOG.warn("Storefile " + sf + " Reader is null");
continue;
}
long size = r.length();
if (size > maxSize) {
// This is the largest one so far
maxSize = size;
largestSf = sf;
}
}
StoreFile.Reader r = largestSf.getReader(); *<= check if the reader is null*
if (r == null) {
LOG.warn("Storefile " + largestSf + " Reader is null");
return null;
}
// Get first, last, and mid keys. Midkey is the key that starts block
// in middle of hfile. Has column and timestamp. Need to return just
// the row we want to split on as midkey.
byte [] midkey = r.midkey();
.....
return null;
}
{code}
> get NPE during MajorCompactionChecker
> --------------------------------------
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: terry zhang
> Attachments: HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5532) get NPE during MajorCompactionChecker
Posted by "terry zhang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
terry zhang updated HBASE-5532:
-------------------------------
Attachment: HBASE-5532.patch
> get NPE during MajorCompactionChecker
> --------------------------------------
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: terry zhang
> Attachments: HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during
MajorCompactionChecker
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223291#comment-13223291 ]
Zhihong Yu commented on HBASE-5532:
-----------------------------------
Which version of HBase were you using ?
> get NPE during MajorCompactionChecker
> --------------------------------------
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: terry zhang
> Attachments: HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during
MajorCompactionChecker
Posted by "Nicolas Spiegelberg (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224761#comment-13224761 ]
Nicolas Spiegelberg commented on HBASE-5532:
--------------------------------------------
Sounds like this is a race condition, so this patch wouldn't be sufficient. We need some way to lock the reader during this query.
> get NPE during MajorCompactionChecker
> --------------------------------------
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: terry zhang
> Attachments: HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5532) get NPE during MajorCompactionChecker
Posted by "terry zhang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
terry zhang updated HBASE-5532:
-------------------------------
Attachment: HBASE-5532-v2.patch
add a read lock during isMajorCompaction() to avoid race condition.
> get NPE during MajorCompactionChecker
> --------------------------------------
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: terry zhang
> Attachments: HBASE-5532-v2.patch, HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during
MajorCompactionChecker
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226622#comment-13226622 ]
stack commented on HBASE-5532:
------------------------------
This looks like patch for 0.90 only?
> get NPE during MajorCompactionChecker
> --------------------------------------
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: terry zhang
> Attachments: HBASE-5532-v2.patch, HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during
MajorCompactionChecker
Posted by "terry zhang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223909#comment-13223909 ]
terry zhang commented on HBASE-5532:
------------------------------------
Hi, Ted. Our is base on 0.90.2 (including many patches from 90.3 till 90.5).
> get NPE during MajorCompactionChecker
> --------------------------------------
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: terry zhang
> Attachments: HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira