[jira] [Comment Edited] (HBASE-26142) NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero

["chenglei (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387925#comment-17387925 ] 

chenglei edited comment on HBASE-26142 at 7/27/21, 9:26 AM:
------------------------------------------------------------

[~anoop.hbase], the default value of {{hbase.hregion.memstore.mslab.indexchunksize.percent}} is 0.1, that is to say , the byte size of the index chunk pool is {{hbase.regionserver.global.memstore.size}}* 0.1 by default


was (Author: comnetwork):
[~anoop.hbase], the default value of {{hbase.hregion.memstore.mslab.indexchunksize.percent}} is 0.1, that is to say , the byte size of the index chunk pool is {hbase.regionserver.global.memstore.size} * 0.1 by default

> NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero
> -------------------------------------------------------------------------------------------
>
>                 Key: HBASE-26142
>                 URL: https://issues.apache.org/jira/browse/HBASE-26142
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 3.0.0-alpha-1, 2.4.0
>            Reporter: chenglei
>            Assignee: chenglei
>            Priority: Critical
>
> By default, we use {{DefaultMemStore}} , which use no  {{IndexChunk}} and {{ChunkCreator.indexChunksPool}} is useless, so we could set  {{hbase.hregion.memstore.mslab.indexchunksize.percent}} to 0 to save memory space, 
> But after running a while, the {{RegionServer}} throws {{NullPointerException}}  and abort:
> {code:java}
>    Caused by: java.lang.NullPointerException
>         at org.apache.hadoop.hbase.regionserver.ChunkCreator$MemStoreChunkPool.access$900(ChunkCreator.java:310)
>         at org.apache.hadoop.hbase.regionserver.ChunkCreator.putbackChunks(ChunkCreator.java:608)
>         at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.recycleChunks(MemStoreLABImpl.java:297)
>         at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.close(MemStoreLABImpl.java:268)
>         at org.apache.hadoop.hbase.regionserver.Segment.close(Segment.java:149)
>         at org.apache.hadoop.hbase.regionserver.AbstractMemStore.clearSnapshot(AbstractMemStore.java:251)
>         at org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1244)
>         at org.apache.hadoop.hbase.regionserver.HStore.access$700(HStore.java:137)
>         at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2461)
>         at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2963)
> {code}
> The problem is caused by line 608 in {{ChunkCreator.putbackChunks}} :{{Chunk.isIndexChunk}} return true for{{DataChunk}}  , and
>  {{ChunkCreator.indexChunksPool}} is null: 
> {code:java}
> 594 synchronized void putbackChunks(Set<Integer> chunks) {
> 595    // if there is no pool just try to clear the chunkIdMap in case there is something
> 596    if (dataChunksPool == null && indexChunksPool == null) {
> 597      this.removeChunks(chunks);
> 598      return;
> 599    }
> 600
> 601   // if there is a pool, go over all chunk IDs that came back, the chunks may be from pool or not
> 602    for (int chunkID : chunks) {
> 603     // translate chunk ID to chunk, if chunk initially wasn't in pool
> 604      // this translation will (most likely) return null
> 605      Chunk chunk = ChunkCreator.this.getChunk(chunkID);
> 606     if (chunk != null) {
> 607        if (chunk.isFromPool() && chunk.isIndexChunk()) {
> 608          indexChunksPool.putbackChunks(chunk);
> {code}
>  For {{DataChunk}} , {{Chunk.isIndexChunk}} return true because  {{Chunk.isIndexChunk}}  determines the type of {{chunk}} based on {{Chunk.size}}
> {code:java}
>  boolean isIndexChunk() {
>     return size == ChunkCreator.getInstance().getChunkSize(ChunkCreator.ChunkType.INDEX_CHUNK);
>   }
> {code}
> and {{ChunkCreator.getChunkSize}} incorrectly return {{DataChunk}} size when {{ChunkCreator.indexChunksPool}} is null:
> {code:java}
>     int getChunkSize(ChunkType chunkType) {
>     switch (chunkType) {
>       case INDEX_CHUNK:
>         if (indexChunksPool != null) {
>           return indexChunksPool.getChunkSize();
>         }
>       case DATA_CHUNK:
>         if (dataChunksPool != null) {
>           return dataChunksPool.getChunkSize();
>         } else { // When pools are empty
>           return chunkSize;
>         }
>       default:
>         throw new IllegalArgumentException(
>                 "chunkType must either be INDEX_CHUNK or DATA_CHUNK");
>     }
>   }
> {code}
> In my opinion, in addition to erroneous implementation of {{ChunkCreator.getChunkSize}}, we would better not determine the type of {{Chunk}} based on {{Chunk.size}}, because
> {{hbase.hregion.memstore.mslab.indexchunksize.percent}} is set by user and the size of {{IndexChunk}} and {{DataChunk}} could be the same.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)