[jira] [Updated] (HBASE-26142) NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero

["Duo Zhang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Duo Zhang updated HBASE-26142:
------------------------------
    Fix Version/s: 2.3.7
                   2.4.6
                   3.0.0-alpha-2
                   2.5.0

> NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero
> -------------------------------------------------------------------------------------------
>
>                 Key: HBASE-26142
>                 URL: https://issues.apache.org/jira/browse/HBASE-26142
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 3.0.0-alpha-1, 2.4.0
>            Reporter: chenglei
>            Assignee: chenglei
>            Priority: Critical
>             Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.6, 2.3.7
>
>
> The default value of {{hbase.hregion.memstore.mslab.indexchunksize.percent}} introduced by HBASE-24892 is 0.1, but when we use {{DefaultMemStore}} by default , which has no  {{IndexChunk}} and {{ChunkCreator.indexChunksPool}} is useless({{IndexChunk}} is only used by {{CompactingMemStore}}), so we set  {{hbase.hregion.memstore.mslab.indexchunksize.percent}} to 0 to save memory space, 
> But after running a while, the {{RegionServer}} throws {{NullPointerException}}  and abort:
> {code:java}
>    Caused by: java.lang.NullPointerException
>         at org.apache.hadoop.hbase.regionserver.ChunkCreator$MemStoreChunkPool.access$900(ChunkCreator.java:310)
>         at org.apache.hadoop.hbase.regionserver.ChunkCreator.putbackChunks(ChunkCreator.java:608)
>         at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.recycleChunks(MemStoreLABImpl.java:297)
>         at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.close(MemStoreLABImpl.java:268)
>         at org.apache.hadoop.hbase.regionserver.Segment.close(Segment.java:149)
>         at org.apache.hadoop.hbase.regionserver.AbstractMemStore.clearSnapshot(AbstractMemStore.java:251)
>         at org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1244)
>         at org.apache.hadoop.hbase.regionserver.HStore.access$700(HStore.java:137)
>         at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2461)
>         at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2963)
> {code}
> The problem is caused by line 608 in {{ChunkCreator.putbackChunks}} : {{Chunk.isIndexChunk}} incorrectly returns true for {{DataChunk}}  and
> unexpectedly invokes {{indexChunksPool.putbackChunks}}， while {{indexChunksPool}}  is null: 
> {code:java}
> 594 synchronized void putbackChunks(Set<Integer> chunks) {
> 595    // if there is no pool just try to clear the chunkIdMap in case there is something
> 596    if (dataChunksPool == null && indexChunksPool == null) {
> 597      this.removeChunks(chunks);
> 598      return;
> 599    }
> 600
> 601   // if there is a pool, go over all chunk IDs that came back, the chunks may be from pool or not
> 602    for (int chunkID : chunks) {
> 603     // translate chunk ID to chunk, if chunk initially wasn't in pool
> 604      // this translation will (most likely) return null
> 605      Chunk chunk = ChunkCreator.this.getChunk(chunkID);
> 606     if (chunk != null) {
> 607        if (chunk.isFromPool() && chunk.isIndexChunk()) {
> 608          indexChunksPool.putbackChunks(chunk);
> {code}
> For {{DataChunk}} , {{Chunk.isIndexChunk}} return true because  {{Chunk.isIndexChunk}}  determines the type of {{chunk}} based on {{Chunk.size}}
> {code:java}
>  boolean isIndexChunk() {
>     return size == ChunkCreator.getInstance().getChunkSize(ChunkCreator.ChunkType.INDEX_CHUNK);
>   }
> {code}
> and {{ChunkCreator.getChunkSize}} incorrectly return {{DataChunk}} size when {{ChunkCreator.indexChunksPool}} is null:
> {code:java}
>     int getChunkSize(ChunkType chunkType) {
>     switch (chunkType) {
>       case INDEX_CHUNK:
>         if (indexChunksPool != null) {
>           return indexChunksPool.getChunkSize();
>         }
>       case DATA_CHUNK:
>         if (dataChunksPool != null) {
>           return dataChunksPool.getChunkSize();
>         } else { // When pools are empty
>           return chunkSize;
>         }
>       default:
>         throw new IllegalArgumentException(
>                 "chunkType must either be INDEX_CHUNK or DATA_CHUNK");
>     }
>   }
> {code}
> In my opinion, in addition to erroneous implementation of {{ChunkCreator.getChunkSize}}, we would better not determine the type of {{Chunk}} based on {{Chunk.size}}, because
> {{hbase.hregion.memstore.mslab.indexchunksize.percent}} is set by user and the size of {{IndexChunk}} and {{DataChunk}} could be the same.Tagged a {{ChunkType}} to {{Chunk}} is a better choice.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)