You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Wellington Chevreuil (Jira)" <ji...@apache.org> on 2024/11/13 11:11:00 UTC

[jira] [Commented] (HBASE-28956) RSMobFileCleanerChore may close the StoreFileReader object which is being used by Compaction thread

    [ https://issues.apache.org/jira/browse/HBASE-28956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17897889#comment-17897889 ] 

Wellington Chevreuil commented on HBASE-28956:
----------------------------------------------

I have merged this into master and branch-3 branches.  [~guluo], mind open a PR for branch-2 with this change so that we run the pre-commits for java11 and java8?

> RSMobFileCleanerChore may close the StoreFileReader object which is being used by Compaction thread
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-28956
>                 URL: https://issues.apache.org/jira/browse/HBASE-28956
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction, mob
>    Affects Versions: 3.0.0-beta-1
>            Reporter: guluo
>            Assignee: guluo
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.0.0
>
>
> For MOB table, RSMobFileCleanerChore is responsible for cleaning MOB files that are no longer referebced by the region located in the current RegionServer.
> RSMobFileCleanerChore get the  the information of MOB files by reading the storefile, as fallow.
> ```java
> // RSMobFileCleanerChore.chore()
> sf.initReader();
> byte[] mobRefData = sf.getMetadataValue(HStoreFile.MOB_FILE_REFS);
> byte[] bulkloadMarkerData = sf.getMetadataValue(HStoreFile.BULKLOAD_TASK_KEY);
> // close store file to avoid memory leaks
> sf.closeStoreFile(true);
> ```
> There is an issue in here, if the StoreFileReader was not created by RSMobFileCleanerChore, but RSMobFileCleanerChore closed it, which will cause the thread that created the object to be unusable, resuting ERROR finally.
>  
> Reproduction:
> This is an occasional problem, but the probability of its occurrence can be increased by making the following modifications.
> 1. Setting hbase.master.mob.cleaner.period from 24h to 10s, and restart hbase.
> 2. Puting some mob data into a MOB table.
> 3. At the same time, executing compaction command for the MOB table, and it is possible that this problem may occur.
>  
> The error logs as follow.
> ERROR: java.io.IOException: Cannot invoke "org.apache.hadoop.hbase.regionserver.StoreFileReader.getMaxTimestamp()" because the return value of "org.apache.hadoop.hbase.regionserver.HStoreFile.getReader()" is null
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:512)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
>         at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
>         at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
> Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.hadoop.hbase.regionserver.StoreFileReader.getMaxTimestamp()" because the return value of "org.apache.hadoop.hbase.regionserver.HStoreFile.getReader()" is null
>         at org.apache.hadoop.hbase.regionserver.DefaultStoreFileManager.lambda$getUnneededFiles$3(DefaultStoreFileManager.java:235)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)