[jira] [Resolved] (HBASE-29363) CompactSplit attempts to split non-default regions

["Duo Zhang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-29363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Duo Zhang resolved HBASE-29363.
-------------------------------
    Fix Version/s: 2.7.0
                   3.0.0-beta-2
                   2.6.3
                   2.5.12
     Hadoop Flags: Reviewed
       Resolution: Fixed

Pushed to all active branches.

Thanks [~charlesconnell]!

> CompactSplit attempts to split non-default regions
> --------------------------------------------------
>
>                 Key: HBASE-29363
>                 URL: https://issues.apache.org/jira/browse/HBASE-29363
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Charles Connell
>            Assignee: Charles Connell
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 2.7.0, 3.0.0-beta-2, 2.6.3, 2.5.12
>
>
> I discovered a problem while running a large cluster with region replicas enabled. From time to time, WAL flushes trigger splits, and when the region being split is a secondary replica, this split causes problems. The {{SplitTableRegionProcedure}} does not permit splitting secondary replicas. The RegionServer will send a {{ReportRegionStateTransitionRequest}} to the HMaster, which will send back an error response, logging as it does so:
> {noformat}
> 2025-05-29T16:40:27,585 [RpcServer.priority.RWQ.Fifo.write.handler=12,queue=0,port=60000] ERROR org.apache.hadoop.hbase.ipc.RpcServer: Unexpected throwable object java.lang.IllegalArgumentException: Can't invoke split on non-default regions directly at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:202) ~[hbase-server-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at 
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.<init>(SplitTableRegionProcedure.java:137) ~[hbase-server-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:1159) ~[hbase-server-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.updateRegionSplitTransition(AssignmentManager.java:1359) ~[hbase-server-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportRegionStateTransition(AssignmentManager.java:1207) ~[hbase-server-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportRegionStateTransition(AssignmentManager.java:1247) ~[hbase-server-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at 
> org.apache.hadoop.hbase.master.MasterRpcServices.reportRegionStateTransition(MasterRpcServices.java:1807) ~[hbase-server-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:16722) ~[hbase-protocol-shaded-2.6-hubspot-SNAPSHOT.jar:2.6-hubspot-SNAPSHOT] at ...
> {noformat}
> As a result, the RegionServer will retry the {{ReportRegionStateTransitionRequest}} infinitely. The RegionServer is doing this in a thread pool task that nothing is waiting on, so it's not really bothered by this. In a large cluster, the HMaster request handling threads its the logging system will eventually become saturated by the volume of these requests coming from many RegionServers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)