[jira] [Comment Edited] (HBASE-25130) Masters in-memory serverHoldings map is not cleared during hbck repair

["Rahul Kumar (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-25130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258189#comment-17258189 ] 

Rahul Kumar edited comment on HBASE-25130 at 1/4/21, 1:01 PM:
--------------------------------------------------------------

Possible approaches I could think of tackling the above issue:

1. Remove the the region entry from serverHolding map in case of mergeOverlap repair.
 * Once deleteMetaRegion() gets executed, we can call HbckRepair to remove the region from serverHoldings.
 * Here HbckRepair would call ServerManager and ServerManager will manage to call AssignmentManager to remove the region entry from serverHolding map via RegionStates.

2. Keep cleaning out the unwanted entries from serverHoldings map via chore cleaner, if the entries for the region is not present in META. This could be an expensive operation, also would it be safe to cleanup based on the above logic ?

[~apurtell] [~vjasani]  Please let me know your feedback on the above approaches or if we can handle it better. Thanks

 


was (Author: rkrahul324):
Possible approaches I could think of tackling the above issue:

1. Remove the the region entry from serverHolding map in case of mergeOverlap repair.
 * Once deleteMetaRegion() gets executed, we can call HbckRepair to remove the region from serverHoldings.
 * Here HbckRepair would call ServerManager and ServerManager will manage to call AssignmentManager to remove the region entry from serverHolding map via RegionStates.

2. Keep cleaning out the unwanted entries from serverHoldings map via chore cleaner, if the entries for the region is not present in META. This could be an expensive operation, also would it be safe to cleanup based on the above logic ?

[~apurtell] [~vjasani]  Looking for feedback on the above approaches or if we can handle it better.

 

> Masters in-memory serverHoldings map is not cleared during hbck repair
> ----------------------------------------------------------------------
>
>                 Key: HBASE-25130
>                 URL: https://issues.apache.org/jira/browse/HBASE-25130
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Sandeep Guggilam
>            Assignee: Rahul Kumar
>            Priority: Major
>
> {color:#1d1c1d}Incase of repairing overlaps, hbck  essentially calls the closeRegion RPC on RS followed by offline RPC on Master to offline all the overlap regions that would be merged into a new region. {color}
> {color:#1d1c1d}However the offline RPC doesn’t remove it from the serverHoldings map unless the new state is MERGED/SPLIT ([https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java#L719]) b{color}{color:#1d1c1d}ut the new state in this case is OFFLINE. {color}
> {color:#1d1c1d}This is actually intended to match with the META entries and would be removed later when the region is online on a different server. However, in our case , the region would never be online on a new server, hence the region info is never cleared from the map that is used by balancer and SCP for incorrect reeassignment.{color}
> {color:#1d1c1d}We might need to tackle this by removing the entries from the map when hbck actually deletes{color}{color:#1d1c1d} the meta entries for this region which kind of matches the in-memory map’s expectation with the META state.{color}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)