[jira] [Commented] (HBASE-24250) CatalogJanitor resubmits GCMultipleMergedRegionsProcedure for the same region

["Huaxiang Sun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-24250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097774#comment-17097774 ] 

Huaxiang Sun commented on HBASE-24250:
--------------------------------------

[~niuyulin] and [~timoha],  I spent some time on this issue today.

To prevent CJ from submitting multiple GCMultipleMergedRegionsProcedure, we can maintain a in-memory hashset for the child region. Pseudo code as follows, it does not handle cases like master switchover nicely as the in-memory hashset is lost, but should be good enough to reduce most duplicate GCMultipleMergedRegionsProcedures.

For split case, it can check the regionMap to prevent duplicate procedure submissions.

What do you think?
{code:java}
HashSet<RegionInfo> newOngoingMergeProcedures = new HashSet<>();
for (Map.Entry<RegionInfo, Result> e : mergedRegions.entrySet()) {
  if (this.services.isInMaintenanceMode()) {
    // Stop cleaning if the master is in maintenance mode
    break;
  }
  // New pseudo code
  newOngoingMergeProcedures.add(e.getKey()); 
  if (this.ongoingMergeProcedures.contains(e.getValue)) {
     continue;
  }

  List<RegionInfo> parents = MetaTableAccessor.getMergeRegions(e.getValue().rawCells());
  if (parents != null && cleanMergeRegion(e.getKey(), parents)) {
    gcs++;
  }
}

 // New pseudo code
this.ongoingMergeProcedures = newOngoingMergeProcedures;{code}
 

> CatalogJanitor resubmits GCMultipleMergedRegionsProcedure for the same region
> -----------------------------------------------------------------------------
>
>                 Key: HBASE-24250
>                 URL: https://issues.apache.org/jira/browse/HBASE-24250
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 2.2.4
>         Environment: hdfs 3.1.3 with erasure coding
> hbase 2.2.4
>            Reporter: Andrey Elenskiy
>            Assignee: niuyulin
>            Priority: Major
>
> If a lot of regions were merged (due to change of region sizes, for example), there can be a long backlog of procedures to clean up the merged regions. If going through this backlog is slower than the CatalogJanitor's scan interval, it will end resubmitting GCMultipleMergedRegionsProcedure for the same regions over and over again.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)