You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ruslan Sabitov (Jira)" <ji...@apache.org> on 2020/07/06 14:47:00 UTC
[jira] [Commented] (HBASE-24376) MergeNormalizer is merging
non-adjacent regions and causing region overlaps/holes.
[ https://issues.apache.org/jira/browse/HBASE-24376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152065#comment-17152065 ]
Ruslan Sabitov commented on HBASE-24376:
----------------------------------------
I faced with the issue in branch-1 (HBase 1.2.0-cdh5.16.2):
2020-07-02 16:41:18,337 INFO org.apache.hadoop.hbase.master.normalizer.MergeNormalizationPlan: Executing merging normalization plan: MergeNormalizationPlan{firstRegion={ENC
ODED => feba09265266f1c3c090bc42cc90becc, NAME => 'tableName,Aw-BEZ0JD4M3HrvA4Yks,1593695478557.feba09265266f1c3c090bc42cc90becc.', STARTKEY => 'Aw-BEZ0JD4M3HrvA4Yks', EN
DKEY => 'D0sYMT716R0tyHPGk8ii'}, secondRegion={ENCODED => 8003ecbf849c4f5e27bf5956ec0729cc, NAME => 'TableName,B_zjT044PCvwQ4I53Q5m,1593695479990.8003ecbf849c4f5e27bf5956
ec0729cc.', STARTKEY => 'B_zjT044PCvwQ4I53Q5m', ENDKEY => 'FCzjFZ4Vhb0hpVtn6VxP'}}
> MergeNormalizer is merging non-adjacent regions and causing region overlaps/holes.
> ----------------------------------------------------------------------------------
>
> Key: HBASE-24376
> URL: https://issues.apache.org/jira/browse/HBASE-24376
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 2.3.0
> Reporter: Huaxiang Sun
> Assignee: Huaxiang Sun
> Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.4.0
>
>
> Currently, we found normalizer was merging regions which are non-adjacent, it will cause inconsistencies in the cluster.
> {code:java}
> 439055 2020-05-08 17:47:09,814 INFO org.apache.hadoop.hbase.master.normalizer.MergeNormalizationPlan: Executing merging normalization plan: MergeNormalizationPlan{firstRegion={ENCODED => 47fe236a5e3649ded95cb64ad0c08492, NAME => 'TABLE,\x03\x01\x05\x01\x04\x02,1554838974870.47fe236a5e3649ded95cb64ad 0c08492.', STARTKEY => '\x03\x01\x05\x01\x04\x02', ENDKEY => '\x03\x01\x05\x01\x04\x02\x01\x02\x02201904082200\x00\x00\x03Mac\x00\x00\x00\x00\x00\x00\x00\x00\x00iMac13,1\x00\x00\x00\x00\x00\x049.3-14E260\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x05'}, secondRegion={ENCODED => 0c0f2aa67f4329d5c4 8ba0320f173d31, NAME => 'TABLE,\x03\x01\x05\x02\x01\x01,1554830735526.0c0f2aa67f4329d5c48ba0320f173d31.', STARTKEY => '\x03\x01\x05\x02\x01\x01', ENDKEY => '\x03\x01\x05\x02\x01\x02'}}
> 439056 2020-05-08 17:47:11,438 INFO org.apache.hadoop.hbase.ScheduledChore: CatalogJanitor-*****:16000 average execution time: 1676219193 ns.
> 439057 2020-05-08 17:47:11,730 INFO org.apache.hadoop.hbase.master.HMaster: Client=null/null merge regions [47fe236a5e3649ded95cb64ad0c08492], [0c0f2aa67f4329d5c48ba0320f173d31]
> {code}
>
> The root cause is that getMergeNormalizationPlan() uses a list of regionInfo which is ordered by regionName. regionName does not necessary guarantee the order of STARTKEY (let's say 'aa1', 'aa1!', in order of regionName, it will be 'aa1!' followed by 'aa1'. This will result in normalizer merging non-adjacent regions into one and creates overlaps. This is not an issue in branch-1 as the list is already ordered by RegionInfo.COMPARATOR in normalizer.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)