[jira] [Commented] (HBASE-25382) Super Split Meta

["Duo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397763#comment-17397763 ] 

Duo Zhang commented on HBASE-25382:
-----------------------------------

In the discussion of this day, we found this rule in the hbase community.

https://hbase.apache.org/book.html#design.invariants.zk.data

So the propose of storing meta region locations in zookeeper failed.

But after rethinking, I realized that, if we consider this rule as a critical one, then the only solution is master local region. Because for both first meta region as ROOT, or a separated ROOT table, you still need to find a way to store the location of the first meta region or the ROOT table. If you store it on zookeeper, then we still break this rule.

Of course, in the current architecture, the rule has already been broken, as we store the location of meta region on zookeeper. And it is the single source of truth. If this node is missing, there is no simple way for us to recover, we need to use HBCK to recover the cluster. If region server is up, maybe we could know where is the meta region as region server will tell us the region it holds, but if all region servers are down, maybe we need to scan the region server WALs to see where is the meta WAL. Anyway, this require extra maintanence operations so I do not think it could/should be done automatically.

So in general, if we think this rule is very critical, then we should stop and try our best to see how we can solve the problem first, and also for the new approaches, we should also consider to fix it, not keep it or make things worse. If we do not think this is a big deal, then let's change the ref guide, and do not use this as a rule to reject people's proposes.

Thanks.

> Super Split Meta
> ----------------
>
>                 Key: HBASE-25382
>                 URL: https://issues.apache.org/jira/browse/HBASE-25382
>             Project: HBase
>          Issue Type: Umbrella
>          Components: meta
>            Reporter: Michael Stack
>            Priority: Major
>
> Currently, the hbase:meta Table is made of one Region only. It is not splittable. We would like to make it so the hbase:meta table can be split just as we for user-space tables as they grow in size.
> Why split the hbase:meta table?
> {quote}A single Region _hbase:meta_ Table hotspots as cluster size grows. At extreme, hotspotting brings on special-casing with deploys dedicating nodes solely to the hosting of the single _hbase:meta_ Region to better carry the higher load (but also to isolate _hbase:meta_ if heavily trafficked neighbor Regions). Splitting, currently disallowed, will enable distributing the _hbase:meta_ Table Regions, and thereby load, across the cluster. A splittable _hbase:meta_ table will also alleviate concerns enlarging the _hbase:meta_ table whether by adding more meta data per Region entry -- e.g. keeping the list of Region HFiles in the _hbase:meta_ table -- or running with more, smaller Regions rather than a few large Regions.
> {quote}
> This is not the first issue to concern itself with meta splitting (HBASE-11288, HBASE-24950). By aggreement. -- see the base of HBASE-11288 -- this issue supplants all previous JIRAs and design efforts. It is a reset. We start by listing requirements in the attached Super Split Meta Design.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)