[jira] [Commented] (HBASE-22938) Fold all the system tables to hbase:meta

["Duo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17175939#comment-17175939 ] 

Duo Zhang commented on HBASE-22938:
-----------------------------------

As splittable meta is not a technical problem now, let's revive this issue.

If we want to purge the zk dependency, the replication peers and wal files should be moved to other storage systems. In HBASE-15867, we proposed to store them in a system table. But then, there is dead lock issue, as when creating a new wal writer, we need to register it into the replication tracking system, but the registration itself needs to write a WAL out as we use a system table now right? Think of the start up of a fresh new cluster with replication enabled...

A possible solution is to also make special wal writer for system tables, as we do not need to replicate system tables, so we can skip registering the wal writer. Then there is another problem, that we have to add new steps to deal with system table assignment in SCP, as if the replication system table is not online, we can not create new wal writers...

So I think a possible way is to fold the replication system table to meta table, with a separated region, and also a separated family. Since we already have steps in SCP to deal with meta regions first, it does not add new complexity to our recovery code.

Of course there is a downside for storing replication information in a system table or meta table, as we need to make an assumption that these tables will not be replicated. This will make the read replicas feature can not 100% work for meta and system tables, as we can not use in-cluster replication for them to spread the edits to the secondary replicas any more.

Thanks.

> Fold all the system tables to hbase:meta
> ----------------------------------------
>
>                 Key: HBASE-22938
>                 URL: https://issues.apache.org/jira/browse/HBASE-22938
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: Duo Zhang
>            Priority: Major
>
> Quote my post on HBASE-15867 here, on how to deal with the dead lock when we want to store replication queues to hbase:replication table.
> {quote}
> We could add a special prefix in the row key for different system tables, and make a special family for it. For example, for all the records in hbase:acl, we could introduce a prefix like ':::acl:::', since we do not allow ':' in either namespace or table name, so it will not conflict with the existing table related records. And the family could be namd as 'acl'.
> And we could make a special split policy that only splits at these special prefixs, so it will not break any assumptions so far, as all the records for the 'system table' are in the same region.
> {quote}
> And I think there are also other advantages, for example the start up logic can be greatly simplified.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)