[jira] [Commented] (HBASE-20166) Make sure the RS/Master can works fine when using table based replication storage layer

["Xu Cang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905680#comment-16905680 ] 

Xu Cang commented on HBASE-20166:
---------------------------------

From high level, this difficulty sounds like HBase needs an external storage to keep the peer info (source/ sink) regardless of HBase service up or down. And HBase startup relies on this info to be present to keep HBase replication logic correct without compromising data integrity.  This "external storage" could be Hfiles, but without letting Regions online, this data cannot be read/understood by HBase.

 

Another thought: 

"try to assign all system table to a rs which only accept regions of system table assignment " 

-- This sounds like a downgrade of high availability for HBase to me. 

 

Thinking it aloud. Does is make sense to have a hybrid solution: letting ZK keeps tracking of peer info (to make source/sink init finish) and store others into HBase table. We could also sync peer info into a table when replication table is online. 

[~openinx]

> Make sure the RS/Master can works fine when using table based replication storage layer
> ---------------------------------------------------------------------------------------
>
>                 Key: HBASE-20166
>                 URL: https://issues.apache.org/jira/browse/HBASE-20166
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Zheng Hu
>            Assignee: Zheng Hu
>            Priority: Major
>         Attachments: HBASE-20166.v1.patch
>
>
> Currently,   we cannot setup the HBase Cluster because the master will list peers before finish its initialization, and if master cannot finish initialization, the meta cannot be online, in other hand, if meta cannot be online, the list peers will never success when using table based replication. a dead loop happen.
> {code}
> 2018-03-09 15:03:50,531 ERROR [M:0;huzheng-xiaomi:46549] helpers.MarkerIgnoringBase(159): ***** ABORTING master huzheng-xiaomi,46549,1520579026550: Unhandled exception. Starting shutdown. *****
> java.io.UncheckedIOException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location for replica 0
> 	at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:55)
> 	at org.apache.hadoop.hbase.replication.TableReplicationPeerStorage.listPeerIds(TableReplicationPeerStorage.java:124)
> 	at org.apache.hadoop.hbase.master.replication.ReplicationPeerManager.create(ReplicationPeerManager.java:335)
> 	at org.apache.hadoop.hbase.master.HMaster.initializeZKBasedSystemTrackers(HMaster.java:737)
> 	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:830)
> 	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2014)
> 	at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:557)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)