[jira] [Issue Comment Deleted] (HBASE-11467) New impl of Registry interface not using ZK + new RPCs on master protocol

["ryan rawson (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-11467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ryan rawson updated HBASE-11467:
--------------------------------

    Comment: was deleted

(was: Attaching draft version of patch (I'm working on improving it) to get initial feedbacks from those who may be interested. Short overview:

I didn't yet figure out what to do with ClusterId, which is used as a text in the secured setup for the handshake. So for now actual code inside connection manager is using default cluster id, but as part of this change I did make an attempt to expose ClusterId at the rest endpoint (for that purpose I moved several classes from hbase-server to hbase-client, namely Client, Cluster and Response, and created REST resource and model in hbase-server module. All this code isn't being used now).
MasterAddressesProvider interface is assumed to provide callers with the address of next master he should attempt to contact, as well as return the full list of all known possible masters. It does not guarantee that next master returned to caller will be up and running at the time callers decides to connect it, so caller still expected to validate it itself (that's what happening in HConnectionImplementation). Also caller shall not assume that call to get next active master costs nothing, as implementation might be doing network-based lookup.
ConfigBasedMasterAddresses provider does simple round robin across all masters he knows of from the config file. Assumptions are that 1) caching of retrieved locations is handled at higher level, in HConnectionImplementation, and this class doesn't care 2) all masters are using the same RPC port, which is retrieved from hbase-site.com, and it's not part of masters quorum property 3) this class doesn't record information about failed servers, handling of failed servers is done separately in RpcClient.FailedServers. This part actually needs to be refined.
HConnectionKey was modified to include now masters quorum string and not ZK-related stuff.
Few new RPC calls (and message types) were added to MasterProtos
Unfortunately, I had to do quite a bit of changes in scaffolding code around mini HBase cluster, because for what I'm doing here I need everyone to know actual master's RPC port, and currently in mini HBase cluster it binds to ephemeral port .
What else..The basic tests (including few humble tests I wrote) are passing, but some LargeTests, including master failover tests, are broken, so I'm working on them now.

I'd prefer to upload the patch to hbase jira as soon as it looks reasonably ready for initial public discussion (to avoid making people wait for too long and also get more feedbacks on design).

Some more notes to emphasize on testing.

Now there are 2 working tests added (1 for config based master addresses provider, and one for RpcRegistry itself, and 1 test for ClusterIdResource REST endpoint, but latter one isn't being used. I'm planning to fix TestMasterFailover integration test and probably add some test cases here to ensure we can handle hops between multiple masters from the client.)

> New impl of Registry interface not using ZK + new RPCs on master protocol
> -------------------------------------------------------------------------
>
>                 Key: HBASE-11467
>                 URL: https://issues.apache.org/jira/browse/HBASE-11467
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Client, Consensus, Zookeeper
>            Reporter: Mikhail Antonov
>            Assignee: Mikhail Antonov
>             Fix For: 2.0.0
>
>         Attachments: HBASE-11467.patch
>
>
> Currently there' only one implementation of Registry interface, which is using ZK to get info about meta. Need to create implementation which will be using  RPC calls to master the client is connected to.
> That includes adding several new methods to master RPC protocol:
> - GetMetaRegionLocation
> - IsTableOnlineState
> - GetCurrentNrHRS
> Additionally, new HTTP endpoint would be added to the master, so anyone can request ClusterId without any authentication.



--
This message was sent by Atlassian JIRA
(v6.2#6252)