[jira] [Commented] (HBASE-11059) ZK-less region assignment

["stack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-11059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018262#comment-14018262 ] 

stack commented on HBASE-11059:
-------------------------------

-public class RegionState implements org.apache.hadoop.io.Writable {
+public class RegionState {

Above is good.  We serialize RegionState ever?  Nvm... I see RegionStatusProto now.

On:

-      ZKUtil.createAndFailSilent(this, assignmentZNode);
+      if (conf.getBoolean("hbase.assignment.usezk", false)) {
+        ZKUtil.createAndFailSilent(this, assignmentZNode);
+      }


Should we remove the assignment znode dir if it exists?  (NVM... I see you do it on startup).

Leave of this sentence I'd say: " The other
+   * way around is not recommended also sophisticated users could do it
+   * somehow."

Put this in release note or lets open issue to ensure the below gets added to the 1.0 migration doc:

+   * For rolling upgrade from using ZK to not using ZK, it can be
+   * done in two steps:
+   * 1. Set hbase.assignment.usezk to true, do a rolling upgrade
+   * so that both masters and regionservers have the new code. Either
+   * master or regionserver can be upgraded at first. The order
+   * is not important for this step.
+   * 2. Set hbase.assignment.usezk to false, do a rolling restart
+   * so that region assignments don't use ZK any more. For this step,
+   * masters should be restarted after regionservers have all
+   * restarted at first so that they won't update meta table
+   * directly and master doesn't know about it.

This message will come out wrong?

+    LOG.info("Finished region assignment in (failover=" + failover
+      + ")" + (System.currentTimeMillis() - startTime) + "ms");

It will have 'failover=true' in between 'in' and time in ms... should it be on end?

It is a pity we have to pass this flag useZKForAssignment down:

-                unassign(regionInfo, rsClosing, expectedVersion, null, true, null);
+                unassign(regionInfo, rsClosing, expectedVersion, null, useZKForAssignment, null);

Can this be done in a 'transaction'?

+      mergingRegions.remove(encodedName);
+      regionOffline(a, State.MERGED);
+      regionOffline(b, State.MERGED);
+      regionOnline(p, sn, 1);

We have transactions right as long as all in the same region?  Which is the case when only single meta.

I skimmed the rest.

This looks amazing Jimmy.  I'd have thought it would have taken way more code.

Looking at tests, I was wondering if the new state transitions could be tested independent of the servers?  That possible at all?  Should be easier now no zk?  Can the state management classes be stood up independent of the Master?

I can see this going in if your testing works out.  We'd have it off on commit.  Need to add doc on new assignment state of affairs.  You have it written up in code.  Would just copy it into releases notes and/or into refguide.  We'd open new blocker against 1.0 to discuss whether to have it on for 1.0 (it'd be great if we could have this on... I think it'll save orders of magnitude more headache than it causes).  If it is on for 1.0, we'd have to doc the rolling upgrade.

> ZK-less region assignment
> -------------------------
>
>                 Key: HBASE-11059
>                 URL: https://issues.apache.org/jira/browse/HBASE-11059
>             Project: HBase
>          Issue Type: Improvement
>          Components: master, Region Assignment
>            Reporter: Jimmy Xiang
>            Assignee: Jimmy Xiang
>         Attachments: hbase-11059.patch, hbase-11059_v2.patch, zk-less_am.pdf
>
>
> It seems that most people don't like region assignment with ZK (HBASE-5487), which causes many uncertainties. This jira is to support ZK-less region assignment. We need to make sure this patch doesn't break backward compatibility/rolling upgrade.



--
This message was sent by Atlassian JIRA
(v6.2#6252)