You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (Created) (JIRA)" <ji...@apache.org> on 2012/03/31 01:57:24 UTC
[jira] [Created] (HBASE-5688) Convert zk root-region-server znode
content to pb
Convert zk root-region-server znode content to pb
-------------------------------------------------
Key: HBASE-5688
URL: https://issues.apache.org/jira/browse/HBASE-5688
Project: HBase
Issue Type: Task
Reporter: stack
Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242929#comment-13242929 ]
Todd Lipcon commented on HBASE-5688:
------------------------------------
bq. Tell me more about your PBUF idea. You mean a first field in each message that is always PBUF
Something like:
{code}
public byte[] protoToZKData(Message m) {
return Bytes.concat(Bytes.toString("PBUF"), m.toByteArray())
}
{code}
i.e literally prepend the bytes to the encoded protobuf.
Then when we read, ensure that the first four bytes are PBUF. If not, we can barf and say "looks like this cluster is from the wrong version of HBase, it's missing the PBUF magic header on ZK node /hbase/root-server". Without the magic number, we'd end up getting a weird decoding exception
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244969#comment-13244969 ]
stack commented on HBASE-5688:
------------------------------
Committed addendum.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.addendum.txt, 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243856#comment-13243856 ]
Hadoop QA commented on HBASE-5688:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520807/5688v4.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 11 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.master.TestSplitLogManager
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1359//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1359//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1359//console
This message is automatically generated.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243774#comment-13243774 ]
jiraposter@reviews.apache.org commented on HBASE-5688:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4600/#review6606
-----------------------------------------------------------
src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
<https://reviews.apache.org/r/4600/#comment14273>
I think prefixedWithPBMagic would be a better name for this method.
src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java
<https://reviews.apache.org/r/4600/#comment14274>
Javadoc would be desirable.
src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java
<https://reviews.apache.org/r/4600/#comment14275>
White space.
- Ted
On 2012-04-01 00:18:54, Michael Stack wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4600/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-01 00:18:54)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Changes the content of the root location znode, root-region-server, to be
bq. four magic bytes ('PBUF') followed by a protobuf message that holds the
bq. ServerName of the server currently hosting root.
bq.
bq. D src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
bq. Removed. Had two methods, one to add root-region-server znode and another
bq. to removed it. Rather, put these methods in RootRegionTracker. It
bq. tracks root-region-server znode. Having all to do w/ root-region-server
bq. is more cohesive. Also makes it so can encapsulate in one class
bq. all to do w/ create, delete, and reading of root-region-server.
bq. We also want to purge the catalog package (See note at head of
bq. CatalogTracker).
bq. M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
bq. M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
bq. M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
bq. Get root region location from RootRegionTracker rather than from RootLocationEditor.
bq. A src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
bq. Utility to do w/ protobuf handling. Has methods to help prefixing
bq. and stripping from serialized protobuf messages some 'magic'.
bq. A src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
bq. PB generated.
bq. M src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
bq. Use new RootRegionTracker method for getting content of znode rather
bq. than do it all here (going via RootRegionTracker, we can keep how
bq. the znode content is serialized private to the RootRegionTracker class.
bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
bq. Has the methods that used to be in RootLocationEditor plus a new
bq.
bq.
bq. This addresses bug hbase-5688.
bq. https://issues.apache.org/jira/browse/hbase-5688
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java c90864a
bq. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java b2a5463
bq. src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 64def15
bq. src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9c215b4
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 2f05005
bq. src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 33e4e71
bq. src/main/protobuf/ZooKeeper.proto PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 533b2bf
bq. src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java fe37156
bq. src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java 2132036
bq. src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4600/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Michael
bq.
bq.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247026#comment-13247026 ]
Hudson commented on HBASE-5688:
-------------------------------
Integrated in HBase-TRUNK-security #157 (See [https://builds.apache.org/job/HBase-TRUNK-security/157/])
HBASE-5688 Convert zk root-region-server znode content to pb; DELETE RLE... causing build to fail because of RAT warnings (Revision 1308697)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.addendum.txt, 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5688:
-------------------------
Attachment: 5688v4.txt
Changes the content of the root location znode, root-region-server, to be
four magic bytes ('PBUF') followed by a protobuf message that holds the
ServerName of the server currently hosting root.
D src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
Removed. Had two methods, one to add root-region-server znode and another
to removed it. Rather, put these methods in RootRegionTracker. It
tracks root-region-server znode. Having all to do w/ root-region-server
is more cohesive. Also makes it so can encapsulate in one class
all to do w/ create, delete, and reading of root-region-server.
We also want to purge the catalog package (See note at head of
CatalogTracker).
M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Get root region location from RootRegionTracker rather than from RootLocationEditor.
A src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
Utility to do w/ protobuf handling. Has methods to help prefixing
and stripping from serialized protobuf messages some 'magic'.
A src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
PB generated.
M src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
Use new RootRegionTracker method for getting content of znode rather
than do it all here (going via RootRegionTracker, we can keep how
the znode content is serialized private to the RootRegionTracker class.
M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
Has the methods that used to be in RootLocationEditor plus a new
getRootRegionLocation method (does not set watcher).
A src/main/protobuf/ZooKeeper.proto
M src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
M src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java
M src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
Get root region location from RootRegionTracker rather than from RootLocationEditor.
A src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java
Test of dataToServerName method in RootRegionTracker.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242917#comment-13242917 ]
stack commented on HBASE-5688:
------------------------------
@Ted Its first cut. Unfinished. Unexplained. Hold off.
@Todd I had something else as first field. Let me fix. On code moves, sure but bulk is collecting the pb serialization and deserialization into one place and trying to hide it. Tell me more about your PBUF idea. You mean a first field in each message that is always PBUF? Thanks.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243008#comment-13243008 ]
stack commented on HBASE-5688:
------------------------------
I can put magic ahead of the pb bytes up in zk, np. You think there should be magic elsewhere too? In info:regioninfo, etc., columns in .META., in whatever we write the fs -- e.g. contents of reference files -- and what about rpc? Should it have magic preamble or you think if its broke on an rpc, its just broke and the on retry it'll be ok? Good stuff Todd.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245746#comment-13245746 ]
Hudson commented on HBASE-5688:
-------------------------------
Integrated in HBase-TRUNK #2706 (See [https://builds.apache.org/job/HBase-TRUNK/2706/])
HBASE-5688 Convert zk root-region-server znode content to pb; DELETE RLE... causing build to fail because of RAT warnings (Revision 1308697)
HBase-5688 Convert zk root-region-server znode content to pb (Revision 1308681)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.addendum.txt, 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244621#comment-13244621 ]
stack commented on HBASE-5688:
------------------------------
Oh, and thanks Jimmy for review.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242957#comment-13242957 ]
Todd Lipcon commented on HBASE-5688:
------------------------------------
less likely to be corrupt. This is a common technique:
http://en.wikipedia.org/wiki/Magic_number_(programming)#Magic_numbers_in_files
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5688:
-------------------------
Attachment: 5688.txt
First cut. Seems to work but includes cleanup I need to check and explain.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242902#comment-13242902 ]
Zhihong Yu commented on HBASE-5688:
-----------------------------------
For RootRegionTracker.java, getRootRegionLocation() ctor's:
{code}
+ * @return server name of null if we failed to get the data.
{code}
Should read 'name or null if ...'
For setRootLocation():
{code}
+ ZKUtil.createAndWatch(zookeeper, zookeeper.rootServerZNode, data);
+ } catch(KeeperException.NodeExistsException nee) {
+ LOG.debug("ROOT region location already existed, updated location");
+ ZKUtil.setData(zookeeper, zookeeper.rootServerZNode,
+ Bytes.toBytes(location.toString()));
{code}
For ZKUtil.setData(), why is data (in PB format) not used ?
Using review board would facilitate further review.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244612#comment-13244612 ]
jiraposter@reviews.apache.org commented on HBASE-5688:
------------------------------------------------------
bq. On 2012-04-01 16:41:00, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 48
bq. > <https://reviews.apache.org/r/4600/diff/1/?file=97843#file97843line48>
bq. >
bq. > I think prefixedWithPBMagic would be a better name for this method.
Disagree.
bq. On 2012-04-01 16:41:00, Ted Yu wrote:
bq. > src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java, line 28
bq. > <https://reviews.apache.org/r/4600/diff/1/?file=97852#file97852line28>
bq. >
bq. > Javadoc would be desirable.
Classname says what it does.
bq. On 2012-04-01 16:41:00, Ted Yu wrote:
bq. > src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java, line 43
bq. > <https://reviews.apache.org/r/4600/diff/1/?file=97852#file97852line43>
bq. >
bq. > White space.
Will fix on commit.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4600/#review6606
-----------------------------------------------------------
On 2012-04-01 00:18:54, Michael Stack wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4600/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-01 00:18:54)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Changes the content of the root location znode, root-region-server, to be
bq. four magic bytes ('PBUF') followed by a protobuf message that holds the
bq. ServerName of the server currently hosting root.
bq.
bq. D src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
bq. Removed. Had two methods, one to add root-region-server znode and another
bq. to removed it. Rather, put these methods in RootRegionTracker. It
bq. tracks root-region-server znode. Having all to do w/ root-region-server
bq. is more cohesive. Also makes it so can encapsulate in one class
bq. all to do w/ create, delete, and reading of root-region-server.
bq. We also want to purge the catalog package (See note at head of
bq. CatalogTracker).
bq. M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
bq. M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
bq. M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
bq. Get root region location from RootRegionTracker rather than from RootLocationEditor.
bq. A src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
bq. Utility to do w/ protobuf handling. Has methods to help prefixing
bq. and stripping from serialized protobuf messages some 'magic'.
bq. A src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
bq. PB generated.
bq. M src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
bq. Use new RootRegionTracker method for getting content of znode rather
bq. than do it all here (going via RootRegionTracker, we can keep how
bq. the znode content is serialized private to the RootRegionTracker class.
bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
bq. Has the methods that used to be in RootLocationEditor plus a new
bq.
bq.
bq. This addresses bug hbase-5688.
bq. https://issues.apache.org/jira/browse/hbase-5688
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java c90864a
bq. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java b2a5463
bq. src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 64def15
bq. src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9c215b4
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 2f05005
bq. src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 33e4e71
bq. src/main/protobuf/ZooKeeper.proto PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 533b2bf
bq. src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java fe37156
bq. src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java 2132036
bq. src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4600/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Michael
bq.
bq.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5688:
-------------------------
Fix Version/s: 0.96.0
Assignee: stack
Release Note:
Changes the content of the root location znode, root-region-server, to be
four magic bytes ('PBUF') followed by a protobuf message that holds the
ServerName of the server currently hosting root.
Status: Patch Available (was: Open)
Trying against hadoopqa. Let me put it up on review board too because would appreciate some review.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243618#comment-13243618 ]
jiraposter@reviews.apache.org commented on HBASE-5688:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4600/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
Changes the content of the root location znode, root-region-server, to be
four magic bytes ('PBUF') followed by a protobuf message that holds the
ServerName of the server currently hosting root.
D src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
Removed. Had two methods, one to add root-region-server znode and another
to removed it. Rather, put these methods in RootRegionTracker. It
tracks root-region-server znode. Having all to do w/ root-region-server
is more cohesive. Also makes it so can encapsulate in one class
all to do w/ create, delete, and reading of root-region-server.
We also want to purge the catalog package (See note at head of
CatalogTracker).
M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Get root region location from RootRegionTracker rather than from RootLocationEditor.
A src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
Utility to do w/ protobuf handling. Has methods to help prefixing
and stripping from serialized protobuf messages some 'magic'.
A src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
PB generated.
M src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
Use new RootRegionTracker method for getting content of znode rather
than do it all here (going via RootRegionTracker, we can keep how
the znode content is serialized private to the RootRegionTracker class.
M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
Has the methods that used to be in RootLocationEditor plus a new
This addresses bug hbase-5688.
https://issues.apache.org/jira/browse/hbase-5688
Diffs
-----
src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java c90864a
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java b2a5463
src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 64def15
src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9c215b4
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 2f05005
src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 33e4e71
src/main/protobuf/ZooKeeper.proto PRE-CREATION
src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 533b2bf
src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java fe37156
src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java 2132036
src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java PRE-CREATION
Diff: https://reviews.apache.org/r/4600/diff
Testing
-------
Thanks,
Michael
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242954#comment-13242954 ]
Zhihong Yu commented on HBASE-5688:
-----------------------------------
bq. somehow got corrupted
The PBUF magic header is not immune to data corruption, right ?
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243769#comment-13243769 ]
jiraposter@reviews.apache.org commented on HBASE-5688:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4600/#review6605
-----------------------------------------------------------
Looks good to me.
- Jimmy
On 2012-04-01 00:18:54, Michael Stack wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4600/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-01 00:18:54)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Changes the content of the root location znode, root-region-server, to be
bq. four magic bytes ('PBUF') followed by a protobuf message that holds the
bq. ServerName of the server currently hosting root.
bq.
bq. D src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
bq. Removed. Had two methods, one to add root-region-server znode and another
bq. to removed it. Rather, put these methods in RootRegionTracker. It
bq. tracks root-region-server znode. Having all to do w/ root-region-server
bq. is more cohesive. Also makes it so can encapsulate in one class
bq. all to do w/ create, delete, and reading of root-region-server.
bq. We also want to purge the catalog package (See note at head of
bq. CatalogTracker).
bq. M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
bq. M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
bq. M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
bq. Get root region location from RootRegionTracker rather than from RootLocationEditor.
bq. A src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
bq. Utility to do w/ protobuf handling. Has methods to help prefixing
bq. and stripping from serialized protobuf messages some 'magic'.
bq. A src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
bq. PB generated.
bq. M src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
bq. Use new RootRegionTracker method for getting content of znode rather
bq. than do it all here (going via RootRegionTracker, we can keep how
bq. the znode content is serialized private to the RootRegionTracker class.
bq. M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
bq. Has the methods that used to be in RootLocationEditor plus a new
bq.
bq.
bq. This addresses bug hbase-5688.
bq. https://issues.apache.org/jira/browse/hbase-5688
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java c90864a
bq. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java b2a5463
bq. src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 64def15
bq. src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9c215b4
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 2f05005
bq. src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 33e4e71
bq. src/main/protobuf/ZooKeeper.proto PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 533b2bf
bq. src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java fe37156
bq. src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java 2132036
bq. src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4600/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Michael
bq.
bq.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244983#comment-13244983 ]
stack commented on HBASE-5688:
------------------------------
Did an svn rm on o.a.h.h.catalog.RootLocationEditor.
svn commit -m 'HBASE-5688 Convert zk root-region-server znode content to pb; DELETE RLE... causing build to fail because of RAT warnings' src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
Was causing the trunk build fail because of RAT complaints the empty file was missing a license.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.addendum.txt, 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242875#comment-13242875 ]
Todd Lipcon commented on HBASE-5688:
------------------------------------
Hey Stack. Looks pretty good. Quick comments:
- might as well start with field id 1 instead of 2 in the protobuf
- I think it's better not to move code in this same patch -- might be good cleanup but a little harder to review. The move makes the change look bigger than it really is.
- for the data in ZK, we might want to consider prefixing all of the entries with a static "PBUF" byte string or something. My thinking is that, while we assume protobufs will last us forever, it will provide a back-compat path if we ever change our mind down the road.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5688:
-------------------------
Resolution: Fixed
Hadoop Flags: Incompatible change,Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks for review T{o,e}dd.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5688) Convert zk
root-region-server znode content to pb
Posted by "stack (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243616#comment-13243616 ]
stack edited comment on HBASE-5688 at 4/1/12 12:18 AM:
-------------------------------------------------------
Changes the content of the root location znode, root-region-server, to be
four magic bytes ('PBUF') followed by a protobuf message that holds the
ServerName of the server currently hosting root.
{code}
D src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
Removed. Had two methods, one to add root-region-server znode and another
to removed it. Rather, put these methods in RootRegionTracker. It
tracks root-region-server znode. Having all to do w/ root-region-server
is more cohesive. Also makes it so can encapsulate in one class
all to do w/ create, delete, and reading of root-region-server.
We also want to purge the catalog package (See note at head of
CatalogTracker).
M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Get root region location from RootRegionTracker rather than from RootLocationEditor.
A src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
Utility to do w/ protobuf handling. Has methods to help prefixing
and stripping from serialized protobuf messages some 'magic'.
A src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
PB generated.
M src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
Use new RootRegionTracker method for getting content of znode rather
than do it all here (going via RootRegionTracker, we can keep how
the znode content is serialized private to the RootRegionTracker class.
M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
Has the methods that used to be in RootLocationEditor plus a new
getRootRegionLocation method (does not set watcher).
A src/main/protobuf/ZooKeeper.proto
M src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
M src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java
M src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
Get root region location from RootRegionTracker rather than from RootLocationEditor.
A src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java
Test of dataToServerName method in RootRegionTracker.
{code}
was (Author: stack):
Changes the content of the root location znode, root-region-server, to be
four magic bytes ('PBUF') followed by a protobuf message that holds the
ServerName of the server currently hosting root.
D src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
Removed. Had two methods, one to add root-region-server znode and another
to removed it. Rather, put these methods in RootRegionTracker. It
tracks root-region-server znode. Having all to do w/ root-region-server
is more cohesive. Also makes it so can encapsulate in one class
all to do w/ create, delete, and reading of root-region-server.
We also want to purge the catalog package (See note at head of
CatalogTracker).
M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Get root region location from RootRegionTracker rather than from RootLocationEditor.
A src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
Utility to do w/ protobuf handling. Has methods to help prefixing
and stripping from serialized protobuf messages some 'magic'.
A src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
PB generated.
M src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
Use new RootRegionTracker method for getting content of znode rather
than do it all here (going via RootRegionTracker, we can keep how
the znode content is serialized private to the RootRegionTracker class.
M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
Has the methods that used to be in RootLocationEditor plus a new
getRootRegionLocation method (does not set watcher).
A src/main/protobuf/ZooKeeper.proto
M src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
M src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java
M src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
Get root region location from RootRegionTracker rather than from RootLocationEditor.
A src/test/java/org/apache/hadoop/hbase/zookeeper/TestRootRegionTracker.java
Test of dataToServerName method in RootRegionTracker.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244968#comment-13244968 ]
stack commented on HBASE-5688:
------------------------------
Committed addendum.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.addendum.txt, 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242946#comment-13242946 ]
Todd Lipcon commented on HBASE-5688:
------------------------------------
If you catch the exception, you can't distinguish between a legitimate error in the data (somehow got corrupted) vs just talking to an earlier cluster which is not protobuf-enabled. Same goes for future compatibility in the case that we ever move away from protobuf to something else.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5688:
-------------------------
Attachment: 5688v5.txt
v5 removes a single white space. Its what I'll commit.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242931#comment-13242931 ]
Zhihong Yu commented on HBASE-5688:
-----------------------------------
bq. a weird decoding exception
I think decoding exception would have similar effect as the PBUF magic.
We can catch the exception and provide more meaningful message.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Attachments: 5688.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5688:
-------------------------
Attachment: 5688.addendum.txt
I just noticed that a field index in the .proto (1) does not agree w/ the generated code (it has 2). Addendum to fix this.
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.addendum.txt, 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245002#comment-13245002 ]
Hudson commented on HBASE-5688:
-------------------------------
Integrated in HBase-TRUNK-security #156 (See [https://builds.apache.org/job/HBase-TRUNK-security/156/])
HBase-5688 Convert zk root-region-server znode content to pb (Revision 1308681)
HBASE-5688 Convert zk root-region-server znode content to pb (Revision 1308560)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
* /hbase/trunk/src/main/protobuf/ZooKeeper.proto
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.addendum.txt, 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5688) Convert zk root-region-server znode
content to pb
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244827#comment-13244827 ]
Hudson commented on HBASE-5688:
-------------------------------
Integrated in HBase-TRUNK #2704 (See [https://builds.apache.org/job/HBase-TRUNK/2704/])
HBASE-5688 Convert zk root-region-server znode content to pb (Revision 1308560)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
* /hbase/trunk/src/main/protobuf/ZooKeeper.proto
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java
> Convert zk root-region-server znode content to pb
> -------------------------------------------------
>
> Key: HBASE-5688
> URL: https://issues.apache.org/jira/browse/HBASE-5688
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.96.0
>
> Attachments: 5688.txt, 5688v4.txt, 5688v5.txt
>
>
> Move the root-region-server znode content from the versioned bytes that ServerName.getVersionedBytes outputs to instead be pb.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira