You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Nitay Joffe (JIRA)" <ji...@apache.org> on 2009/04/19 08:02:47 UTC
[jira] Created: (HBASE-1329) Visibility into ZooKeeper
Visibility into ZooKeeper
-------------------------
Key: HBASE-1329
URL: https://issues.apache.org/jira/browse/HBASE-1329
Project: Hadoop HBase
Issue Type: New Feature
Affects Versions: 0.20.0
Reporter: Nitay Joffe
Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
- Running a REST server. See ZOOKEEPER-36.
- FUSE support. See ZOOKEEPER-25.
- Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
- Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717378#action_12717378 ]
Nitay Joffe commented on HBASE-1329:
------------------------------------
Good idea Lars. Folks please post suggestions for what you would like to have.
Some of my initial thoughts (from above and other chats):
- Add a 'zk' command to the shell. Put all of the ZooKeeper shell commands as sub commands as this. Specifically here are the current ZooKeeper shell commands:
{code}commandMap.put("create", "path data acl");
commandMap.put("delete"," path [version]");
commandMap.put("set"," path data [version]");
commandMap.put("get"," path [watch]");
commandMap.put("ls"," path [watch]");
commandMap.put("getAcl"," path");
commandMap.put("setAcl"," path acl");
commandMap.put("stat"," path [watch]");
commandMap.put("sync"," path");
commandMap.put("setquota"," -n|-b val path");
commandMap.put("listquota"," path");
commandMap.put("delquota"," [-n|-b] path");
commandMap.put("history","");
commandMap.put("redo"," cmdno");
commandMap.put("printwatches", " on|off");
commandMap.put("quit","");
commandMap.put("addauth", " scheme auth");
{code}
- Add higher level commands to 'zk', like get/set master addres, get/set root region server address.
- Add a ZK tree viewer to UI, something like ZOOKEEPER-418.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718366#action_12718366 ]
Lars George commented on HBASE-1329:
------------------------------------
@Jim Makes sense. We could also think of a "logger" style event list that when the UI reloads it shows what happened in between refreshes, similar to the RegionHistorian. That way the UI does not have to be complex at first but still shows you what happened the last x minutes. Also, I agree that a raw ZK node tree should be a standalone app provided by the ZK team (and maybe that issue they have open is doing just that, at least it sounds like it). The HBase UI should not be too overly complicated but still provide all necessary - and relevant (!) - information.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nitay Joffe updated HBASE-1329:
-------------------------------
Attachment: zk.patch
hbase-1329.patch
This patch is the first part, putting the raw ZK commands into our shell.
zk.patch contains edits to ZooKeeper to expose using their ZooKeeperMain class from HBase. Also contains some cleanup to some statics and initializations.
hbase-1329.patch adds ZooKeeperMain to the ruby Admin wrapper around HBaseAdmin with methods to call out to ZooKeeper's command line functions.
All of the zookeeper commands are under the 'zk' method.
Note that this calls out directly to the ZooKeeper functionality, so anything they add will directly be available by updating our jar.
Here is an example of usage:
{code}
shrek:hbase nitay$ bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.0-dev, r6bbf7cb5bdf6c7065e69ba1dfdb7901b0769fba5, Fri Jun 12 22:31:07 PDT 2009
hbase(main):001:0> zk 'help'
09/06/12 22:33:48 INFO zookeeper.ZooKeeperWrapper: Quorum servers: aa0-000-14.u.powerset.com:2188,aa0-000-13.u.powerset.com:2188,aa0-000-12.u.powerset.com:2188,aa0-001-13.u.powerset.com:2188,aa0-000-15.u.powerset.com:2188
ZooKeeper -server host:port cmd args
connect host:port
get path [watch]
ls path [watch]
set path data [version]
delquota [-n|-b] path
quit
printwatches on|off
create path data acl
stat path [watch]
close
history
listquota path
setAcl path acl
getAcl path
sync path
redo cmdno
addauth scheme auth
delete path [version]
setquota -n|-b val path
hbase(main):002:0> zk 'ls','/hbase'
[safe-mode, root-region-server, rs, master, shutdown]
hbase(main):003:0> zk 'get', '/hbase/master'
208.76.44.158:60007
cZxid = 62
ctime = Sat Jun 06 21:30:26 PDT 2009
mZxid = 62
mtime = Sat Jun 06 21:30:26 PDT 2009
pZxid = 62
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 81525328146006018
dataLength = 19
numChildren = 0
hbase(main):005:0> zk 'get', '/hbase/safe-mode'
cZxid = 95
ctime = Sat Jun 06 21:30:31 PDT 2009
mZxid = 95
mtime = Sat Jun 06 21:30:31 PDT 2009
pZxid = 95
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 81525328146006018
dataLength = 0
numChildren = 0
hbase(main):006:0> zk 'get', '/hbase/root-region-server'
208.76.44.141:60027
cZxid = 30
ctime = Tue Jun 02 14:24:17 PDT 2009
mZxid = 97
mtime = Sat Jun 06 21:30:31 PDT 2009
pZxid = 30
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0
dataLength = 19
numChildren = 0
hbase(main):007:0> zk 'ls', '/hbase/rs'
[1244349027581, 1244349027859, 1244349028409, 1244349027545, 1244349027546]
{code}
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329.patch, zk.patch
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nitay Joffe updated HBASE-1329:
-------------------------------
Fix Version/s: 0.20.0
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717763#action_12717763 ]
Andrew Purtell commented on HBASE-1329:
---------------------------------------
+1
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1329) Visibility into ZooKeeper
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1329:
-------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Its kinda ugly but excellent at same time (that you have to quote 'help' is perverse, but hey, its kinda amazing that if you do all the proper quoting, you can do all the zk commands).
I committed it because I want to include in alpha release and because I added a little more help text around zk command.
I like zk_dump.
Good stuff Nitay.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329-v2.patch, hbase-1329.patch, zk.patch, zookeeper-r785019-hbase-1329.jar
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nitay Joffe updated HBASE-1329:
-------------------------------
Attachment: zookeeper-r785019-hbase-1329.jar
hbase-1329-v2.patch
New patch:
- Add zk_dump which produces a dump of everything HBase from ZooKeeper's point of view. This is the higher level API. I'd like to leave 'zk' as a pure tunnel to ZooKeeper's CLI so that we don't clash. We can add parameters to zk_dump (or make other methods) as people like. Is there anything specifically that people would like to see?
- Add zk and zk_dump help to 'tools' help menu.
Stack:
- Yes, this requires a new ZK jar. I've included it in this upload.
- I changed this so that HBA returns HConnection and HConnection returns ZKW. The second one already existed, I only had to add the first.
Some more example usage:
{code}
hbase(main):001:0> zk_dump
09/06/16 17:53:34 INFO zookeeper.ZooKeeperWrapper: Quorum servers: aa0-000-14.u.powerset.com:2188,aa0-000-13.u.powerset.com:2188,aa0-000-12.u.powerset.com:2188,aa0-001-13.u.powerset.com:2188,aa0-000-15.u.powerset.com:2188
HBase tree in ZooKeeper is rooted at /hbase
Cluster up? true
In safe mode? false
Master address: 208.76.44.158:60007
Region server holding ROOT: 208.76.44.141:60027
Region servers:
- 208.76.44.140:60027
- 208.76.44.158:60027
- 208.76.44.141:60027
- 208.76.44.142:60027
{code}
I'm going to put the UI stuff in separate patches, let's get this stuff settled/committed first.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329-v2.patch, hbase-1329.patch, zk.patch, zookeeper-r785019-hbase-1329.jar
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1329) Visibility into ZooKeeper
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1329:
-------------------------
Priority: Blocker (was: Major)
Marking this issue a blocker. We need to do all we can to avoid hbase users having the impression that zookeeper is a black box.
@Patrick -> Thanks for the great suggestions -- IMO, the last one sounds most promising.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Priority: Blocker
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720561#action_12720561 ]
Lars George commented on HBASE-1329:
------------------------------------
Great stuff!
One question, what about the UI? Are we creating a new issue for it? Just asking.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329-v2.patch, hbase-1329.patch, zk.patch, zookeeper-r785019-hbase-1329.jar
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nitay Joffe reassigned HBASE-1329:
----------------------------------
Assignee: Nitay Joffe
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nitay Joffe updated HBASE-1329:
-------------------------------
Status: Patch Available (was: In Progress)
Submitting for +1 approval.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329-v2.patch, hbase-1329.patch, zk.patch, zookeeper-r785019-hbase-1329.jar
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700943#action_12700943 ]
Patrick Hunt commented on HBASE-1329:
-------------------------------------
Here's a couple more areas to investigate. Feel free to ask questions on one of our lists:
http://hadoop.apache.org/zookeeper/mailing_lists.html
These are commands that can be run against the server's command (client) port. They can
give insight into the status of the system:
http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_zkCommands
Our JMX interface provides similar information:
http://hadoop.apache.org/zookeeper/docs/current/zookeeperJMX.html
A perl binding has been created, it's being integrated into 3.2 (avail on cpan):
https://issues.apache.org/jira/browse/ZOOKEEPER-378
We're working on a python binding and set of command line tools:
https://issues.apache.org/jira/browse/ZOOKEEPER-364
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700989#action_12700989 ]
stack commented on HBASE-1329:
------------------------------
Would adding a 'zk' command to the hbase shell work where all that came after 'zk' was passed to the zk servers' command (client) port work?
Would necessitate knowledge of the zk command set. The shell zk help could talk up basic commands. Under the covers, we'd parse the command and pass it in a 'nc' fashion to zk catching the return outputing it to the hbase shell.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720565#action_12720565 ]
Nitay Joffe commented on HBASE-1329:
------------------------------------
@Lars, yes, I've opened HBASE-1532. Thanks for keeping us honest ;).
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329-v2.patch, hbase-1329.patch, zk.patch, zookeeper-r785019-hbase-1329.jar
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719195#action_12719195 ]
stack commented on HBASE-1329:
------------------------------
zk.patch looks good. You going to make up a new zk jar to include in hbase with this change?
You think HBA is right place to go to get a ZookeeperWrapper? Its a weird one. Its an internal thingy. Would be nice if didn't have public API but probably hard to do. Could you get it direct off the connection itself?
Where should the new 'zk' go in our shell help? We have a 'tools' section. We could add it there? 'tools' are for cluster admins.
Good stuff Nitay.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329.patch, zk.patch
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (HBASE-1329) Visibility into ZooKeeper
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HBASE-1329 started by Nitay Joffe.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717365#action_12717365 ]
Lars George commented on HBASE-1329:
------------------------------------
Could we please collect a list here of what information should be made visible out of the ZK data? Also where, i.e. UI or shell and what commands should be added to either of them to change ZK values?
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718340#action_12718340 ]
Jim Kellerman commented on HBASE-1329:
--------------------------------------
@Lars
(my two cents)
Auto refresh in real-time might be impractical for a couple of reasons: events could easily overload a web API and not have an auto refresh interval like 30-60 seconds might cause some events to be overwritten before they are rendered (of course, by using a longer interval, you might miss some important state transitions). What I think is needed is somewhere between the two: obviously some events are more important than others in what needs to be displayed. Some "really important" events might cause a refresh and "less important" events would not. And then also have the web display auto refresh like it does for the current master, table, region, region server displays after a (configurable) longer interval.
It would be cool to drill down into the znode hierarchy, but maybe that is something that ZK should provide in a web based interface.
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720453#action_12720453 ]
Jean-Daniel Cryans commented on HBASE-1329:
-------------------------------------------
Very powerful stuff Nitay, +1
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: hbase-1329-v2.patch, hbase-1329.patch, zk.patch, zookeeper-r785019-hbase-1329.jar
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718173#action_12718173 ]
Lars George commented on HBASE-1329:
------------------------------------
Great start Nitay! Thanks for this. As far as the UI is concerned, is it enough to have a web based tree? Or does it need a constant refresh to show updated values? The above ZK issue mentions something along those lines.
As far as the UI is concerned, what would display? What are the nodes/details etc. from our ZK instance that should be shown?
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
> Assignee: Nitay Joffe
> Priority: Blocker
> Fix For: 0.20.0
>
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1329) Visibility into ZooKeeper
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700996#action_12700996 ]
Patrick Hunt commented on HBASE-1329:
-------------------------------------
This reminds me, there's also the zk shell:
http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_zkMulitServerSetup
see step 6, help is available inside the shell.
We'd be happy to work with you to refactor this if you'd like to embed it as part of your
own shell (perhaps refactoring is not even necessary, take a look). You could set it up to run
any of the zk commands from your own shell by prefixing with "zk " ....
Could include the nc commands as well... although we've been talking about making things
like status available as r/o znodes...
> Visibility into ZooKeeper
> -------------------------
>
> Key: HBASE-1329
> URL: https://issues.apache.org/jira/browse/HBASE-1329
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Nitay Joffe
>
> Currently, to the users of HBase ZooKeeper is some mysterious thing that is for some reason important but no one knows why. We need to provide more visibility into ZooKeeper from HBase.
> This issue is to start a discussion on what sorts of things we can/should provide. Some possible ideas to get started:
> - Running a REST server. See ZOOKEEPER-36.
> - FUSE support. See ZOOKEEPER-25.
> - Admin/surgery tools in HBase shell. Answer questions like "in safe mode?"
> - Status and information on HBase's data in ZooKeeper in the web UI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.