You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2012/06/08 21:48:22 UTC
[jira] [Created] (HBASE-6193) REST Api for HBase administrative
tasks
Eric Yang created HBASE-6193:
--------------------------------
Summary: REST Api for HBase administrative tasks
Key: HBASE-6193
URL: https://issues.apache.org/jira/browse/HBASE-6193
Project: HBase
Issue Type: New Feature
Reporter: Eric Yang
For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
The popular admin task that can take advantage of this setup are:
- Trigger table compaction
- Show process list
- Health check
- Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6193) REST Api for HBase
administrative tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291956#comment-13291956 ]
Andrew Purtell edited comment on HBASE-6193 at 6/8/12 8:36 PM:
---------------------------------------------------------------
-1 We already have a REST gateway for this purpose. Why don't you propose extending that for your needs.
Edit: More detailed explanation: You are proposing what sounds like a HDFS HTTPFS-like approach for rearchitecting the HBase REST interface. In general I'm sympathetic to this idea but for the administrative use cases you have presented, none require such invasive change. If you do want to pursue this approach, it would be helpful to attach a design document that successfully argues why the current REST gateway or an adaptation thereof cannot suffice. Also, if rearchitecting REST in this way, it's not sufficient to only handle administrative tasks. We should not have more than one framework for interacting with HBase RESTfully, it would be a maintenance burden and confusing.
was (Author: apurtell):
-1 We already have a REST gateway for this purpose. Why don't you propose extending that for your needs.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6193) REST Api for HBase administrative
tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291957#comment-13291957 ]
Andrew Purtell commented on HBASE-6193:
---------------------------------------
See also http://search-hadoop.com/m/aeqTh1cykpo1/Rethinking+REST&subj=Rethinking+REST+Re+HBASE+4368+and+friends+. The question was also posed there.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6193) REST Api for HBase
administrative tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292141#comment-13292141 ]
Andrew Purtell edited comment on HBASE-6193 at 6/9/12 12:55 AM:
----------------------------------------------------------------
bq. This jira is not rearchitecting what is already available, but embedded REST api in region server makes a lot of sense for long term maintenance of HBase.
By definition embedding a REST server per daemon is a different architectural approach than using the existing gateway.
Please explain why using the gateway for administrative function is insufficient.
Edit: Also, please explain the statement "embedded REST api in region server makes a lot of sense for long term maintenance of HBase" in detail.
was (Author: apurtell):
bq. This jira is not rearchitecting what is already available, but embedded REST api in region server makes a lot of sense for long term maintenance of HBase.
By definition embedding a REST server per daemon is a different architectural approach than using the existing gateway.
Please explain why using the gateway for administrative function is insufficient.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6193) REST Api for HBase
administrative tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292384#comment-13292384 ]
Andrew Purtell edited comment on HBASE-6193 at 6/9/12 5:01 PM:
---------------------------------------------------------------
bq. Otherwise each new feature implemented in the region server, the rest gateway should be updated with corresponding new feature.
I'm not sure this holds.
There is client activity, and then there is administrative activity.
Certainly for each new feature implemented in the client, the REST interface should be updated, whatever its form. See the RemoteHTable class in o.a.h.hbase.rest.client. I suppose also RemoteHAdmin to provide full API coverage available to Java clients.
Our existing design does the above and adds small InfoServer based servers for other types of administrative activity, like querying JMX exported information in JSON encoding. You may not want to allow users to do that. The current design segregates client and that second class of administrative access well.
I really think you need to produce a design document that covers all of these issues and makes a convincing argument before we can proceed further.
Edit: It bears repeating, but I am not opposed to your approach in principal. However a big design change like this requires justification that I'm just not seeing yet.
was (Author: apurtell):
bq. Otherwise each new feature implemented in the region server, the rest gateway should be updated with corresponding new feature.
I'm not sure this holds.
There is client activity, and then there is administrative activity.
Certainly for each new feature implemented in the client, the REST interface should be updated, whatever its form. See the RemoteHTable class in o.a.h.hbase.rest.client. I suppose also RemoteHAdmin to provide full API coverage available to Java clients.
Our existing design does the above and adds small InfoServer based servers for other types of administrative activity, like querying JMX exported information in JSON encoding. You may not want to allow users to do that. The current design segregates client and that second class of administrative access well.
I really think you need to produce a design document that covers all of these issues and makes a convincing argument before we can proceed further.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6193) REST Api for HBase administrative
tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292141#comment-13292141 ]
Andrew Purtell commented on HBASE-6193:
---------------------------------------
bq. This jira is not rearchitecting what is already available, but embedded REST api in region server makes a lot of sense for long term maintenance of HBase.
By definition embedding a REST server per daemon is a different architectural approach than using the existing gateway.
Please explain why using the gateway for administrative function is insufficient.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6193) REST Api for HBase administrative
tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292384#comment-13292384 ]
Andrew Purtell commented on HBASE-6193:
---------------------------------------
bq. Otherwise each new feature implemented in the region server, the rest gateway should be updated with corresponding new feature.
I'm not sure this holds.
There is client activity, and then there is administrative activity.
Certainly for each new feature implemented in the client, the REST interface should be updated, whatever its form. See the RemoteHTable class in o.a.h.hbase.rest.client. I suppose also RemoteHAdmin to provide full API coverage available to Java clients.
Our existing design does the above and adds small InfoServer based servers for other types of administrative activity, like querying JMX exported information in JSON encoding. You may not want to allow users to do that. The current design segregates client and that second class of administrative access well.
I really think you need to produce a design document that covers all of these issues and makes a convincing argument before we can proceed further.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6193) REST Api for HBase administrative
tasks
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292178#comment-13292178 ]
Eric Yang commented on HBASE-6193:
----------------------------------
REST api gateway may be difficult to secure the system. The proxy approach requires two security implementations for outward facing security and internal security implementation. Where the embedded rest api server in region server would ensure only one security implementation is required. By adding rest api into the region server code, it reduces wrapper code by one layer. Otherwise each new feature implemented in the region server, the rest gateway should be updated with corresponding new feature. It seems to to be higher maintenance with rest gateway approach. In addition, rest proxy increases the latency for HBase response time by one hop. If the rest gateway is busy serving scan query and become none responsive, it is unclear to the administrator if the none responsiveness is caused by admin task or someone had a large scan query for another region server.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6193) REST Api for HBase administrative
tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292382#comment-13292382 ]
Andrew Purtell commented on HBASE-6193:
---------------------------------------
bq. REST api gateway may be difficult to secure the system. The proxy approach requires two security implementations for outward facing security and internal security implementation. Where the embedded rest api server in region server would ensure only one security implementation is required.
The counter argument is you should never let untrusted access onto the cluster directly. This is why I dislike HTTPFS and prefer instead the design of Hoop. The gateway provides an effective security perimeter. Because it is RESTful, security can be layered on top of the HTTP transactions as the system integrator desires via the many options for doing so developed over the years for web servers.
bq. In addition, rest proxy increases the latency for HBase response time by one hop.
The counter argument is REST endpoints built into the RegionServers add the overheads of HTTP processing. And we have to build this new thing into the master that issues HTTP redirects?
bq. If the rest gateway is busy serving scan query and become none responsive
An advantage of the gateway approach is you can scale it independent of the RegionServer tier. So if it's slow, add more workers. Use a proxy that supports sticky sessions to preserve scanner semantics.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6193) REST Api for HBase administrative
tasks
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291956#comment-13291956 ]
Andrew Purtell commented on HBASE-6193:
---------------------------------------
-1 We already have a REST gateway for this purpose. Why don't you propose extending that for your needs.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6193) REST Api for HBase administrative
tasks
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292125#comment-13292125 ]
Eric Yang commented on HBASE-6193:
----------------------------------
Hi Andrew,
This jira is posted based on information of the mailing list thread and gather interests in enhancing HBase REST API. This jira is not rearchitecting what is already available, but embedded REST api in region server makes a lot of sense for long term maintenance of HBase. The gateway proxy approach works too.
> REST Api for HBase administrative tasks
> ---------------------------------------
>
> Key: HBASE-6193
> URL: https://issues.apache.org/jira/browse/HBASE-6193
> Project: HBase
> Issue Type: New Feature
> Reporter: Eric Yang
>
> For enabling HBASE-4368, it may be useful to have embedded REST API server for each region server, and HBase Master can either aggregate or proxy/redirect REST API base on administrative task functions.
> The popular admin task that can take advantage of this setup are:
> - Trigger table compaction
> - Show process list
> - Health check
> - Show black listed servers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira