You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Pete Wyckoff (JIRA)" <ji...@apache.org> on 2008/08/19 02:33:44 UTC
[jira] Created: (HADOOP-3969) Provide Mechanism to optionally
expose public org.apache.hadoop.util.Services APIs
Provide Mechanism to optionally expose public org.apache.hadoop.util.Services APIs
-----------------------------------------------------------------------------------
Key: HADOOP-3969
URL: https://issues.apache.org/jira/browse/HADOOP-3969
Project: Hadoop Core
Issue Type: New Feature
Reporter: Pete Wyckoff
Enhance manageability of Hadoop Services by providing Jute, Thrift, REST, ... APIs to select methods in the Service class (or potentially others).
This will allow external tools written in a myriad of languages to query the state of Hadoop Servers and/or interact with them.
This can be encapsulated in the Service class by defining a very simple interface and then optionally instantiating such an implementation provided in the Configuration.
Some methods to be implemented include all the public methods in Service:
ping()
isRunning()
terminate()
getServiceState()
verifyServiceState()
isTerminated(),
INTERFACE:
package org.apache.hadoop.util;
public interface ExposeServiceAPIs {
/**
* @param service - the service whose APIs are to be exposed
* @param serviceName - a symbolic name for the service
* @param configuration - the hadoop configuration object
**/
public void initialize(Service service, String serviceName, Configuration conf) throws IOException,
public boolean start();
public boolean stop();
} ;
Two straightforward implementations of this would be:
1. Servlet that exposes the APIs via REST
2. Thrift DDL of the service APIs and an implementation in Java + bindings in C++, Java, Perl, Python, Ruby, PHP
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3969) Provide Mechanism to optionally
expose public org.apache.hadoop.util.Services APIs
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631317#action_12631317 ]
Steve Loughran commented on HADOOP-3969:
----------------------------------------
I dont see the lifeycycle itself going into 0.19, as once 0.19 ships, the api becomes frozen, and I dont think we are ready for that. Better to put the changes into SVN_HEAD after 0.19 ships and stabilise things ready for 0.20.
Is there somewhere I can see what the thrift -rest stuff does; I'd be as reluctant to have a binary RPC-pretending to be RESTy as I am to the SOAP stacks that a REST mode but still have a URL for every verb, rather than a model of stateful resources.
FWIW, my use cases for a REST state API would be
-in cluster: have the switches keep an eye on which nodes werent happy through GETs
-long haul health: check the status of things, clients can spin until the cluster is live
-long haul node management: better use HTTP authentication at that point, possibly https.
ultimately: long-haul job submission; monitor status through Atom feeds and XMPP notifications, so I can submit jobs from a roaming laptop.
> Provide Mechanism to optionally expose public org.apache.hadoop.util.Services APIs
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-3969
> URL: https://issues.apache.org/jira/browse/HADOOP-3969
> Project: Hadoop Core
> Issue Type: New Feature
> Reporter: Pete Wyckoff
>
> Enhance manageability of Hadoop Services by providing Jute, Thrift, REST, ... APIs to select methods in the Service class (or potentially others).
> This will allow external tools written in a myriad of languages to query the state of Hadoop Servers and/or interact with them.
> This can be encapsulated in the Service class by defining a very simple interface and then optionally instantiating such an implementation provided in the Configuration.
> Some methods to be implemented include all the public methods in Service:
> {code}
> ping()
> isRunning()
> terminate()
> getServiceState()
> verifyServiceState()
> isTerminated(),
> {code}
> INTERFACE:
> {code}
> package org.apache.hadoop.util;
> public interface ExposeServiceAPIs {
> /**
> * @param service - the service whose APIs are to be exposed
> * @param serviceName - a symbolic name for the service
> * @param configuration - the hadoop configuration object
> **/
> public void initialize(Service service, String serviceName, Configuration conf) throws IOException,
> public boolean start();
> public boolean stop();
> } ;
> {code}
> Two straightforward implementations of this would be:
> 1. Servlet that exposes the APIs via REST
> 2. Thrift DDL of the service APIs and an implementation in Java + bindings in C++, Java, Perl, Python, Ruby, PHP
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3969) Provide Mechanism to optionally
expose public org.apache.hadoop.util.Services APIs
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631088#action_12631088 ]
Pete Wyckoff commented on HADOOP-3969:
--------------------------------------
I like the structure of those. My preferred way of implementing this would be to generate the rest code from thrift with thrift -java -rest services.thrift.ddl , in which case we'd get rest, java, perl, python, erlang, ..., but (a) thrift doesn't yet support -rest for -java that I know of and (b) I know people aren't ready for thrift dependency - but if it were a contrib thing, couldn't be much objection.
If this can get in 19 though, there's no point in musing about thrift and rest since its not implemented and even if i had time, it would require cutting a new thrift release from the non-apache version which isn't going to happen.
-- pete
> Provide Mechanism to optionally expose public org.apache.hadoop.util.Services APIs
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-3969
> URL: https://issues.apache.org/jira/browse/HADOOP-3969
> Project: Hadoop Core
> Issue Type: New Feature
> Reporter: Pete Wyckoff
>
> Enhance manageability of Hadoop Services by providing Jute, Thrift, REST, ... APIs to select methods in the Service class (or potentially others).
> This will allow external tools written in a myriad of languages to query the state of Hadoop Servers and/or interact with them.
> This can be encapsulated in the Service class by defining a very simple interface and then optionally instantiating such an implementation provided in the Configuration.
> Some methods to be implemented include all the public methods in Service:
> {code}
> ping()
> isRunning()
> terminate()
> getServiceState()
> verifyServiceState()
> isTerminated(),
> {code}
> INTERFACE:
> {code}
> package org.apache.hadoop.util;
> public interface ExposeServiceAPIs {
> /**
> * @param service - the service whose APIs are to be exposed
> * @param serviceName - a symbolic name for the service
> * @param configuration - the hadoop configuration object
> **/
> public void initialize(Service service, String serviceName, Configuration conf) throws IOException,
> public boolean start();
> public boolean stop();
> } ;
> {code}
> Two straightforward implementations of this would be:
> 1. Servlet that exposes the APIs via REST
> 2. Thrift DDL of the service APIs and an implementation in Java + bindings in C++, Java, Perl, Python, Ruby, PHP
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3969) Provide Mechanism to optionally
expose public org.apache.hadoop.util.Services APIs
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630985#action_12630985 ]
Steve Loughran commented on HADOOP-3969:
----------------------------------------
1. I'd like to see what you propose as a RESTy API. We could bounce it past the rest-discuss mailing list too
2. longer term, it would be good to take the foundation and see what could be used for a general purpose REST interface to bits of Hadoop
One thing that I've looked at in the past of having a RESTful deployment API is not to have start/stop operations, but purely to export the state as a URI of its own
PUT https://server1/namenode/state <- started/stopped, etc.
GET https://server1/namenode/state returns the state
GET https://server1/namenode/stateis/started returns 200 if in that state, some error -404, 500- if not
GET https://server1/namenode/ping would do the ping and return a page that could be handled with XPath and a load balancer; 500 on a ping failure and 200 if all is well.
In this world the rest of the service configuration would also be something to GET, maybe PUT before you start the service.
> Provide Mechanism to optionally expose public org.apache.hadoop.util.Services APIs
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-3969
> URL: https://issues.apache.org/jira/browse/HADOOP-3969
> Project: Hadoop Core
> Issue Type: New Feature
> Reporter: Pete Wyckoff
>
> Enhance manageability of Hadoop Services by providing Jute, Thrift, REST, ... APIs to select methods in the Service class (or potentially others).
> This will allow external tools written in a myriad of languages to query the state of Hadoop Servers and/or interact with them.
> This can be encapsulated in the Service class by defining a very simple interface and then optionally instantiating such an implementation provided in the Configuration.
> Some methods to be implemented include all the public methods in Service:
> {code}
> ping()
> isRunning()
> terminate()
> getServiceState()
> verifyServiceState()
> isTerminated(),
> {code}
> INTERFACE:
> {code}
> package org.apache.hadoop.util;
> public interface ExposeServiceAPIs {
> /**
> * @param service - the service whose APIs are to be exposed
> * @param serviceName - a symbolic name for the service
> * @param configuration - the hadoop configuration object
> **/
> public void initialize(Service service, String serviceName, Configuration conf) throws IOException,
> public boolean start();
> public boolean stop();
> } ;
> {code}
> Two straightforward implementations of this would be:
> 1. Servlet that exposes the APIs via REST
> 2. Thrift DDL of the service APIs and an implementation in Java + bindings in C++, Java, Perl, Python, Ruby, PHP
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3969) Provide Mechanism to optionally
expose public org.apache.hadoop.util.Services APIs
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pete Wyckoff updated HADOOP-3969:
---------------------------------
Description:
Enhance manageability of Hadoop Services by providing Jute, Thrift, REST, ... APIs to select methods in the Service class (or potentially others).
This will allow external tools written in a myriad of languages to query the state of Hadoop Servers and/or interact with them.
This can be encapsulated in the Service class by defining a very simple interface and then optionally instantiating such an implementation provided in the Configuration.
Some methods to be implemented include all the public methods in Service:
{code}
ping()
isRunning()
terminate()
getServiceState()
verifyServiceState()
isTerminated(),
{code}
INTERFACE:
{code}
package org.apache.hadoop.util;
public interface ExposeServiceAPIs {
/**
* @param service - the service whose APIs are to be exposed
* @param serviceName - a symbolic name for the service
* @param configuration - the hadoop configuration object
**/
public void initialize(Service service, String serviceName, Configuration conf) throws IOException,
public boolean start();
public boolean stop();
} ;
{code}
Two straightforward implementations of this would be:
1. Servlet that exposes the APIs via REST
2. Thrift DDL of the service APIs and an implementation in Java + bindings in C++, Java, Perl, Python, Ruby, PHP
was:
Enhance manageability of Hadoop Services by providing Jute, Thrift, REST, ... APIs to select methods in the Service class (or potentially others).
This will allow external tools written in a myriad of languages to query the state of Hadoop Servers and/or interact with them.
This can be encapsulated in the Service class by defining a very simple interface and then optionally instantiating such an implementation provided in the Configuration.
Some methods to be implemented include all the public methods in Service:
ping()
isRunning()
terminate()
getServiceState()
verifyServiceState()
isTerminated(),
INTERFACE:
package org.apache.hadoop.util;
public interface ExposeServiceAPIs {
/**
* @param service - the service whose APIs are to be exposed
* @param serviceName - a symbolic name for the service
* @param configuration - the hadoop configuration object
**/
public void initialize(Service service, String serviceName, Configuration conf) throws IOException,
public boolean start();
public boolean stop();
} ;
Two straightforward implementations of this would be:
1. Servlet that exposes the APIs via REST
2. Thrift DDL of the service APIs and an implementation in Java + bindings in C++, Java, Perl, Python, Ruby, PHP
> Provide Mechanism to optionally expose public org.apache.hadoop.util.Services APIs
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-3969
> URL: https://issues.apache.org/jira/browse/HADOOP-3969
> Project: Hadoop Core
> Issue Type: New Feature
> Reporter: Pete Wyckoff
>
> Enhance manageability of Hadoop Services by providing Jute, Thrift, REST, ... APIs to select methods in the Service class (or potentially others).
> This will allow external tools written in a myriad of languages to query the state of Hadoop Servers and/or interact with them.
> This can be encapsulated in the Service class by defining a very simple interface and then optionally instantiating such an implementation provided in the Configuration.
> Some methods to be implemented include all the public methods in Service:
> {code}
> ping()
> isRunning()
> terminate()
> getServiceState()
> verifyServiceState()
> isTerminated(),
> {code}
> INTERFACE:
> {code}
> package org.apache.hadoop.util;
> public interface ExposeServiceAPIs {
> /**
> * @param service - the service whose APIs are to be exposed
> * @param serviceName - a symbolic name for the service
> * @param configuration - the hadoop configuration object
> **/
> public void initialize(Service service, String serviceName, Configuration conf) throws IOException,
> public boolean start();
> public boolean stop();
> } ;
> {code}
> Two straightforward implementations of this would be:
> 1. Servlet that exposes the APIs via REST
> 2. Thrift DDL of the service APIs and an implementation in Java + bindings in C++, Java, Perl, Python, Ruby, PHP
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.