You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by st...@apache.org on 2014/12/11 16:17:07 UTC
svn commit: r1644671 - in /incubator/slider/site/trunk/content: docs/api/
docs/api/slider_REST_apis.md index.md
Author: stevel
Date: Thu Dec 11 15:17:06 2014
New Revision: 1644671
URL: http://svn.apache.org/r1644671
Log:
SLIDER-708 Document existing (read-only) REST APIs
Added:
incubator/slider/site/trunk/content/docs/api/
incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md
Modified:
incubator/slider/site/trunk/content/index.md
Added: incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md?rev=1644671&view=auto
==============================================================================
--- incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md (added)
+++ incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md Thu Dec 11 15:17:06 2014
@@ -0,0 +1,206 @@
+# Slider 0.60 REST APIs
+
+## December 2014
+
+This document covers the external REST APIs of Slider
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
+ NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
+ "OPTIONAL" in this document are to be interpreted as described in
+ RFC 2119.
+
+# Introduction and Background
+
+Slider 0.60, the version shipping with HDP2.2 uses Hadoop IPC for communications between the Slider client and the per-instance application master. This uses protobuf-formatted payloads, with the Hadoop IPC layer handling security: authorization, authentication and encryption.
+
+It provides read-only views of generated content and state through a set of web service URLs. All these services are implemented by the Jersey JAX-RS engine, operating in an embedded Jetty Web engine, with the YARN AmWebFilter class redirecting all requests not coming from the RM Proxy IP address to that RM Proxy via a 302 (redirect-as-GET)
+
+Excluding the internal-use-only the AM<->Agent channel operating on a private HTTP port, there are a number of current services
+
+# Read only access to the YARN registry: /ws/v1/slider/registry
+
+This was done primarily to get a browseable view of what the YARN-913 registry was publishing in the absence of any web view built in to that registry. The unit tests against this service actually discovered some JSON marshalling problems with the registry design and Jersey integration.
+
+The root path `/v1/slider/registry` lists the root path of the registry; all entries below it follow the path in the registry.
+
+ /ws/v1/slider/registry: {"nodes":["services","users"]}
+
+
+
+ Every entry in the path serves an instance of `org.apache.slider.server.appmaster.web.rest.registry.PathEntryResource`; a list of child directory entries, and, if present, the Yarn registry service record at that path (`org.apache.hadoop.registry.client.types.ServiceRecord`).
+
+ /ws/v1/slider/registry/users/stevel/services/org-apache-slider/apps-through-agent :
+
+ {
+ "nodes": ["components"],
+ "service": {
+ "type": "JSONServiceRecord",
+ "description": "Slider Application Master",
+ "external": [
+ {
+ "api": "http://",
+ "addressType": "uri",
+ "protocolType": "webui",
+ "addresses": [{"uri": "http://devix.cotham.uk:37283"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.management",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri": "http://devix.cotham.uk:37283/ws/v1/slider/mgmt"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.publisher",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri": "http://devix.cotham.uk:37283/ws/v1/slider/publisher"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.registry",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri": "http://devix.cotham.uk:37283/ws/v1/slider/registry"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.publisher.configurations",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri": "http://devix.cotham.uk:37283/ws/v1/slider/publisher/slider"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.publisher.exports",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri": "http://devix.cotham.uk:37283/ws/v1/slider/publisher/exports"}]
+ }
+ ],
+ "internal": [
+ {
+ "api": "classpath:org.apache.slider.agents.secure",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri": "https://devix.cotham.uk:44586/ws/v1/slider/agents"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.agents.oneway",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri": "https://devix.cotham.uk:36590/ws/v1/slider/agents"}]
+ }
+ ],
+ "yarn:persistence": "application",
+ "yarn:id": "application_1418130391694_0002"
+ }
+ }
+
+The slider code makes no direct use of this registry service except for a pair of tests. The Agents MUST use the core (ZK-based) registry service to locate a restarted AM. The slider client operations to enumerate and retrieve configuration files could use it, but they currently also use the YARN registry directly.
+
+As and when the YARN registry itself implements a REST view (possibly an R/W one), then this view will become obsolete âand possibly disabled.
+
+# Generated configurations: `/ws/v1/slider/publisher`
+
+This service contains a one or more "published configuration sets"
+
+ http://devix.cotham.uk:8088/proxy/application_1418130391694_0002/ws/v1/slider/publisher
+
+ {
+ "uris": {
+ "slider":"http://devix.cotham.uk:37283/ws/v1/slider/publisher/slider",
+ "exports":"http://devix.cotham.uk:37283/ws/v1/slider/publisher/exports"
+ }
+ }
+
+each one contains a list of available configuration documents , such as under /ws/v1/slider/publisher/slider:
+
+ {
+ "configurations":{
+ "core-site":{
+ "description":"Core site settings",
+ "updated":0,
+ "entries":{},"
+ empty":true
+ }
+ }
+ }
+
+The final path is built from the published configuration *and* the desired file format,the latter as a suffix. Thus the following links all server the same content, merely in different formats
+
+ /ws/v1/slider/publisher/slider/core-site.xml
+ /ws/v1/slider/publisher/slider/core-site.properties
+ /ws/v1/slider/publisher/slider/core-site.json
+
+The path for these becomes:
+
+ /ws/v1/slider/publisher/${set-name}/{configuration}.{format}
+
+The value of individual properties can be retrieved as JSON entries under the path
+
+ /ws/v1/slider/publisher/${set-name}/{configuration}/{key}
+
+Thus the response to a GET of `/ws/v1/slider/publisher/slider/core-site/fs.defaultFS`
+
+is
+
+ {"fs.defaultFS":"hdfs://devix.cotham.uk:8020"}
+
+Slider uses this service to publish its configuration (`core-site.xml`), etc, for diagnostics âand the configurations of the deployed applications on the command line
+
+ slider registry --getconf hbase-site.xml --name hbase1 --destdir out
+
+This configuration publishing service is listed in the YARN Registry, and could be implemented by other applications to publish their configurations âincluding the standard YARN services.
+
+# Application state in slider v1 format
+
+The original "v1" formatted slider specification/state is provided, wrapped in HTML, at:
+
+ http://devix.cotham.uk:8088/proxy/application_1418130391694_0002/slideram/spec
+
+This combines the declaration of what is desired with information about what has actually been provided
+
+(this shows the merits of a single file model which can describe both the desired state and the actual state; the same architecture of [the HPL Cells cloud infrastructure](http://www.hpl.hp.com/techreports/2012/HPL-2012-158R1.pdf))
+
+# Slider 0.70: codahale metrics servlets
+
+Slider 0.70 is adding metrics via the Codahale metrics library; as well as (optionally) publishing to Ganglia, the metrics service export metrics,health and liveness information, as well as a thread dump
+
+<table>
+ <tr>
+ <td>path</td>
+ <td>content</td>
+ </tr>
+ <tr>
+ <td>/system/metrics</td>
+ <td>Metrics information, available in JSON if &format=json appended</td>
+ </tr>
+ <tr>
+ <td>/system/health</td>
+ <td>health information from registered "health" classes</td>
+ </tr>
+ <tr>
+ <td>/system/ping</td>
+ <td>minimal liveness probe</td>
+ </tr>
+ <tr>
+ <td>/system/threads</td>
+ <td>thread dump</td>
+ </tr>
+</table>
+
+
+ /system/metrics :
+
+ {
+ "version": "3.0.0",
+ "gauges": {},
+ "counters": {},
+ "histograms": {},
+ "meters": {},
+ "timers": {}
+ }
+
+ /system/health :
+
+ {"AM Health": {"healthy": true}}
+
+As metrics are added, this content will start to be meaningful.
+
Modified: incubator/slider/site/trunk/content/index.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/index.md?rev=1644671&r1=1644670&r2=1644671&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/index.md (original)
+++ incubator/slider/site/trunk/content/index.md Thu Dec 11 15:17:06 2014
@@ -75,12 +75,13 @@ code changes and as of this writing, it
* [Application configuration model](/docs/configuration/index.html)
* [High Availability](/docs/high_availability.html)
* [Troubleshooting](/docs/troubleshooting.html)
+* [REST API](/docs/api/index.html)
## Developing
* [Building](/developing/building.html)
-* [Debugging](/docs/debugging.html)
+* [Debugging](/developing/debugging.html)
* [Testing](/developing/testing.html)
* [Functional Testing](/developing/functional_tests.html)
* [Manual Testing](/developing/manual_testing.html)