You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by mo...@apache.org on 2017/02/10 16:31:09 UTC
svn commit: r1782487 [12/12] - in /knox: site/books/knox-0-12-0/
trunk/books/0.12.0/ trunk/books/0.12.0/dev-guide/
Added: knox/trunk/books/0.12.0/service_yarn.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.12.0/service_yarn.md?rev=1782487&view=auto
==============================================================================
--- knox/trunk/books/0.12.0/service_yarn.md (added)
+++ knox/trunk/books/0.12.0/service_yarn.md Fri Feb 10 16:31:08 2017
@@ -0,0 +1,124 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### Yarn ###
+
+Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager REST APIs allow the
+user to get information about the cluster - status on the cluster, metrics on the cluster, scheduler information,
+information about nodes in the cluster, and information about applications on the cluster. Also as of Hadoop version
+2.5.0, the user can submit a new application as well as kill it (or get state) using the 'Writable' APIs.
+
+The docs for this can be found here
+
+http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html
+
+To enable this functionality, a topology file needs to have the following configuration:
+
+ <service>
+ <role>RESOURCEMANAGER</role>
+ <url>http://<hostname>:<port>/ws</url>
+ </service>
+
+The default resource manager http port is 8088. If it is configured to some other port, that configuration can be
+found in `yarn-site.xml` under the property `yarn.resourcemanager.webapp.address`.
+
+#### Yarn URL Mapping ####
+
+For Yarn URLs, the mapping of Knox Gateway accessible URLs to direct Yarn URLs is the following.
+
+| ------- | ------------------------------------------------------------------------------------- |
+| Gateway | `https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/resourcemanager` |
+| Cluster | `http://{yarn-host}:{yarn-port}/ws}` |
+
+
+#### Yarn Examples via cURL
+
+Some of the various calls that can be made and examples using curl are listed below.
+
+ # 0. Getting cluster info
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster'
+
+ # 1. Getting cluster metrics
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/metrics'
+
+ To get the same information in an xml format
+
+ curl -ikv -u guest:guest-password -H Accept:application/xml -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/metrics'
+
+ # 2. Getting scheduler information
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/scheduler'
+
+ # 3. Getting all the applications listed and their information
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps'
+
+ # 4. Getting applications statistics
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/appstatistics'
+
+ Also query params can be used as below to filter the results
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/appstatistics?states=accepted,running,finished&applicationTypes=mapreduce'
+
+ # 5. To get a specific application (please note, replace the application id with a real value)
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}'
+
+ # 6. To get the attempts made for a particular application
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/appattempts'
+
+ # 7. To get information about the various nodes
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/nodes'
+
+ Also to get a specific node, use an id obtained in the response from above (the node id is scrambled) and issue the following
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/nodes/{node_id}'
+
+ # 8. To create a new Application
+
+ curl -ikv -u guest:guest-password -X POST 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/new-application'
+
+ An application id is returned from the request above and this can be used to submit an application.
+
+ # 9. To submit an application, put together a request containing the application id received in the above response (please refer to Yarn REST
+ API documentation).
+
+ curl -ikv -u guest:guest-password -T request.json -H Content-Type:application/json -X POST 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps'
+
+ Here the request is saved in a file called request.json
+
+ #10. To get application state
+
+ curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
+
+ curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/application_1409008107556_0007/state'
+
+ # 11. To kill an application that is running issue the below command with the application id of the application that is to be killed.
+ The contents of the state-killed.json file are :
+
+ {
+ "state":"KILLED"
+ }
+
+
+ curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
+
Added: knox/trunk/books/0.12.0/websocket-support.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.12.0/websocket-support.md?rev=1782487&view=auto
==============================================================================
--- knox/trunk/books/0.12.0/websocket-support.md (added)
+++ knox/trunk/books/0.12.0/websocket-support.md Fri Feb 10 16:31:08 2017
@@ -0,0 +1,76 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+## Websocket Support ##
+
+### Introduction
+
+Websocket is a communication protocol that allows full duplex communication over single TCP connection.
+Knox provides out-of-the-box support for websocket protocol, currently only text messages are supported.
+
+### Configuration ###
+
+By default websocket functionality is disabled, it can be easily enabled by changing the 'gateway.websocket.feature.enabled' property to 'true' in <KNOX-HOME>/conf/gateway-site.xml file.
+
+ <property>
+ <name>gateway.websocket.feature.enabled</name>
+ <value>true</value>
+ <description>Enable/Disable websocket feature.</description>
+ </property>
+
+Service and rewrite rules need to changed accordingly to match the appropriate websocket context.
+
+### Example ###
+
+In the following sample configuration we assume that the backend websocket URL is ws://myhost:9999/ws. And 'gateway.websocket.feature.enabled' property is set to 'true' as shown above.
+
+#### rewrite ####
+
+Example code snippet from <KNOX-HOME>/data/services/{myservice}/{version}/rewrite.xml where myservice = websocket and version = 0.6.0
+
+ <rules>
+ <rule dir="IN" name="WEBSOCKET/ws/inbound" pattern="*://*:*/**/ws">
+ <rewrite template="{$serviceUrl[WEBSOCKET]}/ws"/>
+ </rule>
+ </rules>
+
+#### service ####
+
+Example code snippet from <KNOX-HOME>/data/services/{myservice}/{version}/service.xml where myservice = websocket and version = 0.6.0
+
+ <service role="WEBSOCKET" name="websocket" version="0.6.0">
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="authentication" name="Anonymous"/>
+ <policy role="rewrite"/>
+ <policy role="authorization"/>
+ </policies>
+ <routes>
+ <route path="/ws">
+ <rewrite apply="WEBSOCKET/ws/inbound" to="request.url"/>
+ </route>
+ </routes>
+ </service>
+
+#### topology ####
+
+Finally, update the topology file at <KNOX-HOME>/conf/{topology}.xml with the backend service url
+
+ <service>
+ <role>WEBSOCKET</role>
+ <url>ws://myhost:9999/ws</url>
+ </service>
Added: knox/trunk/books/0.12.0/x-forwarded-headers.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.12.0/x-forwarded-headers.md?rev=1782487&view=auto
==============================================================================
--- knox/trunk/books/0.12.0/x-forwarded-headers.md (added)
+++ knox/trunk/books/0.12.0/x-forwarded-headers.md Fri Feb 10 16:31:08 2017
@@ -0,0 +1,76 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### X-Forwarded-* Headers Support ###
+Out-of-the-box Knox provides support for some X-Forwarded-* headers through the use of a Servlet Filter. Specifically the
+headers handled/populated by Knox are:
+
+* X-Forwarded-For
+* X-Forwarded-Proto
+* X-Forwarded-Port
+* X-Forwarded-Host
+* X-Forwarded-Server
+* X-Forwarded-Context
+
+If this functionality can be turned off by a configuration setting in the file gateway-site.xml and redeploying the
+necessary topology/topologies.
+
+The setting is (under the 'configuration' tag) :
+
+ <property>
+ <name>gateway.xforwarded.enabled</name>
+ <value>false</value>
+ </property>
+
+If this setting is absent, the default behavior is that the X-Forwarded-* header support is on or in other words,
+'gateway.xforwarded.enabled' is set to 'true' by default.
+
+
+#### Header population ####
+
+The following are the various rules for population of these headers:
+
+##### X-Forwarded-For #####
+
+This header represents a list of client IP addresses. If the header is already present Knox adds a comma separated value
+to the list. The value added is the client's IP address as Knox sees it. This value is added to the end of the list.
+
+##### X-Forwarded-Proto #####
+
+The protocol used in the client request. If this header is passed into Knox its value is maintained, otherwise Knox will
+populate the header with the value 'https' if the request is a secure one or 'http' otherwise.
+
+##### X-Forwarded-Port #####
+
+The port used in the client request. If this header is passed into Knox its value is maintained, otherwise Knox will
+populate the header with the value of the port that the request was made coming into Knox.
+
+##### X-Forwarded-Host #####
+
+Represents the original host requested by the client in the Host HTTP request header. The value passed into Knox is maintained
+by Knox. If no value is present, Knox populates the header with the value of the HTTP Host header.
+
+##### X-Forwarded-Server #####
+
+The hostname of the server Knox is running on.
+
+##### X-Forwarded-Context #####
+
+This header value contains the context path of the request to Knox.
+
+
+