You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by mo...@apache.org on 2017/11/10 16:08:02 UTC
svn commit: r1814879 - in /knox: site/ site/books/knox-0-14-0/
trunk/books/0.14.0/
Author: more
Date: Fri Nov 10 16:08:01 2017
New Revision: 1814879
URL: http://svn.apache.org/viewvc?rev=1814879&view=rev
Log:
KNOX-1083 - Document HttpClient default timeout for hive HA
Modified:
knox/site/books/knox-0-14-0/user-guide.html
knox/site/index.html
knox/site/issue-tracking.html
knox/site/license.html
knox/site/mail-lists.html
knox/site/project-info.html
knox/site/team-list.html
knox/trunk/books/0.14.0/config.md
knox/trunk/books/0.14.0/service_hive.md
Modified: knox/site/books/knox-0-14-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-14-0/user-guide.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/books/knox-0-14-0/user-guide.html (original)
+++ knox/site/books/knox-0-14-0/user-guide.html Fri Nov 10 16:08:01 2017
@@ -472,7 +472,23 @@ https://{gateway-host}:{gateway-port}/{g
<value>false</value>
<description>Enable/Disable port mapping feature.</description>
</property>
-</code></pre><p>If a topology mapped port is in use by another topology or process then an ERROR message is logged and gateway startup continues as normal.</p><h3><a id="Configuration">Configuration</a> <a href="#Configuration"><img src="markbook-section-link.png"/></a></h3><p>Configuration for Apache Knox includes:</p>
+</code></pre><p>If a topology mapped port is in use by another topology or process then an ERROR message is logged and gateway startup continues as normal.</p>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--><h3><a id="Configuration">Configuration</a> <a href="#Configuration"><img src="markbook-section-link.png"/></a></h3><p>Configuration for Apache Knox includes:</p>
<ol>
<li><a href="#Related+Cluster+Configuration">Related Cluster Configuration</a> that must be done within the Hadoop cluster to allow Knox to communicate with various services</li>
<li><a href="#Gateway+Server+Configuration">Gateway Server Configuration</a> - which is the configurable elements of the server itself which applies to behavior that spans all topologies or managed Hadoop clusters</li>
@@ -615,12 +631,12 @@ https://{gateway-host}:{gateway-port}/{g
<tr>
<td>gateway.httpclient.connectionTimeout</td>
<td>The amount of time to wait when attempting a connection. The natural unit is milliseconds but a ‘s’ or ‘m’ suffix may be used for seconds or minutes respectively. The default timeout is system dependent. </td>
- <td>System Dependent</td>
+ <td>System Dependent (-1)</td>
</tr>
<tr>
<td>gateway.httpclient.socketTimeout</td>
<td>The amount of time to wait for data on a socket before aborting the connection. The natural unit is milliseconds but a ‘s’ or ‘m’ suffix may be used for seconds or minutes respectively. The default timeout is system dependent but is likely to be indefinite. </td>
- <td>System Dependent</td>
+ <td>System Dependent (-1)</td>
</tr>
<tr>
<td>gateway.httpserver.requestBuffer</td>
@@ -5687,7 +5703,18 @@ connection.close();
<pre><code><service>
<role>HIVE</role>
</service>
-</code></pre><p>Please note that there is no <code><url></code> tag specified here as the URLs for the Hive servers are obtained from Zookeeper.</p><h3><a id="Yarn">Yarn</a> <a href="#Yarn"><img src="markbook-section-link.png"/></a></h3><p>Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager REST APIs allow the user to get information about the cluster - status on the cluster, metrics on the cluster, scheduler information, information about nodes in the cluster, and information about applications on the cluster. Also as of Hadoop version 2.5.0, the user can submit a new application as well as kill it (or get state) using the ‘Writable’ APIs.</p><p>The docs for this can be found here</p><p><a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html</a></p><p>To enable this functionality, a
topology file needs to have the following configuration:</p>
+</code></pre><p>Please note that there is no <code><url></code> tag specified here as the URLs for the Hive servers are obtained from Zookeeper.</p><p>Also make sure the properties <code>gateway.httpclient.connectionTimeout</code> and <code>gateway.httpclient.socketTimeout</code> are set to acceptable defaults in <code>gateway-site.xml</code> config file e.g.</p>
+<pre><code> <property>
+ <name>gateway.httpclient.connectionTimeout</name>
+ <value>3000</value>
+ <description>Connection timeout after 30 sec</description>
+ </property>
+ <property>
+ <name>gateway.httpclient.socketTimeout</name>
+ <value>3000</value>
+ <description>Socket timeout after 30 sec</description>
+ </property>
+</code></pre><h3><a id="Yarn">Yarn</a> <a href="#Yarn"><img src="markbook-section-link.png"/></a></h3><p>Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager REST APIs allow the user to get information about the cluster - status on the cluster, metrics on the cluster, scheduler information, information about nodes in the cluster, and information about applications on the cluster. Also as of Hadoop version 2.5.0, the user can submit a new application as well as kill it (or get state) using the ‘Writable’ APIs.</p><p>The docs for this can be found here</p><p><a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html</a></p><p>To enable this functionality, a topology file needs to have the following configuration:</p>
<pre><code><service>
<role>RESOURCEMANAGER</role>
<url>http://<hostname>:<port>/ws</url>
Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-06
+ | Generated by Apache Maven Doxia at 2017-11-10
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171106" />
+ <meta name="Date-Revision-yyyymmdd" content="20171110" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – REST API and Application Gateway for the Apache Hadoop Ecosystem</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-10</li>
</ul>
</div>
Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-06
+ | Generated by Apache Maven Doxia at 2017-11-10
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171106" />
+ <meta name="Date-Revision-yyyymmdd" content="20171110" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Issue Tracking</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-10</li>
</ul>
</div>
Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-06
+ | Generated by Apache Maven Doxia at 2017-11-10
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171106" />
+ <meta name="Date-Revision-yyyymmdd" content="20171110" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project License</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-10</li>
</ul>
</div>
Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-06
+ | Generated by Apache Maven Doxia at 2017-11-10
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171106" />
+ <meta name="Date-Revision-yyyymmdd" content="20171110" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project Mailing Lists</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-10</li>
</ul>
</div>
Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-06
+ | Generated by Apache Maven Doxia at 2017-11-10
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171106" />
+ <meta name="Date-Revision-yyyymmdd" content="20171110" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project Information</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-10</li>
</ul>
</div>
Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-06
+ | Generated by Apache Maven Doxia at 2017-11-10
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171106" />
+ <meta name="Date-Revision-yyyymmdd" content="20171110" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Team list</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-10</li>
</ul>
</div>
Modified: knox/trunk/books/0.14.0/config.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/config.md?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/trunk/books/0.14.0/config.md (original)
+++ knox/trunk/books/0.14.0/config.md Fri Nov 10 16:08:01 2017
@@ -1,4 +1,4 @@
-<!---
+<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -13,7 +13,7 @@
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---->
+-->
### Configuration ###
@@ -128,8 +128,8 @@ gateway.keystore.type|Indicates the type
gateway.jdk.tls.ephemeralDHKeySize|jdk.tls.ephemeralDHKeySize, is defined to customize the ephemeral DH key sizes. The minimum acceptable DH key size is 1024 bits, except for exportable cipher suites or legacy mode (jdk.tls.ephemeralDHKeySize=legacy)|2048
gateway.threadpool.max|The maximum concurrent requests the server will process. The default is 254. Connections beyond this will be queued.|254
gateway.httpclient.maxConnections|The maximum number of connections that a single httpclient will maintain to a single host:port. The default is 32.|32
-gateway.httpclient.connectionTimeout|The amount of time to wait when attempting a connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent. | System Dependent
-gateway.httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent but is likely to be indefinite. | System Dependent
+gateway.httpclient.connectionTimeout|The amount of time to wait when attempting a connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent. | System Dependent (-1)
+gateway.httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent but is likely to be indefinite. | System Dependent (-1)
gateway.httpserver.requestBuffer|The size of the HTTP server request buffer. The default is 16K.|16384
gateway.httpserver.requestHeaderBuffer|The size of the HTTP server request header buffer. The default is 8K.|8192
gateway.httpserver.responseBuffer|The size of the HTTP server response buffer. The default is 32K.|32768
Modified: knox/trunk/books/0.14.0/service_hive.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/service_hive.md?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/trunk/books/0.14.0/service_hive.md (original)
+++ knox/trunk/books/0.14.0/service_hive.md Fri Nov 10 16:08:01 2017
@@ -322,3 +322,17 @@ And for the service configuration itself
Please note that there is no `<url>` tag specified here as the URLs for the Hive servers are obtained from Zookeeper.
+Also make sure the properties `gateway.httpclient.connectionTimeout` and `gateway.httpclient.socketTimeout` are set to acceptable defaults in `gateway-site.xml` config file e.g.
+
+ <property>
+ <name>gateway.httpclient.connectionTimeout</name>
+ <value>3000</value>
+ <description>Connection timeout after 30 sec</description>
+ </property>
+ <property>
+ <name>gateway.httpclient.socketTimeout</name>
+ <value>3000</value>
+ <description>Socket timeout after 30 sec</description>
+ </property>
+
+