You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by km...@apache.org on 2013/10/09 20:57:35 UTC
svn commit: r1530762 - in /incubator/knox: site/
site/books/knox-incubating-0-3-0/ trunk/books/0.3.0/
Author: kminder
Date: Wed Oct 9 18:57:34 2013
New Revision: 1530762
URL: http://svn.apache.org/r1530762
Log:
Indicate HBase/Stargate 0.95.2 will not work in secure mode.
Modified:
incubator/knox/site/books/knox-incubating-0-3-0/knox-incubating-0-3-0.html
incubator/knox/site/index.html
incubator/knox/site/issue-tracking.html
incubator/knox/site/license.html
incubator/knox/site/mail-lists.html
incubator/knox/site/project-info.html
incubator/knox/site/team-list.html
incubator/knox/trunk/books/0.3.0/book_getting-started.md
incubator/knox/trunk/books/0.3.0/config_kerberos.md
Modified: incubator/knox/site/books/knox-incubating-0-3-0/knox-incubating-0-3-0.html
URL: http://svn.apache.org/viewvc/incubator/knox/site/books/knox-incubating-0-3-0/knox-incubating-0-3-0.html?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/site/books/knox-incubating-0-3-0/knox-incubating-0-3-0.html (original)
+++ incubator/knox/site/books/knox-incubating-0-3-0/knox-incubating-0-3-0.html Wed Oct 9 18:57:34 2013
@@ -191,7 +191,7 @@
<td>HBase/Stargate </td>
<td>0.95.2 </td>
<td><img src="check.png" alt="y"/> </td>
- <td><img src="question.png" alt="?"/> </td>
+ <td><img src="error.png" alt="n"/> </td>
</tr>
<tr>
<td>Hive (via WebHCat) </td>
@@ -802,11 +802,22 @@ ldapRealm.userDnTemplate=uid={0},ou=peop
</topology>
</code></pre><h3><a id="Secure+Clusters"></a>Secure Clusters</h3><p>See these documents for setting up a secure Hadoop cluster <a href="http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Configuration_in_Secure_Mode">http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Configuration_in_Secure_Mode</a> <a href="http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.3.1/bk_installing_manually_book/content/rpm-chap14.html">http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.3.1/bk_installing_manually_book/content/rpm-chap14.html</a></p><p>Once you have a Hadoop cluster that is using Kerberos for authentication, you have to do the following to configure Knox to work with that cluster.</p><h4><a id="Create+Unix+account+for+Knox+on+Hadoop+master+nodes"></a>Create Unix account for Knox on Hadoop master nodes</h4>
<pre><code>useradd -g hadoop knox
-</code></pre><h4><a id="Create+Kerberos+principal,+keytab+for+Knox"></a>Create Kerberos principal, keytab for Knox</h4><p>One way of doing this, assuming your KDC realm is EXAMPLE.COM</p><p>ssh into your host running KDC</p>
+</code></pre><h4><a id="Create+Kerberos+principal,+keytab+for+Knox"></a>Create Kerberos principal, keytab for Knox</h4><p>One way of doing this, assuming your KDC realm is EXAMPLE.COM, is to ssh into your host running KDC and execute <code>kadmin.local</code> That will result in an interactive session in which you can execute commands.</p><p>ssh into your host running KDC</p>
<pre><code>kadmin.local
add_principal -randkey knox/knox@EXAMPLE.COM
ktadd -norandkey -k /etc/security/keytabs/knox.service.keytab
-</code></pre><h4><a id="Grant+Proxy+privileges+for+Knox+user+in+`core-site.xml`+on+Hadoop+master+nodes"></a>Grant Proxy privileges for Knox user in <code>core-site.xml</code> on Hadoop master nodes</h4><p>Update <code>core-site.xml</code> and add the following lines towards the end of the file.</p><p>Replace FQDN_OF_KNOX_HOST with right value in your cluster. You could use * for local developer testing if Knox host does not have static IP.</p>
+ktadd -k /etc/security/keytabs/knox.service.keytab -norandkey knox/knox@EXAMPLE.COM
+exit
+</code></pre><h4><a id="Grant+Proxy+privileges+for+Knox+user+in+`core-site.xml`+on+Hadoop+master+nodes"></a>Grant Proxy privileges for Knox user in <code>core-site.xml</code> on Hadoop master nodes</h4><p>Update <code>core-site.xml</code> and add the following lines towards the end of the file.</p><p>Replace FQDN_OF_KNOX_HOST with the fully qualified domain name of the host running the gateway. You can usually find this by running <code>hostname -f</code> on that host.</p><p>You could use * for local developer testing if Knox host does not have static IP.</p>
+<pre><code><property>
+ <name>hadoop.proxyuser.knox.groups</name>
+ <value>users</value>
+</property>
+<property>
+ <name>hadoop.proxyuser.knox.hosts</name>
+ <value>FQDN_OF_KNOX_HOST</value>
+</property>
+</code></pre><h4><a id="Grant+proxy+privilege+for+Knox+in+`webhcat-stie.xml`+on+Hadoop+master+nodes"></a>Grant proxy privilege for Knox in <code>webhcat-stie.xml</code> on Hadoop master nodes</h4><p>Update <code>webhcat-site.xml</code> and add the following lines towards the end of the file.</p><p>Replace FQDN_OF_KNOX_HOST with right value in your cluster. You could use * for local developer testing if Knox host does not have static IP.</p>
<pre><code><property>
<name>hadoop.proxyuser.knox.groups</name>
<value>users</value>
@@ -824,12 +835,12 @@ ktadd -norandkey -k /etc/security/keytab
<name>oozie.service.ProxyUserService.proxyuser.knox.hosts</name>
<value>FQDN_OF_KNOX_HOST</value>
</property>
-</code></pre><h4><a id="Copy+knox+keytab+to+Knox+host"></a>Copy knox keytab to Knox host</h4><p>Add unix account for knox on Knox host</p>
+</code></pre><h4><a id="Copy+knox+keytab+to+Knox+host"></a>Copy knox keytab to Knox host</h4><p>Add unix account for the knox user on Knox host</p>
<pre><code>useradd -g hadoop knox
</code></pre><p>Copy knox.service.keytab created on KDC host on to your Knox host /etc/knox/conf/knox.service.keytab</p>
<pre><code>chown knox knox.service.keytab
chmod 400 knox.service.keytab
-</code></pre><h4><a id="Update+krb5.conf+at+/etc/knox/conf/krb5.conf+on+Knox+host"></a>Update krb5.conf at /etc/knox/conf/krb5.conf on Knox host</h4><p>You could copy the <code>templates/krb5.conf</code> file provided in the Knox binary download and customize it to suit your cluster.</p><h4><a id="Update+`krb5JAASLogin.conf`+at+`/etc/knox/conf/krb5JAASLogin.conf`+on+Knox+host"></a>Update <code>krb5JAASLogin.conf</code> at <code>/etc/knox/conf/krb5JAASLogin.conf</code> on Knox host</h4><p>You could copy the <code>templates/krb5JAASLogin.conf</code> file provided in the Knox binary download and customize it to suit your cluster.</p><h4><a id="Update+`gateway-site.xml`+on+Knox+host+on+Knox+host"></a>Update <code>gateway-site.xml</code> on Knox host on Knox host</h4><p>Update <code>conf/gateway-site.xml</code> in your Knox installation and set the value of <code>gateway.hadoop.kerberos.secured</code> to true.</p><h4><a id="Restart+Knox"></a>Restart Knox</h4><p>After you do the above con
figurations and restart Knox, Knox would use SPNego to authenticate with Hadoop services and Oozie. There is not change in the way you make calls to Knox whether you use Curl or Knox DSL.</p><h2><a id="Client+Details"></a>Client Details</h2><p>Hadoop requires a client that can be used to interact remotely with the services provided by Hadoop cluster. This will also be true when using the Apache Knox Gateway to provide perimeter security and centralized access for these services. The two primary existing clients for Hadoop are the CLI (i.e. Command Line Interface, hadoop) and HUE (i.e. Hadoop User Environment). For several reasons however, neither of these clients can <em>currently</em> be used to access Hadoop services via the Apache Knox Gateway.</p><p>This led to thinking about a very simple client that could help people use and evaluate the gateway. The list below outlines the general requirements for such a client.</p>
+</code></pre><h4><a id="Update+krb5.conf+at+/etc/knox/conf/krb5.conf+on+Knox+host"></a>Update krb5.conf at /etc/knox/conf/krb5.conf on Knox host</h4><p>You could copy the <code>templates/krb5.conf</code> file provided in the Knox binary download and customize it to suit your cluster.</p><h4><a id="Update+`krb5JAASLogin.conf`+at+`/etc/knox/conf/krb5JAASLogin.conf`+on+Knox+host"></a>Update <code>krb5JAASLogin.conf</code> at <code>/etc/knox/conf/krb5JAASLogin.conf</code> on Knox host</h4><p>You could copy the <code>templates/krb5JAASLogin.conf</code> file provided in the Knox binary download and customize it to suit your cluster.</p><h4><a id="Update+`gateway-site.xml`+on+Knox+host+on+Knox+host"></a>Update <code>gateway-site.xml</code> on Knox host on Knox host</h4><p>Update <code>conf/gateway-site.xml</code> in your Knox installation and set the value of <code>gateway.hadoop.kerberos.secured</code> to true.</p><h4><a id="Restart+Knox"></a>Restart Knox</h4><p>After you do the above con
figurations and restart Knox, Knox would use SPNego to authenticate with Hadoop services and Oozie. There is no change in the way you make calls to Knox whether you use Curl or Knox DSL.</p><h2><a id="Client+Details"></a>Client Details</h2><p>Hadoop requires a client that can be used to interact remotely with the services provided by Hadoop cluster. This will also be true when using the Apache Knox Gateway to provide perimeter security and centralized access for these services. The two primary existing clients for Hadoop are the CLI (i.e. Command Line Interface, hadoop) and HUE (i.e. Hadoop User Environment). For several reasons however, neither of these clients can <em>currently</em> be used to access Hadoop services via the Apache Knox Gateway.</p><p>This led to thinking about a very simple client that could help people use and evaluate the gateway. The list below outlines the general requirements for such a client.</p>
<ul>
<li>Promote the evaluation and adoption of the Apache Knox Gateway</li>
<li>Simple to deploy and use on data worker desktops to access to remote Hadoop clusters</li>
Modified: incubator/knox/site/index.html
URL: http://svn.apache.org/viewvc/incubator/knox/site/index.html?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/site/index.html (original)
+++ incubator/knox/site/index.html Wed Oct 9 18:57:34 2013
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 8, 2013 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 9, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20131008" />
+ <meta name="Date-Revision-yyyymmdd" content="20131009" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2013-10-08</span>
+ | <span id="publishDate">Last Published: 2013-10-09</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: incubator/knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/incubator/knox/site/issue-tracking.html?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/site/issue-tracking.html (original)
+++ incubator/knox/site/issue-tracking.html Wed Oct 9 18:57:34 2013
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 8, 2013 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 9, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20131008" />
+ <meta name="Date-Revision-yyyymmdd" content="20131009" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2013-10-08</span>
+ | <span id="publishDate">Last Published: 2013-10-09</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: incubator/knox/site/license.html
URL: http://svn.apache.org/viewvc/incubator/knox/site/license.html?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/site/license.html (original)
+++ incubator/knox/site/license.html Wed Oct 9 18:57:34 2013
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 8, 2013 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 9, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20131008" />
+ <meta name="Date-Revision-yyyymmdd" content="20131009" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2013-10-08</span>
+ | <span id="publishDate">Last Published: 2013-10-09</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: incubator/knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/incubator/knox/site/mail-lists.html?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/site/mail-lists.html (original)
+++ incubator/knox/site/mail-lists.html Wed Oct 9 18:57:34 2013
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 8, 2013 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 9, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20131008" />
+ <meta name="Date-Revision-yyyymmdd" content="20131009" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2013-10-08</span>
+ | <span id="publishDate">Last Published: 2013-10-09</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: incubator/knox/site/project-info.html
URL: http://svn.apache.org/viewvc/incubator/knox/site/project-info.html?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/site/project-info.html (original)
+++ incubator/knox/site/project-info.html Wed Oct 9 18:57:34 2013
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 8, 2013 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 9, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20131008" />
+ <meta name="Date-Revision-yyyymmdd" content="20131009" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2013-10-08</span>
+ | <span id="publishDate">Last Published: 2013-10-09</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: incubator/knox/site/team-list.html
URL: http://svn.apache.org/viewvc/incubator/knox/site/team-list.html?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/site/team-list.html (original)
+++ incubator/knox/site/team-list.html Wed Oct 9 18:57:34 2013
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 8, 2013 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.3 at Oct 9, 2013 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20131008" />
+ <meta name="Date-Revision-yyyymmdd" content="20131009" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2013-10-08</span>
+ | <span id="publishDate">Last Published: 2013-10-09</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: incubator/knox/trunk/books/0.3.0/book_getting-started.md
URL: http://svn.apache.org/viewvc/incubator/knox/trunk/books/0.3.0/book_getting-started.md?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/trunk/books/0.3.0/book_getting-started.md (original)
+++ incubator/knox/trunk/books/0.3.0/book_getting-started.md Wed Oct 9 18:57:34 2013
@@ -173,7 +173,7 @@ Only more recent versions of some Hadoop
| WebHDFS | 2.1.0 | ![y] | ![y] |
| WebHCat/Templeton | 0.11.0 | ![y] | ![y] |
| Ozzie | 4.0.0 | ![y] | ![y] |
-| HBase/Stargate | 0.95.2 | ![y] | ![?] |
+| HBase/Stargate | 0.95.2 | ![y] | ![n] |
| Hive (via WebHCat) | 0.11.0 | ![y] | ![n] |
| | 0.12.0 | ![y] | ![?] |
| Hive (via JDBC) | 0.11.0 | ![n] | ![n] |
Modified: incubator/knox/trunk/books/0.3.0/config_kerberos.md
URL: http://svn.apache.org/viewvc/incubator/knox/trunk/books/0.3.0/config_kerberos.md?rev=1530762&r1=1530761&r2=1530762&view=diff
==============================================================================
--- incubator/knox/trunk/books/0.3.0/config_kerberos.md (original)
+++ incubator/knox/trunk/books/0.3.0/config_kerberos.md Wed Oct 9 18:57:34 2013
@@ -29,19 +29,24 @@ Once you have a Hadoop cluster that is u
#### Create Kerberos principal, keytab for Knox ####
-One way of doing this, assuming your KDC realm is EXAMPLE.COM
+One way of doing this, assuming your KDC realm is EXAMPLE.COM, is to ssh into your host running KDC and execute `kadmin.local`
+That will result in an interactive session in which you can execute commands.
ssh into your host running KDC
kadmin.local
add_principal -randkey knox/knox@EXAMPLE.COM
ktadd -norandkey -k /etc/security/keytabs/knox.service.keytab
+ ktadd -k /etc/security/keytabs/knox.service.keytab -norandkey knox/knox@EXAMPLE.COM
+ exit
#### Grant Proxy privileges for Knox user in `core-site.xml` on Hadoop master nodes ####
Update `core-site.xml` and add the following lines towards the end of the file.
-Replace FQDN_OF_KNOX_HOST with right value in your cluster.
+Replace FQDN_OF_KNOX_HOST with the fully qualified domain name of the host running the gateway.
+You can usually find this by running `hostname -f` on that host.
+
You could use * for local developer testing if Knox host does not have static IP.
<property>
@@ -87,7 +92,7 @@ You could use * for local developer test
#### Copy knox keytab to Knox host ####
-Add unix account for knox on Knox host
+Add unix account for the knox user on Knox host
useradd -g hadoop knox
@@ -111,4 +116,4 @@ Update `conf/gateway-site.xml` in your K
#### Restart Knox ####
After you do the above configurations and restart Knox, Knox would use SPNego to authenticate with Hadoop services and Oozie.
-There is not change in the way you make calls to Knox whether you use Curl or Knox DSL.
+There is no change in the way you make calls to Knox whether you use Curl or Knox DSL.