You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oozie.apache.org by bu...@apache.org on 2018/04/09 14:26:52 UTC
svn commit: r1028114 [5/16] - in /websites/staging/oozie/trunk/content: ./
docs/5.0.0/ docs/5.0.0/client/ docs/5.0.0/client/css/
docs/5.0.0/client/images/ docs/5.0.0/client/images/logos/ docs/5.0.0/core/
docs/5.0.0/core/css/ docs/5.0.0/core/images/ doc...
Added: websites/staging/oozie/trunk/content/docs/5.0.0/DG_ActionAuthentication.html
==============================================================================
--- websites/staging/oozie/trunk/content/docs/5.0.0/DG_ActionAuthentication.html (added)
+++ websites/staging/oozie/trunk/content/docs/5.0.0/DG_ActionAuthentication.html Mon Apr 9 14:26:49 2018
@@ -0,0 +1,278 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at Apr 9, 2018
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Oozie - </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="https://oozie.apache.org/" id="bannerLeft">
+ <img src="https://oozie.apache.org/images/oozie_200x.png" alt="Oozie"/>
+ </a>
+ </div>
+ <div class="pull-right"> </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../../" title="Apache">
+ Apache</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../../" title="Oozie">
+ Oozie</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../" title="docs">
+ docs</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="./" title="5.0.0">
+ 5.0.0</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Oozie - </li>
+
+
+
+ <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-04-09</li>
+ <li id="projectVersion" class="pull-right">
+ Version: 5.0.0
+ </li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <p></p>
+<p><a href="./index.html">::Go back to Oozie Documentation Index::</a>
+</p>
+<a name="Action_Authentication"></a>
+<div class="section"><h2> Action Authentication</h2>
+<p><ul><ul><li><a href="#Background">Background</a>
+</li>
+<li><a href="#Oozie_Server_Configuration">Oozie Server Configuration</a>
+</li>
+<li><a href="#Workflow_Changes">Workflow Changes</a>
+</li>
+<li><a href="#Built-in_Credentials_Implementations">Built-in Credentials Implementations</a>
+</li>
+</ul>
+</ul>
+</p>
+<a name="Background"></a>
+<div class="section"><h3>Background</h3>
+<p>A secure cluster requires that actions have been authenticated (typically via Kerberos). However, due to the way that Oozie runs
+actions, Kerberos credentials are not easily made available to actions launched by Oozie. For many action types, this is not a
+problem because they are self contained (beyond core Hadoop components). For example, a Pig action typically only talks to
+MapReduce and HDFS. However, some actions require talking to external services (e.g. HCatalog, HBase Region Server, Hive Server 2)
+and in these cases, the actions require some extra configuration in Oozie to authenticate. To be clear, this extra configuration
+is only required if an action will be talking to these types of external services; running a typical MapReduce, Pig, Hive, etc
+action will not require any of this.</p>
+<p>For these situations, Oozie will have to use its Kerberos credentials to obtain "delegation tokens" (think of it like a cookie) on
+behalf of the user from the service in question. The details of what this means is beyond the scope of this documentation, but
+basically, Oozie needs some extra configuration in the workflow so that it can obtain this delegation token.</p>
+<a name="Oozie_Server_Configuration"></a>
+</div>
+<div class="section"><h3>Oozie Server Configuration</h3>
+<p>The code to obtain delegation tokens is pluggable so that it is easy to add support for different services by simply subclassing
+org.apache.oozie.action.hadoop.Credentials to retrieve a delegation token from the service and add it to the Configuration.</p>
+<p>Out of the box, Oozie already comes with support for some credential types
+(see <a href="./DG_ActionAuthentication.html#Built-in_Credentials_Implementations">Built-in Credentials Implementations</a>
+).
+The credential classes that Oozie should load are specified by the following property in oozie-site.xml. The left hand side of the
+equals sign is the type for the credential type, while the right hand side is the class.</p>
+<p><pre>
+ <property>
+ <name>oozie.credentials.credentialclasses</name>
+ <value>
+ hcat=org.apache.oozie.action.hadoop.HCatCredentials,
+ hbase=org.apache.oozie.action.hadoop.HbaseCredentials,
+ hive2=org.apache.oozie.action.hadoop.Hive2Credentials
+ </value>
+ </property>
+</pre></p>
+<a name="Workflow_Changes"></a>
+</div>
+<div class="section"><h3>Workflow Changes</h3>
+<p>The user should add a <tt>credentials</tt>
+ section to the top of their workflow that contains 1 or more <tt>credential</tt>
+ sections. Each of
+these <tt>credential</tt>
+ sections contains a name for the credential, the type for the credential, and any configuration properties
+needed by that type of credential for obtaining a delegation token. The <tt>credentials</tt>
+ section is available in workflow schema
+version 0.3 and later.</p>
+<p>For example, the following workflow is configured to obtain an HCatalog delegation token, which is given to a Pig action so that the
+Pig action can talk to a secure HCatalog:</p>
+<p><pre>
+ <workflow-app xmlns='uri:oozie:workflow:0.4' name='pig-wf'>
+ <credentials>
+ <credential name='my-hcat-creds' type='hcat'>
+ <property>
+ <name>hcat.metastore.uri</name>
+ <value>HCAT_URI</value>
+ </property>
+ <property>
+ <name>hcat.metastore.principal</name>
+ <value>HCAT_PRINCIPAL</value>
+ </property>
+ </credential>
+ </credentials>
+ ...
+ <action name='pig' cred='my-hcat-creds'>
+ <pig>
+ <job-tracker>JT</job-tracker>
+ <name-node>NN</name-node>
+ <configuration>
+ <property>
+ <name>TESTING</name>
+ <value>${start}</value>
+ </property>
+ </configuration>
+ </pig>
+ </action>
+ ...
+ </workflow-app>
+</pre></p>
+<p>The type of the <tt>credential</tt>
+ is "hcat", which is the type name we gave for the HCatCredentials class in oozie-site.xml. We gave
+the <tt>credential</tt>
+ a name, "my-hcat-creds", which can be whatever you want; we then specify cred='my-hcat-creds' in the Pig action,
+so that Oozie will include these credentials with the action. You can include multiple credentials with an action by specifying
+a comma-separated list of <tt>credential</tt>
+ names. And finally, the HCatCredentials required two properties (the metastore URI and
+principal), which we also specified.</p>
+<p>Adding the <tt>credentials</tt>
+ section to a workflow and referencing it in an action will make Oozie always try to obtain that delegation
+token. Ordinarily, this would mean that you cannot re-use this workflow in a non-secure cluster without editing it because trying
+to obtain the delegation token will likely fail. However, you can tell Oozie to ignore the <tt>credentials</tt>
+ for a workflow by setting
+the job-level property <tt>oozie.credentials.skip</tt>
+ to <tt>true</tt>
+; this will allow you to use the same workflow.xml in a secure and
+non-secure cluster by simply changing the job-level property at runtime. If omitted or set to <tt>false</tt>
+, Oozie will handle
+the <tt>credentials</tt>
+ section normally. In addition, you can also set this property at the action-level or server-level to skip getting
+credentials for just that action or for all workflows, respectively. The order of priority is this:</p>
+<p><ol type="1"><li><tt>oozie.credentials.skip</tt>
+ in the <tt>configuration</tt>
+ section of an action, if set</li>
+<li><tt>oozie.credentials.skip</tt>
+ in the job.properties for a workflow, if set</li>
+<li><tt>oozie.credentials.skip</tt>
+ in oozie-site.xml for all workflows, if set</li>
+<li>(don't skip)</li>
+</ol>
+</p>
+<a name="Built-in_Credentials_Implementations"></a>
+</div>
+<div class="section"><h3>Built-in Credentials Implementations</h3>
+<p>Oozie currently comes with the following Credentials implementations:</p>
+<p><ol type="1"><li>HCatalog and Hive Metastore: <tt>org.apache.oozie.action.hadoop.HCatCredentials</tt>
+</li>
+<li>HBase: <tt>org.apache.oozie.action.hadoop.HBaseCredentials</tt>
+</li>
+<li>Hive Server 2: <tt>org.apache.oozie.action.hadoop.Hive2Credentials</tt>
+</li>
+</ol>
+</p>
+<p>HCatCredentials requires these two properties:</p>
+<p><ol type="1"><li><tt>hcat.metastore.principal</tt>
+ or hive.metastore.kerberos.principal</li>
+<li><tt>hcat.metastore.uri</tt>
+ or hive.metastore.uris</li>
+</ol>
+</p>
+<p><b>Note:</b>
+ The HCatalog Metastore and Hive Metastore are one and the same and so the "hcat" type credential can also be used to talk
+to a secure Hive Metastore, though the property names would still start with "hcat.".</p>
+<p>HBase does not require any additional properties since the hbase-site.xml on the Oozie server provides necessary information to the
+obtain delegation token; though properties can be overwritten here if desired.</p>
+<p>Hive2Credentials requires these two properties:</p>
+<p><ol type="1"><li><tt>hive2.server.principal</tt>
+</li>
+<li><tt>hive2.jdbc.url</tt>
+</li>
+</ol>
+</p>
+<p><a href="./index.html">::Go back to Oozie Documentation Index::</a>
+</p>
+<p></p>
+</div>
+
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright © 2018
+ <a href="http://www.apache.org">Apache Software Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>