You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oozie.apache.org by di...@apache.org on 2021/02/26 14:14:21 UTC
svn commit: r1886952 [5/19] - in /oozie/site/trunk: ./ content/
content/resources/docs/5.2.1/ content/resources/docs/5.2.1/css/
content/resources/docs/5.2.1/fonts/ content/resources/docs/5.2.1/images/
content/resources/docs/5.2.1/images/logos/ content/...
Added: oozie/site/trunk/content/resources/docs/5.2.1/DG_ActionAuthentication.html
URL: http://svn.apache.org/viewvc/oozie/site/trunk/content/resources/docs/5.2.1/DG_ActionAuthentication.html?rev=1886952&view=auto
==============================================================================
--- oozie/site/trunk/content/resources/docs/5.2.1/DG_ActionAuthentication.html (added)
+++ oozie/site/trunk/content/resources/docs/5.2.1/DG_ActionAuthentication.html Fri Feb 26 14:14:19 2021
@@ -0,0 +1,220 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2021-02-26
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20210226" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Oozie – </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="https://oozie.apache.org/" id="bannerLeft">
+ <img src="https://oozie.apache.org/images/oozie_200x.png" alt="Oozie"/>
+ </a>
+ </div>
+ <div class="pull-right"> </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org/" class="externalLink" title="Apache">
+ Apache</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../../" title="Oozie">
+ Oozie</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../" title="docs">
+ docs</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="./" title="5.2.1">
+ 5.2.1</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active "></li>
+
+
+
+ <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2021-02-26</li>
+ <li id="projectVersion" class="pull-right">
+ Version: 5.2.1
+ </li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p>
+<h1>Action Authentication</h1>
+<ul>
+<li><a href="#Background">Background</a></li>
+<li><a href="#Oozie_Server_Configuration">Oozie Server Configuration</a></li>
+<li><a href="#Workflow_Changes">Workflow Changes</a></li>
+<li><a href="#Built-in_Credentials_Implementations">Built-in Credentials Implementations</a></li></ul>
+
+<div class="section">
+<h2><a name="Background"></a>Background</h2>
+<p>A secure cluster requires that actions have been authenticated (typically via Kerberos). However, due to the way that Oozie runs actions, Kerberos credentials are not easily made available to actions launched by Oozie. For many action types, this is not a problem because they are self contained (beyond core Hadoop components). For example, a Pig action typically only talks to MapReduce and HDFS. However, some actions require talking to external services (e.g. HCatalog, HBase Region Server, Hive Server 2) and in these cases, the actions require some extra configuration in Oozie to authenticate. To be clear, this extra configuration is only required if an action will be talking to these types of external services; running a typical MapReduce, Pig, Hive, etc action will not require any of this.</p>
+<p>For these situations, Oozie will have to use its Kerberos credentials to obtain “delegation tokens” (think of it like a cookie) on behalf of the user from the service in question. The details of what this means is beyond the scope of this documentation, but basically, Oozie needs some extra configuration in the workflow so that it can obtain this delegation token.</p></div>
+<div class="section">
+<h2><a name="Oozie_Server_Configuration"></a>Oozie Server Configuration</h2>
+<p>The code to obtain delegation tokens is pluggable so that it is easy to add support for different services by simply subclassing org.apache.oozie.action.hadoop.Credentials to retrieve a delegation token from the service and add it to the Configuration.</p>
+<p>Out of the box, Oozie already comes with support for some credential types (see <a href="DG_ActionAuthentication.html#Built-in_Credentials_Implementations">Built-in Credentials Implementations</a>). The credential classes that Oozie should load are specified by the following property in oozie-site.xml. The left hand side of the equals sign is the type for the credential type, while the right hand side is the class.</p>
+
+<div>
+<div>
+<pre class="source"> <property>
+ <name>oozie.credentials.credentialclasses</name>
+ <value>
+ hcat=org.apache.oozie.action.hadoop.HCatCredentials,
+ hbase=org.apache.oozie.action.hadoop.HbaseCredentials,
+ hive2=org.apache.oozie.action.hadoop.Hive2Credentials
+ </value>
+ </property>
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Workflow_Changes"></a>Workflow Changes</h2>
+<p>The user should add a <tt>credentials</tt> section to the top of their workflow that contains 1 or more <tt>credential</tt> sections. Each of these <tt>credential</tt> sections contains a name for the credential, the type for the credential, and any configuration properties needed by that type of credential for obtaining a delegation token. The <tt>credentials</tt> section is available in workflow schema version 0.3 and later.</p>
+<p>For example, the following workflow is configured to obtain an HCatalog delegation token, which is given to a Pig action so that the Pig action can talk to a secure HCatalog:</p>
+
+<div>
+<div>
+<pre class="source"> <workflow-app xmlns='uri:oozie:workflow:0.4' name='pig-wf'>
+ <credentials>
+ <credential name='my-hcat-creds' type='hcat'>
+ <property>
+ <name>hcat.metastore.uri</name>
+ <value>HCAT_URI</value>
+ </property>
+ <property>
+ <name>hcat.metastore.principal</name>
+ <value>HCAT_PRINCIPAL</value>
+ </property>
+ </credential>
+ </credentials>
+ ...
+ <action name='pig' cred='my-hcat-creds'>
+ <pig>
+ <job-tracker>JT</job-tracker>
+ <name-node>NN</name-node>
+ <configuration>
+ <property>
+ <name>TESTING</name>
+ <value>${start}</value>
+ </property>
+ </configuration>
+ </pig>
+ </action>
+ ...
+ </workflow-app>
+</pre></div></div>
+
+<p>The type of the <tt>credential</tt> is “hcat”, which is the type name we gave for the HCatCredentials class in oozie-site.xml. We gave the <tt>credential</tt> a name, “my-hcat-creds”, which can be whatever you want; we then specify cred=‘my-hcat-creds’ in the Pig action, so that Oozie will include these credentials with the action. You can include multiple credentials with an action by specifying a comma-separated list of <tt>credential</tt> names. And finally, the HCatCredentials required two properties (the metastore URI and principal), which we also specified.</p>
+<p>Adding the <tt>credentials</tt> section to a workflow and referencing it in an action will make Oozie always try to obtain that delegation token. Ordinarily, this would mean that you cannot re-use this workflow in a non-secure cluster without editing it because trying to obtain the delegation token will likely fail. However, you can tell Oozie to ignore the <tt>credentials</tt> for a workflow by setting the job-level property <tt>oozie.credentials.skip</tt> to <tt>true</tt>; this will allow you to use the same workflow.xml in a secure and non-secure cluster by simply changing the job-level property at runtime. If omitted or set to <tt>false</tt>, Oozie will handle the <tt>credentials</tt> section normally. In addition, you can also set this property at the action-level or server-level to skip getting credentials for just that action or for all workflows, respectively. The order of priority is this:</p>
+<ol style="list-style-type: decimal">
+
+<li><tt>oozie.credentials.skip</tt> in the <tt>configuration</tt> section of an action, if set</li>
+<li><tt>oozie.credentials.skip</tt> in the job.properties for a workflow, if set</li>
+<li><tt>oozie.credentials.skip</tt> in oozie-site.xml for all workflows, if set</li>
+<li>(don’t skip)</li>
+</ol></div>
+<div class="section">
+<h2><a name="Built-in_Credentials_Implementations"></a>Built-in Credentials Implementations</h2>
+<p>Oozie currently comes with the following Credentials implementations:</p>
+<ol style="list-style-type: decimal">
+
+<li>HCatalog and Hive Metastore: <tt>org.apache.oozie.action.hadoop.HCatCredentials</tt></li>
+<li>HBase: <tt>org.apache.oozie.action.hadoop.HBaseCredentials</tt></li>
+<li>Hive Server 2: <tt>org.apache.oozie.action.hadoop.Hive2Credentials</tt></li>
+</ol>
+<p>HCatCredentials requires these two properties:</p>
+<ol style="list-style-type: decimal">
+
+<li><tt>hcat.metastore.principal</tt> or hive.metastore.kerberos.principal</li>
+<li><tt>hcat.metastore.uri</tt> or hive.metastore.uris</li>
+</ol>
+<p><b>Note:</b> The HCatalog Metastore and Hive Metastore are one and the same and so the “hcat” type credential can also be used to talk to a secure Hive Metastore, though the property names would still start with “hcat.”.</p>
+<p>HBase does not require any additional properties since the hbase-site.xml on the Oozie server provides necessary information to obtain a delegation token; though properties can be overwritten here if desired.</p>
+<p>Hive2Credentials requires these two properties:</p>
+<ol style="list-style-type: decimal">
+
+<li><tt>hive2.server.principal</tt></li>
+<li><tt>hive2.jdbc.url</tt></li>
+</ol>
+<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright © 2021
+ <a href="https://www.apache.org">Apache Software Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>