You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oozie.apache.org by bu...@apache.org on 2019/12/06 09:10:50 UTC
svn commit: r1053698 [12/17] - in /websites/staging/oozie/trunk/content: ./
docs/5.2.0/ docs/5.2.0/css/ docs/5.2.0/fonts/ docs/5.2.0/images/
docs/5.2.0/images/logos/ docs/5.2.0/images/profiles/ docs/5.2.0/img/
docs/5.2.0/js/
Added: websites/staging/oozie/trunk/content/docs/5.2.0/ENG_Building.html
==============================================================================
--- websites/staging/oozie/trunk/content/docs/5.2.0/ENG_Building.html (added)
+++ websites/staging/oozie/trunk/content/docs/5.2.0/ENG_Building.html Fri Dec 6 09:10:49 2019
@@ -0,0 +1,326 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2019-12-05
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20191205" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Oozie – </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="https://oozie.apache.org/" id="bannerLeft">
+ <img src="https://oozie.apache.org/images/oozie_200x.png" alt="Oozie"/>
+ </a>
+ </div>
+ <div class="pull-right"> </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org/" class="externalLink" title="Apache">
+ Apache</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../../" title="Oozie">
+ Oozie</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../" title="docs">
+ docs</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="./" title="5.2.0">
+ 5.2.0</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active "></li>
+
+
+
+ <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2019-12-05</li>
+ <li id="projectVersion" class="pull-right">
+ Version: 5.2.0
+ </li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p>
+<h1>Building Oozie</h1>
+<ul>
+<li><a href="#System_Requirements">System Requirements</a></li>
+<li><a href="#Oozie_Documentation_Generation">Oozie Documentation Generation</a></li>
+<li><a href="#Passphrase-less_SSH_Setup">Passphrase-less SSH Setup</a></li>
+<li><a href="#Building_with_different_Java_Versions">Building with different Java Versions</a></li>
+<li><a href="#Building_and_Testing_Oozie">Building and Testing Oozie</a>
+<ul>
+<li><a href="#Examples_Running_Oozie_Testcases_with_Different_Configurations">Examples Running Oozie Testcases with Different Configurations</a></li>
+<li><a href="#Build_Options_Reference">Build Options Reference</a></li>
+<li><a href="#Testing_Map_Reduce_Pipes_Action">Testing Map Reduce Pipes Action</a></li>
+<li><a href="#Testing_using_dist_test_and_grind">Testing using dist_test and grind</a></li></ul></li>
+<li><a href="#Building_an_Oozie_Distribution">Building an Oozie Distribution</a></li>
+<li><a href="#IDE_Setup">IDE Setup</a></li></ul>
+
+<div class="section">
+<h2><a name="System_Requirements"></a>System Requirements</h2>
+<ul>
+
+<li>Unix box (tested on Mac OS X and Linux)</li>
+<li>Java JDK 1.8+</li>
+<li><a class="externalLink" href="http://maven.apache.org/">Maven 3.0.1+</a></li>
+<li><a class="externalLink" href="http://hadoop.apache.org/core/releases.html">Hadoop 2.6.0+</a></li>
+<li><a class="externalLink" href="http://hadoop.apache.org/pig/releases.html">Pig 0.10.1+</a></li>
+</ul>
+<p>JDK commands (java, javac) must be in the command path.</p>
+<p>The Maven command (mvn) must be in the command path.</p></div>
+<div class="section">
+<h2><a name="Oozie_Documentation_Generation"></a>Oozie Documentation Generation</h2>
+<p>To generate the documentation, Oozie uses a Doxia plugin for Maven with markdown support.</p>
+
+<div>
+<div>
+<pre class="source">$ mvn install
+</pre></div></div>
+
+<p><a name="SshSetup"></a></p></div>
+<div class="section">
+<h2><a name="Passphrase-less_SSH_Setup"></a>Passphrase-less SSH Setup</h2>
+<p><b>NOTE: SSH actions are deprecated in Oozie 2.</b></p>
+<p>To run SSH Testcases and for easier Hadoop start/stop configure SSH to localhost to be passphrase-less.</p>
+<p>Create your SSH keys without a passphrase and add the public key to the authorized file:</p>
+
+<div>
+<div>
+<pre class="source">$ ssh-keygen -t dsa
+$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys2
+</pre></div></div>
+
+<p>Test that you can ssh without password:</p>
+
+<div>
+<div>
+<pre class="source">$ ssh localhost
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Building_with_different_Java_Versions"></a>Building with different Java Versions</h2>
+<p>Oozie requires a minimum Java version of 1.8. Any newer version can be used but by default bytecode will be generated which is compatible with 1.8. This can be changed by specifying the build property <b>targetJavaVersion</b>.</p></div>
+<div class="section">
+<h2><a name="Building_and_Testing_Oozie"></a>Building and Testing Oozie</h2>
+<p>The JARs for the specified Hadoop and Pig versions must be available in one of the Maven repositories defined in Oozie main ‘pom.xml’ file. Or they must be installed in the local Maven cache.</p>
+<div class="section">
+<h3><a name="Examples_Running_Oozie_Testcases_with_Different_Configurations"></a>Examples Running Oozie Testcases with Different Configurations</h3>
+<p><b>Using embedded Hadoop minicluster with ‘simple’ authentication:</b></p>
+
+<div>
+<div>
+<pre class="source">$ mvn clean test
+</pre></div></div>
+
+<p><b>Using a Hadoop cluster with ‘simple’ authentication:</b></p>
+
+<div>
+<div>
+<pre class="source">$ mvn clean test -Doozie.test.hadoop.minicluster=false
+</pre></div></div>
+
+<p><b>Using embedded Hadoop minicluster with ‘simple’ authentication and Derby database:</b></p>
+
+<div>
+<div>
+<pre class="source">$ mvn clean test -Doozie.test.hadoop.minicluster=false -Doozie.test.db=derby
+</pre></div></div>
+
+<p><b>Using a Hadoop cluster with ‘kerberos’ authentication:</b></p>
+
+<div>
+<div>
+<pre class="source">$ mvn clean test -Doozie.test.hadoop.minicluster=false -Doozie.test.hadoop.security=kerberos
+</pre></div></div>
+
+<p>NOTE: The embedded minicluster cannot be used when testing with ‘kerberos’ authentication.</p>
+<p><b>Using a custom Oozie configuration for testcases:</b></p>
+
+<div>
+<div>
+<pre class="source">$ mvn clean test -Doozie.test.config.file=/home/tucu/custom-oozie-sitel.xml
+</pre></div></div>
+
+<p><b>Running the testcases with different databases:</b></p>
+
+<div>
+<div>
+<pre class="source">$ mvn clean test -Doozie.test.db=[hsqldb*|derby|mysql|postgres|oracle]
+</pre></div></div>
+
+<p>Using <tt>mysql</tt> and <tt>oracle</tt> enables profiles that will include their JARs files in the build. If using <tt>oracle</tt>, the Oracle JDBC JAR file must be manually installed in the local Maven cache (the JAR is not available in public Maven repos).</p></div>
+<div class="section">
+<h3><a name="Build_Options_Reference"></a>Build Options Reference</h3>
+<p>All these options can be set using <b>-D</b>.</p>
+<p>Except for the options marked with <tt>(*)</tt>, the options can be specified in the <tt>test.properties</tt> in the root of the Oozie project. The options marked with <tt>(*)</tt> are used in Maven POMs, thus they don’t take effect if specified in the <tt>test.properties</tt> file (which is loaded by the <tt>XTestCase</tt> class at class initialization time).</p>
+<p><b>hadoop.version</b> <tt>(*)</tt>: indicates the Hadoop version you wish to build Oozie against specifically. It will substitute this value in the Oozie POM properties and pull the corresponding Hadoop artifacts from Maven. The default version is 2.6.0 and that is the minimum supported Hadoop version.</p>
+<p><b>generateSite</b> (*): generates Oozie documentation, default is undefined (no documentation is generated)</p>
+<p><b>skipTests</b> (*): skips the execution of all testcases, no value required, default is undefined</p>
+<p><b>test</b>= (*): runs a single test case, to run a test give the test class name without package and extension, no default</p>
+<p><b>oozie.test.db</b>= (*): indicates the database to use for running the testcases, supported values are ‘hsqldb’, ‘derby’, ‘mysql’, ‘postgres’ and ‘oracle’; default value is ‘hsqldb’. For each database there is <tt>core/src/test/resources/DATABASE-oozie-site.xml</tt> file preconfigured.</p>
+<p><b>oozie.test.properties</b> (*): indicates the file to load the test properties from, by default is <tt>test.properties</tt>. Having this option allows having different test properties sets, for example: minicluster, simple & kerberos.</p>
+<p><b>oozie.test.waitfor.ratio</b>= : multiplication factor for testcases using waitfor, the ratio is used to adjust the effective time out. For slow machines the ratio should be increased. The default value is <tt>1</tt>.</p>
+<p><b>oozie.test.config.file</b>= : indicates a custom Oozie configuration file for running the testcases. The specified file must be an absolute path. For example, it can be useful to specify different database than HSQL for running the testcases.</p>
+<p><b>oozie.test.hadoop.minicluster</b>= : indicates if Hadoop minicluster should be started for testcases, default value ‘true’</p>
+<p><b>oozie.test.job.tracker</b>= : indicates the URI of the JobTracker when using a Hadoop cluster for testing, default value ‘localhost:8021’</p>
+<p><b>oozie.test.name.node</b>= : indicates the URI of the NameNode when using a Hadoop cluster for testing, default value ‘<tt>hdfs://localhost:8020</tt>’</p>
+<p><b>oozie.test.hadoop.security</b>= : indicates the type of Hadoop authentication for testing, valid values are ‘simple’ or ’kerberos, default value ‘simple’</p>
+<p><b>oozie.test.kerberos.keytab.file</b>= : indicates the location of the keytab file, default value ‘${user.home}/oozie.keytab’</p>
+<p><b>oozie.test.kerberos.realm</b>= : indicates the Kerberos real, default value ‘LOCALHOST’</p>
+<p><b>oozie.test.kerberos.oozie.principal</b>= : indicates the Kerberos principal for oozie, default value ‘${user.name}/localhost’</p>
+<p><b>oozie.test.kerberos.jobtracker.principal</b>= : indicates the Kerberos principal for the JobTracker, default value ‘mapred/localhost’</p>
+<p><b>oozie.test.kerberos.namenode.principal</b>= : indicates the Kerberos principal for the NameNode, default value ‘hdfs/localhost’</p>
+<p><b>oozie.test.user.oozie</b>= : specifies the user ID used to start Oozie server in testcases, default value is <tt>${user.name}</tt>.</p>
+<p><b>oozie.test.user.test</b>= : specifies primary user ID used as the user submitting jobs to Oozie Server in testcases, default value is <tt>test</tt>.</p>
+<p><b>oozie.test.user.test2</b>= : specifies secondary user ID used as the user submitting jobs to Oozie Server in testcases, default value is <tt>test2</tt>.</p>
+<p><b>oozie.test.user.test3</b>= : specifies secondary user ID used as the user submitting jobs to Oozie Server in testcases, default value is <tt>test3</tt>.</p>
+<p><b>oozie.test.group</b>= : specifies group ID used as group when submitting jobs to Oozie Server in testcases, default value is <tt>testg</tt>.</p>
+<p>NOTE: The users/group specified in <b>oozie.test.user.test2</b>, <b>oozie.test.user.test3</b> and <b>oozie.test.user.group</b> are used for the authorization testcases only.</p>
+<p><b>oozie.test.dir</b><tt>: specifies the directory where the</tt>oozietests<tt>directory will be created, default value is</tt>/tmp=. The <tt>oozietests</tt> directory is used by testcases when they need a local filesystem directory.</p>
+<p><b>hadoop.log.dir</b>= : specifies the directory where Hadoop minicluster will write its logs during testcases, default value is <tt>/tmp</tt>.</p>
+<p><b>test.exclude</b><tt>: specifies a testcase class (just the class name) to exclude for the tests run, for example =TestSubmitCommand</tt>.</p>
+<p><b>test.exclude.pattern</b><tt>: specifies one or more patterns for testcases to exclude, for example =**/Test*Command.java</tt>.</p></div>
+<div class="section">
+<h3><a name="Testing_Map_Reduce_Pipes_Action"></a>Testing Map Reduce Pipes Action</h3>
+<p>Pipes testcases require Hadoop’s <b>wordcount-simple</b> pipes binary example to run. The <b>wordcount-simple</b> pipes binary should be compiled for the build platform and copied into Oozie’s <b>core/src/test/resources/</b> directory. The binary file must be named <b>wordcount-simple</b>.</p>
+<p>If the <b>wordcount-simple</b> pipes binary file is not available the testcase will do a NOP and it will print to its output file the following message ‘SKIPPING TEST: TestPipesMain, binary ‘wordcount-simple’ not available in the classpath’.</p>
+<p>There are 2 testcases that use the <b>wordcount-simple</b> pipes binary, <b>TestPipesMain</b> and <b>TestMapReduceActionExecutor</b>, the ’SKIPPING TEST…" message would appear in the testcase log file of both testcases.</p></div>
+<div class="section">
+<h3><a name="Testing_using_dist_test_and_grind"></a>Testing using dist_test and grind</h3>
+<p>Testing using <a class="externalLink" href="https://github.com/cloudera/dist_test">dist_test</a> framework with <a class="externalLink" href="https://github.com/cloudera/dist_test/blob/master/docs/grind.md">grind</a> front end might not work using the default 3.0.2 version of the maven dependency plugin. It is necessary to downgrade to version 2.10 using <tt>-Dmaven-dependency-plugin.version=2.10</tt> .</p>
+<p>Maven flags for grind can be specified using <tt>GRIND_MAVEN_FLAGS</tt> environment variable:</p>
+
+<div>
+<div>
+<pre class="source">export GRIND_MAVEN_FLAGS=-Dmaven.dependency.plugin.version=2.10
+grind test --java-version 8
+</pre></div></div>
+</div></div>
+<div class="section">
+<h2><a name="Building_an_Oozie_Distribution"></a>Building an Oozie Distribution</h2>
+<p>An Oozie distribution bundles an embedded Jetty server.</p>
+<p>The simplest way to build Oozie is to run the <tt>mkdistro.sh</tt> script:</p>
+
+<div>
+<div>
+<pre class="source">$ bin/mkdistro.sh [-DskipTests]
+Running =mkdistro.sh= will create the binary distribution of Oozie. The following options are available to customise
+the versions of the dependencies:
+-Puber - Bundle required hadoop and hcatalog libraries in oozie war
+-Dhadoop.version=<version> - default 2.6.0
+-Ptez - Bundle tez jars in hive and pig sharelibs. Useful if you want to use tez
+as the execution engine for those applications.
+-Dpig.version=<version> - default 0.16.0
+-Dpig.classifier=<classifier> - default h2
+-Dsqoop.version=<version> - default 1.4.7
+-Dsqoop.classifier=<classifier> - default hadoop260
+-jetty.version=<version> - default 9.3.20.v20170531
+-Dopenjpa.version=<version> - default 2.2.2
+-Dxerces.version=<version> - default 2.10.0
+-Dcurator.version=<version> - default 2.5.0
+-Dhive.version=<version> - default 1.2.2
+-Dhbase.version=<version> - default 1.2.3
+-Dtez.version=<version> - default 0.8.4
+</pre></div></div>
+
+<p><b>IMPORTANT:</b> Profile hadoop-3 must be activated if building against Hadoop 3</p>
+<p>The following properties should be specified when building a release:</p>
+<ul>
+
+<li>-DgenerateDocs : forces the generation of Oozie documentation</li>
+<li>-Dbuild.time= : timestamps the distribution</li>
+<li>-Dvc.revision= : specifies the source control revision number of the distribution</li>
+<li>-Dvc.url= : specifies the source control URL of the distribution</li>
+</ul>
+<p>The provided <tt>bin/mkdistro.sh</tt> script runs the above Maven invocation setting all these properties to the right values (the ‘vc.*’ properties are obtained from the local git repository).</p></div>
+<div class="section">
+<h2><a name="IDE_Setup"></a>IDE Setup</h2>
+<p>Eclipse and IntelliJ can use directly Oozie Maven project files.</p>
+<p>The only special consideration is that the following source directories from the <tt>client</tt> module must be added to the <tt>core</tt> module source path:</p>
+<ul>
+
+<li><tt>client/src/main/java</tt> : as source directory</li>
+<li><tt>client/src/main/resources</tt> : as source directory</li>
+<li><tt>client/src/test/java</tt> : as test-source directory</li>
+<li><tt>client/src/test/resources</tt> : as test-source directory</li>
+</ul>
+<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright © 2019
+ <a href="https://www.apache.org">Apache Software Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added: websites/staging/oozie/trunk/content/docs/5.2.0/ENG_Custom_Authentication.html
==============================================================================
--- websites/staging/oozie/trunk/content/docs/5.2.0/ENG_Custom_Authentication.html (added)
+++ websites/staging/oozie/trunk/content/docs/5.2.0/ENG_Custom_Authentication.html Fri Dec 6 09:10:49 2019
@@ -0,0 +1,197 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2019-12-05
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20191205" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Oozie – </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="https://oozie.apache.org/" id="bannerLeft">
+ <img src="https://oozie.apache.org/images/oozie_200x.png" alt="Oozie"/>
+ </a>
+ </div>
+ <div class="pull-right"> </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org/" class="externalLink" title="Apache">
+ Apache</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../../" title="Oozie">
+ Oozie</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../" title="docs">
+ docs</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="./" title="5.2.0">
+ 5.2.0</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active "></li>
+
+
+
+ <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2019-12-05</li>
+ <li id="projectVersion" class="pull-right">
+ Version: 5.2.0
+ </li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p>
+<h1>Creating Custom Authentication</h1>
+<ul>
+<li><a href="#Hadoop-Auth_Authentication_Interfaces_and_classes">Hadoop-Auth Authentication Interfaces and classes</a></li>
+<li><a href="#Provide_Custom_Authentication_to_Oozie_Client">Provide Custom Authentication to Oozie Client</a></li>
+<li><a href="#Provide_Custom_Authentication_to_Oozie_Server">Provide Custom Authentication to Oozie Server</a></li></ul>
+
+<div class="section">
+<h2><a name="Hadoop-Auth_Authentication_Interfaces_and_classes"></a>Hadoop-Auth Authentication Interfaces and classes</h2>
+<ol style="list-style-type: decimal">
+
+<li>
+
+<p><tt>org.apache.hadoop.security.authentication.client.Authenticator:</tt> Interface for client authentication mechanisms.</p>
+<p>The following authenticators are provided in hadoop-auth:</p>
+<ul>
+
+<li>KerberosAuthenticator : the authenticator implements the Kerberos SPNEGO authentication sequence.</li>
+<li>PseudoAuthenticator : the authenticator implementation provides an authentication equivalent to Hadoop’s Simple authentication, it trusts the value of the ‘user.name’ Java System property.</li>
+</ul>
+</li>
+<li>
+
+<p><tt>org.apache.hadoop.security.authentication.server.AuthenticationHandler:</tt> Interface for server authentication mechanisms.</p>
+<ul>
+
+<li>KerberosAuthenticationHandler : the authenticator handler implements the Kerberos SPNEGO authentication mechanism for HTTP.</li>
+<li>PseudoAuthenticationHandler : the authenticator handler provides a pseudo authentication mechanism that accepts the user name specified as a query string parameter.</li>
+</ul>
+</li>
+<li>
+
+<p><tt>org.apache.hadoop.security.authentication.server.AuthenticationFilter:</tt> A servlet filter enables protecting web application resources with different authentication mechanisms provided by AuthenticationHandler. To enable the filter, web application resources file (ex. web.xml) needs to include a filter class derived from <tt>AuthenticationFilter</tt>.</p>
+<p>For more information have a look at the appropriate <a class="externalLink" href="https://hadoop.apache.org/docs/r2.7.2/hadoop-auth/index.html">Hadoop documentation</a>.</p>
+</li>
+</ol></div>
+<div class="section">
+<h2><a name="Provide_Custom_Authentication_to_Oozie_Client"></a>Provide Custom Authentication to Oozie Client</h2>
+<p>Apache Oozie contains a default class <tt>org.apache.oozie.client.AuthOozieClient</tt> to support Kerberos HTTP SPNEGO authentication, pseudo/simple authentication and anonymous access for client connections.</p>
+<p>To provide other authentication mechanisms, an Oozie client should extend from <tt>AuthOozieClient</tt> and provide the following methods should be overridden by derived classes to provide custom authentication:</p>
+<ul>
+
+<li>getAuthenticator() : return corresponding Authenticator based on value specified by user at <tt>auth</tt> command option.</li>
+<li>createConnection() : create a singleton class at Authenticator to allow client set and get key-value configuration for authentication.</li>
+</ul></div>
+<div class="section">
+<h2><a name="Provide_Custom_Authentication_to_Oozie_Server"></a>Provide Custom Authentication to Oozie Server</h2>
+<p>To accept custom authentication in Oozie server, a filter extends from AuthenticationFilter must be provided. This filter delegates to the configured authentication handler for authentication and once it obtains an <tt>AuthenticationToken</tt> from it, sets a signed HTTP cookie with the token. If HTTP cookie is provided with different key name, its cookie value can be retrieved by overriding <tt>getToken()</tt> method. Please note, only when <tt>getToken()</tt> return NULL, a custom authentication can be invoked and processed in <tt>AuthenticationFilter.doFilter()</tt>.</p>
+<p>The following method explains how to read it and return NULL token.</p>
+
+<div>
+<div>
+<pre class="source">protected AuthenticationToken getToken(HttpServletRequest request) throws IOException, AuthenticationException {
+ String tokenStr = null;
+ Cookie[] cookies = request.getCookies();
+
+ if (cookies != null) {
+ for (Cookie cookie : cookies) {
+ if (cookie.getName().equals(AuthenticatedURL.AUTH_COOKIE)) {
+ tokenStr = cookie.getValue();
+ LOG.info("Got 'hadoop.auth' cookie from request = " + tokenStr);
+ if (tokenStr != null && !tokenStr.trim().isEmpty()) {
+ AuthenticationToken retToken = super.getToken(request);
+ return retToken;
+ }
+ } else if (cookie.getName().equals("NEWAUTH")) {
+ tokenStr = cookie.getValue();
+ // DO NOT return the token string so request can authenticated.
+ }
+ }
+ }
+ return null;
+ }
+</pre></div></div>
+
+<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright © 2019
+ <a href="https://www.apache.org">Apache Software Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added: websites/staging/oozie/trunk/content/docs/5.2.0/ENG_MiniOozie.html
==============================================================================
--- websites/staging/oozie/trunk/content/docs/5.2.0/ENG_MiniOozie.html (added)
+++ websites/staging/oozie/trunk/content/docs/5.2.0/ENG_MiniOozie.html Fri Dec 6 09:10:49 2019
@@ -0,0 +1,195 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2019-12-05
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20191205" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Oozie – </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script>
+
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="https://oozie.apache.org/" id="bannerLeft">
+ <img src="https://oozie.apache.org/images/oozie_200x.png" alt="Oozie"/>
+ </a>
+ </div>
+ <div class="pull-right"> </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org/" class="externalLink" title="Apache">
+ Apache</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../../" title="Oozie">
+ Oozie</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="../" title="docs">
+ docs</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="">
+ <a href="./" title="5.2.0">
+ 5.2.0</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active "></li>
+
+
+
+ <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2019-12-05</li>
+ <li id="projectVersion" class="pull-right">
+ Version: 5.2.0
+ </li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p>
+<h1>Running MiniOozie Tests</h1>
+<ul>
+<li><a href="#System_Requirements">System Requirements</a></li>
+<li><a href="#Installing_Oozie_Jars_To_Maven_Cache">Installing Oozie Jars To Maven Cache</a></li>
+<li><a href="#Create_Tests_Using_MiniOozie">Create Tests Using MiniOozie</a></li>
+<li><a href="#IDE_Setup">IDE Setup</a></li></ul>
+
+<div class="section">
+<h2><a name="System_Requirements"></a>System Requirements</h2>
+<ul>
+
+<li>Unix box (tested on Mac OS X and Linux)</li>
+<li>Java JDK 1.8+</li>
+<li>Eclipse (tested on 3.5 and 3.6)</li>
+<li><a class="externalLink" href="http://maven.apache.org/">Maven 3.0.1+</a></li>
+</ul>
+<p>The Maven command (mvn) must be in the command path.</p></div>
+<div class="section">
+<h2><a name="Installing_Oozie_Jars_To_Maven_Cache"></a>Installing Oozie Jars To Maven Cache</h2>
+<p>Oozie source tree is at Apache GIT. The code is also mirrored at GitHub. MiniOozie sample project is under Oozie source tree.</p>
+<p>The following command downloads Oozie trunk to local:</p>
+
+<div>
+<div>
+<pre class="source">$ git clone git://git.apache.org/oozie.git
+</pre></div></div>
+
+<p>OR</p>
+
+<div>
+<div>
+<pre class="source">$ git clone git://github.com/apache/oozie.git
+</pre></div></div>
+
+<p>To run MiniOozie tests, the required jars like oozie-core, oozie-client, oozie-core-tests need to be available in remote maven repositories or local maven repository. The local maven cache for the above jars can be created and installed using the command:</p>
+
+<div>
+<div>
+<pre class="source">$ mvn clean install -DskipTests -DtestJarSimple
+</pre></div></div>
+
+<p>The following properties should be specified to install correct jars for MiniOozie:</p>
+<ul>
+
+<li>-DskipTests : ignore executing Oozie unittests</li>
+<li>-DtestJarSimple= : build only required test classes to oozie-core-tests</li>
+</ul>
+<p>MiniOozie is a folder named ‘minitest’ under Oozie source tree. Two sample tests are included in the project. The following command to execute tests under MiniOozie:</p>
+
+<div>
+<div>
+<pre class="source">$ cd minitest
+$ mvn clean test
+</pre></div></div>
+</div>
+<div class="section">
+<h2><a name="Create_Tests_Using_MiniOozie"></a>Create Tests Using MiniOozie</h2>
+<p>MiniOozie is a JUnit test class to test Oozie applications such as workflow and coordinator. The test case needs to extend from MiniOozieTestCase and does the same as the example class ‘WorkflowTest.java’ to create Oozie workflow application properties and workflow XML. The example file is under Oozie source tree:</p>
+<ul>
+
+<li><tt>minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java</tt></li>
+</ul></div>
+<div class="section">
+<h2><a name="IDE_Setup"></a>IDE Setup</h2>
+<p>Eclipse and IntelliJ can use directly MiniOozie Maven project files. MiniOozie project can be imported to Eclipse and IntelliJ as independent project.</p>
+<p>The test directories under MiniOozie are:</p>
+<ul>
+
+<li><tt>minitest/src/test/java</tt> : as test-source directory</li>
+<li><tt>minitest/src/test/resources</tt> : as test-resource directory</li>
+</ul>
+<p>Also asynchronous actions like FS action can be used / tested using <tt>LocalOozie</tt> / <tt>OozieClient</tt> API. Please see <tt>fs-decision.xml</tt> workflow example.</p>
+<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <p >Copyright © 2019
+ <a href="https://www.apache.org">Apache Software Foundation</a>.
+ All rights reserved.
+
+ </p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>