You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gora.apache.org by bu...@apache.org on 2015/03/02 20:24:03 UTC
svn commit: r941978 - in /websites/staging/gora/trunk/content: ./
current/gora-shims.html
Author: buildbot
Date: Mon Mar 2 19:24:03 2015
New Revision: 941978
Log:
Staging update by buildbot for gora
Modified:
websites/staging/gora/trunk/content/ (props changed)
websites/staging/gora/trunk/content/current/gora-shims.html
Propchange: websites/staging/gora/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Mon Mar 2 19:24:03 2015
@@ -1 +1 @@
-1663395
+1663401
Modified: websites/staging/gora/trunk/content/current/gora-shims.html
==============================================================================
--- websites/staging/gora/trunk/content/current/gora-shims.html (original)
+++ websites/staging/gora/trunk/content/current/gora-shims.html Mon Mar 2 19:24:03 2015
@@ -185,25 +185,21 @@ into your software stack. </p>
<li><a href="#gora-shims-distribution">gora-shims-distribution</a><ul>
<li><a href="#description">Description</a></li>
<li><a href="#dependency-definition">Dependency Definition</a></li>
-<li><a href="#direct-dependency">Direct Dependency</a></li>
</ul>
</li>
<li><a href="#gora-shims-hadoop">gora-shims-hadoop</a><ul>
<li><a href="#description_1">Description</a></li>
-<li><a href="#dependency">Dependency</a></li>
-<li><a href="#direct-dependency_1">Direct Dependency</a></li>
+<li><a href="#dependency-definition_1">Dependency Definition</a></li>
</ul>
</li>
<li><a href="#gora-shims-hadoop1">gora-shims-hadoop1</a><ul>
<li><a href="#description_2">Description</a></li>
-<li><a href="#dependency_1">Dependency</a></li>
-<li><a href="#direct-dependency_2">Direct Dependency</a></li>
+<li><a href="#dependency-definition_2">Dependency Definition</a></li>
</ul>
</li>
<li><a href="#gora-shims-hadoop2">gora-shims-hadoop2</a><ul>
<li><a href="#description_3">Description</a></li>
-<li><a href="#dependency_2">Dependency</a></li>
-<li><a href="#direct-dependency_3">Direct Dependency</a></li>
+<li><a href="#dependency-definition_3">Dependency Definition</a></li>
</ul>
</li>
</ul>
@@ -216,19 +212,102 @@ on all of the other shims modules but co
should be used as below.</strong></p>
<h2 id="dependency-definition">Dependency Definition</h2>
<p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-distribution|0.6|bundle">Click Here</a></p>
-<h2 id="direct-dependency">Direct Dependency</h2>
<h1 id="gora-shims-hadoop">gora-shims-hadoop</h1>
<h2 id="description_1">Description</h2>
-<h2 id="dependency">Dependency</h2>
-<h2 id="direct-dependency_1">Direct Dependency</h2>
+<p>This module contains functional Java code for enabling dynamic selection of the Hadoop environment based upon the
+presence of <strong>org.apache.hadoop.util.VersionInfo</strong>. This is a two-part process where we first obtain the <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop/src/main/java/org/apache/gora/shims/hadoop/HadoopShimFactory.java#L79-L94">Hadoop
+Major version</a>.
+We then <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop/src/main/java/org/apache/gora/shims/hadoop/HadoopShimFactory.java#L56-L77">select the appropriate Gora Shim</a>
+based on detecting the Hadoop Major version. This code looks the following:</p>
+<div class="codehilite"><pre><span class="cm">/**</span>
+<span class="cm"> * Get the Hadoop major version number.</span>
+<span class="cm"> *</span>
+<span class="cm"> * @return The major version number of Hadoop.</span>
+<span class="cm"> */</span>
+<span class="n">public</span> <span class="n">String</span> <span class="n">getMajorVersion</span><span class="p">()</span> <span class="p">{</span>
+ <span class="n">String</span> <span class="n">vers</span> <span class="o">=</span> <span class="n">VersionInfo</span><span class="p">.</span><span class="n">getVersion</span><span class="p">();</span>
+ <span class="n">String</span><span class="p">[]</span> <span class="n">parts</span> <span class="o">=</span> <span class="n">vers</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="s">"</span><span class="se">\\</span><span class="s">."</span><span class="p">);</span>
+ <span class="k">if</span> <span class="p">(</span><span class="n">parts</span><span class="p">.</span><span class="n">length</span> <span class="o"><</span> <span class="mh">2</span><span class="p">)</span> <span class="p">{</span>
+ <span class="n">throw</span> <span class="k">new</span> <span class="n">RuntimeException</span><span class="p">(</span><span class="s">"Unable to parse Hadoop version: "</span>
+ <span class="o">+</span> <span class="n">vers</span> <span class="o">+</span> <span class="s">" (expected X.Y.* format)"</span><span class="p">);</span>
+ <span class="p">}</span>
+ <span class="k">return</span> <span class="n">parts</span><span class="p">[</span><span class="mh">0</span><span class="p">];</span>
+<span class="p">}</span>
+
+<span class="cm">/**</span>
+<span class="cm"> * Get the Hadoop shim for the Hadoop version on the class path. In case it</span>
+<span class="cm"> * fails to obtain an appropriate shim (i.e. unsupported Hadoop version), it</span>
+<span class="cm"> * throws a {@link RuntimeException}.</span>
+<span class="cm"> *</span>
+<span class="cm"> * Note that this method is potentially costly.</span>
+<span class="cm"> *</span>
+<span class="cm"> * @return A newly created instance of a {@link HadoopShim}.</span>
+<span class="cm"> */</span>
+<span class="n">public</span> <span class="n">HadoopShim</span> <span class="n">getHadoopShim</span><span class="p">()</span> <span class="p">{</span>
+ <span class="n">String</span> <span class="n">version</span> <span class="o">=</span> <span class="n">getMajorVersion</span><span class="p">();</span>
+ <span class="n">String</span> <span class="n">className</span> <span class="o">=</span> <span class="no">HADOOP_VERSION_TO_IMPL_MAP</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">version</span><span class="p">);</span>
+ <span class="n">try</span> <span class="p">{</span>
+ <span class="n">Class</span><span class="o"><?></span> <span class="n">class1</span> <span class="o">=</span> <span class="n">Class</span><span class="p">.</span><span class="n">forName</span><span class="p">(</span><span class="n">className</span><span class="p">);</span>
+ <span class="k">return</span> <span class="n">HadoopShim</span><span class="p">.</span><span class="k">class</span><span class="p">.</span><span class="n">cast</span><span class="p">(</span><span class="n">class1</span><span class="p">.</span><span class="n">newInstance</span><span class="p">());</span>
+ <span class="p">}</span> <span class="n">catch</span> <span class="p">(</span><span class="n">Exception</span> <span class="n">e</span><span class="p">)</span> <span class="p">{</span>
+ <span class="n">throw</span> <span class="k">new</span> <span class="n">RuntimeException</span><span class="p">(</span>
+ <span class="s">"Could not load Hadoop shim for version "</span> <span class="o">+</span> <span class="n">version</span>
+ <span class="o">+</span> <span class="s">", className="</span> <span class="o">+</span> <span class="n">className</span><span class="p">,</span> <span class="n">e</span><span class="p">);</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+
+
+<h2 id="dependency-definition_1">Dependency Definition</h2>
+<p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop|0.6|bundle">Click Here</a></p>
<h1 id="gora-shims-hadoop1">gora-shims-hadoop1</h1>
<h2 id="description_2">Description</h2>
-<h2 id="dependency_1">Dependency</h2>
-<h2 id="direct-dependency_2">Direct Dependency</h2>
+<p>This module provides all functionality for Hadoop 1.X support in Gora. The Hadoop version is specified within
+the project <a href="https://github.com/apache/gora/blob/master/pom.xml">pom.xml</a> within the <strong>${hadoop1-version}</strong> variable.</p>
+<p>The actual <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop1/src/main/java/org/apache/gora/shims/hadoop1/HadoopShim1.java">code implementation</a>
+is very simple. Essentially it consists of two Java methods</p>
+<div class="codehilite"><pre> <span class="o">/**</span>
+ <span class="o">*</span> <span class="p">{@</span><span class="n">inheritDoc</span><span class="p">}</span>
+ <span class="o">*/</span>
+ <span class="n">public</span> <span class="n">Job</span> <span class="n">createJob</span><span class="p">(</span><span class="n">Configuration</span> <span class="n">configuration</span><span class="p">)</span> <span class="n">throws</span> <span class="n">IOException</span> <span class="p">{</span>
+ <span class="k">return</span> <span class="n">new</span> <span class="n">Job</span><span class="p">(</span><span class="n">configuration</span><span class="p">);</span>
+ <span class="p">}</span>
+ <span class="o">/**</span>
+ <span class="o">*</span> <span class="p">{@</span><span class="n">inheritDoc</span><span class="p">}</span>
+ <span class="o">*/</span>
+ <span class="n">public</span> <span class="n">JobContext</span> <span class="n">createJobContext</span><span class="p">(</span><span class="n">Configuration</span> <span class="n">configuration</span><span class="p">)</span> <span class="p">{</span>
+ <span class="k">return</span> <span class="n">new</span> <span class="n">JobContext</span><span class="p">(</span><span class="n">configuration</span><span class="p">,</span> <span class="n">null</span><span class="p">);</span>
+ <span class="p">}</span>
+</pre></div>
+
+
+<h2 id="dependency-definition_2">Dependency Definition</h2>
+<p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop1|0.6|bundle">Click Here</a></p>
<h1 id="gora-shims-hadoop2">gora-shims-hadoop2</h1>
<h2 id="description_3">Description</h2>
-<h2 id="dependency_2">Dependency</h2>
-<h2 id="direct-dependency_3">Direct Dependency</h2>
+<p>This module provides all functionality for Hadoop 2.X support in Gora. The Hadoop version is specified within
+the project <a href="https://github.com/apache/gora/blob/master/pom.xml">pom.xml</a> within the <strong>${hadoop2-version}</strong> variable.</p>
+<p>The actual <a href="https://github.com/apache/gora/blob/master/gora-shims-hadoop1/src/main/java/org/apache/gora/shims/hadoop1/HadoopShim1.java">code implementation</a>
+is very simple. Essentially it consists of two Java methods
+ /<strong>
+ * {@inheritDoc}
+ <em>
+ * Use the Hadoop 2.x way of creating a {@link Job} object.
+ </em>/
+ public Job createJob(Configuration configuration) throws IOException {
+ Job instance = Job.getInstance(configuration);
+ return instance;
+ }
+ /</strong>
+ * {@inheritDoc}
+ <em>
+ * Use the Hadoop 2.x way of creating a {@link JobContext} object.
+ </em>/
+ public JobContext createJobContext(Configuration configuration) {
+ return new JobContextImpl(configuration, null);
+ }</p>
+<h2 id="dependency-definition_3">Dependency Definition</h2>
+<p><a href="http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop2|0.6|bundle">Click Here</a></p>
</div> <!-- /container (main block) -->