You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by mw...@apache.org on 2018/11/28 19:15:39 UTC
[accumulo-website] branch asf-site updated: Jekyll build from
master:2bab009
This is an automated email from the ASF dual-hosted git repository.
mwalch pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 8ae53ce Jekyll build from master:2bab009
8ae53ce is described below
commit 8ae53ce5bbf8347e2334f3b29d7a8b031ebc12e3
Author: Mike Walch <mw...@apache.org>
AuthorDate: Wed Nov 28 14:15:21 2018 -0500
Jekyll build from master:2bab009
Updated MapReduce docs to include new configuration (#135)
---
docs/2.x/administration/upgrading.html | 1 +
docs/2.x/development/mapreduce.html | 19 +++++++++++++++++++
feed.xml | 4 ++--
search_data.json | 4 ++--
4 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/docs/2.x/administration/upgrading.html b/docs/2.x/administration/upgrading.html
index b4c711f..fc30597 100644
--- a/docs/2.x/administration/upgrading.html
+++ b/docs/2.x/administration/upgrading.html
@@ -479,6 +479,7 @@ distributions of Hadoop.</li>
<li><code class="highlighter-rouge">log4j.properties</code> for Accumulo clients and commands</li>
</ul>
</li>
+ <li><a href="/docs/2.x/development/mapreduce#configuration">New Hadoop configuration is required</a> when reading or writing to Accumulo using MapReduce.</li>
<li>Run the command <code class="highlighter-rouge">accumulo shell</code> to access the shell using configuration in <code class="highlighter-rouge">conf/accumulo-client.properties</code></li>
</ul>
diff --git a/docs/2.x/development/mapreduce.html b/docs/2.x/development/mapreduce.html
index d3c9e0c..76a43f0 100644
--- a/docs/2.x/development/mapreduce.html
+++ b/docs/2.x/development/mapreduce.html
@@ -439,6 +439,25 @@ can be set via these two format classes to do the following:</p>
<li>Restrict the input to a subset of available columns</li>
</ul>
+<h2 id="configuration">Configuration</h2>
+
+<p>Since 2.0.0, Accumulo no longer has the same versions of dependencies (i.e Guava, etc) as Hadoop.
+When launching a MapReduce job that reads or writes to Accumulo, you should build a shaded jar
+with all of your dependencies and complete the following steps so YARN only includes Hadoop code
+(and not all of Hadoop dependencies) when running your MapReduce job:</p>
+
+<ol>
+ <li>
+ <p>Set <code class="highlighter-rouge">export HADOOP_USE_CLIENT_CLASSLOADER=true</code> in your environment before submitting
+your job with <code class="highlighter-rouge">yarn</code> command.</p>
+ </li>
+ <li>
+ <p>Set the following in your Job configuration.</p>
+ <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">job</span><span class="o">.</span><span class="na">getConfiguration</span><span class="o">().</span><span class="na">set</span><span class="o">(</span><span class="s">"mapreduce.job.classloader"</span><span class="o">,</span> <span class="s">"true"</span><span class="o">);</span>
+</code></pre></div> </div>
+ </li>
+</ol>
+
<h2 id="mapper-and-reducer-classes">Mapper and Reducer classes</h2>
<p>To read from an Accumulo table create a Mapper with the following class
diff --git a/feed.xml b/feed.xml
index e166a64..41524c2 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>https://accumulo.apache.org/</link>
<atom:link href="https://accumulo.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Tue, 20 Nov 2018 09:51:13 -0500</pubDate>
- <lastBuildDate>Tue, 20 Nov 2018 09:51:13 -0500</lastBuildDate>
+ <pubDate>Wed, 28 Nov 2018 14:15:11 -0500</pubDate>
+ <lastBuildDate>Wed, 28 Nov 2018 14:15:11 -0500</lastBuildDate>
<generator>Jekyll v3.7.3</generator>
diff --git a/search_data.json b/search_data.json
index 31e1a86..7b96bc9 100644
--- a/search_data.json
+++ b/search_data.json
@@ -51,7 +51,7 @@
"docs-2-x-administration-upgrading": {
"title": "Upgrading Accumulo",
- "content" : "Upgrading from 1.8/9 to 2.0Follow the steps below to upgrade your Accumulo instance and client to 2.0.Upgrade Accumulo instanceIMPORTANT! Before upgrading to Accumulo 2.0, you will need to upgrade to Java 8 and Hadoop 3.x.Upgrading to Accumulo 2.0 is done by stopping Accumulo 1.8/9 and starting Accumulo 2.0.Before stopping Accumulo 1.8/9, install Accumulo 2.0 and configure it by following the 2.0 quick start.There are several changes to scripts and configuration in 2. [...]
+ "content" : "Upgrading from 1.8/9 to 2.0Follow the steps below to upgrade your Accumulo instance and client to 2.0.Upgrade Accumulo instanceIMPORTANT! Before upgrading to Accumulo 2.0, you will need to upgrade to Java 8 and Hadoop 3.x.Upgrading to Accumulo 2.0 is done by stopping Accumulo 1.8/9 and starting Accumulo 2.0.Before stopping Accumulo 1.8/9, install Accumulo 2.0 and configure it by following the 2.0 quick start.There are several changes to scripts and configuration in 2. [...]
"url": " /docs/2.x/administration/upgrading",
"categories": "administration"
},
@@ -107,7 +107,7 @@
"docs-2-x-development-mapreduce": {
"title": "MapReduce",
- "content" : "Accumulo tables can be used as the source and destination of MapReduce jobs. Touse an Accumulo table with a MapReduce job, configure the job parameters to usethe AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameterscan be set via these two format classes to do the following: Authenticate and provide user credentials for the input Restrict the scan to a range of rows Restrict the input to a subset of available columnsMapper and Reducer classesTo [...]
+ "content" : "Accumulo tables can be used as the source and destination of MapReduce jobs. Touse an Accumulo table with a MapReduce job, configure the job parameters to usethe AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameterscan be set via these two format classes to do the following: Authenticate and provide user credentials for the input Restrict the scan to a range of rows Restrict the input to a subset of available columnsConfigurationSince 2.0.0, Ac [...]
"url": " /docs/2.x/development/mapreduce",
"categories": "development"
},