You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by tm...@apache.org on 2018/04/10 04:48:46 UTC

[4/6] impala git commit: IMPALA-6825: Fix HADOOP_CLASSPATH when using Hadoop 3

IMPALA-6825: Fix HADOOP_CLASSPATH when using Hadoop 3

IMPALA-4277 added support for Hadoop 3. However, it omitted a component
from HADOOP_CLASSPATH that is required for the local minicluster to
access AWS.

I tested this manually and with this change the minicluster is able to
access HDFS.

Cherry-picks: not for 2.x

Change-Id: Ibe5784d32c6e8d04f147b26c7d2345609006ab46
Reviewed-on: http://gerrit.cloudera.org:8080/9956
Reviewed-by: Philip Zeyliger <ph...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/f76b1bf1
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/f76b1bf1
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/f76b1bf1

Branch: refs/heads/master
Commit: f76b1bf130df95ec3c4fb512a1117ffa8a04dafb
Parents: 2b7311a
Author: Lars Volker <lv...@cloudera.com>
Authored: Mon Apr 9 11:23:37 2018 -0700
Committer: Impala Public Jenkins <im...@cloudera.com>
Committed: Tue Apr 10 00:39:47 2018 +0000

----------------------------------------------------------------------
 bin/impala-config.sh | 3 +++
 1 file changed, 3 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/impala/blob/f76b1bf1/bin/impala-config.sh
----------------------------------------------------------------------
diff --git a/bin/impala-config.sh b/bin/impala-config.sh
index f909887..0ae7713 100755
--- a/bin/impala-config.sh
+++ b/bin/impala-config.sh
@@ -512,6 +512,9 @@ if [[ $IMPALA_MINICLUSTER_PROFILE == 3 ]]; then
   # Beware of adding entries from $HADOOP_HOME here, because they can change
   # the order of the classpath, leading to configuration not showing up first.
   HADOOP_CLASSPATH="$LZO_JAR_PATH"
+  # Add the path containing the hadoop-aws jar, which is required to access AWS from the
+  # minicluster.
+  HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${HADOOP_HOME}/share/hadoop/tools/lib/*"
 fi
 
 export MINI_DFS_BASE_DATA_DIR="$IMPALA_HOME/cdh-${CDH_MAJOR_VERSION}-hdfs-data"