You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ta...@apache.org on 2018/04/03 02:11:15 UTC
[9/9] impala git commit: IMPALA-6769: Avoid pulling incoherent Hadoop
dependencies.
IMPALA-6769: Avoid pulling incoherent Hadoop dependencies.
Through the following chain of dependencies, Impala was pulling in a
second copy of hadoop-client, mismatched with $IMPALA_HADOOP_VERSION.
To avoid this, we can blacklist the unnecessary dependency on
hive-llap-server, which we don't use.
To prevent this sort of thing from recurring, I used Maven's
"banned dependency plugin" to blacklist
org.apache.{hadoop,hive,kudu,sentry,hbase,parquet} and whitelist the
versions we expect. I confirmed that "mvn compile" would fail
if the new exclusion I added here wasn't present.
This is the relevant snippet of "mvn dependency:tree" showing how we
were pulling this in:
[INFO] +- org.apache.hive:hive-service:jar:2.1.1-cdh6.x-SNAPSHOT:compile
[INFO] | +- org.apache.hive:hive-service-rpc:jar:2.1.1-cdh6.x-SNAPSHOT:compile
[INFO] | +- org.apache.hive:hive-llap-server:jar:2.1.1-cdh6.x-SNAPSHOT:compile
[INFO] | | \- org.apache.slider:slider-core:jar:0.90.2-incubating:compile
[INFO] | | +- org.apache.hadoop:hadoop-client:jar:2.7.1:compile
Change-Id: Ie4b38c4fc5e30ec1464586282f08e3b82f30d7bd
Reviewed-on: http://gerrit.cloudera.org:8080/9860
Reviewed-by: Philip Zeyliger <ph...@cloudera.com>
Tested-by: Impala Public Jenkins
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/89b29d3a
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/89b29d3a
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/89b29d3a
Branch: refs/heads/master
Commit: 89b29d3a89cca041c541476e25435dc0d34344c1
Parents: a026f26
Author: Philip Zeyliger <ph...@cloudera.com>
Authored: Mon Mar 26 15:11:13 2018 -0700
Committer: Impala Public Jenkins <im...@gerrit.cloudera.org>
Committed: Tue Apr 3 02:09:09 2018 +0000
----------------------------------------------------------------------
fe/pom.xml | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/impala/blob/89b29d3a/fe/pom.xml
----------------------------------------------------------------------
diff --git a/fe/pom.xml b/fe/pom.xml
index 07592d2..67ea81c 100644
--- a/fe/pom.xml
+++ b/fe/pom.xml
@@ -194,6 +194,16 @@ under the License.
<groupId>org.apache.hive</groupId>
<artifactId>hive-service</artifactId>
<version>${hive.version}</version>
+ <!--
+ hive-service pulls in an incoherent (not ${hadoop.version}) version of
+ hadoop-client via this dependency, which we don't need.
+ -->
+ <exclusions>
+ <exclusion>
+ <groupId>org.apache.hive</groupId>
+ <artifactId>hive-llap-server</artifactId>
+ </exclusion>
+ </exclusions>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
@@ -802,7 +812,23 @@ under the License.
<bannedDependencies>
<excludes>
<exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+ <!-- Assert that we only use artifacts from only the specified
+ version of these components. -->
+ <exclude>org.apache.hadoop:*</exclude>
+ <exclude>org.apache.hbase:*</exclude>
+ <exclude>org.apache.hive:*</exclude>
+ <exclude>org.apache.kudu:*</exclude>
+ <exclude>org.apache.sentry:*</exclude>
+ <exclude>org.apache.parquet:*</exclude>
</excludes>
+ <includes>
+ <include>org.apache.hadoop:*:${hadoop.version}</include>
+ <include>org.apache.hbase:*:${hbase.version}</include>
+ <include>org.apache.hive:*:${hive.version}</include>
+ <include>org.apache.kudu:*:${kudu.version}</include>
+ <include>org.apache.sentry:*:${sentry.version}</include>
+ <include>org.apache.parquet:*:${parquet.version}</include>
+ </includes>
</bannedDependencies>
</rules>
<fail>true</fail>