You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by bl...@apache.org on 2019/07/08 18:59:09 UTC
[flink] branch master updated: [FLINK-13134][hive] override default
hadoop version from 2.4.1 to 2.7.5 in flink-connector-hive
This is an automated email from the ASF dual-hosted git repository.
bli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new bc6a6fe [FLINK-13134][hive] override default hadoop version from 2.4.1 to 2.7.5 in flink-connector-hive
bc6a6fe is described below
commit bc6a6fe3c68e00f6eaaa7b98da83a4c571f2a376
Author: bowen.li <bo...@gmail.com>
AuthorDate: Sun Jul 7 11:33:08 2019 -0700
[FLINK-13134][hive] override default hadoop version from 2.4.1 to 2.7.5 in flink-connector-hive
Hive 2.3.4 relies on Hadoop 2.7.2 or later version. The default hadoop version in Flink globally is 2.4.1 which misses some classes in 2.7.2 and thus doesn't meet the requirement.
For hadoop 2.7, the minor version supported in flink-shaded-hadoop-2-uber is 2.7.5.
This closes #9011.
---
flink-connectors/flink-connector-hive/pom.xml | 9 +++++++--
pom.xml | 6 +++++-
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/flink-connectors/flink-connector-hive/pom.xml b/flink-connectors/flink-connector-hive/pom.xml
index 7d76a63..3b549b6 100644
--- a/flink-connectors/flink-connector-hive/pom.xml
+++ b/flink-connectors/flink-connector-hive/pom.xml
@@ -63,10 +63,15 @@ under the License.
<!-- Hadoop dependency -->
<!-- Hadoop as provided dependencies, so we can depend on them without pulling in Hadoop -->
+ <!--
+ Hive 2.3.4 relies on Hadoop 2.7.2 and later versions.
+ For Hadoop 2.7, the minor Hadoop version supported for flink-shaded-hadoop-2-uber is 2.7.5,
+ thus override the default hadoop version from 2.4.1 to 2.7.5
+ -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-shaded-hadoop-2-uber</artifactId>
- <version>${hadoop.version}-${flink.shaded.version}</version>
+ <version>${hivemetastore.hadoop.version}-${flink.shaded.version}</version>
<scope>provided</scope>
</dependency>
@@ -434,7 +439,7 @@ under the License.
<id>hive-1.2.1</id>
<properties>
<hive.version>1.2.1</hive.version>
- <hivemetastore.hadoop.version>2.6.0</hivemetastore.hadoop.version>
+ <hivemetastore.hadoop.version>2.6.5</hivemetastore.hadoop.version>
</properties>
</profile>
</profiles>
diff --git a/pom.xml b/pom.xml
index b5047f1..1ab8c51 100644
--- a/pom.xml
+++ b/pom.xml
@@ -140,7 +140,11 @@ under the License.
<minikdc.version>2.7.2</minikdc.version>
<generated.docs.dir>./docs/_includes/generated</generated.docs.dir>
<hive.version>2.3.4</hive.version>
- <hivemetastore.hadoop.version>2.7.2</hivemetastore.hadoop.version>
+ <!--
+ Hive 2.3.4 relies on Hadoop 2.7.2 and later versions.
+ For Hadoop 2.7, the minor Hadoop version supported for flink-shaded-hadoop-2-uber is 2.7.5
+ -->
+ <hivemetastore.hadoop.version>2.7.5</hivemetastore.hadoop.version>
<jamicmp.referenceVersion>1.8.0</jamicmp.referenceVersion>
<japicmp.outputDir>tools/japicmp-output</japicmp.outputDir>
</properties>