You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2019/03/07 22:46:28 UTC
[spark] branch master updated: [MINOR][BUILD] Add 2 maven
properties(hive.classifier and hive.parquet.group)
This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d70b6a3 [MINOR][BUILD] Add 2 maven properties(hive.classifier and hive.parquet.group)
d70b6a3 is described below
commit d70b6a39e1d22aa0ecda26efbcf25c772ba591a5
Author: Yuming Wang <yu...@ebay.com>
AuthorDate: Thu Mar 7 16:46:07 2019 -0600
[MINOR][BUILD] Add 2 maven properties(hive.classifier and hive.parquet.group)
## What changes were proposed in this pull request?
This pr adds 2 maven properties to help us upgrade the built-in Hive.
| Property Name | Default | In future |
| ------ | ------ | ------ |
| hive.classifier | (none) | core |
| hive.parquet.group | com.twitter | org.apache.parquet |
## How was this patch tested?
existing tests
Closes #23996 from wangyum/add_2_maven_properties.
Authored-by: Yuming Wang <yu...@ebay.com>
Signed-off-by: Sean Owen <se...@databricks.com>
---
examples/pom.xml | 2 +-
pom.xml | 8 ++++----
resource-managers/mesos/pom.xml | 1 +
resource-managers/yarn/pom.xml | 1 +
sql/hive/pom.xml | 6 ++----
5 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/examples/pom.xml b/examples/pom.xml
index 0636406..ac148ef 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -110,7 +110,7 @@
<version>3.7.0</version>
</dependency>
<dependency>
- <groupId>com.twitter</groupId>
+ <groupId>${hive.parquet.group}</groupId>
<artifactId>parquet-hadoop-bundle</artifactId>
<scope>provided</scope>
</dependency>
diff --git a/pom.xml b/pom.xml
index ec870d3..0e1913a 100644
--- a/pom.xml
+++ b/pom.xml
@@ -125,6 +125,7 @@
<zookeeper.version>3.4.6</zookeeper.version>
<curator.version>2.7.1</curator.version>
<hive.group>org.spark-project.hive</hive.group>
+ <hive.classifier></hive.classifier>
<!-- Version used in Maven Hive dependency -->
<hive.version>1.2.1.spark2</hive.version>
<!-- Version used for internal directory structure -->
@@ -135,6 +136,7 @@
<parquet.version>1.10.1</parquet.version>
<orc.version>1.5.4</orc.version>
<orc.classifier>nohive</orc.classifier>
+ <hive.parquet.group>com.twitter</hive.parquet.group>
<hive.parquet.version>1.6.0</hive.parquet.version>
<jetty.version>9.4.12.v20180830</jetty.version>
<javaxservlet.version>3.1.0</javaxservlet.version>
@@ -1415,9 +1417,7 @@
<dependency>
<groupId>${hive.group}</groupId>
<artifactId>hive-exec</artifactId>
-<!--
- <classifier>core</classifier>
--->
+ <classifier>${hive.classifier}</classifier>
<version>${hive.version}</version>
<scope>${hive.deps.scope}</scope>
<exclusions>
@@ -1836,7 +1836,7 @@
<scope>${parquet.test.deps.scope}</scope>
</dependency>
<dependency>
- <groupId>com.twitter</groupId>
+ <groupId>${hive.parquet.group}</groupId>
<artifactId>parquet-hadoop-bundle</artifactId>
<version>${hive.parquet.version}</version>
<scope>compile</scope>
diff --git a/resource-managers/mesos/pom.xml b/resource-managers/mesos/pom.xml
index 7b3aad4..107ba36 100644
--- a/resource-managers/mesos/pom.xml
+++ b/resource-managers/mesos/pom.xml
@@ -77,6 +77,7 @@
<dependency>
<groupId>${hive.group}</groupId>
<artifactId>hive-exec</artifactId>
+ <classifier>${hive.classifier}</classifier>
<scope>provided</scope>
</dependency>
<dependency>
diff --git a/resource-managers/yarn/pom.xml b/resource-managers/yarn/pom.xml
index d18df99..df910c1 100644
--- a/resource-managers/yarn/pom.xml
+++ b/resource-managers/yarn/pom.xml
@@ -172,6 +172,7 @@
<dependency>
<groupId>${hive.group}</groupId>
<artifactId>hive-exec</artifactId>
+ <classifier>${hive.classifier}</classifier>
<scope>provided</scope>
</dependency>
<dependency>
diff --git a/sql/hive/pom.xml b/sql/hive/pom.xml
index fe144c7..dc8b733 100644
--- a/sql/hive/pom.xml
+++ b/sql/hive/pom.xml
@@ -37,7 +37,7 @@
<dependencies>
<!-- Added for Hive Parquet SerDe -->
<dependency>
- <groupId>com.twitter</groupId>
+ <groupId>${hive.parquet.group}</groupId>
<artifactId>parquet-hadoop-bundle</artifactId>
</dependency>
<dependency>
@@ -97,9 +97,7 @@
<dependency>
<groupId>${hive.group}</groupId>
<artifactId>hive-exec</artifactId>
-<!--
- <classifier>core</classifier>
--->
+ <classifier>${hive.classifier}</classifier>
</dependency>
<dependency>
<groupId>${hive.group}</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org