You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2019/01/10 08:40:53 UTC
[spark] branch branch-2.4 updated: [SPARK-22128][CORE][BUILD] Add
`paranamer` dependency to `core` module
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new da0b69f [SPARK-22128][CORE][BUILD] Add `paranamer` dependency to `core` module
da0b69f is described below
commit da0b69f4a68303f74350f0efd3ba90745c32768c
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Jan 10 00:40:21 2019 -0800
[SPARK-22128][CORE][BUILD] Add `paranamer` dependency to `core` module
## What changes were proposed in this pull request?
With Scala-2.12 profile, Spark application fails while Spark is okay. For example, our documented `SimpleApp` Java example succeeds to compile but it fails at runtime because it doesn't use `paranamer 2.8` and hits [SPARK-22128](https://issues.apache.org/jira/browse/SPARK-22128). This PR aims to declare it explicitly for the Spark applications. Note that this doesn't introduce new dependency to Spark itself.
https://dist.apache.org/repos/dist/dev/spark/3.0.0-SNAPSHOT-2019_01_09_13_59-e853afb-docs/_site/quick-start.html
The following is the dependency tree from the Spark application.
**BEFORE**
```
$ mvn dependency:tree -Dincludes=com.thoughtworks.paranamer
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) simple ---
[INFO] my.test:simple:jar:1.0-SNAPSHOT
[INFO] \- org.apache.spark:spark-sql_2.12:jar:3.0.0-SNAPSHOT:compile
[INFO] \- org.apache.spark:spark-core_2.12:jar:3.0.0-SNAPSHOT:compile
[INFO] \- org.apache.avro:avro:jar:1.8.2:compile
[INFO] \- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
```
**AFTER**
```
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) simple ---
[INFO] my.test:simple:jar:1.0-SNAPSHOT
[INFO] \- org.apache.spark:spark-sql_2.12:jar:3.0.0-SNAPSHOT:compile
[INFO] \- org.apache.spark:spark-core_2.12:jar:3.0.0-SNAPSHOT:compile
[INFO] \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile
```
## How was this patch tested?
Pass the Jenkins. And manually test with the sample app is running.
Closes #23502 from dongjoon-hyun/SPARK-26583.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit c7daa95d7f095500b416ba405660f98cd2a39727)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
core/pom.xml | 4 ++++
pom.xml | 1 +
2 files changed, 5 insertions(+)
diff --git a/core/pom.xml b/core/pom.xml
index 583a2ff..d8b078e 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -34,6 +34,10 @@
<url>http://spark.apache.org/</url>
<dependencies>
<dependency>
+ <groupId>com.thoughtworks.paranamer</groupId>
+ <artifactId>paranamer</artifactId>
+ </dependency>
+ <dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
</dependency>
diff --git a/pom.xml b/pom.xml
index c8a1b03..f802786 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1978,6 +1978,7 @@
<groupId>com.thoughtworks.paranamer</groupId>
<artifactId>paranamer</artifactId>
<version>${paranamer.version}</version>
+ <scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org