You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2022/11/08 01:36:40 UTC
[spark] branch master updated: [SPARK-41031][BUILD] Upgrade `xz` to 1.9 for `avro` 1.11.1
This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e9503c84c4d [SPARK-41031][BUILD] Upgrade `xz` to 1.9 for `avro` 1.11.1
e9503c84c4d is described below
commit e9503c84c4d8d4b51844a195523ebf064bdf185e
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Mon Nov 7 19:36:29 2022 -0600
[SPARK-41031][BUILD] Upgrade `xz` to 1.9 for `avro` 1.11.1
### What changes were proposed in this pull request?
This pr aims to upgrade `xz` to 1.9 for `avro` 1.11.1.
### Why are the changes needed?
Spark depend on `avro` 1.11.1 and `avro` 1.11.1 use `xz` as an optional dependency, we need to manually check `xz` version when upgrading `avro`.
https://github.com/apache/avro/blob/3a9e5a789b5165e0c8c4da799c387fdf84bfb75e/lang/java/pom.xml#L59
https://github.com/apache/avro/blob/3a9e5a789b5165e0c8c4da799c387fdf84bfb75e/lang/java/avro/pom.xml#L238-L242
The release notes as follows:
- https://git.tukaani.org/?p=xz-java.git;a=blob;f=NEWS;hb=HEAD
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass Github Actions
Closes #38538 from LuciferYang/SPARK-41031.
Authored-by: yangjie01 <ya...@baidu.com>
Signed-off-by: Sean Owen <sr...@gmail.com>
---
dev/deps/spark-deps-hadoop-2-hive-2.3 | 2 +-
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 6 +++++-
3 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-2-hive-2.3 b/dev/deps/spark-deps-hadoop-2-hive-2.3
index 283d93a4e60..6b87c27d4bd 100644
--- a/dev/deps/spark-deps-hadoop-2-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-2-hive-2.3
@@ -267,7 +267,7 @@ xbean-asm9-shaded/4.22//xbean-asm9-shaded-4.22.jar
xercesImpl/2.12.2//xercesImpl-2.12.2.jar
xml-apis/1.4.01//xml-apis-1.4.01.jar
xmlenc/0.52//xmlenc-0.52.jar
-xz/1.8//xz-1.8.jar
+xz/1.9//xz-1.9.jar
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
zookeeper-jute/3.6.2//zookeeper-jute-3.6.2.jar
zookeeper/3.6.2//zookeeper-3.6.2.jar
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3
index fb9beebeaa0..db5af8881c2 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -252,7 +252,7 @@ univocity-parsers/2.9.1//univocity-parsers-2.9.1.jar
velocity/1.5//velocity-1.5.jar
wildfly-openssl/1.0.7.Final//wildfly-openssl-1.0.7.Final.jar
xbean-asm9-shaded/4.22//xbean-asm9-shaded-4.22.jar
-xz/1.8//xz-1.8.jar
+xz/1.9//xz-1.9.jar
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
zookeeper-jute/3.6.2//zookeeper-jute-3.6.2.jar
zookeeper/3.6.2//zookeeper-3.6.2.jar
diff --git a/pom.xml b/pom.xml
index 1c494669455..38ba2b14008 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1482,10 +1482,14 @@
</exclusion>
</exclusions>
</dependency>
+ <!--
+ SPARK-41031: `xz` is marked as optional in the dependency tree of `avro`,
+ we need to manually check `xz` version when upgrading `avro`.
+ -->
<dependency>
<groupId>org.tukaani</groupId>
<artifactId>xz</artifactId>
- <version>1.8</version>
+ <version>1.9</version>
</dependency>
<!-- See SPARK-23654 for info on this dependency;
It is used to keep javax.activation at v1.1.1 after dropping
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org