You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2024/07/19 04:49:24 UTC
(spark) branch master updated: [SPARK-48940][BUILD] Upgrade `Arrow` to 17.0.0
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 1645046fbb83 [SPARK-48940][BUILD] Upgrade `Arrow` to 17.0.0
1645046fbb83 is described below
commit 1645046fbb83de255f2140d7e9e66b509422d512
Author: panbingkun <pa...@baidu.com>
AuthorDate: Thu Jul 18 21:49:12 2024 -0700
[SPARK-48940][BUILD] Upgrade `Arrow` to 17.0.0
### What changes were proposed in this pull request?
The pr aims to upgrade `arrow` from `16.1.0` to `17.0.0`.
### Why are the changes needed?
The full release notes: https://arrow.apache.org/release/17.0.0.html
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #47409 from panbingkun/SPARK-48940.
Authored-by: panbingkun <pa...@baidu.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 13 ++++++-------
pom.xml | 2 +-
2 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3
index f0d50544c84d..0d786bf49b43 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -16,11 +16,11 @@ antlr4-runtime/4.13.1//antlr4-runtime-4.13.1.jar
aopalliance-repackaged/3.0.3//aopalliance-repackaged-3.0.3.jar
arpack/3.0.3//arpack-3.0.3.jar
arpack_combined_all/0.1//arpack_combined_all-0.1.jar
-arrow-format/16.1.0//arrow-format-16.1.0.jar
-arrow-memory-core/16.1.0//arrow-memory-core-16.1.0.jar
-arrow-memory-netty-buffer-patch/16.1.0//arrow-memory-netty-buffer-patch-16.1.0.jar
-arrow-memory-netty/16.1.0//arrow-memory-netty-16.1.0.jar
-arrow-vector/16.1.0//arrow-vector-16.1.0.jar
+arrow-format/17.0.0//arrow-format-17.0.0.jar
+arrow-memory-core/17.0.0//arrow-memory-core-17.0.0.jar
+arrow-memory-netty-buffer-patch/17.0.0//arrow-memory-netty-buffer-patch-17.0.0.jar
+arrow-memory-netty/17.0.0//arrow-memory-netty-17.0.0.jar
+arrow-vector/17.0.0//arrow-vector-17.0.0.jar
audience-annotations/0.12.0//audience-annotations-0.12.0.jar
avro-ipc/1.11.3//avro-ipc-1.11.3.jar
avro-mapred/1.11.3//avro-mapred-1.11.3.jar
@@ -33,7 +33,6 @@ breeze-macros_2.13/2.1.0//breeze-macros_2.13-2.1.0.jar
breeze_2.13/2.1.0//breeze_2.13-2.1.0.jar
bundle/2.24.6//bundle-2.24.6.jar
cats-kernel_2.13/2.8.0//cats-kernel_2.13-2.8.0.jar
-checker-qual/3.42.0//checker-qual-3.42.0.jar
chill-java/0.10.0//chill-java-0.10.0.jar
chill_2.13/0.10.0//chill_2.13-0.10.0.jar
commons-cli/1.8.0//commons-cli-1.8.0.jar
@@ -64,7 +63,7 @@ derbyshared/10.16.1.1//derbyshared-10.16.1.1.jar
derbytools/10.16.1.1//derbytools-10.16.1.1.jar
dropwizard-metrics-hadoop-metrics2-reporter/0.1.2//dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar
esdk-obs-java/3.20.4.2//esdk-obs-java-3.20.4.2.jar
-flatbuffers-java/23.5.26//flatbuffers-java-23.5.26.jar
+flatbuffers-java/24.3.25//flatbuffers-java-24.3.25.jar
gcs-connector/hadoop3-2.2.21/shaded/gcs-connector-hadoop3-2.2.21-shaded.jar
gmetric4j/1.0.10//gmetric4j-1.0.10.jar
gson/2.2.4//gson-2.2.4.jar
diff --git a/pom.xml b/pom.xml
index e00f142db66a..88a1fff8e71c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -225,7 +225,7 @@
./python/pyspark/sql/pandas/utils.py, ./python/packaging/classic/setup.py
and ./python/packaging/connect/setup.py too.
-->
- <arrow.version>16.1.0</arrow.version>
+ <arrow.version>17.0.0</arrow.version>
<ammonite.version>3.0.0-M2</ammonite.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org