You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ya...@apache.org on 2025/05/21 07:04:49 UTC
(spark) branch master updated: [SPARK-51927][SPARK-52213][BUILD] Upgrade jackson to 2.19.0 and upgrade kubernetes-client to version 7.3.0
This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b84d3e6cca95 [SPARK-51927][SPARK-52213][BUILD] Upgrade jackson to 2.19.0 and upgrade kubernetes-client to version 7.3.0
b84d3e6cca95 is described below
commit b84d3e6cca95470d3c26e76861b31308ec236e14
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Wed May 21 15:04:34 2025 +0800
[SPARK-51927][SPARK-52213][BUILD] Upgrade jackson to 2.19.0 and upgrade kubernetes-client to version 7.3.0
### What changes were proposed in this pull request?
The primary objective of this pr is to upgrade Jackson from 2.18.2 to 2.19.0, and simultaneously upgrade the kubernetes-client from 7.2.0 to 7.3.0 to ensure compatibility with Jackson 2.19.0.
### Why are the changes needed?
The new version of Jackson brings several bug fixes:
- https://github.com/FasterXML/jackson-databind/issues/4849
- https://github.com/FasterXML/jackson-databind/issues/4934
- https://github.com/FasterXML/jackson-databind/issues/5052
- https://github.com/FasterXML/jackson-databind/issues/4953
The full release notes as follow:
- https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.19
The release of kubernetes-client 7.3.0 is solely for the purpose of ensuring compatibility with Jackson 2.19.0:
- https://github.com/fabric8io/kubernetes-client/releases/tag/v7.3.0
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- manual check:
```
build/sbt clean package -Phive
python/run-tests --testnames 'pyspark.sql.tests.arrow.test_arrow_python_udf' --python-executables=python3.11
```
```
python/run-tests --testnames 'pyspark.sql.tests.arrow.test_arrow_python_udf' --python-executables=python3.11
Running PySpark tests. Output is in /Users/yangjie01/SourceCode/git/spark-sbt/python/unit-tests.log
Will test against the following Python executables: ['python3.11']
Will test the following Python tests: ['pyspark.sql.tests.arrow.test_arrow_python_udf']
python3.11 python_implementation is CPython
python3.11 version is: Python 3.11.12
Starting test(python3.11): pyspark.sql.tests.arrow.test_arrow_python_udf (temp output: /Users/yangjie01/SourceCode/git/spark-sbt/python/target/83c10dc1-64a2-4b7d-80b4-4977dadd26fa/python3.11__pyspark.sql.tests.arrow.test_arrow_python_udf___gzv2b5u.log)
Finished test(python3.11): pyspark.sql.tests.arrow.test_arrow_python_udf (62s) ... 8 tests were skipped
Tests passed in 62 seconds
Skipped tests in pyspark.sql.tests.arrow.test_arrow_python_udf with python3.11:
test_broadcast_in_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_broadcast_in_udf) ... skip (0.000s)
test_datasource_with_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_datasource_with_udf) ... skip (0.001s)
test_register_java_function (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_register_java_function) ... skip (0.000s)
test_register_java_udaf (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_register_java_udaf) ... skip (0.000s)
test_broadcast_in_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_broadcast_in_udf) ... skip (0.000s)
test_datasource_with_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_datasource_with_udf) ... skip (0.001s)
test_register_java_function (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_register_java_function) ... skip (0.000s)
test_register_java_udaf (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_register_java_udaf) ... skip (0.000s)
```
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #50730 from LuciferYang/SPARK-51927.
Lead-authored-by: yangjie01 <ya...@baidu.com>
Co-authored-by: YangJie <ya...@baidu.com>
Signed-off-by: yangjie01 <ya...@baidu.com>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 70 +++++++++++++++++------------------
pom.xml | 6 +--
2 files changed, 38 insertions(+), 38 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 7c29ccebf5f7..1f33c8efc665 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -104,15 +104,15 @@ ini4j/0.5.4//ini4j-0.5.4.jar
istack-commons-runtime/4.1.2//istack-commons-runtime-4.1.2.jar
ivy/2.5.3//ivy-2.5.3.jar
j2objc-annotations/3.0.0//j2objc-annotations-3.0.0.jar
-jackson-annotations/2.18.2//jackson-annotations-2.18.2.jar
+jackson-annotations/2.19.0//jackson-annotations-2.19.0.jar
jackson-core-asl/1.9.13//jackson-core-asl-1.9.13.jar
-jackson-core/2.18.2//jackson-core-2.18.2.jar
-jackson-databind/2.18.2//jackson-databind-2.18.2.jar
-jackson-dataformat-cbor/2.18.2//jackson-dataformat-cbor-2.18.2.jar
-jackson-dataformat-yaml/2.18.2//jackson-dataformat-yaml-2.18.2.jar
-jackson-datatype-jsr310/2.18.2//jackson-datatype-jsr310-2.18.2.jar
+jackson-core/2.19.0//jackson-core-2.19.0.jar
+jackson-databind/2.19.0//jackson-databind-2.19.0.jar
+jackson-dataformat-cbor/2.19.0//jackson-dataformat-cbor-2.19.0.jar
+jackson-dataformat-yaml/2.19.0//jackson-dataformat-yaml-2.19.0.jar
+jackson-datatype-jsr310/2.19.0//jackson-datatype-jsr310-2.19.0.jar
jackson-mapper-asl/1.9.13//jackson-mapper-asl-1.9.13.jar
-jackson-module-scala_2.13/2.18.2//jackson-module-scala_2.13-2.18.2.jar
+jackson-module-scala_2.13/2.19.0//jackson-module-scala_2.13-2.19.0.jar
jakarta.activation-api/2.1.3//jakarta.activation-api-2.1.3.jar
jakarta.annotation-api/2.1.1//jakarta.annotation-api-2.1.1.jar
jakarta.inject-api/2.0.1//jakarta.inject-api-2.0.1.jar
@@ -159,31 +159,31 @@ jsr305/3.0.0//jsr305-3.0.0.jar
jta/1.1//jta-1.1.jar
jul-to-slf4j/2.0.17//jul-to-slf4j-2.0.17.jar
kryo-shaded/4.0.3//kryo-shaded-4.0.3.jar
-kubernetes-client-api/7.2.0//kubernetes-client-api-7.2.0.jar
-kubernetes-client/7.2.0//kubernetes-client-7.2.0.jar
-kubernetes-httpclient-vertx/7.2.0//kubernetes-httpclient-vertx-7.2.0.jar
-kubernetes-model-admissionregistration/7.2.0//kubernetes-model-admissionregistration-7.2.0.jar
-kubernetes-model-apiextensions/7.2.0//kubernetes-model-apiextensions-7.2.0.jar
-kubernetes-model-apps/7.2.0//kubernetes-model-apps-7.2.0.jar
-kubernetes-model-autoscaling/7.2.0//kubernetes-model-autoscaling-7.2.0.jar
-kubernetes-model-batch/7.2.0//kubernetes-model-batch-7.2.0.jar
-kubernetes-model-certificates/7.2.0//kubernetes-model-certificates-7.2.0.jar
-kubernetes-model-common/7.2.0//kubernetes-model-common-7.2.0.jar
-kubernetes-model-coordination/7.2.0//kubernetes-model-coordination-7.2.0.jar
-kubernetes-model-core/7.2.0//kubernetes-model-core-7.2.0.jar
-kubernetes-model-discovery/7.2.0//kubernetes-model-discovery-7.2.0.jar
-kubernetes-model-events/7.2.0//kubernetes-model-events-7.2.0.jar
-kubernetes-model-extensions/7.2.0//kubernetes-model-extensions-7.2.0.jar
-kubernetes-model-flowcontrol/7.2.0//kubernetes-model-flowcontrol-7.2.0.jar
-kubernetes-model-gatewayapi/7.2.0//kubernetes-model-gatewayapi-7.2.0.jar
-kubernetes-model-metrics/7.2.0//kubernetes-model-metrics-7.2.0.jar
-kubernetes-model-networking/7.2.0//kubernetes-model-networking-7.2.0.jar
-kubernetes-model-node/7.2.0//kubernetes-model-node-7.2.0.jar
-kubernetes-model-policy/7.2.0//kubernetes-model-policy-7.2.0.jar
-kubernetes-model-rbac/7.2.0//kubernetes-model-rbac-7.2.0.jar
-kubernetes-model-resource/7.2.0//kubernetes-model-resource-7.2.0.jar
-kubernetes-model-scheduling/7.2.0//kubernetes-model-scheduling-7.2.0.jar
-kubernetes-model-storageclass/7.2.0//kubernetes-model-storageclass-7.2.0.jar
+kubernetes-client-api/7.3.0//kubernetes-client-api-7.3.0.jar
+kubernetes-client/7.3.0//kubernetes-client-7.3.0.jar
+kubernetes-httpclient-vertx/7.3.0//kubernetes-httpclient-vertx-7.3.0.jar
+kubernetes-model-admissionregistration/7.3.0//kubernetes-model-admissionregistration-7.3.0.jar
+kubernetes-model-apiextensions/7.3.0//kubernetes-model-apiextensions-7.3.0.jar
+kubernetes-model-apps/7.3.0//kubernetes-model-apps-7.3.0.jar
+kubernetes-model-autoscaling/7.3.0//kubernetes-model-autoscaling-7.3.0.jar
+kubernetes-model-batch/7.3.0//kubernetes-model-batch-7.3.0.jar
+kubernetes-model-certificates/7.3.0//kubernetes-model-certificates-7.3.0.jar
+kubernetes-model-common/7.3.0//kubernetes-model-common-7.3.0.jar
+kubernetes-model-coordination/7.3.0//kubernetes-model-coordination-7.3.0.jar
+kubernetes-model-core/7.3.0//kubernetes-model-core-7.3.0.jar
+kubernetes-model-discovery/7.3.0//kubernetes-model-discovery-7.3.0.jar
+kubernetes-model-events/7.3.0//kubernetes-model-events-7.3.0.jar
+kubernetes-model-extensions/7.3.0//kubernetes-model-extensions-7.3.0.jar
+kubernetes-model-flowcontrol/7.3.0//kubernetes-model-flowcontrol-7.3.0.jar
+kubernetes-model-gatewayapi/7.3.0//kubernetes-model-gatewayapi-7.3.0.jar
+kubernetes-model-metrics/7.3.0//kubernetes-model-metrics-7.3.0.jar
+kubernetes-model-networking/7.3.0//kubernetes-model-networking-7.3.0.jar
+kubernetes-model-node/7.3.0//kubernetes-model-node-7.3.0.jar
+kubernetes-model-policy/7.3.0//kubernetes-model-policy-7.3.0.jar
+kubernetes-model-rbac/7.3.0//kubernetes-model-rbac-7.3.0.jar
+kubernetes-model-resource/7.3.0//kubernetes-model-resource-7.3.0.jar
+kubernetes-model-scheduling/7.3.0//kubernetes-model-scheduling-7.3.0.jar
+kubernetes-model-storageclass/7.3.0//kubernetes-model-storageclass-7.3.0.jar
lapack/3.0.3//lapack-3.0.3.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
libfb303/0.9.3//libfb303-0.9.3.jar
@@ -241,7 +241,7 @@ orc-mapreduce/2.1.2/shaded-protobuf/orc-mapreduce-2.1.2-shaded-protobuf.jar
orc-shims/2.1.2//orc-shims-2.1.2.jar
oro/2.0.8//oro-2.0.8.jar
osgi-resource-locator/1.0.3//osgi-resource-locator-1.0.3.jar
-paranamer/2.8//paranamer-2.8.jar
+paranamer/2.8.3//paranamer-2.8.3.jar
parquet-column/1.15.2//parquet-column-1.15.2.jar
parquet-common/1.15.2//parquet-common-1.15.2.jar
parquet-encoding/1.15.2//parquet-encoding-1.15.2.jar
@@ -261,7 +261,7 @@ scala-reflect/2.13.16//scala-reflect-2.13.16.jar
scala-xml_2.13/2.3.0//scala-xml_2.13-2.3.0.jar
slf4j-api/2.0.17//slf4j-api-2.0.17.jar
snakeyaml-engine/2.9//snakeyaml-engine-2.9.jar
-snakeyaml/2.3//snakeyaml-2.3.jar
+snakeyaml/2.4//snakeyaml-2.4.jar
snappy-java/1.1.10.7//snappy-java-1.1.10.7.jar
spire-macros_2.13/0.18.0//spire-macros_2.13-0.18.0.jar
spire-platform_2.13/0.18.0//spire-platform_2.13-0.18.0.jar
@@ -282,7 +282,7 @@ wildfly-openssl/2.2.5.Final//wildfly-openssl-2.2.5.Final.jar
xbean-asm9-shaded/4.27//xbean-asm9-shaded-4.27.jar
xmlschema-core/2.3.1//xmlschema-core-2.3.1.jar
xz/1.10//xz-1.10.jar
-zjsonpatch/7.2.0//zjsonpatch-7.2.0.jar
+zjsonpatch/7.3.0//zjsonpatch-7.3.0.jar
zookeeper-jute/3.9.3//zookeeper-jute-3.9.3.jar
zookeeper/3.9.3//zookeeper-3.9.3.jar
zstd-jni/1.5.7-3//zstd-jni-1.5.7-3.jar
diff --git a/pom.xml b/pom.xml
index 3611aeae8ff5..027011f84935 100644
--- a/pom.xml
+++ b/pom.xml
@@ -182,8 +182,8 @@
<scalafmt.validateOnly>true</scalafmt.validateOnly>
<scalafmt.changedOnly>true</scalafmt.changedOnly>
<codehaus.jackson.version>1.9.13</codehaus.jackson.version>
- <fasterxml.jackson.version>2.18.2</fasterxml.jackson.version>
- <fasterxml.jackson.databind.version>2.18.2</fasterxml.jackson.databind.version>
+ <fasterxml.jackson.version>2.19.0</fasterxml.jackson.version>
+ <fasterxml.jackson.databind.version>2.19.0</fasterxml.jackson.databind.version>
<ws.xmlschema.version>2.3.1</ws.xmlschema.version>
<snappy.version>1.1.10.7</snappy.version>
<netlib.ludovic.dev.version>3.0.3</netlib.ludovic.dev.version>
@@ -237,7 +237,7 @@
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
- <kubernetes-client.version>7.2.0</kubernetes-client.version>
+ <kubernetes-client.version>7.3.0</kubernetes-client.version>
<test.java.home>${java.home}</test.java.home>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org