You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ya...@apache.org on 2025/05/21 07:04:49 UTC

(spark) branch master updated: [SPARK-51927][SPARK-52213][BUILD] Upgrade jackson to 2.19.0 and upgrade kubernetes-client to version 7.3.0

This is an automated email from the ASF dual-hosted git repository.

yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new b84d3e6cca95 [SPARK-51927][SPARK-52213][BUILD] Upgrade jackson to 2.19.0 and upgrade kubernetes-client to version 7.3.0
b84d3e6cca95 is described below

commit b84d3e6cca95470d3c26e76861b31308ec236e14
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Wed May 21 15:04:34 2025 +0800

    [SPARK-51927][SPARK-52213][BUILD] Upgrade jackson to 2.19.0 and upgrade kubernetes-client to version 7.3.0
    
    ### What changes were proposed in this pull request?
    The primary objective of this pr is to upgrade Jackson from 2.18.2 to 2.19.0, and simultaneously upgrade the kubernetes-client from 7.2.0 to 7.3.0 to ensure compatibility with Jackson 2.19.0.
    
    ### Why are the changes needed?
    The new version of Jackson brings several bug fixes:
    - https://github.com/FasterXML/jackson-databind/issues/4849
    - https://github.com/FasterXML/jackson-databind/issues/4934
    - https://github.com/FasterXML/jackson-databind/issues/5052
    - https://github.com/FasterXML/jackson-databind/issues/4953
    
    The full release notes as follow:
    - https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.19
    
    The release of kubernetes-client 7.3.0 is solely for the purpose of ensuring compatibility with Jackson 2.19.0:
    - https://github.com/fabric8io/kubernetes-client/releases/tag/v7.3.0
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    - Pass GitHub Actions
    - manual check:
    
    ```
    build/sbt clean package -Phive
    python/run-tests --testnames 'pyspark.sql.tests.arrow.test_arrow_python_udf' --python-executables=python3.11
    ```
    
    ```
    python/run-tests --testnames 'pyspark.sql.tests.arrow.test_arrow_python_udf' --python-executables=python3.11
    Running PySpark tests. Output is in /Users/yangjie01/SourceCode/git/spark-sbt/python/unit-tests.log
    Will test against the following Python executables: ['python3.11']
    Will test the following Python tests: ['pyspark.sql.tests.arrow.test_arrow_python_udf']
    python3.11 python_implementation is CPython
    python3.11 version is: Python 3.11.12
    Starting test(python3.11): pyspark.sql.tests.arrow.test_arrow_python_udf (temp output: /Users/yangjie01/SourceCode/git/spark-sbt/python/target/83c10dc1-64a2-4b7d-80b4-4977dadd26fa/python3.11__pyspark.sql.tests.arrow.test_arrow_python_udf___gzv2b5u.log)
    Finished test(python3.11): pyspark.sql.tests.arrow.test_arrow_python_udf (62s) ... 8 tests were skipped
    Tests passed in 62 seconds
    
    Skipped tests in pyspark.sql.tests.arrow.test_arrow_python_udf with python3.11:
          test_broadcast_in_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_broadcast_in_udf) ... skip (0.000s)
          test_datasource_with_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_datasource_with_udf) ... skip (0.001s)
          test_register_java_function (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_register_java_function) ... skip (0.000s)
          test_register_java_udaf (pyspark.sql.tests.arrow.test_arrow_python_udf.ArrowPythonUDFTests.test_register_java_udaf) ... skip (0.000s)
          test_broadcast_in_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_broadcast_in_udf) ... skip (0.000s)
          test_datasource_with_udf (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_datasource_with_udf) ... skip (0.001s)
          test_register_java_function (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_register_java_function) ... skip (0.000s)
          test_register_java_udaf (pyspark.sql.tests.arrow.test_arrow_python_udf.AsyncArrowPythonUDFTests.test_register_java_udaf) ... skip (0.000s)
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No
    
    Closes #50730 from LuciferYang/SPARK-51927.
    
    Lead-authored-by: yangjie01 <ya...@baidu.com>
    Co-authored-by: YangJie <ya...@baidu.com>
    Signed-off-by: yangjie01 <ya...@baidu.com>
---
 dev/deps/spark-deps-hadoop-3-hive-2.3 | 70 +++++++++++++++++------------------
 pom.xml                               |  6 +--
 2 files changed, 38 insertions(+), 38 deletions(-)

diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 7c29ccebf5f7..1f33c8efc665 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -104,15 +104,15 @@ ini4j/0.5.4//ini4j-0.5.4.jar
 istack-commons-runtime/4.1.2//istack-commons-runtime-4.1.2.jar
 ivy/2.5.3//ivy-2.5.3.jar
 j2objc-annotations/3.0.0//j2objc-annotations-3.0.0.jar
-jackson-annotations/2.18.2//jackson-annotations-2.18.2.jar
+jackson-annotations/2.19.0//jackson-annotations-2.19.0.jar
 jackson-core-asl/1.9.13//jackson-core-asl-1.9.13.jar
-jackson-core/2.18.2//jackson-core-2.18.2.jar
-jackson-databind/2.18.2//jackson-databind-2.18.2.jar
-jackson-dataformat-cbor/2.18.2//jackson-dataformat-cbor-2.18.2.jar
-jackson-dataformat-yaml/2.18.2//jackson-dataformat-yaml-2.18.2.jar
-jackson-datatype-jsr310/2.18.2//jackson-datatype-jsr310-2.18.2.jar
+jackson-core/2.19.0//jackson-core-2.19.0.jar
+jackson-databind/2.19.0//jackson-databind-2.19.0.jar
+jackson-dataformat-cbor/2.19.0//jackson-dataformat-cbor-2.19.0.jar
+jackson-dataformat-yaml/2.19.0//jackson-dataformat-yaml-2.19.0.jar
+jackson-datatype-jsr310/2.19.0//jackson-datatype-jsr310-2.19.0.jar
 jackson-mapper-asl/1.9.13//jackson-mapper-asl-1.9.13.jar
-jackson-module-scala_2.13/2.18.2//jackson-module-scala_2.13-2.18.2.jar
+jackson-module-scala_2.13/2.19.0//jackson-module-scala_2.13-2.19.0.jar
 jakarta.activation-api/2.1.3//jakarta.activation-api-2.1.3.jar
 jakarta.annotation-api/2.1.1//jakarta.annotation-api-2.1.1.jar
 jakarta.inject-api/2.0.1//jakarta.inject-api-2.0.1.jar
@@ -159,31 +159,31 @@ jsr305/3.0.0//jsr305-3.0.0.jar
 jta/1.1//jta-1.1.jar
 jul-to-slf4j/2.0.17//jul-to-slf4j-2.0.17.jar
 kryo-shaded/4.0.3//kryo-shaded-4.0.3.jar
-kubernetes-client-api/7.2.0//kubernetes-client-api-7.2.0.jar
-kubernetes-client/7.2.0//kubernetes-client-7.2.0.jar
-kubernetes-httpclient-vertx/7.2.0//kubernetes-httpclient-vertx-7.2.0.jar
-kubernetes-model-admissionregistration/7.2.0//kubernetes-model-admissionregistration-7.2.0.jar
-kubernetes-model-apiextensions/7.2.0//kubernetes-model-apiextensions-7.2.0.jar
-kubernetes-model-apps/7.2.0//kubernetes-model-apps-7.2.0.jar
-kubernetes-model-autoscaling/7.2.0//kubernetes-model-autoscaling-7.2.0.jar
-kubernetes-model-batch/7.2.0//kubernetes-model-batch-7.2.0.jar
-kubernetes-model-certificates/7.2.0//kubernetes-model-certificates-7.2.0.jar
-kubernetes-model-common/7.2.0//kubernetes-model-common-7.2.0.jar
-kubernetes-model-coordination/7.2.0//kubernetes-model-coordination-7.2.0.jar
-kubernetes-model-core/7.2.0//kubernetes-model-core-7.2.0.jar
-kubernetes-model-discovery/7.2.0//kubernetes-model-discovery-7.2.0.jar
-kubernetes-model-events/7.2.0//kubernetes-model-events-7.2.0.jar
-kubernetes-model-extensions/7.2.0//kubernetes-model-extensions-7.2.0.jar
-kubernetes-model-flowcontrol/7.2.0//kubernetes-model-flowcontrol-7.2.0.jar
-kubernetes-model-gatewayapi/7.2.0//kubernetes-model-gatewayapi-7.2.0.jar
-kubernetes-model-metrics/7.2.0//kubernetes-model-metrics-7.2.0.jar
-kubernetes-model-networking/7.2.0//kubernetes-model-networking-7.2.0.jar
-kubernetes-model-node/7.2.0//kubernetes-model-node-7.2.0.jar
-kubernetes-model-policy/7.2.0//kubernetes-model-policy-7.2.0.jar
-kubernetes-model-rbac/7.2.0//kubernetes-model-rbac-7.2.0.jar
-kubernetes-model-resource/7.2.0//kubernetes-model-resource-7.2.0.jar
-kubernetes-model-scheduling/7.2.0//kubernetes-model-scheduling-7.2.0.jar
-kubernetes-model-storageclass/7.2.0//kubernetes-model-storageclass-7.2.0.jar
+kubernetes-client-api/7.3.0//kubernetes-client-api-7.3.0.jar
+kubernetes-client/7.3.0//kubernetes-client-7.3.0.jar
+kubernetes-httpclient-vertx/7.3.0//kubernetes-httpclient-vertx-7.3.0.jar
+kubernetes-model-admissionregistration/7.3.0//kubernetes-model-admissionregistration-7.3.0.jar
+kubernetes-model-apiextensions/7.3.0//kubernetes-model-apiextensions-7.3.0.jar
+kubernetes-model-apps/7.3.0//kubernetes-model-apps-7.3.0.jar
+kubernetes-model-autoscaling/7.3.0//kubernetes-model-autoscaling-7.3.0.jar
+kubernetes-model-batch/7.3.0//kubernetes-model-batch-7.3.0.jar
+kubernetes-model-certificates/7.3.0//kubernetes-model-certificates-7.3.0.jar
+kubernetes-model-common/7.3.0//kubernetes-model-common-7.3.0.jar
+kubernetes-model-coordination/7.3.0//kubernetes-model-coordination-7.3.0.jar
+kubernetes-model-core/7.3.0//kubernetes-model-core-7.3.0.jar
+kubernetes-model-discovery/7.3.0//kubernetes-model-discovery-7.3.0.jar
+kubernetes-model-events/7.3.0//kubernetes-model-events-7.3.0.jar
+kubernetes-model-extensions/7.3.0//kubernetes-model-extensions-7.3.0.jar
+kubernetes-model-flowcontrol/7.3.0//kubernetes-model-flowcontrol-7.3.0.jar
+kubernetes-model-gatewayapi/7.3.0//kubernetes-model-gatewayapi-7.3.0.jar
+kubernetes-model-metrics/7.3.0//kubernetes-model-metrics-7.3.0.jar
+kubernetes-model-networking/7.3.0//kubernetes-model-networking-7.3.0.jar
+kubernetes-model-node/7.3.0//kubernetes-model-node-7.3.0.jar
+kubernetes-model-policy/7.3.0//kubernetes-model-policy-7.3.0.jar
+kubernetes-model-rbac/7.3.0//kubernetes-model-rbac-7.3.0.jar
+kubernetes-model-resource/7.3.0//kubernetes-model-resource-7.3.0.jar
+kubernetes-model-scheduling/7.3.0//kubernetes-model-scheduling-7.3.0.jar
+kubernetes-model-storageclass/7.3.0//kubernetes-model-storageclass-7.3.0.jar
 lapack/3.0.3//lapack-3.0.3.jar
 leveldbjni-all/1.8//leveldbjni-all-1.8.jar
 libfb303/0.9.3//libfb303-0.9.3.jar
@@ -241,7 +241,7 @@ orc-mapreduce/2.1.2/shaded-protobuf/orc-mapreduce-2.1.2-shaded-protobuf.jar
 orc-shims/2.1.2//orc-shims-2.1.2.jar
 oro/2.0.8//oro-2.0.8.jar
 osgi-resource-locator/1.0.3//osgi-resource-locator-1.0.3.jar
-paranamer/2.8//paranamer-2.8.jar
+paranamer/2.8.3//paranamer-2.8.3.jar
 parquet-column/1.15.2//parquet-column-1.15.2.jar
 parquet-common/1.15.2//parquet-common-1.15.2.jar
 parquet-encoding/1.15.2//parquet-encoding-1.15.2.jar
@@ -261,7 +261,7 @@ scala-reflect/2.13.16//scala-reflect-2.13.16.jar
 scala-xml_2.13/2.3.0//scala-xml_2.13-2.3.0.jar
 slf4j-api/2.0.17//slf4j-api-2.0.17.jar
 snakeyaml-engine/2.9//snakeyaml-engine-2.9.jar
-snakeyaml/2.3//snakeyaml-2.3.jar
+snakeyaml/2.4//snakeyaml-2.4.jar
 snappy-java/1.1.10.7//snappy-java-1.1.10.7.jar
 spire-macros_2.13/0.18.0//spire-macros_2.13-0.18.0.jar
 spire-platform_2.13/0.18.0//spire-platform_2.13-0.18.0.jar
@@ -282,7 +282,7 @@ wildfly-openssl/2.2.5.Final//wildfly-openssl-2.2.5.Final.jar
 xbean-asm9-shaded/4.27//xbean-asm9-shaded-4.27.jar
 xmlschema-core/2.3.1//xmlschema-core-2.3.1.jar
 xz/1.10//xz-1.10.jar
-zjsonpatch/7.2.0//zjsonpatch-7.2.0.jar
+zjsonpatch/7.3.0//zjsonpatch-7.3.0.jar
 zookeeper-jute/3.9.3//zookeeper-jute-3.9.3.jar
 zookeeper/3.9.3//zookeeper-3.9.3.jar
 zstd-jni/1.5.7-3//zstd-jni-1.5.7-3.jar
diff --git a/pom.xml b/pom.xml
index 3611aeae8ff5..027011f84935 100644
--- a/pom.xml
+++ b/pom.xml
@@ -182,8 +182,8 @@
     <scalafmt.validateOnly>true</scalafmt.validateOnly>
     <scalafmt.changedOnly>true</scalafmt.changedOnly>
     <codehaus.jackson.version>1.9.13</codehaus.jackson.version>
-    <fasterxml.jackson.version>2.18.2</fasterxml.jackson.version>
-    <fasterxml.jackson.databind.version>2.18.2</fasterxml.jackson.databind.version>
+    <fasterxml.jackson.version>2.19.0</fasterxml.jackson.version>
+    <fasterxml.jackson.databind.version>2.19.0</fasterxml.jackson.databind.version>
     <ws.xmlschema.version>2.3.1</ws.xmlschema.version>
     <snappy.version>1.1.10.7</snappy.version>
     <netlib.ludovic.dev.version>3.0.3</netlib.ludovic.dev.version>
@@ -237,7 +237,7 @@
 
     <!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
     <leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
-    <kubernetes-client.version>7.2.0</kubernetes-client.version>
+    <kubernetes-client.version>7.3.0</kubernetes-client.version>
 
     <test.java.home>${java.home}</test.java.home>
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org