You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "panbingkun (via GitHub)" <gi...@apache.org> on 2024/12/05 00:34:16 UTC
[PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
panbingkun opened a new pull request, #49066:
URL: https://github.com/apache/spark/pull/49066
### What changes were proposed in this pull request?
The pr aims to upgrade `kubernetes-client` from `6.13.2` to `7.0.0`.
### Why are the changes needed?
- The full release notes: https://github.com/fabric8io/kubernetes-client/releases/tag/v7.0.0
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1871615374
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -227,7 +235,7 @@ netty-transport-native-kqueue/4.1.115.Final/osx-x86_64/netty-transport-native-kq
netty-transport-native-unix-common/4.1.115.Final//netty-transport-native-unix-common-4.1.115.Final.jar
netty-transport/4.1.115.Final//netty-transport-4.1.115.Final.jar
objenesis/3.3//objenesis-3.3.jar
-okhttp/3.12.12//okhttp-3.12.12.jar
+okhttp/4.12.0//okhttp-4.12.0.jar
Review Comment:
I'm +1 for @pan3793 's comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun closed pull request #49066: [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0
URL: https://github.com/apache/spark/pull/49066
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1870537088
##########
resource-managers/kubernetes/core/pom.xml:
##########
@@ -37,7 +37,7 @@
<dependencies>
<dependency>
<groupId>io.fabric8</groupId>
- <artifactId>volcano-model-v1beta1</artifactId>
+ <artifactId>volcano-model</artifactId>
Review Comment:
<img width="1018" alt="image" src="https://github.com/user-attachments/assets/f3660ddc-3328-4403-af26-a37905bf5666">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1871614384
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -157,32 +158,37 @@ json4s-scalap_2.13/4.0.7//json4s-scalap_2.13-4.0.7.jar
jsr305/3.0.0//jsr305-3.0.0.jar
jta/1.1//jta-1.1.jar
jul-to-slf4j/2.0.16//jul-to-slf4j-2.0.16.jar
+kotlin-stdlib-common/1.8.21//kotlin-stdlib-common-1.8.21.jar
+kotlin-stdlib-jdk7/1.8.21//kotlin-stdlib-jdk7-1.8.21.jar
+kotlin-stdlib-jdk8/1.8.21//kotlin-stdlib-jdk8-1.8.21.jar
+kotlin-stdlib/1.8.21//kotlin-stdlib-1.8.21.jar
Review Comment:
Oh, is this correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1870543039
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -227,7 +235,7 @@ netty-transport-native-kqueue/4.1.115.Final/osx-x86_64/netty-transport-native-kq
netty-transport-native-unix-common/4.1.115.Final//netty-transport-native-unix-common-4.1.115.Final.jar
netty-transport/4.1.115.Final//netty-transport-4.1.115.Final.jar
objenesis/3.3//objenesis-3.3.jar
-okhttp/3.12.12//okhttp-3.12.12.jar
+okhttp/4.12.0//okhttp-4.12.0.jar
Review Comment:
Thanks! fine to me.
From `6.x` to `7.x`, we need to make some changes first, let's run GA first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1870523312
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -227,7 +235,7 @@ netty-transport-native-kqueue/4.1.115.Final/osx-x86_64/netty-transport-native-kq
netty-transport-native-unix-common/4.1.115.Final//netty-transport-native-unix-common-4.1.115.Final.jar
netty-transport/4.1.115.Final//netty-transport-4.1.115.Final.jar
objenesis/3.3//objenesis-3.3.jar
-okhttp/3.12.12//okhttp-3.12.12.jar
+okhttp/4.12.0//okhttp-4.12.0.jar
Review Comment:
The K8s client "Switch default HTTP client from OkHttp to Vert.x" https://github.com/fabric8io/kubernetes-client/issues/6470
instead of upgrading OkHttp4 and pulling Kotlin runtime deps, maybe following the upstream to switch to Vert.x is a better option?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #49066:
URL: https://github.com/apache/spark/pull/49066#issuecomment-2542821442
I have already created a new PR https://github.com/apache/spark/pull/49159 for this feature (switch `default` HttpClient implementation from `OkHttp` to `Vert.x`).
I will close this PR. cc @dongjoon-hyun @pan3793
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1870504191
##########
resource-managers/kubernetes/core/pom.xml:
##########
@@ -37,7 +37,7 @@
<dependencies>
<dependency>
<groupId>io.fabric8</groupId>
- <artifactId>volcano-model-v1beta1</artifactId>
+ <artifactId>volcano-model</artifactId>
Review Comment:
https://github.com/fabric8io/kubernetes-client/pull/6434
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1870523312
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -227,7 +235,7 @@ netty-transport-native-kqueue/4.1.115.Final/osx-x86_64/netty-transport-native-kq
netty-transport-native-unix-common/4.1.115.Final//netty-transport-native-unix-common-4.1.115.Final.jar
netty-transport/4.1.115.Final//netty-transport-4.1.115.Final.jar
objenesis/3.3//objenesis-3.3.jar
-okhttp/3.12.12//okhttp-3.12.12.jar
+okhttp/4.12.0//okhttp-4.12.0.jar
Review Comment:
The K8s client "Switch default HTTP client from OkHttp to Vert.x" in https://github.com/fabric8io/kubernetes-client/issues/6470
instead of upgrading OkHttp4 and pulling Kotlin runtime deps, maybe following the upstream to switch to Vert.x is a better option?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #49066:
URL: https://github.com/apache/spark/pull/49066#issuecomment-2521880280
> For the record, K8s CA seems to fail still for some reason. (Of course, that's the reason of `Draft`).
Well, yes, I'm working on it and it will take some time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50493][BUILD] Upgrade kubernetes-client to 7.0.0 [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49066:
URL: https://github.com/apache/spark/pull/49066#discussion_r1870543039
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -227,7 +235,7 @@ netty-transport-native-kqueue/4.1.115.Final/osx-x86_64/netty-transport-native-kq
netty-transport-native-unix-common/4.1.115.Final//netty-transport-native-unix-common-4.1.115.Final.jar
netty-transport/4.1.115.Final//netty-transport-4.1.115.Final.jar
objenesis/3.3//objenesis-3.3.jar
-okhttp/3.12.12//okhttp-3.12.12.jar
+okhttp/4.12.0//okhttp-4.12.0.jar
Review Comment:
Fine to me. From `6.x` to `7.x`, we need to make some changes first, let's run GA first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org